Current Stuff

What I'm working on

• Counterterrorism

  Problems of counterterrorism (and fraud which is closely related) form a major application focus for my work. Strong data mining techniques can help detect terrorist attacks before they happen (or make sure that the culprits pay a price afterwards). All of my work is unclassified. Working on this problem outside the intelligence community provides a different viewpoint so that useful ideas are not missed; provides trained people who can work in the intelligence sector; and means that I can discuss the issues raised by counterterrorism (e.g. privacy) in public and with the media.

  Fraud is a major problem in developed countries, accounting for perhaps 8-12% of GDP, especially in sectors such as insurance, health, taxation, and customs. On the other hand, each dollar spent on fraud detection has an expected return of $25-29. Data mining for fraud detection has the potential to be as large a market as customer relationship management. I am working on the following topics at present:

  • Anomalies in graphs. Much surveillance data is naturally expressed as graphs, connecting people, places, and things, rather than as attributed data. Recent developments in spectral graph analysis make it possible to discover deep properties from such graph-based data. Some of the useful applications for counterterrorism and fraud are link prediction and anomalous substructure discovery.
  • Anomalies in texts and documents. Writing or speaking always reveals something about the author or speaker. We are looking at how this can be used to select interesting documents in worldwide settings (for example, because they contain unusual patterns suggesting the presence of obfuscation); to detect authorship or properties of authors; to detect deception; and to detection sentiment or intention.
  • Novel information discovery in unstructured data. We have developed the ATHENS system which finds novel information in large hypermedia repositories.

    Enron. We have analysed emails collected in the three years before the collapse of Enron to try and understand how email is used to communicate, and how deceptiveness of various kinds shows up in such data. We have also been able to validate models of deception against this dataset.

  • Anomaly and interestingness detection. For very large datasets, it is often the outliers that are most interesting in counterterrorism and fraud applications. However, in these settings, those who are up to no good are trying not to generate data values that will make them stand out.
  • Ensemble and sampling techniques and their application to intelligence silos and privacy preservation. The problems of separate intelligence gathering systems (`silos') is widely acknowledged. However, data mining also shows us that opinions from separate models can act to cancel variance.
  • Staged exploration and data mining, with intervention points. Many people are rightly concerned about widespread data gathering and analysis. Some of these concerns can be ameliorated by building models of innocence. A staged process can then eliminate most objects/people from further consideration at the first stage, allowing more subtle but expensive techniques to be used at the next stage. This can be repeated as necessary.
  • Belief revision. Understanding when something has changed in a dataset can also be an important signal. For example, terrorists who have been devout often assume more mainstream appearance and behaviour as they move into an attack phase.
  • Group structure; command and control extraction. Understanding the way in which terrorist organisations manage themselves can help to detect attacks or to suggest ways to subvert their goals. Command and control are often implicit in other kinds of connections among members: personal history, but also patterns of telephone calls.

• Treatment Strategies for Childhood Cancers

  At present, treatment strategies for most diseases are determined based only on the disease diagnosis, and perhaps some obvious properties of the patient. We are extending this to allow treament decisions to be made based on subtype of the disease (determined from microarray data), the patient's genetic makeup (determined, for example, from SNPs), other properties of the patient (for example, age), and constraints of the health system. We intend to build a system that will provide guidance to physicians of the treatment-outcome landscape. Such as system should improve patient recovery rates and reduce costs.

• Mineral Exploration

  Mineral exploration typically involves deep drilling which, because of its expense, can only be done at large intervals. We are exploring data mining of geochemical data from surface or near-surface samples to predict the presence of underlying mineralization.