Managing Antiterror Databases

As military and security agencies do everything from coordinating action in Iraq to sifting through telephone and banking records on the home front, they collect mountains of data that are difficult to organize and manipulate, and in which intelligence connections can get lost. Now, database tricks borrowed from the world of particle physics are helping intelligence and law enforcement agencies-and any other organizations that manage large, complex databases-to act on their data more nimbly.

The Stanford Linear Accelerator in Palo Alto, CA, is home to a particle collider that has generated the world’s largest database. The company that designed the database, Objectivity of Mountain View, CA, is now attracting a wave of new customers in the areas of defense, telecommunications, and intelligence systems, says Leon Guzenda, the company’s chief technology officer. TRW of Redondo Beach, CA (recently acquired by Northrop Grumman), says it uses Objectivity’s technology in a data analysis system built for an unnamed U.S. government agency that collects and analyzes scientific data. And SYColeman, a defense contractor in Sherman Oaks, CA, employs the technology in a system for controlling battlefield simulations. While Guzenda says security restrictions prevent him from describing specifics, he says the technology was used to help manage complex military operations in the Iraq war.

These national-security players are all attracted by the same thing: the system’s ability to keep complex relationships straight, even as databases swell to mammoth proportions. The underlying architecture of the database-in which data are stored in the form of “objects” that inherit their properties from parent objects in a vast family tree-can bring important connections to light more efficiently than traditional “relational” approaches, where data are stored in the rows and columns of interlinked tables, says Richard Winter, president of Winter Corporation, a database consulting firm in Waltham, MA. The new system has helped scientists keep track of Stanford’s particle collision data, which collectively takes up some 700 terabytes (700,000 gigabytes) of storage space, or 35 times as much as the contents of the Library of Congress (see “Surveillance Nation, Part Two,” TR May 2003). The “object-oriented” nature of the database makes it easy for scientists to create generations of objects from which more and more of the clutter has been extracted.

The same qualities make the database tool equally well suited to boosting national security. “Let’s say you are monitoring suspected terrorists’ telephone calls, e-mails, and the like. The database grows very large, very fast,” Winter says. In a relational database, the tables storing this information would form an abstract tangle, but a database like Stanford’s would more closely resemble the suspected terrorist network itself. “It can provide a groundbreaking kind of capability,” he says. Which means that, as government databases accumulate video surveillance data, communications intercepts, or battlefield intelligence, they’ll be able to detect where the data collides.