Airline Security's False Hope?

The latest incarnation of security screening for our airports has many friends in high places. U.S. secretary of transportation Norman Mineta described it as “the foundation of aviation security.” The Transportation Security Administration has called the system-known as CAPPS II, meaning the second version of the Computer-Assisted Passenger Prescreening System-the most important single program on the agency’s agenda. Such votes of confidence, however, intensify my fear that this new system-scheduled for deployment in 2004-will do us more harm than good.

Like the CAPPS I system now in place, CAPPS II would sort passengers into two main groups: higher-risk “selectees” who get intense airport screening, and lower-risk non-selectees who are screened less. What differentiates CAPPS II from its predecessor is that it would use personal and demographic data in choosing the selectees. The system would interpret this information using techniques such as data mining-a fast trial-and-error search for patterns and correlations.

Having studied aviation safety and security for a quarter-century, particularly from the standpoint of statistical analysis, I have followed CAPPS II developments closely. This scrutiny leaves me with several misgivings.

The data-mining technology at the heart of the system works best when patterns in existing data can be expected to persist into the future. CAPPS II thus seeks to exploit commonalities among past terrorists as a means of identifying future ones. The limits of this approach, however, were indicated in the 2002 Washington sniper crisis, when experts who “mined” data about previous serial killers believed strongly that the perpetrators were white.

The value of the screening system depends largely on the assumption that potential terrorists could not understand the selection process well enough to outwit it. But someone wishing to beat the system need not know the exact formula; all he or she needs to determine is the likelihood of being personally targeted for high scrutiny. This can be accomplished very simply with trial runs at the airport: a would-be terrorist who breezes through check-in several times without being designated a selectee could reasonably expect that the same will happen the next time he or she arrives-with deadly weapons.

Stung by accusations that CAPPS II would entail ethnic and political profiling and violate personal privacy, the Transportation Security Administration has gone into retreat mode. Recent press reports have suggested that CAPPS II will attach no significance to a bad credit record, that passengers will have the right to appeal their “selectee” status, and that-astoundingly-CAPPS II might not even apply at all to U.S. citizens. Such concessions could enormously degrade the system’s predictive accuracy.

Still, CAPPS II could yield some security improvement over CAPPS I-as long as two conditions are met. The first is that CAPPS II designates the same percentage of passengers selectees as CAPPS I: that should ensure that the number of actual terrorists caught in the net goes up. The second condition is that security processing does not slacken under CAPPS II-for either selectees or non-selectees.

Alas, the evidence suggests that neither of these conditions will be met. The Transportation Security Administration says travelers “may well notice” that fewer passengers are designated selectees under CAPPS II, and has described non-selectees under the system as people who “clearly pose no threat of terrorism.” In other words, the agency envisions that CAPPS II will select fewer people but nab practically all terrorists. But I see no reason that, if the overall selection rate declines from, say, 5 percent to 2 percent, the selection rate among terrorists will rise: the new system probably isn’t that much better than CAPPS I. If security processing for non-selectees becomes less demanding (and after all, what sort of screening is needed for people who “pose no threat of terrorism?”), then a terrorist erroneously assigned to the low-risk group could have a greater chance of success under the new system than under the existing one.

Used wisely, CAPPS II could be a moderately helpful weapon in the antiterrorist arsenal. Unaccountably, however, many security planners see CAPPS II as the preeminent weapon, and their excess confidence in the system suggests that its forecasts will get more weight than is prudent. This supposed foundation of security could therefore pose a new security threat to U.S. air travelers.

This article originally appeared in the MIT Technology Insider, a monthly newsletter covering MIT research and commercial spinoff activity.