On Answers
Four kinds of search engines.
Search engines are knowledge systems that we ask, “What is known?” The answers we get reflect the questions the systems’ designers allow, which in turn reflect designers’ conceptions of what is knowable and useful to know.
The first search engines were not machines, and they didn’t satisfy their users. The most famous of them all, the Oracle of Apollo at Delphi on the slopes of Mount Parnassus, issued prophecies for more than a thousand years. We possess more than 500 of the results of queries put to the Pythia, the priestess who presided over the Oracle. With exceptions, her answers were not helpful.
King Croesus of Lydia once asked the Oracle if he should wage war on the Persians, whose empire was expanding westward after a successful revolt against their rulers, the Medes. According to the historian Herodotus, the Pythia answered that if he did, he would destroy a great empire. Cautious, Croesus sent a large fee to the Delphians, and refined his search terms. He pressed: Would his reign be a long one? The answer, according to my battered Penguin translation by A. R. Burn, came back:
“When comes the day that a mule shall sit on the Median throne, then, tender-footed Lydian, by pebbly Hermus run and abide not, nor think it shame to be a coward.”
Opaque–but Herodotus writes, “This reply gave Croesus more pleasure than anything he had yet heard; for he did not suppose that a mule was likely to become king of the Medes, and that meant that he and his line would remain in power forever.” Alert readers will have guessed the end. Croesus went to war; the empire he destroyed was his own. Cyrus, the king of the Persians, was half Persian and half Mede, and thus a kind of mule.
The answers of the Delphic Oracle abound in these sorts of tricky occlusions. Whoever designed the system at Delphi believed or pretended to believe that the future was known to the god Apollo, who chose (as a demonstration of the mutability of human affairs) to deliver through his priestesses prophecies that were obscure, but that retrospectively provided dramatic satisfaction. A rationalist will suspect that obscure answers had another function: they could apply equally well to different outcomes. In any case, the turbidity of the Oracle’s answers was its virtue.
In this month’s cover story (”Search Me”), Technology Review’s chief correspondent, David Talbot, describes how the Web is usually searched: “Among all the leaders in Web search … the core approach has remained the same. They create massive indexes of the Web–that is, their software continually ‘crawls’ the Web, collecting phrases, keywords, titles, and links.” Talbot examines some of the technical limitations of this method. But the notion that a search should produce a list of links to Web pages represents a view of what is knowable and what is useful to know that is as specific as that which made the Delphic Oracle. Traditional search is chaotically democratic. It assumes that the consensus view is the best, while rewarding the wayward answer by exposing it to the curious. The truths of traditional search are provisional. Popularity is virtue.
Our story describes a new kind of search engine, Wolfram Alpha. In fact, its inventor, the physicist and entrepreneur Stephen Wolfram, dislikes the word search: he calls it a “computational knowledge engine.” Alpha, writes Talbot, is “meant to compute answers rather than list Web pages.” It consists of “three elements … a constantly expanding collection of data sets, an elaborate calculator, and a natural-language interface for queries.”
Alpha, too, represents a particular point of view–that of its creator. Wolfram’s monumental book, A New Kind of Science (2002), explains how the complex world can be reduced to simple rules, and how those rules are computable. Alpha will be the first major application of his theories: an experiment to see how much of what is known can be expressed in straightforward answers.
About these fundamental questions, views differ. Ivan Herman of the World Wide Web Consortium tells Talbot, “Although I have graduated as a mathematician … I am not sure you can handle all of the miseries of this world by mathematical formula and computation.” Another critic provides an example: “Imagine a question like ‘Who are the most dangerous terrorists?’ … Is someone a terrorist? How do we assess danger? And danger to whom? It’s computationally very difficult to do that kind of reasoning.”
Perhaps, speculates Daniel Tunkelang, the cofounder of the search company Endeca, there is a better way to approach the problem of building a search engine (see “To Search, Ask”). “What we need is human-computer information retrieval. … Rather than guessing what users need, these tools provide users with opportunities to clarify and elaborate their intent. If the engine isn’t sure what users want, it just asks them.”
Now there’s an alternative that is somehow shocking: Ask the questioner. Write and tell me what you think at jason.pontin@technologyreview.com.
