Google and the Coming Search Wars, Revisited

As some will know, I wrote about the future of Google for the January 2005 issue of Technology Review.

In that article, I argued that the search market will become enormously larger and more diverse than presently, encompassing many forms of personal, internal corporate, for-sale proprietary, and public data stored in a wide variety of systems, ranging from PCs to iPods to corporate servers. 

Then I asserted that Google’s leadership position remains fragile, given the absence of barriers to switching and an impending challenge from Microsoft.  In particular, the emerging requirement for search interoperability across many data formats and systems, and the diversity of search-related innovation, imply that search engines must become standardized platforms with open interfaces.

I concluded that unless Google created such a platform and made it generally accessible by providing Application Program Interfaces (APIs) to its search engine, it would both lose much of its potential market and render itself vulnerable to a Microsoft attack. 

Conversely a proprietary but open architecture based upon public APIs would enable Google to attract more users, create interoperability across many systems, and generate switching costs.

In part, the article proved to be prescient.  On March 17, Google announced an initiative to create developer communities based upon APIs, some of them newly released.  While Google still has not made available an API for its core search engine services, it may be moving in this direction.

The article also generated several criticisms.  One of these, which with some ironic license I have called the New Age Theory, runs as follows.  With the Internet, we have entered a new and better era, one in which openness will triumph over narrow, proprietary self-interest.   Because the Internet is a vast commons, the greatest fortunes will be made by those who are most open.  Moreover large users, by now fully educated about the dangers of lockin, no longer tolerate proprietary control.

Alas, while this vision of Eden is very seductive, the evidence doesn’t completely support it.  On the Web, most users and advertisers are dispersed and unorganized, and large websites behave accordingly.  Proprietary control is alive and well on the Web; indeed, the only thing more proprietary than a proprietary standard is a closed system with no interfaces at all, which is essentially what Google has traditionally operated (as have Yahoo and AOL).    Amazon and EBay both have, in essence, open but proprietary APIs.  None of the most successful firms on the Web have nonproprietary architectures that others can freely clone.

But while the New Age Theory doesn’t apply to the Web, it does increasingly apply to the software market.  After completing the Google article, I had embarked upon research for my upcoming article, on open source software and the challenge it poses to Microsoft and the traditional software industry.

And here I found an unexpected convergence between another criticism advanced against the Google article and my new research.

On the one hand, my research on open source reconfirmed my view that standardization, platforms, and APIs and will prove just as important in the future, and in the search industry, as they have in traditional PC software and many other sectors.  On the other hand, I also found strong reasons to think that the open source movement is changing the nature of standardization contests, and represents a powerful threat to Microsoft’s control over mass market software.  Consequently, as new standards emerge, Microsoft may not be the one to control them.

Several people responding to my Google article argued that I had simply overestimated Microsoft, which, they said, was now at best mature and possibly in decline.  I increasingly feel that they were correct.  By relying too heavily on its monopoly control of Windows and Office, Microsoft has painted itself into a corner.  Both Microsoft and its products are now large, aging, complex, and very expensive, rendering them vulnerable to attacks from below.  Open source software, with its low cost, transparency, and decentralized development model, threatens the very foundations of Microsoft’s power.

Large computer vendors, corporate users, and governments have become increasingly frustrated by Microsoft’s behavior, and they are actively funding subversion, most notably in the form of the Linux operating system and applications based upon it.  And, as I shall discuss at length in my forthcoming article, they’re winning. 

The result is that Microsoft now faces increasingly serious threats to the entire spectrum of its mature businesses while it simultaneously tries to enter growth areas such as the Web, mobile devices, and the game industry.  Thusfar, with the partial exceptions of the Xbox game system and the MSN portal, Microsoft’s progress have been singularly unimpressive.  When I have asked knowledgeable friends recently about this, most of them say the same thing:  Microsoft has lost its edge.  It’s over.

There seem to be three reasons for this. 

First, Microsoft has gotten addicted to the money derived from repeated forced upgrades.  As a result, it is now a high-priced incumbent with large, aging products – an inherently poor position when faced with a less expensive, newer competitor such as Linux. 

Second, Microsoft has become a large, politicized, often complacent company.  Many of its senior employees are now so wealthy that they don’t have to work hard, take orders, or worry.  The CEO is a former Procter & Gamble brand manager, not a technologist, and the company is so large and complex that politics inevitably distorts information flows.

Third, in growth markets such as the Web, Microsoft cannot afford to be as leisurely as it was in the 1980s when it faced slow-moving competitors with flawed business models such as IBM, Lotus, Novell, Apple, and Sun.  Now it is chasing Google and Yahoo, who are growing much faster than Microsoft, and whose combined revenues in 2005 will exceed $10 billion.

In my article on Google, I therefore may have given Microsoft too much credit.  I still think that Google does indeed have the vulnerabilities that I discussed, and that if Microsoft attacked forcefully, Google would be in serious trouble.  I also still believe that there will be increasing demand for common, interoperable search functions across many platforms, and that widely available software with open APIs would be valuable for both the search industry and for users. 

But if Microsoft is too paralyzed and complacent to act, Google will succeed whether or not it provides an open platform.  Unlike the Linux case, in the search arena there do not presently seem to be other powerful actors likely to step in to discipline the dominant player. 

As a result, if Google chooses not to release an API for its search engine, the search industry may not evolve the standards and open interfaces that would provide interoperability across PCs, corporate servers, the Web, proprietary content providers, and portable consumer devices.

Alternatively, Google might provide such interoperability itself, but only within its own closed universe.  Thus either short term profitability or internal corporate politics could lead Google to decide against the deployment of truly open search architectures.

This would be a shame, but it could happen, and things like it have happened before.  Much of the success of Linux, for example, derives from the fact that in the 1980s, the UNIX operating system split into multiple, incompatible dialects, each one controlled by a different hardware vendor (Sun Microsystems, IBM, DEC, Silicon Graphics, Hewlett Packard). 

This brought high profits for over a decade, followed by disaster when an open, industry-wide alternative finally appeared. Most industry analysts now believe that Sun is in irreversible decline at the hands of Linux. 

In the search industry, too, market forces would probably eventually bring regime change, but it could take decades.  Only time will tell what Google will do, and whether users will be well served; my own hope is that Google will conclude that enlightened self-interest leads it in the direction of an open architecture.