Amazon: Giving Away the Store

Visit Amazon Light at www.kokogiak.com/amazon4, and you’ll see a plain search box that allows you to locate any product in Amazon.com’s database. Click on an item, and you’ll be taken to a page with the usual product image, price information, and customer reviews, and, of course, the familiar “Buy This” button. Amazon Light’s pages are deliberately less cluttered than those at Amazon itself, but the family relationship is obvious.

Look closer, however, and you’ll spot some distinctly non-Amazonian features. If the item you’re viewing is a DVD, for example, there will be a button that lets you see in a single click whether the same disc is for rent at Netflix. If it’s a CD, you can check whether Apple’s iTunes music store has a downloadable version. And if it’s a book, Amazon Light will even tell you whether it’s on the shelf at your local public library.

What’s going on here? Surely, executives at Seattle-based Amazon would never condone an online service that encourages people to buy things from sites other than Amazon?

Actually, they would. Amazon Light, created by former Amazon programmer Alan Taylor and hosted on his personal website, kokogiak.com, is one of thousands of independent sites incorporating the product data and programming tools that Amazon has been sharing freely since July 16, 2002. That’s the day Amazon celebrated its seventh anniversary–and unveiled a startling new project, called Amazon Web Services, that promises to change, once again, the way retailers of all stripes think about reaching their customers.

While companies such as Google and Microsoft are also experimenting with the idea of letting outsiders tap into their databases and use their content in unpredictable ways (see “What’s Next for Google?”), none is proceeding more aggressively than Amazon. The company has, in essence, outsourced much of its R&D, and a growing portion of its actual sales, to an army of thousands of software developers, who apparently enjoy nothing more than finding creative new ways to give Web surfers access to Amazon merchandise–and earning a few bucks in the process. The result: a syndicate of mini-Amazons operating at very little cost to Amazon itself and capturing customers who might otherwise have gone elsewhere. It’s as if Starbucks were to recruit 50,000 of its most loyal caffeine addicts to strap urns of coffee to their backs each morning and, for a small commission, spend the day dispensing the elixir to their officemates.

“Amazon is pouring so many resources into their Web services that it’s almost frightening,” says Paul Bausch, one of the inventors of the well-known weblogging tool Blogger and, more recently, the author of O’Reilly Media’s Amazon Hacks, a collection of tips for tapping into Amazon’s rich database. “They are extremely aggressive, and that separates them from Google and from other people who are still just experimenting with the technology. They really believe that this is where their business is heading.”

Making the Web Safe for Computers
The strategy behind Amazon Web Services is to give programmers virtually unlimited access to the very foundation of Amazon’s business–its product database–whether they are inside or outside the company’s walls. Developers can grab product data, reformat it, add related services, and use it to attract eyeballs to their own sites. If they feel like it, they’re even free, like Taylor, to create parallel-universe Amazons that have the added features they crave. Amazon demands only one thing in return: that visitors to these satellite sites complete any purchases through Amazon.com itself. The site owners, meanwhile, earn a decent commission on each sale.

Exposing the world’s largest product database–along with the editorial content and personalization functions that make Amazon.com so uncannily useful–is such a counterintuitive business strategy* that analogies are hard to come by. In a way, the risk the company is taking is like the one Apple Computer has always avoided by refusing to license its Macintosh operating system to other manufacturers. Some observers still think Apple missed its golden opportunity to mount a comeback against Microsoft, while others maintain that the Mac OS is Apple, and that putting it on other machines would have diluted the brand. In Amazon’s case, outside programmers could find cleverer ways of using Amazon’s data than Amazon itself and end up sucking away so much traffic that Amazon’s own site cedes e-retailing’s center stage.

And yet Amazon’s move also reflects the spirit of the age–or at least, the spirit among the Web’s technical avant-garde. As Amazon CEO Jeff Bezos put it in a speech at October’s Web 2.0 conference in San Francisco, “Web 1.0 was making the Internet for people; Web 2.0 is making the Internet better for computers.”

Web services, whose name became a marketing meme years before the technology to make them work had actually matured, boil down to the simple but powerful idea that the Web should be more than just a way for human users to call up preformatted documents. It can also be a medium for software programs to communicate and share data with one another. Through a nonproprietary formatting scheme called the Extensible Markup Language, or XML, a string of data can be labeled according to type–as, say, a phone number, a price, or a book title. Web software can then harvest the data from a remote site and re-present it to the end user in any way a programmer wishes. That means a company with a trove of data it wants to share need only put it on the public Web and give programmers a few simple tools for accessing it.

Though these tools have a geeky and daunting name–“application programming interfaces,” or APIs–their function is easy to understand: they are short hooks of computer code that allow smaller programs to communicate with large software systems, such as Microsoft Windows. In the case of Web services, an API amounts to a set of commands programmers can use to interact with large, database-driven Web sites.

For a company that makes its APIs public, the real trick is figuring out how it will be rewarded. And in this respect, too, Amazon has done exceedingly well, signing up thousands of programmers who are sending it millions of hits a day, though the company won’t disclose how much it’s earning through their satellite sites.

There isn’t one tutelary genius responsible for Web services’ ascent at Amazon. Even Bezos was reportedly awakened to their possibilities only after a bit of prodding from friendly outsiders.

Rather, this is a tale of clever Amazon software engineers and marketing professionals who were attuned to emerging technologies like XML, the Simple Object Access Protocol (SOAP), and the Semantic Web, and who saw how their efforts in the 1990s to increase Amazon.com’s accessibility led inexorably to the Web services model. Increasingly, it’s also the tale of tens of thousands of outside Web developers with a fixation on new technologies and an urge–with Amazon’s blessing–to see just how far they can stretch that model.

The Information Hub
There are two versions of the story of Amazon Web Services’ genesis. One is told by Tim O’Reilly, a revered futurist and CEO of the software-manual publisher O’Reilly Media in Sebastopol, CA. O’Reilly says he began touting the idea of Web services as early as 2000 but wasn’t seeing it blossom into the next-generation Internet programming language he envisioned. “I went up to see Jeff Bezos with a Web services pitch,” O’Reilly wrote in his weblog in July 2002. “Amazon isn’t just an e-commerce-site. It’s become the information hub of the publishing industry. How about giving us some tools for building out services based on that hub?” Once O’Reilly enumerated the ways Amazon itself would benefit, he says, Bezos became intrigued. Bezos then “told me a day or two later that he’d discovered that his skunk-works team already had a Web services API in the works. But he says that without my presentation he ‘might have done something stupid like shutting the project down.’”

Naturally, Amazon software engineers tell a different story. Robert Frederick, a 31-year-old senior technical manager at Amazon, was part of the skunk-works team that Bezos consulted after his meeting with O’Reilly. Frederick dates the origins of the project all the way back to his hiring in 1999, when he was assigned to figure out how the rich information on Amazon’s website could be simplified and translated for display on the tiny screens of devices such as cell phones, PDAs, and pagers. These devices used an alphabet soup of different display formats, such as WML, CHTML, and XHTML. “To expose this information in varying markup languages was extremely challenging, given the infrastructure that [Amazon had] put in place,” says Frederick. But his team eventually came up with special Web server software that could retrieve product data and reformat it for a few select devices such as the Palm VII, one of the first wireless organizers.

This project, dubbed Amazon Anywhere, succeeded so well that Frederick’s bosses asked him to equip Amazon’s entire battalion of Web servers with the software, the better to help partners present Amazon data in more ways. “In order to do this,” Frederick says, “I advocated the separation of display logic from our business logic.” More geek-speak–but that insight was perhaps the key to Amazon Web Services. It meant that Amazon needed to stop viewing its website as one giant application capable of storing product information, managing user accounts, sending product details to customers’ Web browsers, processing purchases, and the like. Rather, Frederick said, Amazon.com should be reconceived as a set of independent parts, including the database, shared APIs for accessing it and repackaging the data in XML format, and the final layout as displayed in browsers. In a series of meetings with senior managers in 2001 and early 2002, Frederick argued that the more these things were separated, the easier it would be for Amazon’s partners to build their own online storefronts that would draw on the Amazon infrastructure.

A few Amazon managers reacted cautiously. But “the funny thing is that it did not take a great deal of convincing,” recalls Frederick. “The main concern was that we were going to expose valuable information and concepts that we had spent years developing.” Absolutely true. But Frederick and his supporters argued that Amazon’s payback would be the innovative applications dreamed up by external developers. In theory, these applications would vastly increase the variety of contexts in which Web surfers might encounter products from Amazon.

“Once we had incorporated XML, we realized that ‘Wow, the future is here. We are the first to do this,’” says Jeff Barr, program manager for Amazon Web Services. “So we decided to formalize it and invite the world’s developers to take a look, understand what it was all about, and start creating and innovating.”

Pizzazz–and Profits
Developers accepted the invitation, and downloaded Amazon’s Web services APIs, help manuals, and sample code in droves (see “Developing Interest,” right). “I was amazed that they did what they did, because companies are usually so reluctant to expose their databases,” says Paul Bausch, who had become interested in Web services while building Blogger at San Francisco-based Pyra Labs. After Google acquired Pyra in 2003, Bausch returned to the life of an independent software developer and had so much fun playing with Amazon’s new programming interfaces that he ended up writing Amazon Hacks. Among other things, the book teaches readers how to embed Amazon product information in their Web pages and (on a less serious note) how to add pizzazz to their personal archives of MP3 songs by grabbing CD cover art from Amazon.

In one way, Amazon Web Services is simply an extension of longstanding collaborations such as Amazon Associates, in which external website owners can earn commissions of up to 10.5 percent for referrals. But previously, affiliate sites could only show simple links to Amazon products; with Web services, the possibilities became much broader (see “Amazon Everywhere,” below).

At one end of the spectrum are sites like Taylor’s Amazon Light that simply proffer Amazon in a different skin. At the other end are noncommercial sites where Amazon data isn’t the main attraction but is used to enhance other offerings. One example is AllConsuming.net, which appeals to bibliophiles by scouring thousands of blogs every hour, listing the most talked-about books, and providing snippets from each blogger’s comments. Designed by Erik Benson, another ex-Amazonian, the site dips into the Amazon database for cover images and other details about the listed books and offers links to each book’s page at Amazon.

Benson says AllConsuming isn’t making him rich, but it does bring in enough commissions to cover its own hardware and bandwidth costs (about $400 a month). Other companies earn far more by acting as middlemen. Monsoon of Portland, OR, for example, builds software that helps its own customers use Amazon’s Web services to simplify inventory management.

By November 2004, the number of developers participating in Amazon Web Services had grown to 65,000. To keep up with their demands, the company has kept updating its APIs to open up more types of product information and more functions, such as wish lists and advanced searches. How many purchases originate each day with Amazon’s growing web of syndicated storefronts? The company won’t say, but experts have estimated that sites using the company’s Web services send 10 million requests a day to Amazon’s servers.

Amazon is far from the only company exploring Web services. IBM, for example, has opened its Websphere server software to outside developers and expects to invest $1 billion this year in new business-to-business Web services, according to Michael Liebow, director of Web services for IBM Global Services, the company’s consulting wing. One IBM creation: a system that uses XML and other standards to tie together the disparate databases used by merchants, banks, and credit-card firms, helping to resolve disputed credit-card charges faster.

In fact, Web services’ biggest impact may not be the syndication of individual businesses’ information, as in Amazon’s case, but the standardization of business processes across whole industries, such as finance, electronics, or automobiles, according to Liebow. “Amazon is unique,” he says. “It’s kind of a closed system, and there’s a level of control.”

That’s true–and Amazon reserves the right to shut down its Web services at any time. But doing so would destroy the rare symbiosis that has emerged from years of careful community building. “Developers are another kind of customer for us,” says Jeff Barr. “The work they do is going to bring even more diverse types of customers.” In other words, by sharing not just its data but also its retailing tools and a modest slice of the profits, Amazon has turned a programming subculture typically ruled by anticorporate suspicion and paranoia into a wellspring of evangelism, not to mention a funnel for revenue. Surely, that’s one for the books.