Facing Doubters, IBM Expands Plans for Watson

IBM’s computer system Watson vanquished human contests on the TV quiz show Jeopardy! The question now: can it defeat the complexities of the real world?

IBM thinks so. The company says it plans to greatly expand its efforts to commercialize Watson by putting another 1,500 engineers and marketers to work on the project. It will also combine Watson with other “cognitive computing” technologies and invest a further $1 billion into a business it says will define the future of how companies use data.

“We have learned enough about the benefit of the cognitive systems that we think it has much further to go in solving business problems,” says Rob High, vice president and chief technology officer of the newly expanded IBM unit, now being called Watson Group.

IBM’s expansion plans come amid questions about Watson’s real commercial prospects and how well the underlying technology is performing. On Wednesday, the Wall Street Journal reported that Watson has generated revenues of less than $100 million, far short of the company’s goals, and that IBM has been scrambling to get its plans for the technology back on track after various stumbles.

The problem is that despite big claims, Watson has not always fared as well in real-world circumstances as it did on Jeopardy!, where its combination of machine-learning strategies and an ability to process natural language—or ordinary speech—allowed it to defeat human contestants Brad Rutter and Ken Jennings in 2011.

IBM has aggressively marketed the idea that Watson’s eerie smarts will revolutionize everything from cancer care to call centers. But real-world problems aren’t as tidy as the game show (see “With Watson, IBM Seeks to Sell Medical Knowledge”). For Watson to perform just as well in new areas, it requires a big effort to train the system and adapt it to new information, leading some critics to say that IBM moved too quickly to commercialize the cognitive system, originally developed inside its R&D division, IBM Research.

High, who has been with the project since 2012, says IBM made the right call. “You have to put this stuff into action and refine it. You need examples to work on. You do hit speed bumps, but I don’t think it was premature, it was exactly right,” he says.

Under the reorganization of the Watson program, High says, the number of IBM employees working on Watson technologies, including engineers, salespeople, and consultants, will increase fourfold or fivefold to 2,000. The Watson Group will also be elevated inside IBM and report directly to the chairman and CEO, Virginia Rometty.

According to IBM, the new Watson program will include other “cognitive” technologies, including voice and image recognition, feature extraction, and visualization tools. “We are adding the ability to Watson to evaluate images and respond to them. We are adding the ability to listen and hear. It will be a Watson system that can hear, see, and talk,” says High.

IBM’s $1 billion investment plans include $100 million to fund startups so they can build “cognitive apps” that work with Watson, via a cloud-computing service. That effort, announced in November but largely still on paper, aims to create an “ecosystem” around the computing platform.

IBM believes cognitive systems are the next frontier in business computing. These systems can understand speech and language, learn from examples, and even imitate human reasoning to a degree. IBM announced details of its expanded efforts on Wednesday in New York City, also the site of a new technology center for software developers that the company announced.

While IBM already has a few customers for Watson, including insurance company WellPoint, it also created huge expectations by running TV advertisements and promoting the Watson idea in thousands of news articles (see “Watson’s New Job: IBM Salesman”).

“Part of my problem with the approach that Watson has taken to health care is that almost everything that we know about it comes from the marketing and publicity departments,” says Peter Szolovits, head of a clinical decision making group in MIT’s computer science division. “They have been very closed-mouthed about what they are doing.”

One possible reason for that: Watson was running into problems outside of the narrow limits of the game show.

Among IBM’s biggest plans for Watson has been creating a system that can read medical records and recommend treatments, particularly for cancer patients. But so far, the system does not do that very well. Steven D’Amato, executive director and clinical pharmacy specialist at the Maine Center for Cancer Medicine, which collaborates with IBM, says that while the technology is exciting, from what he’s seen it could take time, maybe years, to perfect. “Watson is still a medical student and it is not ready for prime time,” says D’Amato.

Watson is already able to recommend the correct treatments for cancer patients, doctors say. But it does so consistently only when it is fed clearly structured data on a patient’s case. Where the system has struggled, experts say, is with its vaunted ability to make sense of language on its own.

In data presented to the American Society of Clinical Oncology last year, researchers at the Memorial Sloan-Kettering Cancer Center, which has been working with IBM to build the cancer assistant, reported that Watson correctly identified all the key data in a patient’s records less than half the time, not accurately enough to inform direct medical decisions.

Cancer turns out to be a harder problem than Jeopardy!, since unlike crisp game-show questions, doctors’ case notes are a maze of jargon, abbreviations, and inconsistently used terminology. And while IBM researchers spent several years and millions of dollars tuning its system to win on Jeopardy!, its commercialization teams don’t appear to have approached cancer with the same degree of rigor. Sloan-Kettering spent much of its time manually coding cases so Watson could understand and learn from them.

What Watson can do—given the right data—is pull up relevant literature and also consistently recommend the same course of treatment that’s suggested in the written medical guidelines that doctors consult. But following guidelines is also something that less sophisticated software can do. “Watson can easily duplicate a guideline recommendation. But we are not looking for an electronic version of guidelines,” says Mark Kris, a lung cancer specialist at Sloan-Kettering. “You don’t need Watson to do that. We want a machine that docs can turn to as an advisor and colleague.”

Over the past year, IBM has made efforts to readjust its approach. According to the Wall Street Journal report, the company has sought to repair initially rocky results in another collaboration with MD Anderson Cancer Center.

Kris and others say IBM is still planning to launch a product for cancer centers by the end of the spring—only a few months later than the company had initially planned. IBM declined to comment.

That system will be able to make recommendations for treating several cancers based on manually organized inputs—structured data—and will also interpret text notes for two cancers, lung and breast, with reasonable accuracy. While that falls short of the highest hopes for what a system like Watson can do, it may be good enough for a commercial product.

“We do have further to go, but so does this business,” says IBM’s High. “This is the right time to move forward with a bigger investment.”