Steve Jobs couldn’t hide his frustration. Asked at a technology conference in 2010 whether Apple might finally turn its attention to television, he launched into an exasperated critique of TV. Cable and satellite TV companies make cheap, primitive set-top boxes that “squash any opportunity for innovation,” he fumed. Viewers are stuck with “a table full of remotes, a cluster full of boxes, a bunch of different [interfaces].” It was the kind of technological mess that cried out for Apple to clean it up with an elegant product. But Jobs professed to have no idea how his company could transform the TV.

Scarcely a year later, however, he sounded far more confident. Before he died on October 5, 2011, he told his biographer, ­Walter Isaacson, that Apple wanted to create an “integrated television set that is completely easy to use.” It would sync with other devices and Apple’s iCloud online storage service and provide “the simplest user interface you could imagine.” He added, tantalizingly, “I finally cracked it.”

Precisely what he cracked remains hidden behind Apple’s shroud of secrecy. Apple has had only one television-related product—the black, hockey-puck-size Apple TV device, which streams shows and movies to a TV. For years, Jobs and Tim Cook, his successor as CEO, called that device a “hobby.” But under the guise of this hobby, Apple has been steadily building hardware, software, and services that make it easier for people to watch shows and movies in whatever way they wish. Already, the company has more of the pieces for a compelling next-generation TV experience than people might realize.

And as Apple showed with the iPad and iPhone, it doesn’t have to invent every aspect of a product in order for it to be disruptive. Instead, it has become the leader in consumer electronics by combining existing technologies with some of its own and packaging them into products that are simple to use. TV seems to be at that moment now. People crave something better than the fusty, rigidly controlled cable TV experience, and indeed, the technologies exist for something better to come along. Speedier broadband connections, mobile TV apps, and the availability of some shows and movies on demand from Netflix and Hulu have made it easier to watch TV anytime, anywhere. The number of U.S. cable and satellite subscribers has been flat since 2010.

Apple would not comment. But it’s clear from two dozen interviews with people close to Apple suppliers and partners, and with people Apple has spoken to in the TV industry, that television—the medium and the device—is indeed its next target.

The biggest question is not whether Apple will take on TV, but when. The company must eventually come up with another breakthrough product; with annual revenue already topping $156 billion, it needs something very big to keep growth humming after the next year or two of the iPad boom. Walter Price, managing director of Allianz Global Investors, which holds nearly $1 billion in Apple shares, met with Apple executives in September and came away convinced that it would be years before Apple could get a significant share of the $345 billion worldwide market for televisions. But at $1,000, the bare minimum most analysts expect an Apple television to cost, such a product would eventually be a significant revenue generator. “You sell 10 million of those, it can move the needle,” he says.

Cook, who replaced Jobs as CEO in August 2011, could use a boost, too. He has presided over missteps such as a flawed iPhone mapping app that led to a rare apology and a major management departure. Seen as a peerless operations whiz, Cook still needs a revolutionary product of his own to cement his place next to Saint Steve. Corey Ferengul, a principal at the digital media investment firm Apace Equities and a former executive at Rovi, which provided TV programming guide services to Apple and other companies, says an Apple TV will be that product: “This will be Tim Cook’s first ‘holy shit’ innovation.”

What Apple Already Has

Rapt attention would be paid to whatever round-edged piece of brushed-aluminum hardware Apple produced, but a television set itself would probably be the least important piece of its television strategy. In fact, many well-connected people in technology and television, from TV and online video maven Mark Cuban to venture capitalist and former Apple executive Jean-Louis Gassée, can’t figure out why Apple would even bother with the machines.

For one thing, selling televisions is a low-margin business. No one subsidizes the purchase of a TV the way your wireless carrier does with the iPhone (an iPhone might cost you $200, but Apple’s revenue from it is much higher than that). TVs are also huge and difficult to stock in stores, let alone ship to homes. Most of all, the upgrade cycle that powers Apple’s iPhone and iPad profit engine doesn’t apply to television sets—no one replaces them every year or two.

But even though TVs don’t line up neatly with the way Apple makes money on other hardware, they are likely to remain central to people’s ever-increasing consumption of video, games, and other forms of media. Apple at least initially could sell the screens as a kind of Trojan horse—a way of entering or expanding its role in lines of business that are more profitable, such as selling movies, shows, games, and other Apple hardware.

That’s essentially the justification for the Apple TV product, the $99 hockey puck that streams TV shows and movies on demand for $1.99 and up. For most of its six years on the market, the device hasn’t been a big seller. Nor have many of the other TV add-ons, such as Google TV (see “Searching for the Future of Television,” January/February 2011), or even TiVo. But Cook has upgraded the way he talks about Apple TV: in October he called it a “beloved” hobby, perhaps because sales had almost doubled, to five million units, in the previous fiscal year. One reason it’s more appealing is that in July, Apple added Hulu Plus to the small list of Apple TV apps. That made it possible, for $8 a month, to watch current shows on demand the day after they air on TV.

But selling a TV set could also give Apple a way to enhance the role iPads and iPhones are playing in living rooms. Apps ranging from Apple’s own Remote to personalized programming guides such as NextGuide are turning them into far more capable portals into the TV than cable remote controls. In fact, Ben Reitzes, an analyst at the investment bank Barclays, believes that Apple’s TV strategy actually revolves less around the TV set than around the iPad as universal remote. He thinks the appeal of an iPad remote would help maintain Apple’s tablet dominance, especially as the company extends iPads to become a “central command” for lights, heating systems, and other features of the digital home.

A potentially much bigger advantage for Apple is a feature called AirPlay in the latest Mac and mobile iOS software. It allows whatever is showing on Macs, iPhones, and iPads to be “mirrored” to a TV set. Although not many iOS television apps support AirPlay yet, viewers can use Macs sold since mid-2011 to mirror shows from the free Hulu site, network websites, and even—perish the thought—pirate video sites. Suddenly, viewers can watch a lot of current shows on their HDTVs quickly and wirelessly—and, most important, without a cable subscription.

What Apple Still Needs

But Apple isn’t likely to disrupt the TV business solely by helping people get around the traditional cable and satellite providers. Instead, it will try to work with them—and give them an incentive to come along. Stewart Alsop, a partner in the VC firm Alsop Louie Partners and a former member of TiVo’s board, says Apple could use a tough-love approach: “Apple is the one company in the world that’s powerful enough to take on monopolies and force them to change.”

Making friends with them has proved to be Apple’s most difficult challenge so far—and a solution is hard to discern. A few media conglomerates that run cable and broadcast channels, such as Walt Disney, Time Warner, and Viacom, remain extremely profitable. TV advertising generates $72 billion annually in the United States alone. Plus, the cable and satellite operators that distribute programming to homes gross $103 billion a year in pay TV subscriptions, sending $28 billion of that back to the media companies. Pay TV operators such as Comcast are also large Internet service providers, giving them influence over how far online TV services can go.

So unlike music labels before them—which were weakened by piracy and thus were more willing to grant Apple the right to sell individual songs for 99 cents apiece—they have no need to sell their content cheap. In particular, producers won’t give Apple access to their live shows without a guarantee of the same big bucks they get from cable and satellite operators. The TV companies are wary of even letting Apple create a new TV user interface for their customers, the key to making an Apple television something special.

The classic Apple approach to such a situation would be to come up with a superior, or at least more elegant, product and force companies in related fields to play along. But Apple used to do that by having Steve Jobs charm and cajole recalcitrant partners. Jobs also understood the entertainment business and knew the players. He built Pixar into one of the world’s most successful movie studios, then served on Disney’s board after Disney bought Pixar in 2006. And even he struggled to persuade TV companies. CBS CEO Leslie Moonves, for one, said he rejected overtures in 2011 by Jobs himself for an Apple TV subscription service. Today, Eddy Cue, Apple’s senior vice president of Internet software and services, is the company’s most important TV dealmaker.

So how can Apple get more leverage and force the pay TV industry to deal? One possibility is that the troika of iPads, Apple TV, and the TV set—made by Apple or not—could bring about even more sweeping changes to make the television-watching experience more interactive. Google tried to do something similar two years ago with its Google TV service, but at least initially the results were too geeky, requiring a keyboard and clunky navigation. Its recently introduced voice-driven remote control feature, using Android smartphones and tablets, only underscores the fact that people want a less brain-taxing experience in the living room.

But a new generation of “dual-screen” apps could provide the best of both traditional TV and the Internet, says Jeremy Allaire, chairman of Brightcove, a provider of online video services. Major League Baseball’s iPad app, for instance, plays a game on the TV through Apple TV while you check out relevant stats and chat with friends on your tablet. Essentially, says Allaire, whose company helps software developers create these apps, iPads and iPhones serve as the real brains of the TV.

Apple could also let people use voice-driven commands to find shows and change channels using Siri, its intelligent personal assistant. You could toss that annoying cable remote and just tell your TV what you want to watch. Moreover, Apple’s iCloud storage service could be used as a digital video recorder in the sky, as Jobs hinted to Isaacson. Already iCloud can store TV shows bought on iTunes and feed them to any Apple device.

If such enhancements make Apple TV ever more useful, then “at some point, the growing Apple TV installed base will gain enough mass to become a viable distribution channel, an alternative to traditional cable TV,” suggests onetime Macintosh executive Gassée, now a general partner at the venture firm ­Allegis Capital. “When this happens, someone will crack.” That is, a cable channel such as ESPN will offer its must-have sports coverage on Apple TV, and others will feel forced to follow—thus paving the way for a credible Apple television.

Apple has also explored building a cable set-top box—possibly a souped-up Apple TV using a CableCard, a small card that plugs into a DVR or other TV device and allows subscribers to view cable channels without a separate box. Although it would be working within the current cable industry model, Apple would provide a more intuitive interface, like the iPad’s, and users would be able to watch live and on-demand shows through an Internet-based DVR service.

This might work. Comcast and Time Warner Cable executives have said they’re open to new program guide interfaces from Apple and other companies so long as cable subscribers keep paying them. Cable subscribers could be “authenticated” through Apple’s set-top box, or eventually a TV, to prove they’re subscribers—a system like the one HBO uses, for example, with the HBO Go app that streams its shows to mobile devices.

But Apple may not have time to wait for them to deal. Competition from Google (which is experimenting with a pay TV and Internet service in Kansas City), Amazon (which has a streaming video service and plans to produce original series), or Microsoft (whose Xbox gaming console is as much a video delivery device as a game machine) may force Apple to stake out territory in the living room more forcefully, and soon. Maybe, just maybe, a sleek Apple-designed flat screen, combined with a more elegant user interface, iPads serving as slick remotes, Apple’s existing iTunes library, and shows from outside services such as Netflix and Hulu, would be compelling enough for consumers while Eddy Cue keeps doing lunch in Hollywood. “The reality from a consumer standpoint,” says Piper ­Jaffray analyst Gene Munster, is that Apple “needs to revolutionize the interface and the design” before it can reorganize TV content. Munster expects to see an Apple television this November, at least two years after he initially predicted. “We don’t need a nuclear event around content for an Apple television to be successful,” he says.

It’s worth noting that with its iPads and iPhones, Apple is already selling the TV screen of choice for a rising number of peripatetic viewers. You can’t use these devices to watch everything you get on cable, but TV apps offer shows from CBS and HBO, some free, plus live baseball games and other programming. More than half of tablet owners under 35 watch TV on them at least weekly, according to a survey in August by the consulting firm Altman Vilandrie.

In other words, don’t just think of an Apple TV as the big screen in the living room, or else you might miss where Apple really envisions the 75-year-old medium going next: everywhere. “Whatever it is, it’s not going to be just on that big box,” says ­Jeremy Toeman, CEO of Dijit, developer of the TV programming guide app NextGuide. Before long, he says, “everything you have with a screen will become a television set.” Jobs may have “cracked” television, but Apple could blow it wide open.