Little Big Screen

Mike Foster’s Palm Vx could cost him money today-lots of it. The professional speaker, whose specialty is telling businesses how to exploit new technology, has a chance to make $240,000. But he must respond promptly to an international organization that has requested a speaker for 24 of its chapters. At $10,000 a pop, it’s an opportunity Foster can’t pass up. But critical information he needs to close the deal is locked in an e-mail attachment that is slow to open. The e-mail itself is next to impossible to view on his PDA’s low-resolution screen. Someone else might reach the meeting planner first and get the job-and the payday.

“It’s a big game, and the quicker you get your information, the better you can respond,” says Foster, who might not be able to connect his laptop to the Internet until he arrives in Chicago at 10:30 at night. To make matters worse, the airline lost some of his baggage, including a keyboard accessory that would let him type an intelligible reply, and responding with the Palm’s handwriting recognition software isn’t generally as fast as typing.

Make no mistake-this isn’t a knock on the Palm Vx specifically. As people become dependent on digital organizers, cell phones, pagers and other portable devices to connect to the Internet, many are frustrated as the flood of information online narrows to a trickle. The interfaces, including screens that display data, as well as keyboards and voice recognition systems that deliver it, are too small, too slow and too awkward to process information effectively. Web sites become unusable, e-mails constrained, and graphics are eliminated. As a result of these clunky interfaces, a bottleneck problem that previously existed between Internet service providers and their subscribers-a link known as the last mile-also occurs between a handheld’s processor and its owner.

That critical distance, known to insiders as the “last three feet,” could put at risk the billions of dollars invested in wireless connectivity. Even if vendors solve the challenges of the last mobile mile, it could all be for naught if they cannot cover the distance between the hand and the face. The dilemma has engineers, programmers and designers going back to the drawing board for new ways to twist open the information valve. Microdisplays, foldable keyboards and better voice recognition systems are just some of the solutions. But will people use them? And are handheld devices equipped to operate them? Let’s take a look.

Size Matters

When it comes to portable devices, people’s expectations outstrip what they demand from a desktop. “Users using a mobile device are much more sensitive to the time it takes to perform a task than they are when they are sitting at their desk,” says Mike Flom, CEO of Portable Internet in Park Ridge, NJ. Yet the screens and displays of mobile devices remain thoroughly inferior to those of PCs. “I don’t care how many bits per second you’ve got, if I’ve got a [five-centimeter by eight-centimeter] display, there’s only so much I can do with that,” says Eric Schaffer, founder and CEO of Human Factors International, a Fairfield, IA-based software usability consulting firm.

Not only are the screens small, but their resolution remains stubbornly low-only a fraction of that available even on the cheapest computer monitors. As a result, each letter or image needs more screen space to be legible. That limitation plus a miniscule screen adds up to one big frustration.

One solution: larger screens. The extra space might come from a flexible screen that unfolds like a map. Plug it into a pocket computer and you have as much visual real estate as you’d find on a desktop, or at the very least on your cumbersome laptop. Indeed, that development could spell the end of the laptop. “If I had a palm-sized device thatI could fold out to be a full-sized display with a competent processorand one of those fold-up keyboards,” asks Flom, “do I need a laptop anymore?”

Pocket-sized, foldable screens are only a few years away. Both E Ink of Cambridge, MA, and Gyricon Media, a Xerox spinoff in Palo Alto, CA, have been developing thin displays with “electronic ink” technology (see “Electronic Paper Turns the Page,” TR March 2001). Electrostatic charges orient white microscopic particles suspended in tiny spheres. An underlying circuit controls the charge and whether or not the spheres show as white or dark. The companies are also looking into eventually developing flexible color displays.

Although electronic ink is on the technology horizon, it isn’t quite ready for mobile electronics use. The prototypes being developed by E Ink and Gyricon Media provide reflective displays, relying on the environment for illumination. That may be fine outdoors or in a bright room, but not in dim surroundings.

But electronic ink is not the only little big screen solution in the pipeline. Another approach keeps the display small-two centimeters or less in diagonal-but offers good resolution and an effective screen size many times its physical dimension. The solution makes use of magnifying lenses mounted in monocular units or goggles. The magnifiers pump up visuals to the scale seen in conventional full-sized monitors. InViso of Sunnyvale, CA, for example, makes a handheld device that, in conjunction with glasses, offers images 800 pixels across and 600 pixels high-the same resolution as a standard 35.5-centimeter desktop monitor. Large companies are also entering the market, with Sony’s Glasstron and the Eye-Trek from Olympus both giving the viewer an image equivalent to a 132-centimeter screen seen from about two meters away.

Of course, this approach raises the question: why not just cram more pixels into tinier screens for better resolution? The answer has to do with the physics underlying the most common technology used in handheld screens-liquid crystal displays. These displays are a multilayered sandwich of components backlit by a light source. The light must pass through electrode strips to illuminate the pixels on the surface of the screen. Whether a pixel gets illuminated is controlled by a thin-film transistor connected to the electrode strip. A current flowing from the transistor makes the strip opaque; no current lets light through. But as the pixels shrink, they soon become smaller than the transistors, which then block out the light, whether they’re on or off.

Organic light-emitting diodes, a new display technology, eliminate the need for backlighting (see “A Bright Future for Displays,” TR April 2001). When charged by electrodes, these organic materials emit their own light. Currently, Motorola has a mobile phone on the market that uses organic diodes developed by Eastman Kodak. And many other big-name corporations are experimenting with the technology as well, including eMagin, IBM’s Almaden Research Center, Uniax and Cambridge Display Technology.

While many companies are struggling with the engineering challenges inherent in small screens, one ingenious solution does away with screens altogether. Microvision, a Bothell, WA, manu-facturer, actually projects an image, pixel by pixel, directly onto the viewer’s retina. The approach is similar to cathode-ray projection in a television. But instead of electron beams selectively lighting spots on a screen, light projects onto the retina, illuminating tiny points and creating the illusion of a 43- to 53-centimeter screen floating in space at arm’s length. To view a Web page, people using mobile phones equipped with retinal displays might hold the devices close to their eyes, or they might wear goggles with retinal scanning devices built into the temples. The technology is just starting to hit the real world. The Microvision head-mounted prototype “Nomad”-scheduled for market release this fall-creates a clear, see-through image at arm’s length even under the brightest daylight conditions. Eurocontrol, the organization that oversees air traffic control in Western and Eastern Europe, is testing Nomad in a flight control-tower simulator. Freiburg, Germany-based Stryker Leibinger, a manufacturer of surgical instruments, purchased 10 Nomads in late March for clinical evaluation. And the Mayo Clinic in Rochester, MN, is also considering using Nomad in the operating room.

Designing display systems that offer the experience of a full monitor is one thing, but deciding to use them is quite an-other. The sticking point is the social acceptability of wearing headgear that can make someone look like a lost cyborg. Donning such a device “is a very big issue,” according to Robert Steinbugler, manager of the IBM corporate strategic design program. “What you typically find is that a person has to have seen a value for doing that, or seen the idea on someone else before trying it themselves.” Cell-phone users may have made it acceptable to wear an earphone to make calls while walking, but covering an eye like some high-tech pirate might still be considered strange, even in a large city (see “Cyborg Seeks Community,” TR May/June 1999).

Squeezing In

If people get over their self-consciousness about looking like cyborgs, then head-mounted displays only help relieve part of the problem of wireless connection to the Internet. Displays provide “output”-the information coming from the network to the user. The other half of the circle is “input”-sending responses back out into cyberspace. And it ain’t easy. “Output is hard,” notes Michael Karasick, chief technology officer of IBM’s pervasive computing division. “Input is harder.”

Just think of what it’s like using the keypad of your cell phone to send typed messages. You must press the same key three, even four, times to type a single letter. Some people may thrive on these difficulties: Adam Lavine, CEO of FunMail in Pleasanton, CA, reports from his Asian travels that the Japanese have a word, which translates roughly as “lightning thumb,” referring to people who are particularly adept at typing messages on mobile phones. But for those of us with ordinary thumbs, something more comfortable is in order. And it’s coming. Targus and other companies sell folding keyboards that work with PDAs. Last fall, Electro-Textiles demonstrated a prototype PDA fabric keyboard that could actually be rolled up for storage.

Some of us won’t ever have fingers dexterous enough to do our talking for us. But few of us have trouble talking, and that’s where voice recognition comes in. A number of cellular-phone manufacturers, including Motorola and Nokia, have been offering voice recognition for contact lookup for a few years. Samsung has incorporated voice navigation into a wristwatch-like phone. Conversay, which provided the voice technology for Samsung, has also ported its products to the Microsoft Windows CE Pocket PC platform.

However, voice recognition doesn’t necessarily make for a good user experience. “In a device, you need to make sure you’re not trying to force it to do things more easily done with a single button click,” says Williamson.

This could be tough in an area where restricted vocabularies-think “yes” and “no”-are more commonplace than sophisticated sentences. A general interface that would allow anyone to talk freely is overly optimistic, according to Dr. George White, senior vice president of technology at NetByTel, a Boca Raton, FL-based vendor of voice commerce systems. “True speech recognition requires enormous intelligence, which we can’t get into these little devices, and it will always be that way,” says White. But that doesn’t mean more sophisticated voice recognition will never be mobile. For complex voice recognition, White and others think the mobile device will do some preprocessing, then pass the sound over the Internet to a server that can handle more elaborate processing.

Each approach to more fluid input has its champions, but some experts think the ultimate solution will not come from a single technology but from a combination of approaches. Omar Javaid, chairman of New York mobile commerce consulting firm Mobilocity, shares White’s skepticism about voice recognition. “We call it the Star Trek interface,” he says. “In all likelihood, no single method of input will serve, and mobile electronics will probably best serve with a combination of all these methods and others as they become available.”

The ultimate interface, some argue, would be the complete integration of human and machine, with chips and sensors implanted under the skin to detect the user’s every intent. Issues of social acceptance aside, there may be practical reasons to keep devices at arm’s length. Doug Armstrong, CEO of AppForge, which manufactures mobile and wireless software development technology, remembers being part of a Navy research project that was monitoring the brain activity of pilots to try to let them “think” the weapons on their F-14s and F-22s into action. But the result was like telling someone not to think of the word “persimmon”-the pilots would think the trigger word even when they did not want to fire.

At this point, tech consultant Mike Foster, whose Palm may have cost him a payday, would consider almost any high-tech addition to improve his experience. “I can see myself stopping in a restaurant and putting all that stuff on,” he says, but only if it were sturdy and durable. In the meantime, he’d settle for a better screen. “I understand the physical limitations, but at some point having 640 by 480 on a Palm in color, that would be great,” Foster muses. “I’d pay three times the price for that.”

For now, Foster will continue trudging through the Internet, wishing under his breath for improved displays, better keyboards, friendlier interfaces. If vendors build it, he will buy it-as will others, no doubt. But so far, interfaces for handhelds are nothing more than tiny nozzles, and that makes the wireless Internet industry a little nervous, as they watch the flood of data gushing down the pipes.