Facial Recognition Is Only the Beginning: Here’s What to Expect Next in Biometrics on Your Phone

With the introduction of the iPhone X, Apple shined a new spotlight on a decades-old technology that has never worked well on the phone: facial recognition for things like unlocking and paying with the device. It’s a big bet, but one that may very well pay off—and pave the way for easier-to-use, more secure biometric technology of all kinds in phones of the future.

Apple has a penchant for adding future-facing features to the iPhone a little later than other phone makers, but it markets them so slickly that they seem like shiny must-haves if you don’t already have them. Facial-recognition technology has been around for years, and is available in some phones, like Samsung’s Galaxy S8. What’s out there typically hasn’t worked well, though—for one thing, it’s been shown to be easily spoofed with something as simple as a photo of the phone owner’s face.

Apple says its version of the technology, called Face ID and available when the phone ships in November, uses a suite of sensors to map your face in 3-D. An infrared light illuminates your face, and a projector projects an array of infrared dots at it. An IR camera snaps an image of these dots, which the phone uses to authenticate you against an already-stored image of your face. The company claims its Face ID feature is so secure that there’s a one in a million chance that someone could spoof you—not only does it require a measure of the user’s attention to unlock the phone, but Apple also says it trained the feature on realistic-looking 3-D masks so it would not be tricked by them.

The technology behind Face ID is not novel. Anil Jain, a Michigan State University professor who studies biometric recognition and computer vision, notes that it uses an existing tactic called structured light to capture your visage in three dimensions—something he employed for object recognition back in the 1980s.

That doesn’t mean it’s not a big deal if the company can get it to work well, though. There are different ways to perform facial recognition, but generally the accuracy of it depends on factors such as the quality of the image of your face at authentication time, light conditions, time between the enrollment image and verification, and visibility of occluding objects like a scarf or sunglasses. Beyond the work the company has done to keep the wrong people out of the phone, Apple claims that Face ID will let the right person in even in the dark, while wearing glasses or a hat, and after growing a beard.

“Having everything working on a mobile phone is quite amazing,” Jain says.

It could also lead more companies to add similar three-dimensional facial-recognition technology into their phones, either by trying to cram all the hardware for it into their own phones, or by using software-based facial recognition from a company like FaceTec (which is something that individual app developers could add to their apps, too).

“Everyone’s still in the mode of copying what Apple did largely, and the 3-D does have the effect of wiping out simple spoofs—you can’t hold up a picture anymore and expect it to do anything,” says Kevin Bowyer, a University of Notre Dame professor who studies biometrics and object recognition.

And Face ID—which Apple touts as being less vulnerable to bad actors than its Touch ID fingerprint sensing—could be more secure than using a fingerprint sensor on a phone simply because such sensors tend to be so small that they require you to input several partial prints from the same finger. A study from Michigan State and New York University earlier this year used computer simulations to show that such prints could be compromised by dummy prints.

Rather than wiping out fingerprint sensors, though, the inclusion of facial recognition on the iPhone may lead to future phones with a greater breadth of biometric sensors. Iris scanning, for instance, commonly uses near-infrared illumination, which the iPhone X has (such a feature is already included in the aforementioned Samsung handset).

Jain says it’s conceivable that smartphones will eventually include sensors for face, iris, and fingerprint recognition—a rarity now. The cost isn’t that high for the hardware, he reasons, and perhaps you’d use them in different combinations for different transactions—the user might decide which they want to use, or for a big purchase on the phone a merchant might want you to use all three.

“There is no reason why all three can’t be in the mobile phone within the next few years,” he says.