This AI Learns Your Fashion Sense and Invents Your Next Outfit

Artificial intelligence might just spawn a whole new style trend: call it “predictive fashion.”

In a paper published on the ArXiv, researchers from the University of California, San Diego, and Adobe have outlined a way for AI to not only learn a person’s style but create computer-generated images of items that match that style. The system could let retailers create personalized pieces of clothing, or could even be used to help predict broader fashion trends.

The paper details two different algorithms. First, the researchers trained a convolutional neural network (CNN) to learn and classify a user’s preferences for certain items, using purchase data scraped from Amazon in six categories: shoes, tops, and pants for both women and men. This type of recommender model is common in the online retail world, usually showing up in an “Other items you might like” area at the bottom of a page.

The team then used that information to train a generative adversarial network (GAN), a type of artificial intelligence that is especially proficient when it comes to generating realistic images. A GAN works by having two networks train on the same data. One of the networks generates fake images based on that data set, while the other network uses the same data to determine whether an image is real. This method lets the network improve its results. For this research, the GAN created multiple images of items for each user.

GANs, which were created by Ian Goodfellow, one of MIT Technology Review’s 35 Innovators Under 35 for 2017, have been in the news recently: after a different research team trained them on real images of Hollywood stars, the networks were able to create eerily believable fake celebrity faces. The faces weren’t all perfect, though—some had blurred areas or were missing features like eyebrows. There were fewer such problems with the fashion project, largely because the images used to train the networks were all shot from the same angle on white backgrounds, which makes generating convincing images much easier—something that would be essential if they were ever to be used to sell clothing.

Adding GANs to recommender systems could help online retailers figure out what customers want beyond the items that already exist. Still, researchers would need to figure out quite a few things before that could happen, including how to turn the two-dimensional computer-generated images into 3-D renderings that could be used to produce a piece of clothing.

“It’s not like we are generating a sewing pattern,” says Julian McAuley, a computer scientist at UC San Diego and one of the paper’s authors.

The team’s GAN also has a way to go before it can replace a stylist or even suggest a new outfit. At the moment, for a shopper who liked blue shirts, the GAN creates more blue shirts—hardly a revelation. Preference for black pants did feed into the GAN to create khakis, but the system can’t create a set of shoes that would go well with a certain pair of pants yet.

“You’d have to read the tea leaves a little bit if you want to call that style or not,” McAuley says.

Despite the current limitations, fashion seems ripe for an AI invasion; it’s an arena that has great data sets on customers’ interests, and there is a lot of money at stake. Amazon, for one, is already working on AI systems to provide a leg up in spotting fashion trends, and it has also done some work with GANs (see “Amazon Has Developed an AI Fashion Designer”). Alibaba, meanwhile, just debuted FashionAI, a technology that can recommend items to shoppers on the basis of what they brought into the dressing room.

Costa Colbert, the chief scientist at Vue.ai, a fashion AI startup that recently revealed a method for creating fake fashion models using GANs, says that as promising as the UCSD and Adobe research appears to be, it requires so much data that it might be helpful only for the biggest names in online retail.

“If all the person does is come in and click one thing, you aren’t going to be able to do much,” Colbert says.

GANs will continue to make waves in online fashion, however. Colbert points out that some companies already let shoppers send in their personal measurements to get customized pieces. GANs could be a cheap, fast way to show users all the different options available—and, of course, sell more items.