This VR Exhibit Lets You Connect with the Human Side of War

Sun streams through a grid of skylights, carving the gallery’s wooden floor into a checkerboard. When I look up, I can see wispy clouds passing overhead. Large photos hang on the gallery walls. They’re pictures of a landscape devastated by war and portraits of men fighting in those wars.

I hear footsteps behind me. I turn around and watch two figures enter the room and take up stations in front of the portraits. They’re the men from the pictures.

An unseen narrator explains that the shorter one, Jean de Dieu, was a child soldier recruited by the Democratic Forces for the Liberation of Rwanda (FDLR). It’s a Hutu group waging war against Rwanda from its base in the eastern part of the Democratic Republic of the Congo. The other, Patient, is a sergeant in the Congolese army, which is allied with Rwanda’s ruling Tutsi ethnic group.

I know they’re both virtual characters, re-created through 3-D scanning and computer graphics. But they’re startlingly realistic—far more lifelike than anything I’ve seen in a game or movie.

As I approach Jean de Dieu, who looks sad and tired, a conversation begins. The narrator asks: Who is your enemy? What is violence for you? What makes your enemy inhuman? Jean answers in halting, vulnerable tones. I listen to his story of being forced into a refugee camp at age 11 and seeing Congolese militia kill his parents, their brains splattering onto him. Of course he’d hate the Tutsi, and everyone aligned with them.

Now the narrator quizzes Patient. He says the army pursues the FDLR because its soldiers rob, rape, and murder Congolese citizens. “He has no human values and can no longer change his mind,” Patient says of his despised FDLR enemy. “He wants to stay in the forest as part of the rebellion like a savage. Only beasts live in the forest.”

But Patient and Jean de Dieu also tell the narrator something else: they just want to live in peace with their neighbors and families. And as I walk through three more rooms and meet more combatants—gang members in El Salvador, a reservist in Israel and a Palestinian fighter in Gaza—I hear that shared hope flicker through in answer after answer. These men all have different stories, different traumas, and different allegiances. But their dreams are the same. Abu Khaled, in Gaza, says 23 of his family members have died during the Israeli occupation, but he still hopes for “peace and brotherhood” in the region.

After 40 minutes, I’m guided to a spot on the floor that resembles a Star Trek transporter pad. An assistant helps me remove my Oculus Rift VR headset and backpack, and I’m back on the ground floor of the MIT Museum, where this ambitious virtual-reality exhibit, “The Enemy,” made its North American premiere in the fall of 2017.

The exhibit—or maybe “experience” is a better word—is the creation of the Belgian-Tunisian photojournalist Karim Ben Khelifa. He interviewed and filmed the fighters and then worked with Fox Harrell, a professor of digital media and artificial intelligence at MIT, and French partners Camera Lucida, France Télévisions Nouvelles Ecritures, and Emissive to bring them to life inside the virtual gallery.

Part of what’s groundbreaking about “The Enemy” is the sheer size of the simulation: the museum cleared out a 3,000-square-foot space so that up to 15 Oculus-wearing visitors at a time could roam freely in the virtual world. The fidelity of the characters and their movements is also striking. You can see the stubble on their chins and the tattoos on their arms and torsos. Thanks to eye-tracking sensors, each figure’s gaze is locked onto yours, cementing the illusion that the fighters are speaking directly to you. The technology works well enough to disappear, allowing you to form direct, empathetic connections with Jean, Patient, Abu, and their fellow combatants.

Which is exactly what Ben Khelifa wanted. “My interest was, can you look at these people in the eyes?” he told me. “Can they look you in the eyes? And what is happening when two people look at one another in the eyes? There is a connection, whether we want it or not.”

Right now, the “The Enemy” is accessible only to museum visitors, but Ben Khelifa says he wants those trapped in conflict zones, especially young people, to experience it too. If the installation can help people see that every conflict is grounded, to some extent, in stereotypes and misunderstandings, they might come to understand one another better and stop fighting, he believes. It’s a noble goal—but will all future VR producers have such benevolent aims?

Blown away

The idea that VR might be a medium for a new kind of journalism took hold around 2015, when the New York Times released its first VR documentary, “The Displaced,” about three young war refugees. Technically, the pieces produced by the Times’ VR studio are 360° films. Viewers can look in different directions, but otherwise, they watch passively. Sticklers reserve the term “virtual reality” for simulated 3-D environments in which users can move around at will and control objects, as gamers can on platforms such as HTC Vive, PlayStation VR, and Oculus Rift. That’s the type of virtual reality that Ben Khelifa, a freelancer who has covered conflicts in Iraq, Libya, Syria, Israel, Yemen, Somalia, and many other countries, wanted to employ for “The Enemy.”

Ben Khelifa says he was worried that traditional war images have lost their power. Take the famous photo of Alan Kurdi, the three-year-old refugee boy whose body washed ashore in Turkey in 2015. “Every single parent in the world should react to this and say, ‘That could be my kid,’” Ben Khelifa says. But though the image saddened millions, it didn’t move nations to intervene in Syria. “We don’t have the same emotional relation with photos that we used to have,” he says.

A virtual-reality re-creation of a fighter, speaking in his own words, might help viewers feel the impact of war more deeply, Ben Khelifa believed. So he went to Israel and Gaza, where he found soldiers willing to be videotaped. While they talked, he scanned them with a Microsoft Kinect and photographed them from multiple angles. He says his experience as a photojournalist helped him get the subjects to open up. “These fighters understand that I’ve been through a lot of fighting too—without holding a gun, but holding my camera,” Ben Khelifa says. “And I think there is—I wouldn’t call it a brotherhood, but an understanding that we both know what war is.”

In April 2015, at New York’s Tribeca Film Festival, Ben Khelifa showed a prototype of “The Enemy,” featuring only Abu Khaled and an Israeli soldier named Gilad. “People were just blown away by the realism of the fighters,” he says. But these early figures didn’t walk, turn their heads, or react to users. “From there, what I’ve been realizing is, the more the fighters are modified to recognize your presence, the more you recognize the presence of the fighter,” he says. “You spend less time wondering if he’s real or not. And you get to listen.”

A few years earlier Ben Khelifa had met MIT’s Fox Harrell, whose book Phantasmal Media explores how creators of VR and other computational media can build experiences that mutate depending on the user’s actions. Harrell says he’s fascinated by the narrative techniques of the 1950 Kurosawa film Rashomon, which retells the story of a brutal rape and murder from multiple perspectives. “I’ve been interested in how you can use algorithmic processes in AI to trigger these kinds of effects,” he says.

For “The Enemy,” Harrell helped Ben Khelifa and his team of developers in France build a system that surveys visitors before the experience and then monitors them on camera and via the Oculus headset as they interact with each fighter. Visitors’ responses determine the order in which they experience the three conflicts, the message they receive in the final gallery, and even the weather visible through the skylights.

John Durant, the director of the MIT Museum, says “The Enemy” took the museum into untested territory, both technologically and politically. “It was very appealing, because a lot of us talk about the ways in which technology may or may not contribute to addressing certain kinds of social and political issues, and sometimes people talk about it more than actually experiencing it and trying it,” he says.

The poignant stories told by Amilcar and Jorge, members of two rival gangs in San Salvador, give that section of the exhibit a sticking power that a photo essay just wouldn’t have, Durant says. “Most of the people who are likely to visit this museum don’t have the experience of growing up as members of a gang where a kind of tribal loyalty is perhaps the most fundamental thing you know,” he says. “So it takes some effort, honestly, to try and think about what the world might be like from that point of view. I think ‘The Enemy,’ to me, made it much easier.”

Visitors to the museum report similar revelations. “I’m from Colombia … I’ve lived close to war,” one visitor wrote in the guest book. “Forgiveness is gonna be always the hardest part. For forgiveness to appear, there’s gotta be compassion, and that is what ‘The Enemy’ brought me. Thank you.”

Brainwashing

VR has, in fact, begun to compete with old-fashioned photojournalism and TV news. VR producers have been flocking to Southeast Asia lately to document the plight of the Rohingya, a Muslim-majority ethnic group under assault in Buddhist-majority Myanmar. A refugee featured in a searing Al Jazeera VR film recounted how security forces in Myanmar had killed her husband and raped her. An Emmy-nominated VR film shot inside a Rohingya confinement camp by the anti-atrocity group the Nexus Fund showed prisoners languishing with little food or medical care. “I can’t put everybody on a plane and take them to Myanmar, but I know that if I could and they could see this in person, there’s nothing they wouldn’t do to help,” Nexus Fund executive director Sally Smith told CNN.

But if VR is an empathy machine, where will all that empathy be directed in the future? Here in the United States, meddlers have hijacked Google, YouTube, Facebook, and Twitter to generate outrage and spread falsehoods, with political consequences we are only beginning to understand. VR’s immersiveness and realism pull even more directly on our heartstrings. There’s nothing to stop Buddhist extremists in Myanmar, for instance, from making VR films designed to further inflame passions against the Rohingya. “Am I scared by it? Yeah,” Ben Khelifa says. “If you can create empathy, you can brainwash people too.”

In “The Enemy,” the VR storytelling is even-handed to a fault. In fact, if the piece has a limitation, it’s that it refuses to judge the merits of each fighter’s cause. But that limitation is also a strength. The parallel questions put to each combatant allow the visitor to construct “this kind of model of what’s the same and what’s different” for each fighter, Harrell explains. “And that can be some impetus to thinking beyond the preconceptions you had of the conflict.”

Without this kind of commitment to fairness and factuality, VR could easily devolve into a propaganda tool. But that’s true of all journalism. We’re fortunate that a creator with Ben Khelifa’s vision and conscience is showing the way. 

Wade Roush is a technology journalist and the producer and host of Soonish, a podcast about technology and the future.

"The Enemy” was produced by Camera Lucida, France Télévisions, the National Film Board of Canada, Emissive, and Dpt, and was staged at the MIT Museum in late 2017. It will continue its North American tour in Montreal and other Canadian cities. For tour dates visit theenemyishere.org.