How Obama Wrangled Data to Win His Second Term

This is Part 2 of our in-depth profile of the big data techniques that gave Barack Obama a second term in office. Read Part 1.

The Experiments

When Jim Messina arrived in Chicago as Obama’s newly minted campaign manager in January of 2011, he imposed a mandate on his recruits: they were to make decisions based on measurable data. But that didn’t mean quite what it had four years before. The 2008 campaign had been “data-driven,” as people liked to say. This reflected a principled imperative to challenge the political establishment with an empirical approach to electioneering, and it was greatly influenced by David Plouffe, the 2008 campaign manager, who loved metrics, spreadsheets, and performance reports. Plouffe wanted to know: How many of a field office’s volunteer shifts had been filled last weekend? How much money did that ad campaign bring in?

But for all its reliance on data, the 2008 Obama campaign had remained insulated from the most important methodological innovation in 21st-century politics. In 1998, Yale professors Don Green and Alan Gerber conducted the first randomized controlled trial in modern political science, assigning New Haven voters to receive nonpartisan election reminders by mail, phone, or in-person visit from a canvasser and measuring which group saw the greatest increase in turnout. The subsequent wave of field experiments by Green, Gerber, and their followers focused on mobilization, testing competing modes of contact and get-out-the-vote language to see which were most successful.

The first Obama campaign used the findings of such tests to tweak call scripts and canvassing protocols, but it never fully embraced the experimental revolution itself. After Dan Wagner moved to the DNC, the party decided it would start conducting its own experiments. He hoped the committee could become “a driver of research for the Democratic Party.”

To that end, he hired the Analyst Institute, a Washington-based consortium founded under the AFL-CIO’s leadership in 2006 to coördinate field research projects across the electioneering left and distribute the findings among allies. Much of the experimental world’s research had focused on voter registration, because that was easy to measure. The breakthrough was that registration no longer had to be approached passively; organizers did not have to simply wait for the unenrolled to emerge from anonymity, sign a form, and, they hoped, vote. New techniques made it possible to intelligently profile nonvoters: commercial data warehouses sold lists of all voting-age adults, and comparing those lists with registration rolls revealed eligible candidates, each attached to a home address to which an application could be mailed. Applying microtargeting models identified which nonregistrants were most likely to be Democrats and which ones Republicans.

The Obama campaign embedded social scientists from the Analyst Institute among its staff. Party officials knew that adding new Democratic voters to the registration rolls was a crucial element in their strategy for 2012. But already the campaign had ambitions beyond merely modifying nonparticipating citizens’ behavior through registration and mobilization. It wanted to take on the most vexing problem in politics: changing voters’ minds.

The expansion of individual-level data had made possible the kind of testing that could help do that. Experimenters had typically calculated the average effect of their interventions across the entire population. But as campaigns developed deep portraits of the voters in their databases, it became possible to measure the attributes of the people who were actually moved by an experiment’s impact. A series of tests in 2006 by the women’s group Emily’s List had illustrated the potential of conducting controlled trials with microtargeting databases. When the group sent direct mail in favor of Democratic gubernatorial candidates, it barely budged those whose scores placed them in the middle of the partisan spectrum; it had a far greater impact upon those who had been profiled as soft (or nonideological) Republicans.

That test, and others that followed, demonstrated the limitations of traditional targeting. Such techniques rested on a series of long-standing assumptions—for instance, that middle-of-the-roaders were the most persuadable and that infrequent voters were the likeliest to be captured in a get-out-the-vote drive. But the experiments introduced new uncertainty. People who were identified as having a 50 percent likelihood of voting for a Democrat might in fact be torn between the two parties, or they might look like centrists only because no data attached to their records pushed a partisan prediction in one direction or another. “The scores in the middle are the people we know less about,” says Chris Wyant, a 2008 field organizer who became the campaign’s general election director in Ohio four years later. “The extent to which we were guessing about persuasion was not lost on any of us.”

One way the campaign sought to identify the ripest targets was through a series of what the Analyst Institute called “experiment-informed programs,” or EIPs, designed to measure how effective different types of messages were at moving public opinion.

The traditional way of doing this had been to audition themes and language in focus groups and then test the winning material in polls to see which categories of voters responded positively to each approach. Any insights were distorted by the artificial settings and by the tiny samples of demographic subgroups in traditional polls. “You’re making significant resource decisions based on 160 people?” asks Mitch Stewart, director of the Democratic campaign group Organizing for America. “Isn’t that nuts? And people have been doing that for decades!”

An experimental program would use those steps to develop a range of prospective messages that could be subjected to empirical testing in the real world. Experimenters would randomly assign voters to receive varied sequences of direct mail—four pieces on the same policy theme, each making a slightly different case for Obama—and then use ongoing survey calls to isolate the attributes of those whose opinions changed as a result.

In March, the campaign used this technique to test various ways of promoting the administration’s health-care policies. One series of mailers described Obama’s regulatory reforms; another advised voters that they were now entitled to free regular check-ups and ought to schedule one. The experiment revealed how much voter response differed by age, especially among women. Older women thought more highly of the policies when they received reminders about preventive care; younger women liked them more when they were told about contraceptive coverage and new rules that prohibited insurance companies from charging women more.

When Paul Ryan was named to the Republican ticket in August, Obama’s advisors rushed out an EIP that compared different lines of attack about Medicare. The results were surprising. “The electorate [had seemed] very inelastic,” says Terry Walsh, who coördinated the campaign’s polling and paid-media spending. “In fact, when we did the Medicare EIPs, we got positive movement that was very heartening, because it was at a time when we were not seeing a lot of movement in the electorate.” But that movement came from quarters where a traditional campaign would never have gone hunting for minds it could change. The Obama team found that voters between 45 and 65 were more likely to change their views about the candidates after hearing Obama’s Medicare arguments than those over 65, who were currently eligible for the program.

A similar strategy of targeting an unexpected population emerged from a July EIP testing Obama’s messages aimed at women. The voters most responsive to the campaign’s arguments about equal-pay measures and women’s health, it found, were those whose likelihood of supporting the president was scored at merely 20 and 40 percent. Those scores suggested that they probably shared Republican attitudes; but here was one thing that could pull them to Obama. As a result, when Obama unveiled a direct-mail track addressing only women’s issues, it wasn’t to shore up interest among core parts of the Democratic coalition, but to reach over for conservatives who were at odds with their party on gender concerns. “The whole goal of the women’s track was to pick off votes for Romney,” says Walsh. “We were able to persuade people who fell low on candidate support scores if we gave them a specific message.”

At the same time, Obama’s campaign was pursuing a second, even more audacious adventure in persuasion: one-on-one interaction. Traditionally, campaigns have restricted their persuasion efforts to channels like mass media or direct mail, where they can control presentation, language, and targeting. Sending volunteers to persuade voters would mean forcing them to interact with opponents, or with voters who were undecided because they were alienated from politics on delicate issues like abortion. Campaigns have typically resisted relinquishing control of ground-level interactions with voters to risk such potentially combustible situations; they felt they didn’t know enough about their supporters or volunteers. “You can have a negative impact,” says Jeremy Bird, who served as national deputy director of Organizing for America. “You can hurt your candidate.”

In February, however, Obama volunteers attempted 500,000 conversations with the goal of winning new supporters. Voters who’d been randomly selected from a group identified as persuadable were polled after a phone conversation that began with a volunteer reading from a script. “We definitely find certain people moved more than other people,” says Bird. Analysts identified their attributes and made them the core of a persuasion model that predicted, on a scale of 0 to 10, the likelihood that a voter could be pulled in Obama’s direction after a single volunteer interaction. The experiment also taught Obama’s field department about its volunteers. Those in California, which had always had an exceptionally mature volunteer organization for a non-­battleground state, turned out to be especially persuasive: voters called by Californians, no matter what state they were in themselves, were more likely to become Obama supporters.

With these findings in hand, Obama’s strategists grew confident that they were no longer restricted to advertising as a channel for persuasion. They began sending trained volunteers to knock on doors or make phone calls with the objective of changing minds.

That dramatic shift in the culture of electioneering was felt on the streets, but it was possible only because of advances in analytics. Chris Wegrzyn, a database applications developer, developed a program code-named Airwolf that matched county and state lists of people who had requested mail ballots with the campaign’s list of e-mail addresses. Likely Obama supporters would get regular reminders from their local field organizers, asking them to return their ballots, and, once they had, a message thanking them and proposing other ways to be involved in the campaign. The local organizer would receive daily lists of the voters on his or her turf who had outstanding ballots so that the campaign could follow up with personal contact by phone or at the doorstep. “It is a fundamental way of tying together the online and offline worlds,” says Wagner.

Wagner, however, was turning his attention beyond the field. By June of 2011, he was chief analytics officer for the campaign and had begun making the rounds of the other units at headquarters, from fund-raising to communications, offering to help “solve their problems with data.” He imagined the analytics department—now a 54-person staff, housed in a windowless office known as the Cave—as an “in-house consultancy” with other parts of the campaign as its clients. “There’s a process of helping people learn about the tools so they can be a participant in the process,” he says. “We essentially built products for each of those various departments that were paired up with a massive database we had.”

The Flow

As job notices seeking specialists in text analytics, computational advertising, and online experiments came out of the incumbent’s campaign, Mitt Romney’s advisors at the Republicans’ headquarters in Boston’s North End watched with a combination of awe and perplexity. Throughout the primaries, Romney had appeared to be the only Republican running a 21st-century campaign, methodically banking early votes in states like Florida and Ohio before his disorganized opponents could establish operations there.

But the Republican winner’s relative sophistication in the primaries belied a poverty of expertise compared with the Obama campaign. Since his first campaign for governor of Massachusetts, in 2002, Romney had relied upon ­TargetPoint Consulting, a Virginia firm that was then a pioneer in linking information from consumer data warehouses to voter registration records and using it to develop individual-level predictive models. It was TargetPoint’s CEO, Alexander Gage, who had coined the term “microtargeting” to describe the process, which he modeled on the corporate world’s approach to customer relationship management.

Such techniques had offered George W. Bush’s reëlection campaign a significant edge in targeting, but Republicans had done little to institutionalize that advantage in the years since. By 2006, Democrats had not only matched Republicans in adopting commercial marketing techniques; they had moved ahead by integrating methods developed in the social sciences.

Romney’s advisors knew that Obama was building innovative internal data analytics departments, but they didn’t feel a need to match those activities. “I don’t think we thought, relative to the marketplace, we could be the best at data in-house all the time,” Romney’s digital director, Zac Moffatt, said in July. “Our idea is to find the best firms to work with us.” As a result, Romney remained dependent on TargetPoint to develop voter segments, often just once, and then deliver them to the campaign’s databases. That was the structure Obama had abandoned after winning the nomination in 2008.

In May a TargetPoint vice president, Alex Lundry, took leave from his post at the firm to assemble a data science unit within Romney’s headquarters. To round out his team, Lundry brought in Tom Wood, a University of Chicago postdoctoral student in political science, and Brent McGoldrick, a veteran of Bush’s 2004 campaign who had left politics for the consulting firm Financial Dynamics (later FTI Consulting), where he helped financial-services, health-care, and energy companies communicate better. But Romney’s data science team was less than one-tenth the size of Obama’s analytics department. Without a large in-house staff to handle the massive national data sets that made it possible to test and track citizens, Romney’s data scientists never tried to deepen their understanding of individual behavior. Instead, they fixated on trying to unlock one big, persistent mystery, which Lundry framed this way: “How can we get a sense of whether this advertising is working?”

“You usually get GRPs and tracking polls,” he says, referring to the gross ratings points that are the basic unit of measuring television buys. “There’s a very large causal leap you have to make from one to the other.”

Lundry decided to focus on more manageable ways of measuring what he called the information flow. His team converted topics of political communication into discrete units they called “entities.” They initially classified 200 of them, including issues like the auto industry bailout, controversies like the one surrounding federal funding for the solar-power company Solyndra, and catchphrases like “the war on women.” When a new concept (such as Obama’s offhand remark, during a speech about our common dependence on infrastructure, that “you didn’t build that”) emerged as part of the election-year lexicon, the analysts added it to the list. They tracked each entity on the National Dialogue Monitor, TargetPoint’s system for measuring the frequency and tone with which certain topics are mentioned across all media. TargetPoint also integrated content collected from newspaper websites and closed-caption transcripts of broadcast programs. Lundry’s team aimed to examine how every entity fared over time in each of two categories: the informal sphere of social media, especially Twitter, and the journalistic product that campaigns call earned press coverage.

Ultimately, Lundry wanted to assess the impact that each type of public attention had on what mattered most to them: Romney’s position in the horse race. He turned to vector autoregression models, which equities traders use to isolate the influence of single variables on market movements. In this case, Lundry’s team looked for patterns in the relationship between the National Dialogue Monitor’s data and Romney’s numbers in Gallup’s daily tracking polls. By the end of July, they thought they had identified a three-step process they called “Wood’s Triangle.”

Within three or four days of a new entity’s entry into the conversation, either through paid ads or through the news cycle, it was possible to make a well-informed hypothesis about whether the topic was likely to win media attention by tracking whether it generated Twitter chatter. That informal conversation among political-class elites typically led to traditional print or broadcast press coverage one to two days later, and that, in turn, might have an impact on the horse race. “We saw this process over and over again,” says Lundry.

They began to think of ads as a “shock to the system”—a way to either introduce a new topic or restore focus on an area in which elite interest had faded. If an entity didn’t gain its own energy—as when the Republicans charged over the summer that the White House had waived the work requirements in the federal welfare rules—Lundry would propose a “re-shock to the system” with another ad on the subject five to seven days later. After 12 to 14 days, Lundry found, an entity had moved through the system and exhausted its ability to move public opinion—so he would recommend to the campaign’s communications staff that they move on to something new.

Those insights offered campaign officials a theory of information flows, but they provided no guidance in how to allocate campaign resources in order to win the Electoral College. Assuming that Obama had superior ground-level data and analytics, Romney’s campaign tried to leverage its rivals’ strategy to shape its own; if Democrats thought a state or media market was competitive, maybe that was evidence that Republicans should think so too. “We were necessarily reactive, because we were putting together the plane as it took off,” Lundry says. “They had an enormous head start on us.”

Romney’s political department began holding regular meetings to look at where in the country the Obama campaign was focusing resources like ad dollars and the president’s time. The goal was to try to divine the calculations behind those decisions. It was, in essence, the way Microsoft’s Bing approached Google: trying to reverse-engineer the market leader’s code by studying the visible output. “We watch where the president goes,” Dan ­Centinello, the Romney deputy political director who oversaw the meetings, said over the summer.

Obama’s media-buying strategy proved particularly hard to decipher. In early September, as part of his standard review, Lundry noticed that the week after the Democratic convention, Obama had aired 68 ads in Dothan, Alabama, a town near the Florida border. Dothan was one of the country’s smallest media markets, and Alabama one of the safest Republican states. Even though the area was known to savvy ad buyers as one of the places where a media market crosses state lines, Dothan TV stations reached only about 9,000 Florida voters, and around 7,000 of them had voted for John McCain in 2008. “This is a hard-core Republican media market,” Lundry says. “It’s incredibly tiny. But they were advertising there.”

Romney’s advisors might have formed a theory about the broader media environment, but whatever was sending Obama hunting for a small pocket of votes was beyond their measurement. “We could tell,” says McGoldrick, “that there was something in the algorithms that was telling them what to run.”

Tomorrow: Part 3—The Community