The Other Side of the Desk

It’s one of many rites of passage that junior faculty face: the first lecture to their professor peers. And in August 2011, it was Julie (Arnold) Shah’s turn. As one of the newly hired assistant profs in MIT’s Aero-Astro department, Shah prepared a formal presentation about her innovative work in human-robot collaboration and delivered it at the department’s last faculty lunch before the start of the academic year.

She scanned the tables and the extra chairs that lined the walls in conference room 33-206. Almost every seat was full. She spotted familiar faces, including department chair Jaime Peraire, robotics whiz Nick Roy, and top-ranking faculty such as Edward Greitzer, who’s been at the Institute since 1977. But much of the 20-minute presentation was a blur.

“I was so nervous,” Shah says. “I felt like an 18-year-old talking to my professors.”

This was no ordinary case of new-job nerves. Because not that long ago, Peraire, Greitzer, and other faculty seated around the room were her professors. Shah, 30, has been at MIT since she was a freshman, completing bachelor’s, master’s, and doctoral degrees. Peraire had her in his classroom during her sophomore year and remembers her as one of his “smart students.” Roy guest-lectured in some of Shah’s graduate courses. Greitzer taught her how to give a solid technical presentation in his experimental design class.

Multimedia

• Julie Shah photo gallery

Shah is not the first Aero-Astro graduate to be instantly hired as a professor. (Institute Professor Sheila Widnall ’60, SM ’61, ScD ’64, for example, became MIT’s first female professor of engineering the same year she got her doctorate.) But it’s not a typical career track. Institute-­wide, the percentage of alumni on the faculty has dropped from 37 percent in 1991 to 27 percent in 2011, and it’s relatively rare for a newly minted MIT PhD to join the faculty right after graduating. (In 2011, fewer than 2 percent of faculty members had been hired the same year they’d earned an MIT doctorate, down from 3 percent in 1991.) Like most departments at MIT, Aero-Astro prefers to hire faculty from the outside—or at least those with outside experience. Mixing cultures from different academic institutions creates a richer environment for all, says ­Peraire. “We tend not to hire our own unless there is a strong reason to do so.”

For Shah, making the transition from student to scholar means reminding herself to call Greitzer “Ed” instead of “Professor.” Changing the way she spends her days, from sitting quietly at her desk, tweaking the algorithms for the robot decision-­making software she’d designed, to steering a half-dozen grad students pursuing multiple research ideas. Settling into her new office with a sweet view of the Wright Brothers Wind Tunnel—an office she first set foot in a decade ago to meet her advisor, Olivier de Weck, SM ’99, PhD ’01. Today, de Weck directs MIT’s Strategic Engineering Systems group and calls Shah a “world leader” in her field. But back then, he was the new professor sitting behind the desk, and she was a sophomore coming to talk about her budding interest in research.

“It’s incredible to be on the other side of the desk,” she says.

At 18, when Shah came to MIT, she never expected to one day fill the professor’s chair. Growing up in New Jersey, she wanted to be an astronaut, drawing rocket ships on a chalkboard even before she knew how to write. When she declared her major during her sophomore year and met with de Weck, he was struck by her passion for aerospace—and her sunny, optimistic, forward-looking disposition. Throughout college, Shah pictured herself someday working at NASA; one of her prized mementos is an autographed photo of Apollo 17 astronaut Eugene Cernan, the last man to walk on the moon. She didn’t even consider becoming a professor until she had her first graduate teaching assistantship and discovered she enjoyed the process of organizing and distilling scientific knowledge for students.

MIT brought Shah back partly on the strength of her work in the burgeoning field of human-robot collaboration. For her doctoral thesis project, she designed software called Chaski, which lets robots decide what to do and when; the industry jargon for this is “plan execution system.” Typically, plan execution systems delineate all a robot’s tasks in advance, sequencing them one by one. But Chaski, which mimics the best aspects of how human teams work together, allows a robot to watch, anticipate, and nimbly adapt to the shifting demands of its human coworkers—and it takes a fraction as much computing time as previous systems.

“What is pathbreaking about [Shah’s work] is the basic notion that humans and robots could be ‘friends’ or ‘peers’ in the workplace, helping each other collaboratively,” de Weck says. “This is not at all the prevailing view.” In most manufacturing areas, he explains, humans and robots are viewed as “enemies”; at the least, it’s thought that they can’t work side by side, in part for safety reasons. “Julie’s vision is that robots can serve as very effective assistants to humans—for example, in space exploration, in medical surgery, and in industrial manufacturing,” he says. “This requires that humans and robots become aware of each other’s locations, movements, and most importantly, intentions and future moves.”

Shah is not the only one working in this area. Many major industrial-robot companies are now designing “inherently safe” robots, she says. At Rethink Robotics, for example, professor emeritus Rodney Brooks is making industrial robots that are safe enough to work with humans (see “This Robot Could Transform Manufacturing”). Her lab is going a step further, designing robots that Shah says can work in teams with people by learning their work styles and using flexible decision making—machines that are “not just safe enough to work alongside people, but smart enough to figure out how to assist people.”

A Stint in Seattle

MIT’s teaching offer (which was extended before she’d defended her thesis) came with a strong recommendation: that Shah spend an academic year away from MIT, at a place where she did not know anyone. She lined up a postdoc with Boeing Research and Technology in Seattle, where she gained manufacturing experience that gave her a clearer picture of the possible real-world applications for her work.

Airplane manufacturing is a hands-on and staggeringly complex affair. Picture a massive factory floor with jets arrayed nose to tail in varying stages of production; a beehive of people hand-assembling hundreds of thousands of parts, seaming every inch of fuselage, tightening every bolt, soldering every wire. It would not be practical to adopt traditional automobile-style factory automation, where for safety’s sake robots are physically separated from people, repetitively riveting, bolting, and welding away in massive cages. Just imagine trying to move a massive jet engine or wing assembly from the robot cage to the human production line.

But what if people and robots could work more closely together, each doing what they do best? The robot could pick a tool or a part and hand it to a worker, for example, freeing the worker for finer-grained assembly tasks. One of the many obstacles to this model in aviation (and in other types of manual assembly) is that humans are unpredictable—one may like to do a task one way, the next another way. Robots need to be able to learn about these differences and adapt to them on the fly. The speedy adaptive learning algorithms that Shah has developed for her Chaski software could be applied to just this type of problem.

While still at Boeing, Shah began laying the groundwork for the CSAIL research group she would launch at MIT. (Coincidentally, she’d been named the Boeing Career Development Assistant Professor, which meant she had funding to do so.) The Institute had offered her a yet-to-be-renovated space in the Gas Turbine Laboratory building for her Interactive Robotics Group (IRG, pronounced “urg”). The room had old turbine test-bed equipment piled up to the 20-foot-high ceiling. Still, she could envision the perfect robotics lab: a playground for pint-size industrial robots in the back; specialized motion-capture cameras mounted high around the perimeter to track people’s hand and body movements during robot experiments and share them with robot coworkers in real time; grad-student desks along the wall of windows. When she visited MIT in March 2011 to court her first grad students, one asked Shah for a peek at the future lab. “Here’s how I’m going to lose him,” she recalls thinking. “I’m going to show him the space, and he’s going to run for the hills.” But the student, Matthew Gombolay, says that like Shah, he wasn’t fazed by the mess—he was excited. He had seen the blueprints of the room, but seeing it in person made it come to life: “I could visualize the space and see it all coming together.”

From a student’s perspective, becoming a founding member of a new professor’s lab does take a bit of vision, and not just because of the décor. You forgo the certainty of an already-famous professor’s name on your résumé to gamble on the brilliance of an as-yet-untested researcher. If all goes well, the professor’s and the lab’s reputations grow (and yours along with them). Yet there’s always the chance that your mentor won’t get tenure, leaving you in the lurch in the midst of your PhD.

Part of Shah’s appeal to Gombolay and the other three inaugural IRG members was that they would get to work more closely with her than they would with the chief of a more established lab, and in turn, they would have a greater say in the lab’s direction. “Julie herself pulled me in,” says Ron Wilcox, a master’s student in Shah’s lab. “She might not have the reputation yet but would eventually, and I knew that I’d get to spend significant blocks of time with her.”

By early October 2011, the renovation was complete, and the job of setting up the lab began. Students did just about everything from scratch—building the desks (Shah, too, pitched in), installing the motion-capture system, and figuring out how to get the robots up and running. (It took months for the first of them to arrive.) The students weighed in on every detail, from the cappuccino machine to the lab’s logo design, which shows a human hand and a robot hand joined in a handshake. The experience has given them a deeper emotional attachment to the lab. “You feel it is part of yourself, and in a sense it is,” says PhD student Stefanos Nikolaidis.

That fall shah also began co-teaching her first undergraduate class, Real-Time Systems and Software; she would be taking it over from Roy, who had taught it for years. The enrollment was just four students, juniors and seniors. A rule of thumb is that for every hour in front of the class, you need to spend three to four preparing your lecture, Shah says, and she easily took that much time and more. She hand-wrote all her notes, shuffling the order for the best flow, and then she would get in early on the morning of each lecture and reshuffle them again. Though she understood the subject matter, she had never taken this class, so she could not predict when students were likely to be perplexed. One such occasion, though disheartening to Shah at the time, led to one of her lab’s most exciting research advances in its first year.

The source of the confusion was the so-called “readers-writers problem,” a fundamental problem in real-time systems engineering: how to design software so that two different processes running at the same time don’t overwrite memory and are able to share resources. At the midterm exam review, Shah posed a question that shifted the readers-writers problem to the robotics context. If you have a drilling robot and a bolting robot, she asked, how do you program them so that they don’t get in each other’s way?

Her students looked at her with blank stares. They weren’t making the connection between robots vying for physical space and software vying for memory, and to Shah’s mind, this was not a good sign. “They’re not recognizing a canonical example that they should know,” she recalls thinking. “This reflects poorly on my teaching ability that they’re not getting this.”

While she was mulling it over later, she had an insight: maybe she could apply software techniques used in real-time processor scheduling to a fundamental problem of factory automation, the “job shop scheduling problem.” Basically, how do you know whether a given number of workers—be they human or machine—have enough resources to complete multiple, interrelated time-sensitive tasks within the time available? Shah made some notes, tried it out on paper, and then enlisted Gombolay to help.

“The current state-of-the-art systems are only able to solve the problem for one worker and 100 tasks,” he says. “We invented a method for solving the problem of one worker and 10,000 tasks.” Building on this, they were able to schedule 10 robots and 1,000 tasks in less than a minute, Shah says. Beyond being fast, the algorithm schedules tasks at the elusive “nearly optimal” level of efficiency. Shah credits Gombolay for the advance: “He spun some brilliance,” she says. MIT has since filed for patent protection on the invention.

Culture Breeds Creativity

Wooing brilliant students, setting up shop, and assigning creative (yet manageable) projects are all part of starting and sustaining a successful lab. But there’s one other essential ingredient: lab culture. Shah sought to create a positive, collaborative one that allows students to balance work and play. On a campus where many labs tend to keep nontraditional hours, with people coming and going on their own schedules (often late at night), she set hours of 9 a.m. to 5 p.m. so that her students would be in the lab at the same time, sharing energy and ideas.

Shah encourages her lab to get things done early through steady, high-quality work, rather than pulling all-nighters to meet deadlines. There are still some late nights, of course. In December 2011, for example, master’s student Ron Wilcox approached Shah about submitting his project to the RSS robotics conference, a prestigious venue that accepts only 33 percent of submissions. With the deadline less than two months away, this was an ambitious goal, especially for a new lab that didn’t even have a robot yet. Wilcox (who had studied theoretical physics as an undergrad and had only four months of robotics experience) was developing an algorithm that would let a robot observe human workers and subtly adjust the timing and sequence of its task plan to match their different working styles. If he could pull it off, the project would be a showcase for IRG’s approach to fluid human-robot collaboration. “Well, it’s possible,” Shah told him, “but there’s a lot to be done.”

Wilcox worked over Christmas break to develop his algorithms with help from his lab mates, who were excited about the prospect of producing the group’s first paper. On January 17, a gray and ­drizzly Tuesday, Gombolay drove Wilcox and Nikolaidis to the Connecticut headquarters of ABB, a maker of industrial robots, to pick up one that the company was donating to Shah’s lab. It’s a miniature version of the massive, orange caged robots found on factory floors; Shah’s students have nicknamed it Abbie. They drove straight back to Cambridge, carried Abbie up the stairs to the lab, plugged the robot in, and turned it on. They were able to put it through its paces using a simple joystick controller. Cheers and high fives all around, but other celebrating would have to wait: they had just two weeks to get Wilcox’s algorithm working and videotape demos. Thanks to the lab culture of getting things done early, Nikolaidis had already developed the software interface to the robot, using a robotics simulator. It was easy to port that over and then integrate Wilcox’s algorithm; all along, they had been planning, writing, and editing the paper, awaiting only the description of the demos. They submitted it late on the evening before the conference deadline.

Wilcox learned in April that his paper had been accepted, and Shah joined her students for a triumphant dinner at an Irish pub near the lab. “[It] just blew us all away,” she says of her new students’ coup. “I hadn’t set the bar for them at that level. Now they have set the bar for themselves.”

The students say Shah’s passion for the work, the constructive feedback she gives them, and the way she encourages collaboration all help motivate them. “I always feel an invisible force pushing me forward,” Nikolaidis says. But there’s an equally strong force that pushes the students out of the lab to play pool or eat hot wings: ­Wilcox, whom Shah appointed its social chair. (He planned IRG’s Thanksgiving dinner on the whiteboard, getting everyone signed up to bring a different dish.) “When you’re happy at work, you produce better,” Wilcox says, explaining Shah’s philosophy. “If it’s a pain in the butt to walk in in the morning, you’re not focusing very much. If you’re going to hang out with friends, you can really get some work done.”

Shah herself does not have the luxury of a 40-hour workweek—no new faculty member does. She says she’s had to go from single-tasking student to parallel-­processing professor whose days are divided across multiple endeavors—preparing lectures, running a lab, writing proposals, editing papers, attending meetings, sitting in on thesis defenses for students who will one day follow in her shoes. “Your satisfaction comes from keeping all these balls in the air,” Shah says. “It feels like half a dozen full-time jobs all the time. I’ve upped that number—I used to say it feels like three full-time jobs.” But she says the workload isn’t burdensome because she enjoys what she’s doing.

Now the tenure clock is ticking; Shah, like other new professors, has less than seven years to make her mark. And the expectations for new faculty are extremely high. “It is always challenging to start a new research program, develop new courses, and establish yourself as a leading figure in the field,” Peraire says. “I must say, however, that Julie is off to a very good start.”

It has certainly been a fast start: in July 2012, as her lab was preparing to double in size with the addition of four new students and a postdoc, Shah flew to Australia with Wilcox for the RSS conference, where she lectured in the University of Sydney’s Great Hall. Leading robotics researchers from around the world—including Roy and others from MIT—filled the seats as sunlight streamed through stained-glass windows depicting Chaucer, Newton, and other giants of letters and science. Introducing herself as “Professor Julie Shah,” she easily launched into a polished five-minute rundown of her new lab’s exciting findings. Gone was the nervousness of that first lecture before her former professors, just 11 months earlier. “I feel confident in the role of being a professor now,” she says. “I feel like I own it now. This is my job.”