"Long before it's in the papers"
April 28, 2009

RETURN TO THE WORLD SCIENCE HOME PAGE

Robotic helicopters teach themselves stunts

Sept. 2, 2008
Courtesy Stanford University
and World Science staff

Com­put­er sci­en­tists have de­vel­oped an ar­ti­fi­cial in­tel­li­gence sys­tem that en­a­bles robotic mini-he­li­copters to teach them­selves to fly stunts by watch­ing oth­er he­li­copters. The re­sult: an au­ton­o­mous hel­i­cop­ter than can per­form a com­plete, com­plex air­show.

They’re “by far the most dif­fi­cult aer­o­bat­ic ma­neu­vers flown by any com­put­er con­trolled hel­i­cop­ter,” said An­drew Ng, pro­fes­sor at Stan­ford Uni­ver­s­ity in Cal­i­for­nia di­rect­ing the re­search by a group of grad­u­ate stu­dents.

A self-teach­ing ro­bo­tic heli­copter crea­ted by Stan­ford Uni­ver­sity re­search­ers. Vi­deo here.

Computer scientists have developed an artificial intelligence system that enables robotic helicopters to teach themselves to fly stunts by watching other helicopters. The result: an autonomous helicopter than can perform a complete, complex airshow. They’re “by far the most difficult aerobatic maneuvers flown by any computer controlled helicopter,” said Andrew Ng, professor at Stanford University in California directing the research by a group of graduate students. The show is a demonstration of “apprenticeship learning,” in which robots learn by observing an expert, rather than by having software engineers peck away at their keyboards in an attempt to write instructions from scratch. Stanford’s artificial intelligence system learned how to fly by “watching” the four-foot-long helicopters flown by expert radio control pilot Garett Oku. “Garett can pick up any helicopter, even ones he’s never seen, and go fly amazing aerobatics. So the question for us is always, why can’t computers do things like this?” Coates said. They can, it turns out. On a recent morning in an empty field at the edge of campus, Abbeel and Coates sent up one of their helicopters to demonstrate autonomous flight. The bright red aircraft is an off-the-shelf radio control helicopter, with instrumentation added by the researchers. For five minutes, the chopper, on its own, ran through a dizzying series of stunts beyond the capabilities of a full-scale piloted helicopter and other autonomous remote control helicopters. The artificial-intelligence helicopter performed a smorgasbord of maneuvers: traveling flips, rolls, loops with pirouettes, stall-turns with pirouettes, a knife-edge, an Immelmann, a slapper, an inverted tail slide and a hurricane, described as a “fast backward funnel.” The pièce de résistance may have been the “tic toc,” in which the helicopter, while pointed straight up, hovers with a side-to-side motion as if it were the pendulum of an upside down clock. “I think the range of maneuvers they can do is by far the largest” in the autonomous helicopter field, said Eric Feron, a Georgia Tech aeronautics and astronautics professor who worked on autonomous helicopters while at the Massachusetts Institute of Technology. “But what’s more impressive is the technology that underlies this work. In a way, the machine teaches itself how to do this by watching an expert pilot fly. This is amazing.” Writing software for robotic helicopters is daunting, in part because the craft, unlike an airplane, is inherently unstable. “the helicopter doesn’t want to fly. It always wants to just tip over and crash,” said Oku, the pilot. To scientists, a flying helicopter is an “unstable system” that comes unglued without constant input. Abbeel compares flying a helicopter to balancing a long pole in the palm of your hand: “If you don’t provide feedback, it will crash.” Early in their research, graduate students Pieter Abbeel and Adam Coates tried to write computer code that would specify the commands for the desired trajectory of a helicopter flying a specific maneuver. While this hand-coded approach succeeded with novice-level flips and rolls, it flopped with the complex tic-toc.” It might seem an autonomous helicopter could fly stunts by simply replaying the exact finger movements of an expert pilot using the joy sticks on the helicopter’s remote controller. That approach, however, is doomed because of uncontrollable variables such as wind gusts. When the Stanford researchers decided their autonomous helicopter should be capable of flying airshow stunts, they realized that even defining their goal was difficult. How do you define “flying well?” The answer, it turned out, was: whatever an expert radio control pilot does at an airshow. So the researchers had Oku and other pilots fly entire airshow routines while every movement of the helicopter was recorded. As Oku repeated a maneuver several times, the trajectory of the helicopter inevitably varied slightly with each flight. But the learning formulas created by Ng’s team were able to discern the ideal trajectory the pilot was seeking. There is interest in using autonomous helicopters to search for land mines in war-torn areas or to map out the hot spots of California wildfires in real time, allowing firefighters to quickly move toward or away from them. Firefighters now must often act on information that is several hours old, Abbeel said. “In order for us to trust helicopters in these sort of mission-critical applications, it’s important that we have very robust, very reliable helicopter controllers that can fly maybe as well as the best human pilots in the world can,” Ng said.