"Long before it's in the papers"
January 28, 2015

RETURN TO THE WORLD SCIENCE HOME PAGE


Robotic helicopters teach themselves stunts

Sept. 2, 2008
Courtesy Stanford University
and World Science staff

Com­put­er sci­en­tists have de­vel­oped an ar­ti­fi­cial in­tel­li­gence sys­tem that en­a­bles robotic mini-he­li­copters to teach them­selves to fly stunts by watch­ing oth­er he­li­copters. The re­sult: an au­ton­o­mous hel­i­cop­ter than can per­form a com­plete, com­plex air­show.

They’re “by far the most dif­fi­cult aer­o­bat­ic ma­neu­vers flown by any com­put­er con­trolled hel­i­cop­ter,” said An­drew Ng, pro­fes­sor at Stan­ford Uni­ver­s­ity in Cal­i­for­nia di­rect­ing the re­search by a group of grad­u­ate stu­dents.

A self-teach­ing ro­bo­tic heli­copter crea­ted by Stan­ford Uni­ver­sity re­search­ers. Vi­deo here.


The show is a demon­stra­t­ion of “ap­pren­tice­ship learn­ing,” in which robots learn by ob­serv­ing an ex­pert, rath­er than by hav­ing soft­ware en­gi­neers peck away at their key­boards in an at­tempt to write in­struc­tions from scratch. 

Stan­ford’s ar­ti­fi­cial in­tel­li­gence sys­tem learn­ed how to fly by “watch­ing” the four-foot-long he­li­copters flown by ex­pert ra­dio con­trol pi­lot Garett Oku. 

“Garett can pick up any hel­i­cop­ter, even ones he’s nev­er seen, and go fly amaz­ing aer­o­bat­ics. So the ques­tion for us is al­ways, why can’t com­put­ers do things like this?” said Stan­ford grad­uate stud­ent Ad­am Coates. 

They can, it turns out. One morn­ing in a field at the edge of cam­pus, Abbeel and Coates sent up one of their he­li­copters to demon­strate au­ton­o­mous flight. The bright red air­craft is an off-the-shelf ra­dio con­trol hel­i­cop­ter, with in­stru­menta­t­ion added by the re­search­ers. 

For five min­utes, the chop­per, on its own, ran through a diz­zy­ing se­ries of stunts be­yond the ca­pa­bil­i­ties of a full-scale pi­loted hel­i­cop­ter and oth­er au­ton­o­mous re­mote con­trol he­li­copters, re­search­ers said. The robot per­formed a smor­gas­bord of ma­neu­vers: trav­el­ing flips, rolls, loops with pirou­ettes, stall-turns with pirou­ettes, a knife-edge, an Im­mel­mann, a slap­per, an in­verted tail slide and a hur­ri­cane, de­scribed as a “fast back­ward fun­nel.” 

The pièce de ré­sis­tance may have been the “tic toc,” in which the hel­i­cop­ter, while point­ed straight up, hov­ers with a side-to-side mo­tion as if it were the pen­du­lum of an up­side down clock. 

“I think the range of ma­neu­vers they can do is by far the largest” in the au­ton­o­mous hel­i­cop­ter field, said Er­ic Feron, a Geor­gia Tech aer­o­naut­ics and as­tro­nau­tics pro­fes­sor who worked on au­ton­o­mous he­li­copters while at the Mas­sa­chu­setts In­sti­tute of Tech­nol­o­gy. “But what’s more im­pres­sive is the tech­nol­o­gy that un­der­lies this work. In a way, the ma­chine teaches it­self how to do this by watch­ing an ex­pert pi­lot fly. This is amaz­ing.” 

Writ­ing soft­ware for robotic he­li­copters is daunt­ing, in part be­cause the craft, un­like an air­plane, is in­her­ently un­sta­ble. “the hel­i­cop­ter does­n’t want to fly. It al­ways wants to just tip over and crash,” said Oku, the pi­lot. To sci­en­tists, a fly­ing hel­i­cop­ter is an “unsta­ble sys­tem” that comes un­glued with­out con­stant in­put. Abbeel com­pares fly­ing a hel­i­cop­ter to bal­anc­ing a long pole in the palm of your hand: “If you don’t pro­vide feed­back, it will crash.” 

Early in their re­search, grad­u­ate stu­dents Coates and Pie­ter Abbeel and tried to write com­put­er code that would spec­i­fy the com­mands for the de­sired tra­jec­to­ry of a hel­i­cop­ter fly­ing a spe­cif­ic ma­neu­ver. While this hand-coded ap­proach suc­ceeded with novice-level flips and rolls, it flopped with the com­plex tic-toc.” 

It might seem an au­ton­o­mous hel­i­cop­ter could fly stunts by simply re­play­ing the ex­act fin­ger move­ments of an ex­pert pi­lot us­ing the joy sticks on the hel­i­cop­ter’s re­mote con­troller. That ap­proach, how­ev­er, is doomed be­cause of un­con­trolla­ble vari­a­bles such as wind gusts.

When the Stan­ford re­search­ers de­cid­ed their au­ton­o­mous hel­i­cop­ter should be ca­pa­ble of fly­ing air­show stunts, they real­ized that even de­fin­ing their goal was dif­fi­cult. How do you de­fine “fly­ing well?” The an­swer, it turned out, was: what­ev­er an ex­pert ra­dio con­trol pi­lot does at an air­show. 

So the re­search­ers had Oku and oth­er pi­lots fly en­tire air­show rou­tines while eve­ry move­ment of the hel­i­cop­ter was recorded. As Oku re­peat­ed a ma­neu­ver sev­er­al times, the tra­jec­to­ry of the hel­i­cop­ter in­evitably var­ied slightly with each flight. But the learn­ing for­mu­las cre­at­ed by Ng’s team were able to dis­cern the ide­al tra­jec­to­ry the pi­lot was seek­ing.

There is in­ter­est in us­ing au­ton­o­mous he­li­copters to search for land mines in war-torn ar­eas or to map out the hot spots of Cal­i­for­nia wild­fires in real time, al­low­ing fire­fight­ers to quickly move to­ward or away from them. Fire­fight­ers now must of­ten act on in­forma­t­ion that is sev­er­al hours old, Abbeel said. “In or­der for us to trust he­li­copters in these sort of mission-critical ap­plica­t­ions, it’s im­por­tant that we have very ro­bust, very relia­ble hel­i­cop­ter con­trollers that can fly may­be as well as the best hu­man pi­lots in the world can,” Ng said.


* * *

Send us a comment on this story, or send it to a friend









 

Sign up for
e-newsletter
   
 
subscribe
 
cancel

On Home Page         

LATEST

  • St­ar found to have lit­tle plan­ets over twice as old as our own

  • “Kind­ness curricu­lum” may bo­ost suc­cess in pre­schoolers

EXCLUSIVES

  • Smart­er mice with a “hum­anized” gene?

  • Was black­mail essen­tial for marr­iage to evolve?

  • Plu­to has even cold­er “twin” of sim­ilar size, studies find

  • Could simple an­ger have taught people to coop­erate?

MORE NEWS

  • F­rog said to de­scribe its home through song

  • Even r­ats will lend a help­ing paw: study

  • D­rug may undo aging-assoc­iated brain changes in ani­mals

Computer scientists have developed an artificial intelligence system that enables robotic helicopters to teach themselves to fly stunts by watching other helicopters. The result: an autonomous helicopter than can perform a complete, complex airshow. They’re “by far the most difficult aerobatic maneuvers flown by any computer controlled helicopter,” said Andrew Ng, professor at Stanford University in California directing the research by a group of graduate students. The show is a demonstration of “apprenticeship learning,” in which robots learn by observing an expert, rather than by having software engineers peck away at their keyboards in an attempt to write instructions from scratch. Stanford’s artificial intelligence system learned how to fly by “watching” the four-foot-long helicopters flown by expert radio control pilot Garett Oku. “Garett can pick up any helicopter, even ones he’s never seen, and go fly amazing aerobatics. So the question for us is always, why can’t computers do things like this?” Coates said. They can, it turns out. On a recent morning in an empty field at the edge of campus, Abbeel and Coates sent up one of their helicopters to demonstrate autonomous flight. The bright red aircraft is an off-the-shelf radio control helicopter, with instrumentation added by the researchers. For five minutes, the chopper, on its own, ran through a dizzying series of stunts beyond the capabilities of a full-scale piloted helicopter and other autonomous remote control helicopters. The artificial-intelligence helicopter performed a smorgasbord of maneuvers: traveling flips, rolls, loops with pirouettes, stall-turns with pirouettes, a knife-edge, an Immelmann, a slapper, an inverted tail slide and a hurricane, described as a “fast backward funnel.” The pièce de résistance may have been the “tic toc,” in which the helicopter, while pointed straight up, hovers with a side-to-side motion as if it were the pendulum of an upside down clock. “I think the range of maneuvers they can do is by far the largest” in the autonomous helicopter field, said Eric Feron, a Georgia Tech aeronautics and astronautics professor who worked on autonomous helicopters while at the Massachusetts Institute of Technology. “But what’s more impressive is the technology that underlies this work. In a way, the machine teaches itself how to do this by watching an expert pilot fly. This is amazing.” Writing software for robotic helicopters is daunting, in part because the craft, unlike an airplane, is inherently unstable. “the helicopter doesn’t want to fly. It always wants to just tip over and crash,” said Oku, the pilot. To scientists, a flying helicopter is an “unstable system” that comes unglued without constant input. Abbeel compares flying a helicopter to balancing a long pole in the palm of your hand: “If you don’t provide feedback, it will crash.” Early in their research, graduate students Pieter Abbeel and Adam Coates tried to write computer code that would specify the commands for the desired trajectory of a helicopter flying a specific maneuver. While this hand-coded approach succeeded with novice-level flips and rolls, it flopped with the complex tic-toc.” It might seem an autonomous helicopter could fly stunts by simply replaying the exact finger movements of an expert pilot using the joy sticks on the helicopter’s remote controller. That approach, however, is doomed because of uncontrollable variables such as wind gusts. When the Stanford researchers decided their autonomous helicopter should be capable of flying airshow stunts, they realized that even defining their goal was difficult. How do you define “flying well?” The answer, it turned out, was: whatever an expert radio control pilot does at an airshow. So the researchers had Oku and other pilots fly entire airshow routines while every movement of the helicopter was recorded. As Oku repeated a maneuver several times, the trajectory of the helicopter inevitably varied slightly with each flight. But the learning formulas created by Ng’s team were able to discern the ideal trajectory the pilot was seeking. There is interest in using autonomous helicopters to search for land mines in war-torn areas or to map out the hot spots of California wildfires in real time, allowing firefighters to quickly move toward or away from them. Firefighters now must often act on information that is several hours old, Abbeel said. “In order for us to trust helicopters in these sort of mission-critical applications, it’s important that we have very robust, very reliable helicopter controllers that can fly maybe as well as the best human pilots in the world can,” Ng said.