Why researchers are teaching this AI to fly a plane

In February, for the first time, a Black Hawk helicopter flew without a human on board. The autonomous military helicopter project involved both DARPA and Sikorsky, which manufactures the UH-60 helicopters.

Meanwhile, in some places, companies like Zipline and Wing are delivering goods by drone. Other companies are working on electric air taxis to transport people or cargo, and of course normal air traffic – commercial flights departing from major airports, general aviation aircraft leaving others – is also flying. Factor in helicopters, hot air balloons, etc., and a lot can happen up there.

With all the commotion in the skies, Carnegie Mellon researchers are working on an artificial intelligence pilot system that can perform tasks such as predicting what another plane might be doing, or keeping an eye out for nearby planes using cameras on an airplane. The idea is that an AI like this could help fly drones, assist a human pilot, or even one day fly an airplane on its own.

Currently, in a flight simulator, the AI ​​is able to understand what another aircraft does or could do, then determine how to land the plane safely accordingly. Think of how a driver behind the wheel of a car notices another vehicle approaching an intersection and begins to preemptively plan what to do if the other car should execute a stop sign, for example.

In this case, the AI ​​is looking for another plane, not a car, of course. “It’s basically looking at their behavior for 10 seconds,” says Jay Patrikar, a doctoral student at Carnegie Mellon University’s Robotics Institute. “He’s trying to judge, ‘They’re there. What are they potentially likely to do? »

In that sense, it’s like an AI that can play chess, Patrikar says, thinking in advance about what his move would be if his opponent were to take a certain action.

[Related: The Air Force plans to test an AI copilot on its cargo planes]

Artificial intelligence systems need data to learn. In this case, the team is collecting data from two real-world airports, both located in Pennsylvania. One has an air traffic control tower, and the other does not. Patrikar says that at these airports, the data they collect includes visual information from cameras located on a hangar or near the taxiway, voice communications from radios, weather data, and more. “We’re recording the entirety of it,” he says. The idea is that the AI ​​can learn cause and effect by paying attention to all this information.

“He knows the causality of things,” he adds. This means that the AI ​​could learn, for example, that “it’s because of the weather that they [a pilot] decided to do this particular thing. The training the AI ​​received in these scenarios helped it learn how to navigate a simulated landing, Patrikar says.

Additionally, an AI bringing a plane in for a landing at a small, uncontrolled airport must both follow FAA rules as well as other standards when interacting with other planes, Patrikar points out. “One of the ways humans trust each other is our common understanding of rules — our social norms,” he says. People on a busy sidewalk can decide how to pass each other by moving to the right, for example, and rules like this apply in aviation that the AI ​​pilot must follow.

[Related: This company is retrofitting airplanes to fly on missions with no pilots]

A related job in the real world, not in simulation, is installing cameras on aircraft like a Cessna 172 or a hexacopter drone. These cameras and AI are able to spot other aircraft in the area, identify them and determine how far away they are with an accuracy rate of over 90% at a distance of 700 meters (approximately 2,300 feet ). This kind of technology could help a human pilot in a small plane visually spot other traffic in the area. “I would like to have this system in my plane,” says Patrikar, who has a private pilot’s license. After all, artificial intelligence doesn’t blink.

Certainly, Carnegie Mellon researchers are not the only ones exploring the new frontier of artificial intelligence capable of piloting or helping to pilot planes. Drone company Zipline has been working on a way to use microphones on its drones to listen for other aircraft in the area and then take evasive action to avoid any potential collisions. And notably, a company called Merlin Labs has also developed a digital pilot that could replace a human co-pilot. As an example, he is working with the Air Force to equip C-130J cargo planes with their system, so that instead of a human crew of two pilots, the plane could be piloted by a single human paired with an artificial co-pilot.