Pilot In A Real Aircraft Just Fought An AI-Driven Virtual Enemy Jet For The First Time

Donning an augmented reality headset in the cockpit, a veteran F-22 pilot just had a dogfight with a projection of a Chinese J-20 fighter.

byThomas Newdick|
U.S. Homeland photo


Two U.S. companies have recently completed what they say is the world’s first dogfight between a real aircraft and an artificial intelligence-driven virtual fighter jet. The experiment, run by Red 6 and EpiSci, is the first step toward similar technology being provided to U.S. military fighter pilots, which would allow them to battle virtual adversaries as part of augmented reality training. You can read all about this potentially highly disruptive technology in this recent War Zone exclusive feature

The concept could help dramatically reduce the costs of air combat training compared to the adversary aircraft that currently have to physically fly against fighter pilots. It could also help solve a rash of other tactical challenges when it comes to replicating realistic foreign threats for fleet aviators. 

The live-flight augmented reality dogfight involved a Berkut 540 experimental plane and a simulated, reactive adversary aircraft in the form of a computer-generated projection inside the Berkut pilot's augmented reality helmet-mounted display. The adversary was a representation of the Chinese J-20 stealth fighter, created in augmented reality by EpiSci’s Tactical AI technology. The unusual aerial contest took place out of Camarillo Airport, California.

“With this first-ever within-visual-range dogfight against an AI bandit, EpiSci’s Tactical AI demonstrated the ability to work on a real aircraft, with flight-ready hardware and sensors,” said Chris Gentile, EpiSci’s Vice President for Tactical Autonomous Systems. “While fielding autonomous systems that control fighters may be in the future, this system is ready now to bring next-generation capability to our training programs, providing immediate benefit to the USAF’s ability to develop and maintain world-class fighter pilots. By introducing them to this technology now, they’ll be even more prepared to use a range of unmanned tools in the future.”

The Berkut 540 used in the latest demonstration., Mike Killian

EpiSci drew upon its previous work in the U.S. Defense Advanced Research Projects Agency’s (DARPA) Alpha Dogfight program to create its Tactical AI technology in a hybrid AI system. In this way, the kind of AI-driven simulation that was previously found only in traditional ground-based simulators can be introduced to the cockpit — in this case, presenting the pilot in the real aircraft with a simulated adversary flying a J-20 fighter.

The demonstration also used the Airborne Tactical Augmented Reality System (ATARS) developed by Red 6, which includes the display and control systems needed to inject augmented reality into the real world of the cockpit, and then for these virtual entities to interact with the surroundings as if they were a part of the real world.

The Founder and CEO of Red 6, Daniel Robinson., Mike Killian

“Red 6’s ATARS system enables real pilots to fly real airplanes connected together in an augmented world,” explained Dan Robinson, CEO and founder of Red 6. “With the additional integration of Tactical AI into our platform, we are now able to interact and respond to any threat aircraft. This opens spectacular possibilities for training,” Robinson added. Robinson was at the controls of the real aircraft in the mock dogfight.

ATARS is billed as the first wide field-of-view, full-color augmented reality solution for “dynamic outdoor environments.” It doesn’t get much more dynamic than a fighter jet and that’s where Red 6 aims to bring this technology. Ultimately, it aims to provide pilots flying real-world aircraft to perceive synthetic threats in real-time and in high-speed environments. This, of course, requires augmented reality to be blended seamlessly with the environments both inside and outside the cockpit. 

Daniel Robinson wearing the AR headset. , Mike Killian

There is also the potential to employ Tactical AI in scenarios beyond replicating enemies in air combat exercises. More generally, it could serve to present various other presentations for military pilots at different levels of instruction, for example, simulating flying in a larger formation, including alongside unmanned loyal wingmen

“Pilots of the future will need to be comfortable teaming with AI — and training is the ideal place to introduce this technology,” said EpiSci's Gentile. Now, after this pioneering aerial battle between a real aircraft and an AI-driven virtual foe, this ambition is one step closer to becoming a reality.

Contact the author: thomas@thedrive.com