Next Big Challenge For AI Dogfighting Is Keeping Track Of Enemies Up Close

Readers can get fully up to speed on the landmark live mock dogfight test in The War Zone's initial reporting here, but to recap quickly here, the X-62A test jet, a heavily modified two-seat F-16D Viper also known as the Variable-stability In-flight Simulator Test Aircraft (VISTA), squared off against a crewed F-16 for the first time last September. The X-62A flew the dogfight in a fully autonomous mode using an AI and machine learning-driven software package, though a pilot was in the cockpit the entire time as a safety measure. The flight test was conducted as part of a program called Air Combat Evolution (ACE), which the Defense Advanced Research Projects Agency (DARPA) is leading. The U.S. Air Force, as well as several private contractors and academic institutions, are also participating in ACE.

"We built up in safety [with]… the maneuvers, first defensive, then offensive then high-aspect nose-to-nose engagements where we got as close as 2,000 feet at 1,200 miles per hour," Air Force Lt. Col. Maryann Karlen, the Deputy Commandant of the U.S. Air Force Test Pilot School (USAF TPS) said in a video presentation, seen below, which was released last week.

Despite over a century of military aviation advancements, dogfighting remains a visual affair where a human pilot's on-the-spot judgment, intuition, and stereo vision are critical. An aircraft's sensor suite, including radar, electro-optical and infrared cameras, and electronic warfare and support measures systems, can provide a wealth of data about enemy contacts. However, their utility steadily diminishes, if it does not largely collapse, as the distance to the target closes to very close-in maneuvering range. A radar in the nose of an aircraft, for instance, can only 'see' what's in a cone-shaped area in front of it. Even existing 360-degree camera systems have two-dimensional limitations and can be hampered by environmental conditions. Datalinked information from external sources can be extremely valuable for building situational awareness or even guiding weapons, but it also has limited in fidelity. External tracks of the enemy and friendly fighters can merge together at very close range.

"Lose sight, lose the fight" is a therefore understandably common adage, but especially for an AI-driven jet that needs high-quality telemetry close in to know where it is in relation to the adversary aircraft. A true adversary is going to very uncooperative in providing that kind of information.

With all this in mind, as it turns out, perhaps unsurprisingly, there are major caveats to last year's still very significant autonomous dogfighting milestone.

"From the early days in the ACE program, we always knew that ... perception in the program was going to be a challenge area," Air Force Lt. Col. Ryan Hefron, the ACE program manager, told The War Zone and other outlets during a press roundtable last Friday. "What the original plan always was in ACE was to use a data link between the two aircraft to pass state information."

Hefron was responding to a specific question from Aviation Week's Pentagon editor Brian Everstine about how the so-called "autonomy agents" loaded into the X-62A's mission systems maintained general situational awareness during the dogfight. Everstine also pointed out that the AI-driven algorithms had complete situational awareness during DARPA's AlphaDogfight Trials, which wrapped up in 2020 and fed directly into ACE. The AlphaDogfight Trials took place in entirely simulated environments.

"We're a research program. We understand that there are limitations in some of the research at times," Hefron continued. "So, we added some flight pods to the F-16, the human-flown F-16, as well as the VISTA aircraft. And we used that to... create a data link between the aircraft so that they were passing state information that would be ingested on the VISTA platform and then made available to the agents as part of what we call their observation space."

"So, that's them [the autonomy agents] having observations about the environment around them," the head of ACE added. "What I would also like to say is that, you know, while that may sound like, 'Oh, hey, you're just, you know, piping truth data to the to the agent,' there is still noise in that data. ... It's not as clean per se as what you might see in a simulation environment. So it's kind of the next step along a pathway to being able to ... sense."

"We really have a lot of basic science and engineering questions to answer. And so the incremental build-up is really the hallmark of tests, right?" Air Force Col. James Valpiani, the commandant of the USAF TPS, who was also at the roundtable last Friday and has been involved in ACE, said. "We want to isolate variables to understand how the aircraft performs with an understanding of all the other contexts, and then slowly open up the envelope of things that we're testing. So, to have an aircraft that receives perfect state data, … there are [still] many things that we learned, many deviations, many differences from the simulation environments – the sim-to-real gap."

This all makes immediate sense from a safety perspective. With so many unknowns in this first-of-its-kind dogfight, making sure the X-62A would be capable of performing various tasks autonomously as expected when given as much situational data as possible is a logical and necessary first step. DARPA and the Air Force have repeatedly stressed that the main point of ACE is really about building trust in autonomy and doing so in safe and ethical incremental steps. For the reasons outlined earlier on, dogfighting was chosen as what officials have described as the core "challenge problem" because of how demanding and dangerous it is already with crewed aircraft. Developing the required technologies and capabilities for an autonomous aircraft to engage in such maneuvers thereore has much broader implications.

"If you think about self-driving vehicles, and the early days of developments there, ... you may have had a self-driving vehicle that could perform well, on a closed-circuit test track that it maybe it was trained on," Lt. Col. Hefron used as a tangential example of the processes involved. "But as soon as you take it out of that environment, and you put it into... downtown San Francisco that... autonomous vehicle, it'll fail in that condition."

"It's dangerous, it's challenging, but it's also well characterized – it's all of those things, right? I mean, it's a trifecta in the sense that it's a problem where human trust is acute. We need to trust these platforms when we're putting them within 500 feet and 1,200 miles an hour of closure apart," Col. Valpiani further explained about the choice to use dogfighting as the central focus of ACE. "But it's not a problem that we don't understand. We understand dogfighting very, very well. We understand the energy management component. When it's F-16 versus F-16, we understand the tactics. We understand what it looks like to win and also what it looks like to lose."

Valpiani said separately during the roundtable last week that he could not speak as to who won or lost last year's dogfight between the X-62A and the crewed F-16 for "national security reasons."

"We understand what human errors look like so that we can potentially see how that differs from what [an] AI agent's errors look like. And we have well-codified ethical and safety rules that humans have to comply with," Valpiani added. "When you start to place an AI agent in this space, you can see dramatic differences, as well as similarities. And … those are very important points of learning."

The video below shows the X-62A flying in formation with an F-16C and an F-22 Raptor stealth fighter during a test flight in March 2023.

There is also just a matter of practicality. The X-62A simply does not have any kind of organic sensor suite that would give it the uninterrupted 360-degree situational awareness that truly autonomous dogfighting would require. This is something that will have to be addressed when it comes to the development of future autonomous platforms. Arrays of small conformal radars, electro-optical or infrared cameras, and other sensors could be utilized to provide the necessary situational and spatial data, in effect working together to build telemetry for creating a solid digital three dimensional 'picture' of what is going on directly around the jet during a fast developing dogfight.

A distributed network of sensors, including on individual drones operating in a cooperative swarm, as well as on other offboard platforms, could also be used to help create a more complete situational picture.

The commercial aviation sector, as well as the military side of that industry, has made significant advances in automated 'sense and avoid' capabilities over the past couple of decades, including when it comes to uncrewed platforms. Some of these technologies could be ported over to help dogfight problem, especially paired with a far more dynamic 'thinking' AI agent backbone that benefits from deep machine learning. Even sensors and software models used for self-driving cars could be leveraged to help make better sense of what's happening around a combat drone engaged in this kind of combat.

Regardless, simply installing something like an array of simple cameras around an aircraft may not provide the required three-dimensional situational awareness to support robust autonomous aerial dogfighting capabilities. Two dimensional data does not give range and closure information, although some of this may be able to be emulated in software through machine learning. Still, three dimensional data would be of the greatest value for this application.

"There is a plan and there is a development pathway in the future to incorporate sensing, and the complexity that that [sic] adds to the picture," according to Col. Valpiani. However, "doing this in a safe and responsible way is to take it one step at a time to truly understand how each aspect works before moving on to the next problem."

"The mission environments that you're employing it [in] or deploying in is a critical challenge that we see as kind of that next challenge that we have to address in the air domain," Hefron also said. The head of ACE acknowledged that his program is not the only won working to overcome these challenges, as well, and talked specifically about the Air Force's separate Project VENOM (Viper Experimentation and Next-Gen Operations Mode). A total of six F-16s are being modified under Project VENOM to help support further autonomous flight research and development work. This effort will also allow for more experimentation with multiple autonomous platforms working collaboratively together, as you can read more about here.

"The VENOM aircraft are the platforms that will have the ability to sense the environment around them and do some of the things that we presently have some limitations on VISTA with," according to Hefron. "Not to say ... that we won't continue to work on [ACE using] VISTA, because we absolutely will. There are certain things that the [X-62A] aircraft is capable of performing that won't be, I'll say, well suited on the VENOM birds... They each have their own unique place, I'll say, as testbeds."

ACE and Project VENOM are among a larger array of programs and activities that are feeding into the Air Force's larger vision for future autonomous capabilities, particularly the service's Collaborative Combat Aircraft advanced drone program. The rest of the U.S. military is also becoming increasingly interested in new and improving autonomous capabilities, and not just limited to the air domain. There could be implications in all of this or the commercial aviation sector, as well.

Altogether, significant challenges clearly remain after the breakthrough dogfight last year, particularly when it comes to allowing an AI piloted fighter to dogfight successfully with a true enemy opponent. It will be very interesting to see what milestones ACE and these other autonomous research and development efforts reach next and solving this issue will surely be at the top of their to do lists.

Contact the author: joe@twz.com