The Revolution Of AI-Enabled Autonomous Piloting With Shield AI’s Brandon Tseng

From its small AI-driven Nova drones that can help ground forces clear urban environments in close-quarters combat to its MQ-35 V-BAT drone that can be launched and recovered vertically and can scan vast areas of territory in a swarm – all leverage the company’s Hivemind AI pilot for command and control.

The very same ‘AI pilot’ has also proved its mettle in real-world fighter aircraft test flights linked to advanced U.S. Air Force programs, and now the same system is set to control a host of Kratos drones, all done with a keen eye on the emerging world of the Collaborative Combat Aircraft (CCA) and intelligent affordable mass with V-BAT Teams and their recent acquisition of Sentient Vision.

The War Zone talked in depth with Shield AI’s co-founder, president, and chief growth officer Brandon Tseng. Previously, Brandon served in the U.S. Navy for seven years, including as a SEAL. He founded Shield AI with the mission to protect service members and civilians with intelligent systems, driven by his first hand experiences in the military.

The War Zone: Can you give us a bit about your background and how you created Shield AI?

Brandon: My background is as a mechanical engineer. I graduated from the U.S. Naval Academy and my first job was as Surface Warfare Officer aboard USS Pearl Harbor [LSD-52]. I did that for about a year before I became a U.S. Navy SEAL, and I did that for about six years.

During my time in the Navy I deployed twice, once to Afghanistan and once to the Pacific Theater, both times as a Navy SEAL. I also deployed one time to the Persian Gulf as a Surface Warfare Officer. My experiences in the military, primarily serving in combat in Afghanistan, are what led me to found Shield AI.

I was trying to figure out what I wanted to do after the Navy. As an engineer, I was always interested in technology. My older brother is an entrepreneur and he has been very successful, and he is also our co-founder and CEO at Shield AI. My Dad is also a small business owner, so I got the entrepreneurial bug, I would say from both of them, and decided to do something with AI and autonomy.

My brother had already been working on this dishwashing robot, and to cut a long story short, he had already got me onto the AI bug. 

I remember growing up and seeing the internet evolve and I was thinking what an incredible journey that must have been for those entrepreneurs, the technologists of the world to build that. And so I was thinking about what was going to be the next really amazing technology and I decided it was going to be AI.

Consumer problems didn't really resonate with me and I kept coming back to national security problems. Besides, there wasn't really anybody focused or driving the solution set there. That's how I got started – the idea of bringing the best of AI and autonomy to the national security sector to power, command, and maneuver all of our assets with the mission of protecting service members and civilians and, you know, ultimately deterring conflict.

The War Zone: Let’s talk about your products, your artificial intelligence journey and Shield AI’s Hivemind system and how you developed this.

Brandon: So when we started the company in 2015, I asked myself what the military of 2035 might look like and what role AI and autonomy might play in that military? I don't want to call it an original thought – lots of people convened on this. My conclusion was that every single asset should be powered, commanded, maneuvered, by AI by 2035. That was my vision. 

That it would be a shame if we have people piloting and commanding military assets, because AI will do it better, it will do it faster and you won't stand a chance against adversaries that have AI powered systems that are AI commanded and maneuvered – so that’s fighter jets flown by AI pilots.

But where do you start? At the time it was just me, with no money and an idea. I started working with my brother and we decided we had to stair step our way there. To start we focused on a problem that I was very familiar with, which was clearing buildings of threats. That's when we decided to take our Hivemind AI pilot that we created and put it onto a quadcopter that we built. Hivemind is an homage to one of my favorite video games – StarCraft. It's a sci-fi real time strategy game. Hivemind is self-driving technology for aircraft. 

And why is that important? It enables aircraft to accomplish missions fully autonomously without GPS, without communications, without a remote pilot. What do I mean by fully autonomous mission execution? In the example of our NOVA quadcopter, it would go inside buildings and it would find threats, it would map out the building, it would clear the building all by itself without anybody piloting it. GPS didn't matter, communications didn't matter, it would accomplish the mission.

The War Zone: What can you tell us about Hivemind itself, how it works and its architecture?

Brandon: When you build autonomy, getting the architecture right is so, so important. It's like when you're building a skyscraper, you have to get the architecture right from the get go or you're gonna build a skyscraper that falls on itself. A lot of companies mess up because they don't appreciate this. One of the great things that we had from the get go was someone who had a lifetime worth of experience focusing on how to architect and build resilient intelligence systems.

We got the architecture right from the get go and we've just been building off of that and I think it's why you see Shield AI able to do amazing things like fly six different aircraft now. I can't name another company that has aircraft that operate without GPS or communications to win on the battlefield. The reason we were able to get that out as a product was because we had the right architecture from the start.

Beyond architecture are the design tools, the infrastructure of the pipelines that engineers work with every single day to build on top of the architecture that's been laid out for them. I can say that Shield AI is right now leading the autonomy architecture for some of the most strategic efforts inside the Department of Defense.

Our Chief Technology Officer was formerly a professor at Carnegie Mellon University. He taught at the Resilient Intelligence Systems Laboratory which he built into the largest faculty laboratory at the Robotics Institute at CMU. So he has taught thousands of students about autonomy and AI. The way he breaks all this down is really nice. He says every autonomous system is about perception, cognition, and action. How's the system perceiving? How is it thinking? How is it acting?

If you then break down those elements, for example look at the perception module – what's making that up? You have a series of different modules, you have a state estimation module, you have a mapping module, you have a sensing module or a scene understanding module. So that’s about how the system is actually perceiving everything that’s going on. It builds out a map, a reference map of itself on the world. You've probably seen self-driving cars and the images that they're building, well we do the exact same thing in terms of seeing and sensing. Whether that’s computer vision algorithms that recognize a person or a door, or a window, that's a surface there, missile system, that's a friendly pilot. Whatever it is, that would be the sensing aspect.

The cognitive aspect is how it actually thinks in its mission and how it thinks about problems as a single agent, as a multi-agent system. So when I talk about the mission set, we refer to this sometimes as global reasoning. What is the mission? Is my mission clearing a building? Is it dogfighting? Is it suppression of enemy air defenses? And this is no different than I would claim the missions of self-driving cars. It could be navigating on the freeway, learning to park, learning to use on ramps, off ramps, learning what you do when pedestrians are on the road or bicyclists are on the road.

All of those are just different mission subsets, which is no different than what we have in our cognition module as we think about autonomy. Then the action aspect is just controls and how you actually accomplish what your brain and what your sensors are telling you to do.

This is happening, whether it's a single agent system or a multi-agent system, as we think about V-BAT teams, whether it's teams of CCAs, you name it – perception, cognition, action, and those different subcomponent modules are embedded onto every aircraft on the edge system.

An "edge system" executes computation directly on the aircraft itself, or in its immediate operational environment, rather than relying on cloud-based resources or centralized servers at a tactical operations center. The compute is happening "on the edge." This approach is inherently more robust because it minimizes the reliance on a centralized system and critical to operating in highly contested environments with GPS and comms jamming.

The War Zone: Shield AI’s MQ-35 V-BAT Unmanned Aerial System (UAS) has received a lot of positive attention recently, and you are developing that capability dramatically. What can you tell us about the latest advances with V-BAT Teams with Hivemind AI enabling those drones to work together as a swarm?

Brandon: For V-BAT we've been focused on the mission of maritime domain awareness and suppression of enemy air defenses. The latter being about how you find radar systems and surface-to-air missile systems so that you can tie that back into long-range kill chains completely autonomously with teams of aircraft and without the need for GPS or communications.

Hivemind is a core central system with core architecture, core components on every single one of these aircraft. That's part of our strategy – to build an AI backbone that we can leverage across many different aircraft. So 90% of what we do is leveraged across aircraft to aircraft. So the learnings from quadcopters help build V-BATs that help build our AI pilot for fighter jets and it works up the chain too. The teaming work that we're doing for fighter jets informs and advances the work that we're doing on V-BAT and vice versa.

We are doing real life flights, testing, and validation. We've demonstrated in front of a couple of customers and right now we're in the product refinement period for delivery in quarter four [Q4] of this year. The first mission set will be maritime domain awareness. I think pretty much every four star [general] in Indo-Pacom is about how do you provide comprehensive coverage 24 hours a day, seven days a week, over a 30,000 square mile geography without the need for GPS or communications in order to find, fix, and put at risk targets of interest in that area. You're talking about large area coverage in a specific geography, in an electronic warfare environment, and tied back to long range kill chains. If our adversaries know that they are being targeted, they will not be able to maneuver on the ground. That's the mission set that we're working on. 

We're starting with four V-BATs, which are probably not going to be able to necessarily cover 30,000 square miles immediately. But the system is not actually the number of aircraft that are flown – it’s not inhibited by the algorithms on board in simulation. We are flying 16, 30, any number of aircraft at any given time. It's more of an operational constraint. If you talk to the Navy about aircraft carriers, how often do they need to put up 40 jets at any given time. They don't do it often. There's a big operational lift to do it and the number of missions that you actually need 100 aircraft up in the sky at any given time is relatively few.

So, for a go-to-market motion for us for this being the first time that swarming will ever be deployed with a customer as a real capability. I would claim human beings are the first swarm ever. The first autonomous form like swarming is actually a really old warfighting concept as old as warfare itself. The more equipment that you have the better you tend to do in battle. But it's not an autonomous robotic swarm. I would claim this is the first autonomous robotics swarm that we are employing, and that customers will be employing this year. 

The product is called V-BAT Resilient Teams. I can't comment on who the customer is going to be but it is going to deploy this year for that maritime domain awareness mission primarily to find, fix and put targets of interest at risk and be tied back to long range kill chains. We will deploy a team of four MQ-35 V-BATs that will work together as a swarm.

The War Zone: Where do you see the real value in using AI pilots, and what is the range of applications you foresee.

Brandon: I tell people that self-driving technology for aircraft enables mission execution, with no remote pilot, no communications, and no GPS. It enables the concept of teaming or swarming where these aircraft can execute the commander's intent. They can execute a mission, working together dynamically, reading and reacting to each other, to the battlefield, to the adversarial threats, and to civilians on the ground.

The other value proposition that I claim is that you don't have to train human pilots to fly aircraft. And there is a shortage of pilots. Commander, Ninth Air Force [Air Forces Central] U.S. Central Command's leader Gen. Grynkewich has said he doesn’t care about 1,000 drones or 10,000 drones. He says we have to field hundreds of thousands, if not millions, of drones. We're not going to produce pilots for a million drones! So you have to use AI and autonomy to be able to fly those aircraft.

The other value proposition I think of is the system – the fleet of aircraft always gets better. You always have the best AI pilot on an aircraft at any given time. We win 99.9% of engagements with our fighter jet AI pilot, and that's the worst that it will ever be, which is superhuman. So when you talk about fleet learning, that will be on every single aircraft, you will always have the best quadcopter pilot, you'll always have the best V-BAT pilot, you'll always have the best CCA pilot, you name it. It'll be dominant. You don't want the second best AI pilot or the third best, because it truly matters that you're winning these engagements at incredibly high rates.

The War Zone: You acquired AlphaDogfight from Heron Systems and that has been melded into Hivemind, and that’s led to some very high-profile work with regard to programs like the X-62 Variable Stability In-flight Simulator Test Aircraft (VISTA) F-16.

Brandon: Our original vision was to get AI pilots on everything, including fighter jets. In 2019, we're talking about a specific set of technologies that were tailored for that specific application – and either building this capability or buying it. We were cognizant of the AlphaDogfight Trials and when we started talking to the Heron Systems team including Brett Darcy, who is now our executive vice president of product management, it was clear that there were great synergies with very little overlap in terms of capabilities that had been built. We decided to join forces to work on it.

We bought them in 2021 and AlphaDogfight started to integrate their capabilities into our design tools, our infrastructure, our pipelines, and in 2022 we undertook the first ever autonomous dogfights of F-16s in the real world. In 2023, we did some other really amazing stuff which I can't talk about yet, except to say that the work is what got us a nomination as a finalist for this year’s Collier Trophy, which is awarded annually for the greatest achievement in aeronautics or astronautics in America.

The War Zone: What can you tell us about VISTA in terms of how Hivemind worked with the jet? 

Brandon: We were flying Basic Fighter Maneuver missions completely autonomously. The pilot would just push a button, take his hands off the controls and the aircraft just did everything. That was achieved solely by Hivemind – our AI system – which was the only thing activated at that time. Some other providers flew as well in terms of other versions of AI, but the reason we were chosen first and the reason we were downselected as the only company to continue with the project was not only because we had the best AI pilot and won AlphaDogfight, but also because we are simply the most advanced and furthest along in orders of magnitude.

Shield AI was the only company downselected for the follow on efforts for that program and we have accumulated more flight hours flying fighter jets autonomously in live-fly real training scenarios – including air-to-air missions – than any other company. 

Our focus at this stage in the game is less about research efforts and more about product refinement and getting the product out there – an AI pilot that's ready to rock that we are going to launch as a product in Q4 of 2025 so the government can purchase it for jet aircraft.

The War Zone: Could you elaborate on any of your involvement in the Defense Advanced Research Projects Agency’s (DARPA) Air Combat Evolution (ACE) program, which also focused on exploring how AI and machine learning could help automate various aspects of air-to-air combat?

Brandon: I’d just say that it was a way to burn down risk for sixth-generation uncrewed fighter jets.

The War Zone: Bearing the VISTA work in mind, and bringing the Hivemind journey up to date, you’ve been selected by NAVAIR to integrate your Hivemind AI pilot onto the Kratos BQM-177A drone for AI-powered crewed-uncrewed teaming (CU-T). Can you maybe explain a little around how you go about integrating onto a platform like that.

Brandon: There's definitely a level of integration that happens. It's not like just downloading an app on your iPhone, but it's also not like complex surgery. It's somewhere in between those two.

You have to adapt certain modules, that perception, cognition, action framework to work with the platform. You need to understand what sensor suite is being run. Hivemind interacts with different mission systems. So it interacts with the autopilot, it's interacting with the mission systems, it’s doing everything a human pilot would do if they were flying the aircraft.

The War Zone: You are also putting Hivemind onto the MQM-178 Firejet as the sixth type of aircraft that Hivemind has flown, with XQ-58 Valkyrie flights coming this year too, followed by the BQM-177 as the eighth platform for Hivemind. 

Brandon: Hivemind has now successfully flown on and controlled the Kratos MQM-178 Firejet and we are tracking to fly the Valkyrie this year too. The Firejet project was ready to fly after just 120 days of effort – that was lightning fast. It took three years of effort to fly on the F-16! We're just getting faster and faster in terms of integrating the autonomous software AI pilot capabilities from aircraft to aircraft. This is truly portable autonomy software, which the military has been talking about for at least three or four years.

We are doing a lot with Kratos, but I am agnostic to the OEM [Original Equipment Manufacturer]. I want Hivemind to fly on every single CCA. I want to be flying with Boeing, Lockheed Martin, Northrop Grumman, General Atomics, you name it. I just want the best AI pilot flying the best jets that are out there.

The War Zone: In terms of when you have weapons involved, is there a ground station or is someone actually monitoring all this stuff? Are humans in the loop? How does this work?

Brandon: Absolutely, 100% we have a ground control station. A great example is when we were demonstrating V-BAT Teams to customers. We had the engineer who is in charge of the flight operation, and at the same time he’s briefing about 20 government customers, just fielding question after question. Everything's going on and finally one of the customers says, “wait a second, are you the one who's also flying these aircraft right now?” He says, ”yeah” – but he isn’t really flying them. They are doing their thing while he’s briefing, the aircraft are seeing different things and making maneuvers on their own. It was a huge moment of realization for our defense customers. This is real and it's here today. One person can command a hundred, a thousand, maybe ten thousand assets. It's mind boggling.

The War Zone: What reactions to AI are you seeing from operators that have a legacy mission, meaning that they've done these things manually in the past. I'm talking about a willingness to embrace autonomy replacing traditional roles.

Brandon: The willingness is extremely high. I tell people being a Navy SEAL is a really amazing job until you're asked to go fight inside a tunnel system where casualty rates are typically high. That job is a lot of fun, but there are few things that are more terrifying than close quarters combat, which has killed more service members than any mission set in the past 20 years!

It's really cool to be a fighter pilot until you're asked to go up against an integrated air defense system where you know the number of surface-to-air missiles, radar systems, and you just don't stand a chance going up against them. I see a willingness from our customers to figure out how to solve the problem in a better way.

This relates to deployments of V-BATs or AI-piloted CCAs – you're going to have those things out in front because it reduces the risk to mission, the risk to loss of human life, astronomically. I think you're seeing a high willingness in the military and the operators to move forward under this paradigm – robots first would be the mantra.

The War Zone: The willingness to put people into harm’s way has also changed massively. That must also play into this.

Brandon: It's a good call out because, for example in the U.S. there has to be the will of the population to go to conflict. The willingness of the population to see loss of human life on the battlefield is not high, nor should it be. We want our forces to have the technological offset capabilities because it matters when you're fighting a conflict.

The War Zone: So, where do you think this is all going to lead? Do you see other opportunities that aren’t necessarily linked to the military?

Brandon: On the defense aviation side, it will lead to millions of millions of autonomous drones and launched effects. That's really powerful because it will deter conflict and if it comes to conflict, it will help win. But I think it will be the next great strategic deterrence.

I regularly consider the commercial aviation aspects of what we are doing and I've had extensive conversations with Boeing, Airbus, and Embraer about commercial aviation starting in 2030. Of course, there’s some regulatory items associated – the Federal Aviation Administration is not racing to put AI pilots up in the sky and OEMs are not racing to put AI pilots onto their aircraft. 

But everybody believes AI pilots are coming to the commercial aviation industry. There's a shortage of pilots in the airlines, but I don't think you or I are getting on a plane without a pilot on board, however maybe by 2033-2035 you might have a passenger plane that’s going to have only one pilot on board, with an AI copilot. That’s totally possible.

The other aspect could be around what you see happening in urban air mobility – there is a lot of money flowing into that space. Urban air mobility autonomy is key to getting that business model to work. You simply can't have a pilot flying four people around and the operator generating lots of profit in a sustainable way. For those types of air taxis you need autonomy to make a largely profitable business at scale.

The War Zone: Any other big plans for Hivemind?

Brandon: We have just announced that we are acquiring Sentient Vision Systems in Australia. The company is a leader in AI-enabled real-time situational awareness. This builds on our joint development of the ViDAR [Visual Detection and Ranging]-enabled wide area motion imagery product Sentient Observer, which we plan to fly this year. So this merges Sentient’s ViDAR and Shield’s Hivemind AI pilot, creating the world’s most advanced AI-piloted ISR sensor packages. 

The combination of AI pilots, Sentient Observer, and teams of affordable drones like the V-BAT will provide the same land and maritime domain awareness that Group 5 drones and crewed aircraft like the P-8 Poseidon provide at a fraction of the price. Sentient Vision is the final piece of that puzzle. 

ViDAR is Sentient’s AI system, which uses an electro-optic or infrared sensor to detect and classify targets in the imagery stream that would be invisible to a human operator or to a conventional radar. All of a sudden you're getting a huge leap in ability to observe the battlefield. If you consider the MQ-4 Triton, it's an amazing aircraft and it carries an incredible sensor package that enables combatant commanders to see massive areas of interest. But it’s massively expensive.

Now, in Sentient Observer, you have an all-seeing eye for a fraction of the price in terms of what these incredibly exquisite drones can do. If you go back to our original point, you need the ability to find, fix, and track targets of interest over thousands of square miles. For example, traditional drones with an electro-optical/infrared sensor can probably scan about 300 square miles every hour. Doing that with Sentient Observer, you can do 2,000 square miles every hour.

We have to empower our customers, we have to empower our industry to be able to accomplish these mission sets. So we work closely with others, and I think you'll see Sentient Observer is really just the start.

Contact the editor: tyler@twz.com