New VR / AI Tech Allowing One Person to Control a “Swarm” of 130 Robots Includes “significant risk of error”

By B.N. Frank

Heads up, literally!!! Especially if you live in an urban environment!!! The Pentagon “expects future battles to take place in massive urban environments” and may use risky Artificial Intelligence (A.I.) technology including drones, uncrewed planes, and quadcopters to address them.

From Popular Science:

An inside look at how one person can control a swarm of 130 robots

Virtual reality and artificial intelligence helped with the daunting task.

By Kelsey D. Atherton | Published Jan 20, 2022 8:00 PM

Last November, at Fort Campbell, Tennessee, half a mile from the Kentucky border, a single human directed a swarm of 130 robots. The swarm, including uncrewed planes, quadcopters, and ground vehicles, scouted the mock buildings of the Cassidy Range Complex, creating and sharing information visible not just to the human operator but to other people on the same network. The exercise was part of DARPA’s OFFensive Swarm-Enabled Tactics (OFFSET) program.

If the experiment can be replicated outside the controlled settings of a test environment, it suggests that managing swarms in war could be as easy as point and click for operators in the field.

“The operator of our swarm really was interacting with things as a collective, not as individuals,” says Shane Clark, of Raytheon BBN, who was the company’s main lead for OFFSET. “We had done the work to establish the sort of baseline levels of autonomy to really support those many-to-one interactions in a natural way.”

Piloting even one drone can be so taxing that it’s not rare to see videos of first-time flights leading immediately to crashes. Getting to the point where a single human can control more than a hundred drones takes some skill—and a lot of artificial intelligence.

In total, the swarm operator directed 130 vehicles in the physical world, as well as 30 simulated drones operating in the virtual environment. These 30 virtual drones were integrated into the swarm’s planning and appeared as indistinguishable from the others in the program to the human operator, and to the rest of the swarm. As apparitions of pure code, tracked by the swarm AI, these virtual drones flew in formation with the physical drones, and maneuvered around as though they really existed in physical space.

Part of the swarm. US Army / Jerry Woller

How a human commanded the swarm

For the person directing the swarm, the entire array of robots appeared as a virtual reality strategy game, mapped onto the real world.

Wearing a Microsoft VR headset, the operator interacted “with this 3D VR interface that has what’s called a sand-table view,” says Clark. “It can be very similar to the sort of God’s Eye, real-time strategy game view, where they see all the platforms laid out over a 3D map of the environment. When something is detected, they’ll get an icon geolocated to the space. So if one of the platforms detected a hazard right outside of the building, they’d see a red flashing icon appear in space where that belongs.”

With the headset on, the operator was able to assign the swarm missions and see what the swarm had already scouted, but they was not giving direct orders to individual drones. The swarm AI, receiving orders and processing sensor information, was the intermediary between human control and the movement of robots in physical space.

At present, when a squad wants to use a drone to scout an area, it takes a dedicated pilot controlling it by tablet to see and direct the robot. The new approach has drawbacks and advantages: Because the operator’s VR headset blocks out the surrounding world, the human operator is vulnerable and oblivious to their immediate surroundings, except as they are displayed inside the headset. At the same time, the human operator is directing the actions of dozens of robots, shifting the one-to-one pilot-to-drone ratio drastically. Provided the human swarm operator can stay safe and the network can stay functional, the drones can capture useful information and relay it to nearby soldiers in the field. The new technology allows for a tremendous shift in what can be known about a battlefield in real time.

How the swarm operated

The physical drone models included custom drone planes made by Johns Hopkins Applied Physics Lab. In the swarm, their role was primarily passing over the entire training area and initial mapping, though the swarm also directed them to provide camera coverage as needed. On the ground were small wheeled robots made by Aion Robotics. In between were four different types of quadcopters, including 75 3DR Solo drones, light-show drone models from UVify, and two sizes of quadcopter by ModalAI.

All of the robots involved were either commercial models or custom models built from commercial, off-the-shelf parts. This was not a swarm built to military specifications—these were commercial, hobbyist, and custom-built machines working together through dedicated software. That keeps the cost of the swarm down to about $2,000 for each robot in it.

These six types of robots reported telemetry information to each other, letting the swarm know where each part of the whole was in space. That shared information let the swarm collaborate on flight paths in real time, moving pieces into place as needed while avoiding collision.

“We work hard to do some processing [of sensor readings] on platforms, so we don’t need to send a lot back on the network,” says Clark. “So we never send back a raw picture. We send back, ‘ah this picture is a door’ or ‘this picture is a hazard’ or ‘a source of intel.’ That processing happens on board. We just send back a notification.”

Automated processing makes swarm interactions possible. It can also, depending on the nature of what is processed, come with a significant risk of error.

Each of the robots was equipped with a little LTE modem, and to ensure the robots were communicating with each other and the human operator, the team set up an LTE cell tower, like the kind that regularly processes cell phone signals. This let the swarm share location to Android Tactical Assault Kit (ATAK), a program that runs on phones and lets soldiers in the field receive and send battle-relevant information.

“During the field exercise, as a matter of convenience, we had safety spotters out on the range looking for unsafe conditions or flyaways, and all of their phones were receiving all of the swarm traffic and their phone would vibrate and warn them if something got too close to them, because it’s too hard to track all of them in the air at the same time,” says Clark.

By only filtering the proximate drone location to people in the field, the swarm shared only what the observers needed to immediately know. In a real-life scenario, with swarms moving in support of battle, soldiers will at most need to know if a nearby drone is friendly, and won’t want to spend a lot of time sorting out where exactly the rest of the swarm is.

As the Pentagon expects future battles to take place in massive urban environments, getting more situational awareness could mean the difference between squad survival and ambush.

Watch a video from the human swarm operator’s perspective below:

On a side note, research has also determined that wearing Virtual Reality (VR) headsets can cause behavioral changes, balance problems (see 1, 2), cognitive problems, eye problems (soreness, vision changes), headaches, skin irritation and other short-term and/or long-term health issues. Research has determined that children absorb 2-5 times more harmful radiation than adults while using VR headsets.

Nevertheless, VR, AR (Augmented Reality), and now Mixed Reality (MR) headsets are increasingly being promoted for a variety of purposes including agricultural (as is on livestock!), education (see 1, 2), medical (see 1, 2, 3), employee and military training (see 1, 2), and of course, recreational.

Activist Post reports regularly about Virtual Reality (VR) and other unsafe technology. For more information, visit our archives and the following websites:

Top image: Pixabay

Become a Patron!
Or support us at SubscribeStar
Donate cryptocurrency HERE

Subscribe to Activist Post for truth, peace, and freedom news. Follow us on Telegram, HIVE, Flote, Minds, MeWe, Twitter, Gab, What Really Happened and GETTR.