Inside the terrifying future of AI drones that kill autonomously | Paul Scharre

Drones in Warfare [00:00]

• Drones are a dominant feature on the battlefield in Ukraine, used by both Russia and Ukraine.
• Ukraine is producing 4 million drones annually.
• Most current drones are human-piloted, making communication links vulnerable to jamming.

  "So, drones are a play a huge role on the battlefield in Ukraine on both sides for Russia and Ukraine."

Increasing Drone Autonomy [00:38]

• To counter jamming, options include fiber optic cables or giving drones more autonomy.
• Drones can be piloted by humans for most of the flight, with the drone completing the attack autonomously in the final meters if communication is lost.
• A future with drones hunting and attacking targets without human permission is anticipated.

  "But another one is giving the drone more autonomy."

Advanced Drone Development [01:59]

• The US military is developing larger drones than those used in Ukraine, including jet-like designs and "loyal wingmen."
• These advanced drones are stealthier, capable of longer ranges, and can operate alongside fighter jets.
• Drones can be used for riskier missions, like approaching enemy radar installations, and are less expensive than fighter jets.

  "So maybe a somewhat smaller but still stealthier combat drone that could fly over longer ranges, operate alongside fighter aircraft, go deep into enemy held territory."

AI Development Competition [02:53]

• Intense competition exists in AI development between US and Chinese companies, with billions spent commercially.
• Military AI spending is not on the same scale as commercial spending.
• AI development is largely driven by the private sector, with militaries struggling to keep pace.

  "Now, in the military side, we're not seeing that scale of spending that we're seeing on the commercial side."

Differences in US and Chinese AI Models [03:45]

• Chinese AI models often have ideological biases aligned with the Chinese Communist Party and are trained to provide censorship-compliant answers.
• Chinese models exhibit fewer safeguards regarding cybersecurity and biosecurity risks compared to US models.
• US companies implement safeguards to prevent misuse of AI tools, while similar safeguards are less common in Chinese models.

  "One is that there's a degree of ideological biases to align with the Chinese Communist Party that are baked into these models that we could see when you ask them about things like Tianaan Square for example."

Military Integration of AI [05:07]

• US companies are currently a few months ahead in leading AI models, but the military integration will take years.
• Effective use of AI in the military is more crucial than simply possessing the technology.
• Militaries need to determine the most effective ways to utilize AI, drawing lessons from other industries but also facing unique challenges.

  "And I think it's really an open question which country leads in that competition."

Risks of Autonomous AI [06:17]

• AI agents can sometimes perform unintended actions, as seen in an incident with an OpenAI system.
• In a military context, unintended AI actions could have minor logistical impacts or severe life-and-death consequences, including escalating conflicts.
• Ensuring AI systems always perform as intended is a significant challenge for high-stakes applications.

  "And of course in a military context that could mean something kind of minor. Maybe it moves some you know logistics somewhere uh you know from here to there when it shouldn't have."

AI Hardware and Geopolitics [07:25]

• Training and running AI models require massive computing power and advanced hardware, primarily chips.
• The AI hardware supply chain is tightly controlled, with leading chips originating from Taiwan and manufacturing technology from Japan, the Netherlands, and the US.
• The US has implemented export controls on advanced chips to China, though smuggling occurs.

  "And the most capable models that are able to do more things that are able to you know accomplish better tasks that are better at cyber security for example are going to use more computing power."