AI Almost Started a U.S.–China War — and No One Seems to Care

The Intercept | Sam Biddle

A U.S. Special Operations Command analyst using a large language model to generate intelligence reports nearly instigated a military conflict with China this past spring. The artificial intelligence system produced an entirely false report claiming a Chinese vessel in the Middle East was transporting nuclear weapons components, prompting rapid American preparations for a forceful interception.

This near-miss occurred amid aggressive Pentagon efforts to accelerate the integration of experimental autonomous technologies into safety-critical national security infrastructure. While Silicon Valley executives warn of hypothetical existential threats from rogue superintelligence, they simultaneously secure lucrative defense contracts to deploy unproven software. Speed takes priority over reliability. Consequently, the rapid deployment of unreliable targeting systems raises the risk of accidental escalation between nuclear-armed states. Experts warn that automating target generation at scale reproduces errors faster than human supervisors can verify them, transforming software hallucinations into immediate geopolitical crises.

Comment

The integration of large language models into the intelligence cycle of U.S. Special Operations Command introduces severe systemic vulnerabilities. When these models generate false target intelligence for U.S. Special Operations Command, the compressed decision-making window leaves human analysts with insufficient time to verify raw data. Speed-driven automation compromises analytical rigour. This operational friction is exacerbated by the lack of explainability in the proprietary algorithms supplied to the Pentagon by commercial developers like OpenAI. Consequently, tactical commanders operating under U.S. Central Command risk executing kinetic interdictions based on unverified, hallucinated data.

The downstream consequence of this automation bias is the erosion of strategic deterrence stability between Washington and Beijing. If U.S. Strategic Command relies on similar automated intelligence pipelines, the threshold for accidental nuclear escalation drops precipitously during a crisis. Ultimately, the rapid adoption of unverified AI tools by the Pentagon creates a structural vulnerability where a software hallucination can trigger a retaliatory strike by the People's Liberation Army Rocket Force before diplomatic channels can intervene.

Strategic Question for Discussion

If U.S. Special Operations Command continues to integrate unverified large language models into its intelligence pipelines, does the traditional concept of human-in-the-loop oversight remain a viable safeguard against accidental escalation, or does it become an operational fiction?

The pattern suggests that human-in-the-loop oversight becomes largely symbolic when the speed and volume of AI-generated intelligence outpace human cognitive processing limits. My assessment is that analysts facing compressed timelines will increasingly default to automation bias, accepting algorithmic outputs without independent verification. Consequently, the safeguard of human intervention is effectively neutralized, transforming nominal oversight into a rubber-stamping exercise of automated target generation.

Share your assessment in the comments below.

To read in details click here