Between Extinction and Hubris: A Dialectical Approach to AI Risk, Control, and Governance

Inss.ndu.edu | Elise Annett, James Giordano

Artificial intelligence discourse remains sharply divided between catastrophic extinction warnings from frontier developers and critical reminders from scholars like Timnit Gebru regarding immediate harms. Dr. Elise Annett and Dr. James Giordano argue that these conflicting positions require dialectical conciliation focused on verifiable capabilities and operational interactions rather than speculative superintelligence.

Current agentic systems execute complex sequences autonomously, creating near-term risks as humans progressively cede agency for operational expediency. Department of Defense Directive 3000.09 mandates appropriate judgment, yet organizational inertia and compressed decision cycles threaten to render human oversight performative. The authors propose a pragmatic governance framework based on continuous TEVV, capability-based rules, and graduated autonomy thresholds. Effective management requires aligning institutional behavior, technical evaluation, and international norms to prevent uncontrolled machine-speed escalation in military and intelligence domains before systemic vulnerabilities precipitate strategic crises.

Comment

Unchecked delegation to agentic systems under Department of Defense Directive 3000.09 risks creating a structural illusion of human control during rapid targeting cycles. When software agents execute multi-step planning and tool invocation faster than human operators can verify context, nominal presence in the loop collapses into passive rubber-stamping. This operational velocity creates vulnerabilities to subtle adversarial manipulation within NIST TEVV-Athlon testing frameworks that static validation protocols fail to capture. Consequently, military command structures face severe friction when forced to arbitrate between compressed decision timelines and the rigid verification requirements necessary to prevent unintended escalation.

Strategic Question for Discussion

If agentic AI systems continue to compress planning and execution timelines within military targeting cycles, does the requirement for appropriate human judgment under Department of Defense Directive 3000.09 remain operationally viable, or does it transition into a performative safeguard?

The available evidence points toward a growing mismatch between rigid oversight mandates and machine-speed execution. As agentic autonomy expands, nominal human-in-the-loop structures risk becoming symbolic hurdles rather than substantive brakes on escalation.

Share your assessment in the comments below.

To read in details click here