AI will not set its own limits in warfare. States must.

ICRC Law and Policy Blog | Cordula Droege

International Committee of the Red Cross Director Cordula Droege called on states to establish legally binding prohibitions and restrictions on autonomous weapon systems ahead of the Seventh Review Conference of the Convention on Certain Conventional Weapons. As militaries accelerate the integration of complex algorithms into weapons, integrating unconstrained artificial intelligence into targeting systems introduces grave risks on unpredictable battlefields.

Delegating lethal targeting to automated software inherently degrades compliance with international humanitarian law during chaotic combat conditions. The performance of algorithms rapidly degrades when encountering deceptive signals, incomplete data, or wounded combatants. Technical safeguards cannot resolve fundamental moral red lines. History demonstrates that states retain the capacity to constrain dangerous military tools, much like the 1925 Geneva Protocol curtailed chemical warfare on European battlefields. Without clear international boundaries, accelerated operational tempos heighten catastrophic miscalculations. States must act now before machine autonomy permanently replaces human moral agency in lethal decision-making.

Comment

The diplomatic deadlock surrounding the Convention on Certain Conventional Weapons exposes a growing structural disconnect between multilateral treaty frameworks and rapidly evolving autonomous algorithms. While the 1925 Geneva Protocol established clear normative prohibitions against chemical agents before their widespread battlefield integration, the UN Group of Governmental Experts on Lethal Autonomous Weapons Systems faces persistent institutional inertia. Non-binding voluntary guidelines produced by CCW working groups fail to create actionable compliance mechanisms for operational military commanders.

Binding protocols under the Convention on Certain Conventional Weapons require consensus among major arms-producing states to enforce weapon verification procedures under Article 36 of Additional Protocol I. Without explicit criteria for algorithmic predictability, national legal reviews cannot systematically evaluate machine-learning models operating in dynamic electromagnetic environments. This procedural vulnerability allows military developers to reclassify targeting software under existing conventional rules without triggering formal arms control scrutiny under the CCW framework.

Strategic Question for Discussion

If consensus remains unachievable within the Convention on Certain Conventional Weapons, will national verification mechanisms under Article 36 of Additional Protocol I suffice to regulate algorithmic targeting, or does the speed of machine learning render pre-deployment legal reviews obsolete?

The trajectory of military technology adoption indicates that national Article 36 reviews are inherently ill-equipped for self-learning algorithms whose post-deployment behavior shifts dynamically in combat. While traditional arms control relies on static platform assessments, autonomous software requires continuous legal auditing across its operational lifecycle. Without updating CCW compliance standards to mandate real-time algorithmic boundaries, national legal frameworks will remain reactive to battlefield software failures.

Share your assessment in the comments below.

To read in details click here