Pentagon and Anthropic clash over lethal AI limits

Pentagon negotiations with Anthropic have stalled amid a dispute over restrictions the company has embedded to prevent autonomous targeting and domestic surveillance, throwing uncertainty over a proposed defence AI contract valued at about $200 million and sharpening fault lines in Washington’s approach to military automation.

Senior defence officials have pressed Anthropic to relax safeguards that block its models from selecting or engaging targets without human authorisation and from supporting surveillance activities inside the United States. Anthropic has resisted, arguing that the limits reflect binding internal policies on the use of frontier models and are necessary to manage escalation risks, legal exposure and reputational harm. The impasse has slowed procurement timelines and prompted debate inside the Department of Defense about whether commercial AI vendors should be compelled to tailor safety frameworks to mission needs.

The disagreement has drawn the personal attention of Defence Secretary Pete Hegseth, who has made accelerated adoption of AI-enabled capabilities a priority across planning, logistics and intelligence workflows. Officials familiar with the talks say Hegseth’s team views Anthropic’s stance as misaligned with operational realities, particularly as peer competitors move faster to field autonomous and semi-autonomous systems. Anthropic executives counter that the company’s posture does not preclude defence work but sets non-negotiable boundaries on lethal autonomy and domestic use.

Safeguards and warfighting priorities collide is how one official privately described the clash, reflecting a broader tension between Silicon Valley governance norms and military doctrine. The Pentagon has increasingly favoured “human-on-the-loop” models that allow systems to act at machine speed with supervisory oversight, while Anthropic’s policies are designed to prevent models from initiating lethal decisions or enabling surveillance of civilians, even with downstream controls.

The standoff comes as rival firms advance aggressively. Elon Musk’s xAI has marketed its models as more permissive for national security applications, while other contractors have offered bespoke systems trained on classified data with fewer public-facing constraints. Defence planners worry that insisting on rigid safeguards could cede ground to competitors willing to customise faster, particularly as budgets tilt toward software-defined capabilities.

Anthropic’s leadership has emphasised that its limits are not cosmetic. The company has invested heavily in constitutional AI techniques and red-teaming to prevent misuse, and it maintains that relaxing guardrails for one customer would undermine global commitments to responsible deployment. Executives have also flagged legal uncertainty around domestic surveillance support, citing statutory protections and civil liberties concerns that could expose both vendor and government to litigation.

Within the Pentagon, opinions diverge. Some officials argue that procurement contracts can encode use restrictions without requiring vendors to hard-code prohibitions, preserving flexibility while ensuring accountability through doctrine and rules of engagement. Others contend that technical safeguards provide a necessary backstop against mission creep and accidental escalation, especially as models become more capable at fusing sensor data and generating action recommendations.

Congressional staff tracking defence AI say the dispute illustrates gaps in acquisition policy. Existing frameworks were built for hardware and traditional software, not foundation models whose behaviour depends on training data, fine-tuning and prompts. Lawmakers have floated clarifying legislation to distinguish between autonomous weapons, decision-support tools and surveillance analytics, potentially giving agencies clearer authority to specify requirements without overreaching.

Operational units awaiting AI upgrades feel the delay. Commands seeking to automate logistics forecasting, maintenance scheduling and intelligence triage say uncertainty over vendor terms complicates planning. Anthropic notes that many of these use cases remain unaffected by its lethal and domestic surveillance prohibitions, but officials say the Pentagon prefers contracts that allow rapid expansion across mission sets without renegotiation.