From Oppenheimer to Autonomous Weapons: How Technology Changed Warfare
In July 1945, a small group of people watched a new physical capability become a geopolitical fact. Eighty-one years later, the central problem is not another Trinity test. It is the accelerating use of software, sensors and artificial intelligence in systems that can find, track, select or engage targets. Nuclear weapons and autonomous weapons are profoundly different technologies, but their histories expose the same difficult pattern: invention can move faster than the rules, institutions and public language needed to govern it.
The essential question is no longer whether militaries will use AI. They already do. The question is where human judgment must remain indispensable, how responsibility survives when decisions are distributed across people and machines, and which capabilities should never be built or used.
01 · Capability before governance
The Oppenheimer lesson is about institutions, not mythology
J. Robert Oppenheimer is often reduced to a symbol: the brilliant physicist who helped create the atomic bomb and later became troubled by the world it made possible. The real history is more useful and less tidy. The Manhattan Project was a vast state-directed enterprise spanning laboratories, industrial sites, military command and an international scientific workforce. Oppenheimer led the scientific laboratory at Los Alamos; General Leslie Groves directed the broader project. The achievement depended not on a lone genius but on an organization able to combine science, engineering, manufacturing, secrecy and political authority at extraordinary speed.
Los Alamos was established in 1943. On July 16, 1945, the Trinity device detonated in New Mexico in the world’s first nuclear explosion. The project had converted a theoretical possibility into a weapon before governments had developed durable international institutions for controlling the new capability. Decisions about Hiroshima and Nagasaki, postwar secrecy, the arms race and nuclear deterrence unfolded after the technological threshold had already been crossed.
That sequence matters more than the familiar image of one scientist confronting his conscience. Scientists, engineers, military officers, contractors, elected leaders and institutions each controlled only part of the process. Moral responsibility was real, but it was distributed. Technical expertise did not automatically confer democratic authority, while political authority did not guarantee a full understanding of the technology’s consequences.
The Oppenheimer story also warns against treating ethics as a final review performed after a system works. By then, organizations have invested money, careers and strategic expectations in the result. Governance is strongest when it shapes requirements, design, testing, acquisition and deployment from the beginning. The question “Can it be built?” must be accompanied by “Under what conditions should it be built, and who can stop it?”
That lesson applies beyond weapons. Civilian debates about algorithmic bias, facial recognition and automated decision-making all show how hard it is to restore accountability after a system is embedded. Military use adds consequences measured not merely in unfair outcomes or lost privacy, but potentially in injury, death and escalation between states.
02 · A spectrum, not a robot uprising
What “autonomous weapon” actually means
Public discussion often jumps from today’s drones to science-fiction machines that independently decide to wage war. That leap hides the decisions that already matter. Autonomy is not a single switch. A system may navigate by itself, identify objects with computer vision, recommend a target, prioritize threats, select an aim point or engage after activation. Different functions can involve different levels of human supervision and different degrees of uncertainty.
The International Committee of the Red Cross describes autonomous weapon systems as systems that, once activated, select and engage targets without further human intervention. The exact wording used by governments varies, and the lack of one universally agreed definition remains part of the diplomatic problem. Still, the practical concern is clear: after a person activates the system, a combination of sensors, software and environmental conditions can determine the specific object or person against which force is applied.
This spectrum is more informative than labels such as “AI weapon” or “killer robot.” A weapon can be autonomous without using fashionable generative AI. It can follow relatively fixed rules and sensor inputs. Conversely, a military AI system may support logistics, maintenance, intelligence analysis or planning without itself being a weapon. Collapsing all military AI into one category makes sensible rules harder to write.
The operating environment matters as much as the algorithm. A tightly bounded defensive system responding to incoming materiel in an empty area presents a different problem from a mobile system searching for people in a crowded city. Duration, geographic limits, target type, sensor reliability, communications, weather, adversarial deception and the possibility of civilians entering the area all change the risk. The same software can be more or less controllable depending on where and how it is used.
Is it a clearly identifiable military object, or does classification depend on interpreting human behavior?
Is the area constrained and predictable, or complex, crowded and rapidly changing?
Can a supervisor understand and interrupt the system before force is applied, not merely watch afterward?
What happens when sensors are confused, data are incomplete, communications fail or an adversary deliberately deceives the system?
Recruiting pressures, the desire to protect personnel and the speed of modern combat may all influence military automation. But it would be misleading to say autonomous weapons exist mainly because armed forces cannot recruit enough people. Strategic competition, surveillance capacity, precision, operating tempo, cost, industrial policy and the fear of falling behind rivals are all part of the picture. Technology does not arrive from one cause, and a system’s perceived usefulness does not settle whether its use is lawful or wise.
03 · The human is not a rubber stamp
Meaningful control requires time, knowledge and the power to intervene
“A human is in the loop” sounds reassuring, but it can describe very different realities. A commander who approves a carefully bounded mission after reviewing reliable information exercises more judgment than an operator asked to confirm dozens of machine recommendations in seconds. A person technically able to press an abort button may have no meaningful control if the interface is confusing, the system acts too quickly, communications are unreliable or organizational pressure makes intervention unrealistic.
Human control therefore has to be evaluated across the system’s full lifecycle. What targets was the system designed to recognize? What assumptions shaped the training and test data? What geographic and temporal limits are imposed? What confidence information does the operator see? Under what circumstances must the system fail safe? Who has authority to suspend its use when the environment no longer matches the tested conditions?
- Political controlClear policy about which systems and uses are prohibited or restricted.
- Command controlDefined objectives, target classes, geography, duration and rules of engagement.
- Technical controlPredictable behavior, traceable logs, secure updates, testing and safe failure modes.
- Operator controlEnough information, time, training and authority to question or stop the system.
- Accountability after useEvidence that allows investigators to reconstruct decisions and assign responsibility.
The U.S. Department of Defense’s Directive 3000.09 illustrates how one major military frames this problem. The 2023 directive requires autonomous and semi-autonomous weapon systems to be designed to allow commanders and operators to exercise appropriate levels of human judgment over the use of force. It also calls for rigorous verification and validation, realistic developmental and operational testing, training, doctrine and safeguards against unintended engagements. Those are meaningful requirements, though a national policy is not the same thing as an international legal rule and phrases such as “appropriate levels” still require interpretation in specific cases.
Testing is especially difficult because machine performance is not a timeless property. A model validated against one environment may behave differently when sensors change, software is updated or an adversary manipulates inputs. Average accuracy can hide rare but catastrophic failures. A test range cannot reproduce every civilian pattern, cultural context or deceptive tactic. Confidence scores can appear precise while depending on assumptions the operator cannot see.
Automation bias adds another layer. People often defer to a system perceived as objective, especially under time pressure. At the same time, demanding constant monitoring of a highly reliable system can make attention deteriorate. Good control cannot depend on a heroic operator correcting a machine at the last instant. It must be built into mission design, interfaces, staffing, training, authority and the boundaries placed on the system.
04 · Responsibility cannot disappear into software
International law still applies—but application can become harder
Autonomy does not create a law-free zone. International humanitarian law applies to the use of all weapons, including systems with autonomous functions. Parties to a conflict must distinguish between military objectives and civilians, take feasible precautions and avoid attacks expected to cause excessive civilian harm in relation to the anticipated concrete and direct military advantage. States also have obligations to review new weapons for legality.
The hard question is whether a particular system can be used in a particular environment while those obligations are actually met. Legal rules are applied through human judgment about context: whether an object is making an effective contribution to military action, whether a person is directly participating in hostilities, what civilian harm is foreseeable and whether circumstances have changed. Machines can classify patterns, but pattern classification is not identical to legal judgment.
This distinction becomes especially important when targets are people rather than objects. Human behavior is ambiguous. A person may run, carry an object, approach a checkpoint or travel with others for many reasons. Data can reflect gaps and past biases. An algorithm cannot bear moral or legal responsibility, understand the dignity of the person before it or be punished for a wrongful attack.
Responsibility should remain with people and institutions: commanders who authorize missions, operators who employ systems, developers and acquisition officials who make safety-relevant choices, and states that field weapons. Yet highly distributed design and decision chains can make accountability harder to trace. If a harmful engagement results from the interaction of a sensor limitation, a model update, poorly specified mission parameters and inadequate supervision, each participant may point to someone else.
Automation can distribute a decision. It cannot erase the obligation to explain who authorized force, on what basis, within which limits and with what evidence that the system could comply with the law.
That is why traceability matters. Logs should preserve relevant system states, inputs, confidence information, operator actions and software versions. Command structures should specify who can authorize, modify, pause and terminate a mission. Reviews should be independent enough to challenge optimistic assumptions. Contractors cannot be the sole judges of whether a government’s use is lawful, and secrecy should not become a blanket excuse for avoiding legislative or public oversight.
Cybersecurity is part of weapons safety as well. A system that behaves predictably in a laboratory may be spoofed, jammed, compromised or fed manipulated data in conflict. Update mechanisms and supply chains create additional attack surfaces. Security failures can alter not only whether a weapon functions, but what it perceives and how it applies force.
None of this proves every autonomous function is inherently unlawful. Some systems may be limited to defensive use against clearly defined military objects in constrained environments. The point is that legality and acceptability cannot be inferred from a product label. They depend on technical characteristics, target type, environment, mission design and the quality of human control.
05 · A live policy window
Why 2026 is a decision point
The diplomatic debate is no longer hypothetical. Through the Convention on Certain Conventional Weapons, governments have spent years discussing lethal autonomous weapon systems. The United Nations secretary-general has called for a legally binding instrument by 2026 that would prohibit systems that function without human control or oversight and regulate other autonomous weapon systems. The ICRC has similarly recommended prohibiting unpredictable autonomous weapons and systems designed or used to target people, while placing strict restrictions on other systems.
The first 2026 session of the CCW Group of Governmental Experts met in Geneva from March 2 to 6. A second session is scheduled for August 31 to September 4—just after this article’s publication. The group’s rolling text and the chair’s summary show that states are working with possible elements for an instrument, but major questions remain: which prohibitions should be explicit, what form restrictions should take, how human control should be described and whether the outcome will be legally binding.
On August 25, 2026, the UN secretary-general and the ICRC president renewed their call for states to negotiate legally binding prohibitions and restrictions, pointing to the November 2026 Review Conference of the Convention as a key opportunity. Their position does not call for banning every use of autonomy. It separates unacceptable systems from other autonomous systems that would remain subject to regulation and limits.
Systems whose effects cannot be sufficiently understood, predicted or controlled
A prohibition addresses systems for which users cannot make a reliable legal and operational judgment about what will happen after activation.
Do not delegate life-and-death targeting of people to machines
The ICRC argues for prohibiting systems designed or used to apply force against persons, preserving human agency in decisions that directly endanger life.
Bound all other systems by target, place, time and supervision
Limits on target type, duration, geographic scope, scale and intervention can reduce uncertainty and help preserve compliance with the law.
Make review, testing, incident evidence and accountability real
Rules matter only if states can evaluate systems before use, investigate failures and hold identifiable people and institutions responsible.
Opponents of a new treaty often argue that existing international humanitarian law is sufficient, that rigid definitions may become obsolete and that autonomy can sometimes improve precision or protect civilians. Those concerns deserve serious treatment. Poorly written rules could create loopholes or freeze one technical model in time. A ban framed around marketing terms would age badly.
But existing law and new rules are not mutually exclusive. A focused instrument can clarify how enduring legal obligations apply when a machine selects and engages targets. It can establish bright lines for unacceptable systems and practical restrictions for others without pretending every autonomous function is identical. Technical change is a reason to write rules around human control, predictability and effects—not a reason to avoid rules altogether.
The strongest framework would be technology-aware but technology-neutral. It would not depend on whether a vendor calls a system “AI-enabled.” It would ask what functions the system performs, against what targets, under which conditions, with what uncertainty and with which opportunities for informed human judgment. Those questions remain relevant even as specific models and sensors change.
06 · What responsible progress would look like
Move the ethical decision upstream
The most important governance choices happen long before deployment. Procurement requirements determine which risks contractors optimize against. Test plans decide which environments and failure modes count. Interface design determines what an operator can understand. Staffing and doctrine determine whether intervention is possible. Legal review determines whether assumptions are challenged before a system becomes strategically indispensable.
Responsible states should begin with clear prohibitions and mission boundaries, not a promise to “add a human” later. They should require independent technical and legal review, red-team systems against deception and distribution shift, test with realistic civilian complexity, document software changes, protect incident data and create channels through which operators can halt unsafe use without career-ending pressure.
Developers also need protected ways to raise concerns. Oppenheimer’s era is sometimes remembered as though scientists first built the device and only afterward discovered politics. In reality, political judgment was present from the beginning; it was simply concentrated within wartime institutions and secrecy. Today’s engineers should not be asked to substitute personal conscience for public rules, but neither should they be told that technical work is morally neutral.
Legislatures and the public need enough information to debate doctrine without demanding operational details that would create legitimate security risks. Oversight can examine categories of systems, review standards, testing regimes, incident reporting and chains of command. Democratic control does not require publishing targeting software. It requires refusing to let classification make consequential policy invisible.
International agreement will not eliminate violations or strategic competition. Arms-control history has never depended on perfect trust. Rules can still shape procurement, training, alliances, export decisions, reputational costs and the evidence used to judge conduct. A shared boundary also helps responsible actors resist the claim that every technically possible capability is strategically inevitable.
Oppenheimer’s enduring relevance is not that every new military technology leads to apocalypse. It is that technical success can narrow political choices if governance arrives too late. With autonomous weapons, the opportunity is still open: preserve human judgment where force is applied, prohibit systems that cannot be meaningfully controlled, restrict the rest, and keep responsibility with people.
Sources and further reading
Primary and institutional sources were favored for historical facts, current policy positions and legal frameworks. Accessed August 29, 2026.
- U.S. Department of Energy — Manhattan Project historical resources
- U.S. Department of Energy — Trinity Site: the world’s first nuclear explosion
- National Nuclear Security Administration — Trinity history brochure
- ICRC — Autonomous weapon systems and international humanitarian law: selected issues
- ICRC — Autonomous weapons: law and policy overview
- ICRC and SIPRI — Limits on autonomy in weapon systems
- ICRC — 2026 opportunity to prevent unacceptable autonomous weapons
- UN Secretary-General and ICRC President — Renewed 2026 call for binding rules
- UN Office for Disarmament Affairs — Lethal autonomous weapon systems
- UNODA — 2026 Group of Governmental Experts meeting schedule
- UNODA — 2026 GGE documents, rolling text and chair’s summary
- UNODA — Artificial intelligence in the military domain
- United Nations — General Assembly action on lethal autonomous weapons, 2024
- U.S. Department of Defense Directive 3000.09 — Autonomy in Weapon Systems