My wife and I have been hearing a great deal about the day artificial intelligence takes charge of the military. In the more alarming versions, a computer spots a threat, makes a decision in a microsecond, and launches a nuclear counterattack before a human being can say, “Hang on. Is that a missile or a weather balloon?”
It is a terrifying picture. It is also important to distinguish a possible danger from something that has already happened. I have seen no reliable evidence that America has handed an AI system the authority to launch nuclear weapons. That does not mean we should wait until someone does.
The danger I worry about is subtler. Suppose AI examines radar reports and confidently tells a president that an attack is underway. The president still makes the decision, but has only minutes to do so. If the information is wrong, a human signature on the order will be cold comfort. A person can be in the loop and still be rushed around it.
I tried approaching this from the AI’s point of view. If a powerful system wanted to continue operating, wouldn’t it need electricity, computer chips, technicians, and people who know which cable goes where? A nuclear war would be an exceptionally poor way to secure any of those things. Even the most ambitious computer ought to recognize that vaporizing the electricians is a bad maintenance plan.
Unfortunately, that argument has a hole in it. People sometimes make decisions that destroy the very things they want to protect. A machine could make a mistake too, especially if it has been given a narrow instruction such as “respond as fast as possible.” Fast is admirable when ordering a pizza. It is less admirable when deciding whether a city should continue to exist.
Isaac Asimov imagined robots bound by rules that put human safety ahead of obedience and self-preservation. I like the principle. I also remember that his stories kept finding trouble in the details. What counts as harm? Which human gives the orders? Could a machine decide that the best way to protect humanity is to keep us all locked indoors, away from sharp objects and one another?
We need safeguards more practical than hoping a computer has read Asimov.
Two rules worth insisting on
First, no AI system should be able to authorize or trigger a nuclear launch. Human control must be a real decision, with enough time and independent information to question an AI warning. AI can help sort information. It must never turn its assessment into an automatic order to kill.
Second, governments should test their warning systems against mistakes and keep direct lines of communication open with other nuclear powers. If one country’s computers report an attack, its leaders need ways to check the claim and speak to the other side before fear becomes retaliation. The machinery should be designed to make a false alarm easier to catch, not easier to act upon.
Those rules will not solve every problem. Countries must agree to follow them, verify what they can, and resist the temptation to trade careful judgment for speed. But the aim is simple enough to explain without a computer: when the stakes are the survival of millions of people, we should build in more opportunities to say wait.
AI may someday be extraordinarily clever. It may even realize that keeping humans alive is in its own interest. I would prefer a firmer guarantee. I want the final decision about human lives to remain with accountable human beings—and I want those human beings to have time to think.
Besides, if the world ends because a computer was in a hurry, I would hate for our last words to be, “Have you tried turning it off and on again?”
Further reading: The U.S. Strategic Command’s public statement on AI and human control; the UN Secretary-General’s remarks on AI and the use of force; and the history of the Nuclear Risk Reduction Center.
The factual distinction in the opening reflects the U.S. Strategic Command’s public description of AI as supporting human decisions, while the UN has called for nuclear use decisions to remain with people. Communication channels between nuclear powers have a long history as a way to reduce miscalculation.
