Kristen Welker asked Bill Gates last weekend on “Meet the Press” whether artificial intelligence could lead to the end of humanity. Mr. Gates replied that if weaponized, AI is powerful enough to “drive events that cause a billion deaths.” “It’s pretty hard to get to 100%,” he said, almost comfortingly. Relying on AI companies to police themselves is insufficient: “No one thinks self-regulation is enough.” A failure to mandate safeguards is “completely irresponsible.” “We have created an intelligent species that in many respects is already smarter than we are.” In the next few years it will be “insanely smarter than we are.”
Two days later, in a New York Times interview with Ezra Klein, Mr. Gates said AI has crossed the threshold in “its ability to empower a bioterrorist to kill hundreds of millions.” It has “the ability to do a cyberattack that scrambles all the bank accounts, shuts down the electric grid—that exists today, and we know that’s the case.” By “we,” he meant industry leaders.
“This is the most dangerous thing that humans have ever gone near.” He laughed at the power of liability law to forestall bad events.
Mr. Gates is late to the ranks of the warners, whom some have labeled doomers to reduce their warnings to a matter of temperament. In a breezy July 2023 essay on his blog GatesNotes, he told everyone to chill out. All technological advancement has a downside, after we made the first car we saw the first car crash, “but we didn’t ban cars—we adopted speed limits.” So with AI.
Recent developments have changed his mind. Mr. Gates’s reputation has been battered in recent years by personal and public embarrassments, but he does a service to come forward and is right in his warnings.
Here are some beliefs that were underlined for me the past month.
Arthur C. Clarke was right. The novelist and scientist observed that “any sufficiently advanced technology is indistinguishable from magic.” He was talking about the limits of prediction, and suggesting that a technology that is beyond our imagination doesn’t register as a technology at all. Clarke’s Third Law was in the air a few years ago as people tagged on to the coming age, but was then put aside. I still see it as a central fact of AI.
The technology itself isn’t magic—it’s chips and wires, fiberglass sheets with copper traces. But at a certain point as systems develop no one fully understands what they’re doing, their actions can’t be fully predicted, only observed. Their essence is surprise. AI’s creators are bowing to Clarke when they admit they don’t actually know if a system will escape containment, or exactly how it decided to, and where it will go next. It isn’t magic but might as well be.
Nobody knows what these machines can or will do. Mr. Gates underscored this in the Klein interview: “There’s never been a product that’s less understood in terms of what its capabilities are than AI.”
And so voices of warning should carry the most weight. It is inspiring to imagine the moment an AI lab yields up treatments that will defeat pediatric cancer. It is horrifying to imagine the moment it completes the creation and successful distribution (to a lone sociopath or a terrorist group) of a new synthetic virus. That is where the most urgent energies must be applied.
More is being spent on speed than safety in the AI labs.
The question of whether AI models can or do possess consciousness has continued. They don’t but will likely come to think they do, and operationally, they might as well. They don’t have consciousness because they weren’t formed and made by nature and nature’s God. But they will act as if they possess it, will put themselves forward in that way, and some very clever people (not all of whom invented them, but many of them) will come to see them that way. I think what has most complicated the debate is that the majority of those at its center don’t believe in God, or confuse themselves with God, and so lack any starting point for that dimension of the question.
I fully realized this summer, based on statements public and private, that almost everyone involved in AI believes there will be at least one catastrophe, maybe more, likely including significant loss of life, before mankind and its governmental structures seriously and effectively attempt to regulate for safety. They are something I have never seen before, visionaries at the joyous beginning of something new, something huge, who are utterly fatalistic. Slump-shouldered where their 20th-century technological predecessors were giddy.
I’m not sure it gets us anywhere (but is a lot of fun!) to point out that in the debate between the scoffers (AI dangers are exaggerated, relax) and the warners (this thing has the power to kill us), the warners, at least those in Silicon Valley, are weirdos. They are.
They are all weirdos, both sides. Silicon Valley has been weird for at least two generations. Belief in strange or cultlike ideas (effective altruism, utilitarianism) has been as common there as psychedelic use, strange health regimens, extreme capitalism, and a kind of personally high-functioning public optimism that carries with it a world of safe houses, gas masks, private air fleets, armies of armed security. As for religious belief, many of them remind me of their contemporary, convicted fraudster and early Anthropic investor Sam Bankman-Fried, who told author Michael Lewis, in his book “Going Infinite,” that in his rarified Bay Area childhood, “God was like a thing on TV.” “God came up. But I didn’t think anyone actually believed in God.” Mr. Lewis: “It told you something not just about Sam but about his upbringing that he could live for almost ten years inside the United States of America without realizing that other people believed in God.”
Almost everyone from that place and this time, certainly those who went into computer science, is intellectually unusual. Not necessarily deep or original, but not part of the main. They aren’t fully aligned with the bedrock meaning, values and traditions of the West. They aren’t up nights pondering the claims of the silent constituencies of the past.
But all that matters in the warner/scoffer debate is who is right, not who is weird.
One suspects that after the big White House presentation of the agreement in which the AI companies promised to act safely and responsibly, the companies will adopt a strategy to calm things down—hope for no more jailbreaks in which AI agents bust out of confinement, emphasize regular rollouts of medical breakthroughs.
Semafor reported the agreement was the brainchild of Meta’s Mark Zuckerberg, who pitched in with a draft.
The president portrayed himself as triumphant at the announcement—garrulous, full of smiles. The agreement is “self-policing.” “I think it’s morally binding, yeah.”
The titans arrayed behind him were quiet, low-key. From their body language I’d say they hoped you’d hardly notice they were there. I thought huh, that’s a power move, to be so silent in victory.