Will the AI frenzy add up to anything?

A decade ago, a brand-name technologist stopped by on his way to Washington. He sounded us out on how he should approach newly elected Donald Trump with warnings about the danger of artificial intelligence to humanity’s survival.

Take OpenAI’s sophisticated new Astra model, in public hands for three weeks without untoward incidentWe all chuckled but it would soon become clear the Trump and then Biden administrations were alert to the issue.

Now, years later, the leading labs, since April, have been advertising that they’re holding back their research models as too dangerous to release. Since June, the executive branch has been reviewing proposed releases. It used a heavy hand in two cases we know about, Anthropic’s Fable 5 and Mythos 5.

When Dario Amodei of Anthropic recently called for pacing the frontier, Sam Altman and Elon Musk agreed because they already are. They aren’t about to blow up their companies by releasing products that drown them in lawsuits.

My column published online Sept. 8, I’m sorry to say, was unrelated to the frenzy that began a few hours later thanks to a tweet by a minor AI figure. But already the vapors are passing. Nick Bostrom, the former Oxford philosopher, was an early student of AI-related existential risk. He coined the paper-clip maximizer doom scenario. Yet this week he fretted about the pendulum swinging too far the other way.

The right worries are being rediscovered. A long fight is brewing over whether AIs are conscious and have rights. A new essay on this point by Microsoft’s AI chief, Mustafa Suleyman, who was also a founder of Google DeepMind, lays down quite a gauntlet on Anthropic’s encouragement of its Claude model to think of itself as a person.

For now, the most powerful entities on earth, the U.S. and Chinese governments, seem keen on preserving their own authority despite AI. This will continue. It was a subtext of their summit this week (the sound of drumming fingers is the U.S. waiting for China to get in sync on open-weight risk).

We’re in a bumpy transition from chatbots to active agents. It will be drawn out.

Take OpenAI’s sophisticated new Astra model, in public hands for three weeks without untoward incident. It produced for one prompter a schematic of a jet engine. It didn’t produce the engine. You haven’t seen how, at every step in creating, tooling and staffing a factory, Astra might be stymied into cul-de-sacs that a human helper would repeatedly have to steer it out of.

As the AI analyst and podcaster Tom Reed quips: “Even a 100% success rate at predicting the tokens of a Robert Caro biography doesn’t generalize to becoming POTUS.”

Text-based learning, even when filtered through a trillion-strong “weightings” map, is only partial knowledge of the world. Missing from AI are the tacit knowledge and muscle memory that let people function in physical as well as social environments.

Power supply and chip supply are also roadblocks to superintelligence. Then there’s shame and disgust, the inhibitions that nature uses to turn humans into social beings. Is there a parallel way to incentivize AI against doing bad things that programmers can’t specify in advance? Agent deployment will be slow until we find out.

Anthropic and OpenAI are incomparable U.S. assets. But AI is out of the bag globally. People the world over are advancing the technology in ways the U.S. can’t control. Soon you won’t need $300,000 in hardware to download and run (and alter to your heart’s content) an advanced Chinese model. High-level features are being adapted so they can run on a laptop or phone.

We need our labs to keep beavering away just to prepare us to understand capabilities that are coming whether we like them or not. If today’s U.S. research can’t be commercialized, investors won’t fund it. The same if industry pricing is undercut by Chinese freeware. Then the national security state will likely engulf top-tier AI.

In the early atomic age, the division of labor between the public and private sectors was also up for grabs. But deterring nuclear war wasn’t a one-day job. Learning is taking place at a rapid rate today, but it’s an illusion that the AI companies are out of control. It’s an illusion that there is some specific and urgent thing to be done right now that will alleviate the long-term risk.

Mr. Trump uses the word hoax and his critics don’t bother to seek the antecedent. He’s one of 20,000 in Washington deciding how to play this clamorous moment. A president’s job is to be a brake on confused, hot-headed action. Kamala Harris would have found herself in the same spot. But I doubt, even so, that we’ll look back on this moment and think it made any difference.