President Trump will host Chinese President Xi Jinping in Washington on September 24, and both leaders confirm artificial intelligence will be on the agenda. Trump has made his position clear: the United States must not fall behind Beijing. “We’re leading China in AI,” he said Sunday. “Whoever wins AI wins.”
But a growing number of the people actually building these systems are sounding alarms that complicate the picture. Jacob Coxon, a researcher who worked at both OpenAI and Anthropic, resigned from Anthropic last week and told WIRED that colleagues describe the next year or two as “crunch time for humanity.” His concern: competition among AI labs is pushing systems to become more powerful before anyone has figured out how to keep them under reliable human control.
Days later, Anthropic CEO Dario Amodei published an essay titled “We Must Pace the Frontier,” arguing the industry needs to deliberately slow the growth of AI capabilities so safety measures can keep pace. “We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote. He is not calling for a halt to AI development. Instead he wants independent evaluators placed inside frontier labs, coordinated safety standards among democratic nations, and eventually international agreements on the most dangerous capabilities.
What started as one company’s warning quickly picked up support from competitors. OpenAI’s Sam Altman backed the call for independent evaluators. Elon Musk responded with three words: “Dario is right.” Google DeepMind’s Demis Hassabis signaled agreement, and Microsoft CEO Satya Nadella said superintelligence that doesn’t help humanity and doesn’t stay under human control isn’t worth building.
Skepticism Is Warranted, But So Is the Warning
There’s reason to question the motives behind some of this. Anthropic is reportedly preparing an IPO that could value the company at nearly $2 trillion, and Nvidia CEO Jensen Huang has accused parts of the industry of hyping fears while selling the products that supposedly fix them. Talking up how dangerous your technology is can also be a way of talking up how powerful it is. Still, Altman’s decision to push OpenAI’s own IPO back to 2027 over safety concerns doesn’t fit neatly with the theory that this is all marketing.
Treasury Secretary Scott Bessent has laid out the competing pressure bluntly: “We can’t pause. You can’t, because the Chinese won’t pause.” Trump echoed that logic Sunday, dismissing safety warnings as overblown while saying Washington could still add guardrails.
The administration is trying to do both at once. Trump’s Genesis Mission links federal scientific data and supercomputing resources to AI research across more than 15 agencies. A June national security directive pushes faster AI adoption while requiring systems to stay “robust, steerable, controllable” and accountable to humans. A separate executive order sets up voluntary pre-release cybersecurity review of frontier models.
Congress Is Split, Not United
Lawmakers have not settled on a shared approach. House Speaker Mike Johnson rejected an emergency moratorium on AI development, warning that regulatory delay hands China the advantage. Former White House AI adviser David Sacks argued that labs wanting to slow down are free to do so on their own — but shouldn’t use federal power to force smaller rivals to brake with them. Other members of Congress have floated everything from outright bans on superintelligent systems to narrower rules requiring developers to test and disclose catastrophic risks. There is broad agreement in Washington that something significant is happening. There is no agreement on what to do about it.
Beijing’s Role Complicates Any Slowdown
On September 8, U.S. cyber and law enforcement officials accused Chinese AI firms — including DeepSeek, Moonshot AI, and Alibaba — of copying American AI technology “at an industrial scale.” Beijing denies the accusation and describes the practice as a standard industry technique.
China’s Foreign Ministry responded Monday to Amodei’s slowdown proposal by calling it fearmongering and warning that confrontation would disrupt global AI governance. Washington cannot compel Beijing to accept the same limits it might impose on American companies, and Amodei himself has acknowledged that unilaterally slowing U.S. labs could simply hand the advantage to projects tied to the Chinese Communist Party.
Reuters has reported preparations for U.S.-China discussions on AI safety, including AI-directed cyberattacks and incident information-sharing, though the White House says no separate meeting has been formally scheduled.
Any real agreement would run into a hard verification problem. Nuclear arms control could be checked against physical evidence — warheads, missiles, enrichment plants. Frontier AI lives mostly in software, model weights, and computing infrastructure, which is far harder to inspect or confirm.
The September 24 summit puts Trump and Xi in front of a question that goes beyond who builds the most powerful system first. It’s whether either government can actually control what gets built — and that’s no longer just a technical question. It’s a matter of national strategy.
