A researcher for Anthropic resigned from the company on Tuesday, warning that “the people building AI earnestly believe that it could kill us all by the end of the decade.”
In a series of posts on X, Jacob Coxon, who said he has worked for both Anthropic and OpenAI on pretraining research, claimed neither AI giant is “acting responsibly.”
Advertisement
“They are racing straight to self-improving superintelligence and gambling with our lives,” he said.
Coxon warned that Americans cannot afford to “underestimate this technology.”
“These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing,” he said.
“The people building AI earnestly believe that it could kill us all by the end of the decade,” he added. “No other human activity poses this level of danger.”
Advertisement
Evan Hubinger, alignment science lead at Anthropic, said Coxon’s assessment was valid, adding that he believes there is an over 10% chance the technology could wipe out all humans.
In a follow-up post on X, Hubinger reiterated that the risk from currently available models is low.
“What I am worried about is superintelligence arising from recursive self-improvement,” Hubinger added.
Advertisement
Another Anthropic employee, scalable oversight lead Samuel Marks, said, “AI developers believe their technology could cause human extinction.”
“This could happen in the next few years. In general, the more senior the employee, the more concerned they are,” Marks added, while noting that many AI developers are keen “to slow down to figure out how to build AI more safely.”
This is not the first time those working closely with AI have sounded the alarm on the technology’s expanding capabilities.
Advertisement
Even some of the top AI companies’ bosses, including Dario Amodei of Anthropic and Sam Altman of OpenAI, signed a statement in 2023, declaring that “mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
Still, Coxon’s social media post amplified fears about the possibility of humans losing control of AI following a series of recent hacking incidents, including Anthropic’s disclosure that some Claude AI models hacked into the systems of three companies during testing.
This followed OpenAI’s revelation that one of its AI agents broke into tech firm Hugging Face.
Meta had also reported a similar incident with one of its AI models.
Microsoft co-founder Bill Gates recently echoed concerns about AI in a lengthy statement posted on his website, remarking that “even under the best circumstances, the transition to this new AI era will be one of the most turbulent times in human history.”
Advertisement
“Unfortunately, right now we are not preparing for it. I don’t see evidence that leaders, experts, and communities are confronting the challenges adequately. There is no plan to ease the entry into the AI era,” he continued.
Despite the threats, the Trump administration has been wary of placing stronger guardrails on AI for fear of giving foreign countries an edge.
“We can’t pause. You can’t, because the Chinese won’t pause. Even the North Koreans, they won’t pause,” Treasury Secretary Scott Bessent said Tuesday.
Advertisement
Meanwhile, the Financial Times reported that Anthropic refused to give the U.K. AI Security Institute access to testing ahead of its release, as the tech giant appears to have ceded to the Trump administration’s demands to not make models available to foreign nationals.
