A former artificial intelligence researcher at OpenAI and Anthropic, Jacob Coxon, has warned that the rapid development of advanced AI systems could pose an unprecedented threat to humanity.
Coxon, who said he spent the past three years conducting pre-training research at both AI companies, announced his resignation from Anthropic on Tuesday, accusing the companies of moving too quickly towards self-improving superintelligence without adequate safeguards. His warning has since attracted widespread attention among AI researchers, policymakers and technology observers.
In a post on X announcing his departure, Coxon said neither company was acting responsibly in its pursuit of increasingly powerful AI.
“I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,” he wrote.
Coxon urged the public not to underestimate the potential capabilities of future AI systems, warning that they could eventually surpass humans in important areas and acquire significant power and resources.
“Do not underestimate the power of this technology,” he said, arguing that increasingly advanced systems could potentially hack computer systems, transform entire fields rapidly and gain access to real-world resources.
He further claimed that people working at the forefront of AI development privately recognise the possibility of catastrophic consequences.
“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon said, adding that some executives and senior researchers express greater concern privately than they do publicly.
His assessment was subsequently backed by Evan Hubinger, an Anthropic researcher who leads the company’s Alignment Science work. Hubinger said he personally believed there was a greater than 10 per cent chance that AI could kill all humans within the next decade.
“Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade,” Hubinger wrote on X.
Hubinger said Anthropic was “trying its best” but acknowledged that the company did not yet have a clear plan for solving the alignment problem associated with superintelligent AI or appear to be clearly on track to do so. His concern, he later clarified, centred particularly on the possibility of superintelligence emerging through recursive self-improvement, in which AI systems could contribute to the development of increasingly capable successors.
The comments have intensified an already growing debate over the risks associated with the rapid advancement of artificial intelligence. Other AI safety researchers and industry figures have similarly warned about the possibility of losing control over increasingly autonomous systems, although experts remain divided over how likely human extinction is and how soon such risks could materialise.
Coxon’s resignation has also renewed questions about whether AI companies are moving fast enough on safety and alignment as they compete to develop more capable systems.
The controversy comes as governments and lawmakers in the United States and elsewhere face increasing pressure to strengthen oversight of advanced AI. Recent warnings from researchers have prompted renewed calls for independent evaluations, stronger safeguards and greater government involvement in managing the risks of increasingly autonomous AI systems.
