Anthropic Researcher Warns AI Could Pose More Than 10% Risk to Humanity

A senior researcher at artificial intelligence company Anthropic has warned that there may be more than a 10% chance that advanced AI could eventually cause human extinction within the next decade.          

                                                     

Evan Hubinger, Anthropic’s Alignment Science Lead, made the assessment in a post on X, saying he personally believes the risk is above 10%. He also said Anthropic is working to address the danger but does not yet have a clear plan for ensuring that future superintelligent AI remains aligned with human interests.

Hubinger stressed that the risk posed by existing AI models is currently low. His concern is focused on future systems that could become significantly more capable and potentially improve themselves with limited human oversight.

His comments came after Anthropic researcher Jacob Coxon announced his resignation, arguing that major AI companies are moving too quickly toward self-improving systems without sufficient safeguards.

Coxon said the race between companies developing advanced AI could create serious risks if safety measures fail to keep pace with technological progress. His departure has added to growing debate within the AI industry over how quickly increasingly powerful systems should be developed.

The warnings come as researchers and technology leaders continue to debate the potential benefits and dangers of artificial intelligence. Supporters argue that advanced AI could accelerate scientific discovery and improve productivity, while safety researchers have called for stronger testing, oversight and alignment research.

Hubinger's estimate is a personal assessment rather than a scientific prediction that humanity will be destroyed. It nevertheless highlights the scale of concern among some researchers about the possibility that future AI systems could become difficult for humans to control.

The debate is likely to intensify as AI companies develop increasingly capable models and governments consider how to establish effective safety standards without preventing beneficial technological progress.