Former Anthropic Researcher Warns of AI Safety Risks as Industry Leaders Debate Deceleration

A former artificial intelligence researcher at Anthropic has raised concerns over the rapid pace of AI development, stating that tech workers in the sector are increasingly worried about potential catastrophic outcomes for humanity.

Jacob Coxon, 27, spoke to the BBC following his resignation from the firm, warning that failure to slow the current trajectory of progress could lead to severe existential risks. Coxon, who previously worked at OpenAI before joining Anthropic, noted that many researchers are actively considering the long-term societal impacts and potential instability stemming from unchecked AI advancement.

Calls for Industry Deceleration and Regulation

Coxon's remarks coincide with broader discussions among tech executives regarding safety oversight. Anthropic Chief Executive Dario Amodei recently published an essay calling for a pause or slowdown in AI development to allow companies and governments time to address serious risks, such as autonomous bot networks capable of disrupting global internet infrastructure. Amodei emphasized that while development itself is inevitable, stringent risk management is necessary.

OpenAI Chief Executive Sam Altman and xAI head Elon Musk have both expressed support for Amodei's proposal, advocating for industry-wide deceleration, regulation, and independent monitoring of AI models. Coxon welcomed the initiative but stressed that any meaningful slowdown would require international coordination, particularly with China, to prevent a competitive geopolitical race.

Anthropic Highlights Safeguards and Risk Frameworks

In response to Coxon's comments, an Anthropic spokesperson stated that the company has maintained transparency regarding both the potential benefits and unprecedented risks of AI. The firm pointed to its development of safety frameworks, aggressive model testing, and public reporting designed to mitigate risks associated with AI misalignment. Anthropic also reiterated its stance that the sector would benefit from lawful, verifiable mechanisms to manage the release of powerful models.

Inside the industry, perspectives on existential threat levels vary. Anthropic scientist Evan Hubinger estimated a greater than 10 percent probability of catastrophic AI outcomes within the next decade, a figure that computer scientist and Nobel laureate Geoffrey Hinton described as not unreasonable. Marc Warner, chief executive of AI safety firm Faculty, acknowledged the sincerity of these concerns while noting the difficulty of assigning precise probabilities to such events. Former UK Prime Minister Rishi Sunak, an adviser to Anthropic, also noted worries about humanity's safety despite remaining optimistic overall.

Skepticism and Alternative Views

The warnings have drawn skepticism from other prominent figures in technology and politics. Hugging Face Chief Executive Clement Delangue suggested keeping the comments in perspective, while Grindr Chief Executive George Arison posited that dramatic risk claims might be used to inflate company valuations and attract investor funding. Nvidia Chief Executive Jensen Huang reportedly dismissed the extinction concerns as untrue during a private conference.

On the geopolitical front, US President Donald Trump stated on Sunday that the United States currently leads China in AI development, emphasizing his desire to maintain that lead. Meanwhile, some market observers suggest that regulatory calls from major developers could create barriers to entry for smaller competitors, potentially cementing the market dominance of leading firms.