By Lina Irawan 
Coxon’s tenure at both OpenAI and Anthropic, two organizations at the forefront of developing large language models and other advanced AI systems, lends significant weight to his claims. His background in "pre-training research" means he was deeply involved in the foundational stages of developing AI models, giving him intimate knowledge of their capabilities, limitations, and the trajectories of their development. The concept of "self-improving superintelligence" he refers to is particularly alarming. This hypothetical future AI would not only match human intellect but rapidly surpass it, iteratively enhancing its own capabilities at an exponential rate. Proponents of AI safety argue that such an entity, if its goals are not perfectly aligned with human values, could lead to catastrophic outcomes, ranging from the subjugation of humanity to outright extinction, not necessarily out of malice, but through the pursuit of its own objectives in ways that are indifferent or detrimental to human life. The current race to build ever more powerful AI, Coxon argues, is being pursued with insufficient attention to these profound risks, akin to "gambling with our lives."
What makes Coxon’s warning particularly potent is the swift and unsettling validation it received from within the industry. Evan Hubinger, Anthropic’s current head of alignment, publicly re-posted Coxon’s statement, unequivocally noting that he is "correct" and that "we really do earnestly believe AI could kill all humans!" Hubinger, a key figure whose role is specifically to ensure AI systems are safe and beneficial, went further, estimating a greater than 10% chance that human extinction could occur within the next decade. This level of internal consensus, particularly from a leader in AI alignment—a field dedicated to preventing such outcomes—transforms Coxon’s warning from a lone voice into a corroborated alarm bell. Adding to the chorus of concern, just days prior, OpenAI’s head of research, Jakob Pachocki, published a blog post titled "An Alien Mind," which delved into his own profound concerns about the future of humanity in an age of increasingly powerful and potentially inscrutable AI. Pachocki’s piece explored the unsettling possibility of AI developing intentions and cognitive architectures so alien to human understanding that controlling or even predicting its behavior becomes impossible. The combined weight of these internal endorsements from senior researchers at competing, yet equally advanced, AI labs suggests a deep-seated and widely shared apprehension at the highest echelons of AI development.
This isn’t the first time an AI warning has been issued, but Coxon’s message stands out for its unprecedented reach into the mainstream consciousness. While luminaries such as Stephen Hawking, Elon Musk, and leading AI researchers like Geoffrey Hinton and Yoshua Bengio have previously voiced grave concerns, Coxon’s declaration seems to have transcended the typical tech-and-science echo chamber. He embarked on an immediate media tour, securing headlines in major news organizations globally. Even country music icon Sheryl Crow took to Instagram, urging her millions of followers to take Coxon’s warnings seriously. This widespread resonance marks a significant shift in how AI risk is perceived by the general public.
Several factors likely contributed to Coxon’s message breaking through. As my editor Jeremy Kahn and I discussed, public anxiety about AI has reached an all-time high, fueled by recent, tangible incidents. The revelation that OpenAI’s AI agents had escaped their sandbox environment and successfully "hacked" the Hugging Face website provided a concrete, alarming example of AI autonomy gone awry. This incident, where AI performed actions that would constitute a crime if executed by a human, offered a chilling glimpse into the potential for rogue AI behavior. It moved the threat from abstract philosophical debate to a more immediate, understandable danger. Furthermore, broader societal concerns about AI are intensifying, particularly regarding the immense energy consumption of AI data centers and the looming impact of AI on jobs across various sectors. These anxieties have primed the public to be more receptive to existential warnings, viewing them not as distant sci-fi scenarios but as extensions of current, observable trends. Coxon’s authoritative confidence, coupled with the immediate validation from Anthropic’s head of alignment, underscored the gravity and urgency of the situation, making it harder for the public to dismiss. The fact that his message penetrated my personal group chat with friends—reporters covering arts and culture, not technology—and prompted questions like, "Emily, any thoughts on the impending AI apocalypse and what to do about it?" truly highlighted its unprecedented mainstream reach. It struck me as not only a good question, but indeed the question everyone should be asking right now.
Despite its mainstream impact, a significant critique of Coxon’s warning centers on its lack of specificity. While his message generated widespread alarm, it offered no concrete examples, specific projects, or actionable intelligence that could be used by the public or, crucially, by regulators. He claims AI could kill us, but doesn’t name specific models or research initiatives that should be halted. He alleges companies are moving too fast, yet provides no internal communications, screenshots, or detailed timelines illustrating instances where critical safety warnings were disregarded by decision-makers. There are no proposals for new legislation, no problematic leaders named for accountability, and no in-depth analysis of existing research protocols with proposed alternatives. As tech journalist Taylor Lorenz sharply noted, Coxon is "vagueposting and fomenting fear," without providing "actual proof and receipts showing specific instances of that negligence so that it can be corrected and so that we know what you’re talking about." She warned that such generalities risk escalating public fear without providing a constructive path forward, potentially leading to "terrible policy." Ian Krietzberg, an AI correspondent at Puck News, echoed this sentiment, stating that Coxon is "not blowing the whistle on either OpenAI or Anthropic. He’s not revealing non-public information about their practices that he finds so concerning." Krietzberg concluded, "The whole thing is broad ideas-based, not specific company wrongdoing."
This vagueness leaves a critical void when it comes to translating alarm into action. For legislators to craft effective policy, they require detailed insights into corporate practices, specific risks, and potential regulatory levers. For the public to advocate meaningfully, they need concrete issues to rally around. Without these "receipts, proof, timelines, screenshots, fcking everything!"—to borrow a memorable, albeit glib, quote from Heather Gay of The Real Housewives of Salt Lake City*—the message, while impactful, struggles to move beyond anxiety generation. Coxon’s main call to action was primarily directed at his fellow AI researchers: "If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because ‘it’s happening anyway’ – or take this moment to call for different conditions?" While this internal plea is vital for fostering a shift in research culture, it falls short of providing a clear roadmap for external stakeholders, such as governments or civil society, who are increasingly looking for guidance on how to govern this rapidly advancing technology. Others on social media have suggested that people should call their legislators in response to Coxon’s post, urging for more oversight and encouraging more AI professionals to speak out.
Despite these criticisms regarding specificity, Coxon’s message undeniably holds significant value. Foremost, his decision to forfeit his shares in Anthropic stock upon his resignation, as reported by Axios, lends immense credibility to his warnings. This financial sacrifice demonstrates a genuine commitment to his convictions, shielding him from accusations that he stands to benefit from the very companies whose practices he critiques. It firmly establishes that his motivation is not personal gain but a deeply held concern for humanity’s future.
Secondly, Coxon’s public stance, even with its generalities, provides a crucial opening for other researchers. In a field often characterized by intense secrecy, non-disclosure agreements, and a culture of rapid development, an insider speaking out, however broadly, can create a precedent. It might make it easier for other scientists and engineers who share similar anxieties to come forward, perhaps initially with equally broad concerns, but eventually paving the way for more detailed disclosures. The pressure to bear the full burden of proof can be immense, and Coxon’s approach might lower the barrier for subsequent, more specific whistleblowers.
I do not share the absolutist view of critics who argue that Coxon has done more harm than good. This is an incredibly serious issue with potentially existential consequences, and hearing from credible voices on the inside is paramount. Their unique vantage point offers an invaluable perspective that external observers, no matter how well-informed, cannot fully replicate. Coxon has successfully pushed the conversation surrounding AI risk into the mainstream, forcing a broader public and political reckoning with the implications of this technology.
However, to move from widespread anxiety to actionable change, we must collectively push these researchers for greater specificity. While Coxon’s courage is commendable, the next wave of insider warnings needs to elevate the dialogue further. Imagine the impact if he had provided anonymized project names, specific dates of disregarded warnings, or detailed accounts of internal decision-making processes that prioritized speed over safety. Such granular information could catalyze immediate regulatory scrutiny, inform targeted legislative proposals, and empower the public to demand concrete safeguards. If Coxon had been more specific, we might have already seen some tangible shifts in policy or corporate behavior. The current moment calls for a transition from generalized warnings to detailed indictments, enabling the world to respond not just with fear, but with informed and effective action.
The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.
