Top AI researchers call for oversight of automated model development

Jack Clark: Jack Clark is a co-founder of Anthropic and author of the Import AI newsletter, focusing on AI research trends and implications. He has recently highlighted evidence of recursive self-improvement risks and the need for policy preparation in public statements. As a co-author, he supports the paper's recommendation that governments demand clearer insights into internal automation of model development at AI companies. Yoshua Bengio: Yoshua Bengio is a prominent AI pioneer and co-chair of the UN’s Independent International Scientific Panel on AI. He recently addressed the UN Security Council, urging international licensing and oversight mechanisms for frontier AI systems due to risks of catastrophic misuse and loss of control. His participation in the paper reinforces the authors' push for greater visibility into automated AI R&D within leading labs. Jakub Pachocki: Jakub Pachocki serves as chief scientist at OpenAI, where he oversees key aspects of advanced AI model development. In recent weeks, he has publicly warned about the potential for AI systems to accelerate their own capabilities through recursive self-improvement and alignment challenges. As a co-author of the new paper, he contributes to calls for greater policy visibility into how labs are automating their internal R&D processes. Geoffrey Hinton: Geoffrey Hinton is a leading AI researcher known as the 'Godfather of AI' and a Nobel Prize recipient in physics. He has recently briefed U.S. Senate members on the urgent need for regulatory guardrails, emphasizing that AI has reached the stage of designing better AI systems. In the context of this paper, his involvement underscores the call for policymakers to seek more transparency on companies' progress toward recursive self-improvement. Regulation: Prominent AI researchers have recently briefed lawmakers and addressed international bodies on the need for oversight of systems capable of self-improvement. Transparency: The new paper emphasizes that policymakers require greater visibility into labs' internal progress on automating AI research and development. Intelligence Explosion: Recent public warnings from AI leaders describe the risks of rapid capability acceleration through automated R&D processes.