More than 100 AI experts and evaluators have signed a public letter urging foundation model providers like Anthropic and OpenAI to grant third-party evaluators the autonomy and protections needed to assess AI safety risks independently.
The letter, published Friday and shared exclusively with CNBC, was organized by the AI Evaluator Forum, a consortium including luminaries such as Geoffrey Hinton and representatives from Johns Hopkins University, Stanford University, and METR. Signatories argue that evaluators require scientific objectivity, transparency, independence, and robust protections to conduct credible oversight of frontier AI models.
The push follows Anthropic CEO Dario Amodei’s proposal over the weekend to grant some evaluators employee-like access to inspect and audit cutting-edge models and their development processes. OpenAI CEO Sam Altman has also signaled support for third-party oversight, aligning with broader industry calls for regulatory frameworks to mitigate AI risks.
Anthropic outlines development metrics to track AI progress
In a separate development, Anthropic announced three new metrics to monitor AI development pace, days after Amodei publicly advocated for a coordinated slowdown in AI advancement. The company’s blog post outlined methodologies for measuring AI-led research, oversight of AI agents, and compute allocation, aiming to enhance transparency.
Anthropic stated its goal is to "minimize the gap between what frontier labs know and what the public knows" by publicly reporting these metrics. The metrics include assessments of whether Claude models operate autonomously, the number of AI agents performing research (approximately 30,000 on its most-used platform), and compute resource allocation across projects.
Amodei’s slowdown proposal has garnered support from industry leaders, including Elon Musk, Sam Altman, and Demis Hassabis, amid growing concerns about AI’s potential harms. The plan seeks to temper model capability improvements without sacrificing commercial or geopolitical advantages.
Key demands from evaluators
The AI Evaluator Forum’s letter specifies minimum conditions for third-party oversight, including:
- Unfiltered communication with company boards and oversight bodies.
- Public release of findings by evaluators.
- Protection from retaliation by AI labs.
- Access to the same systems, data, and tools as internal assessors.
- Diverse expertise to evaluate risks such as biological threats or loss of control.
The consortium includes METR, which recently investigated a security incident involving OpenAI and Hugging Face, and AVERI, led by former OpenAI researcher Miles Brundage. The letter follows industry pledges to support more rigorous third-party safety testing.
Industry and policy responses
While some leaders advocate for government regulation, others, including former AI czar David Sacks under the Trump administration, have opposed such measures. The debate underscores tensions between industry self-regulation and external oversight, with calls for accountability growing alongside AI advancements.