Anthropic is launching a Program Funding the development of new types of benchmarks capable of evaluating the performance and impact of AI models, including generative models such as our own Cloud,
Anthropic’s program, unveiled Monday, will make payments to third-party organizations that, as the company said in a blog post, “can effectively scale advanced capabilities in AI models.” Interested individuals can submit applications for evaluation on a rolling basis.
“Our investment in these assessments is meant to elevate the entire field of AI safety, providing valuable tools that benefit the entire ecosystem,” Anthropic wrote on its official blog. “Developing high-quality, safety-related assessments remains challenging, and demand is outpacing supply.”
As we have Thrown light on First, AI had a benchmarking problem. The most frequently cited benchmarks for AI today do a poor job of understanding how the average person actually uses the system being tested. There are also questions about whether some benchmarks, particularly those released before the advent of modern generative AI, even measure what they claim, given their age.
The much higher-level, more difficult solution proposed by Anthropic is setting challenging standards by focusing on AI safety and societal implications through new tools, infrastructure, and methods.
The company specifically calls for tests that assess the model’s ability to accomplish tasks such as conducting cyberattacks, “enhancing” weapons of mass destruction (such as nuclear weapons), and manipulating or deceiving people (such as through deepfakes or misinformation). As for AI risks related to national security and defense, Anthropic says it is committed to developing a kind of “early warning system” to identify and assess risks, though it does not explain in the blog post what such a system might involve.
Anthropic has also said that its new program aims to support benchmarks and research into “end-to-end” tasks, examining AI’s potential to assist scientific studies, converse across multiple languages and reduce implicit biases, as well as self-censor toxicity.
To achieve all this, Anthropic envisions a new platform that allows subject-matter experts to develop their own evaluations and large-scale testing of models involving “thousands” of users. The company says it has hired a full-time coordinator for the program and may buy or expand projects it thinks have the potential to scale.
“We offer a variety of funding options based on the needs and stage of each project,” Anthropic wrote in the post, though an Anthropic spokesperson declined to provide any further details about those options. “Teams will have the opportunity to interact directly with Anthropic’s domain experts from the Frontier Red Team, Fine-Tuning, Trust and Security, and other related teams.”
Anthropic’s effort to support the new AI benchmark is admirable – assuming, of course, that it has enough cash and manpower behind it. But given the company’s business ambitions in the AI race, it may be hard to rely on it completely.
In the blog post, Anthropic is quite transparent about the fact that it wants some of the evaluations it funds to be compliant with standards. AI Security Classification it advanced (With some input from third parties like the nonprofit AI research organization METR). This is within the company’s prerogative. But it could force program applicants to accept definitions of “safe” or “risky” AI they may not agree with.
Part of the AI community may also object to Anthropic’s references to “catastrophic” and “deceptive” AI risks, such as the risks of nuclear weapons. Many experts Says there is little evidence that AI as we know it will ever achieve world-ending, human-beating capabilities, if it ever does. Claims of imminent “super intelligence” only serve to distract from the pressing AI regulatory issues of the day, such as AI delusional Experts say these are the trends.
In her post, Anthropic wrote that she hopes her program will “serve as a catalyst for progress toward a future where comprehensive AI evaluation will be an industry standard.” It’s a mission many are aiming for open, Corporate-Unaffiliated Efforts to create better AI benchmarks may be worth identifying with. But it remains to be seen whether those efforts are worth working with an AI vendor whose loyalty ultimately lies with shareholders.
anthropic-looks-to-fund-a-new-more-comprehensive-generation-of-ai-benchmarks-techcrunch
