NTI | bio and Concordia AI, with technical input from SecureBio, have launched the AIxBio Technical Working Group on Evaluation Practice. Convened through the AIxBio Global Forum, the Working Group will bring together experts from organizations in China, the European Union, the United Kingdom, and the United States that develop or conduct biological capability evaluations of AI systems.
Biological capability evaluations increasingly inform frontier AI risk management by assessing capabilities that could lower barriers to biological misuse. Model developers use the findings when deciding whether to strengthen safeguards, restrict access, increase monitoring, or change deployment plans.
As these evaluations become more widely used, findings produced in one setting may inform risk assessments and related decisions in another, even when evaluation methods and reporting practices differ. Results reflect not only the capabilities of the model being evaluated, but how those capabilities are tested. Without consistent reporting, differences arising from system configuration or evaluation design may be mistakenly attributed to the underlying model itself.
The Working Group will compare published evaluation approaches across jurisdictions and institutional settings to establish a common technical basis for designing evaluations and interpreting their results. Building on this analysis, it will develop shared terminology and a minimum reporting baseline for evaluation results. Its work will:
Methodological findings, voluntary reporting conventions, and research priorities will be published to support more consistent interpretation and use of evaluation evidence by AI developers, governments, and third-party evaluators.
By bringing together experts who conduct this work to examine assessment design and comparability, the group will complement the AIxBio Forum’s Working Group on Horizon Scanning, Risk Assessment, and Evaluations.
Organizations and individuals working on biological capability evaluations are invited to contact Greg Butchello, at [email protected] to discuss participation or share related work.
Sign up for our newsletter to get the latest on nuclear and biological threats.
New YouGov polling commissioned by NTI and ReThink Media finds that nearly 80 percent of registered voters, across party lines, say it is in the interest of U.S. national security to work with China to mitigate catastrophic threats associated with nuclear risks, AI development, and bioterrorism.
More than 35 leading experts highlight the risks posed by rapidly advancing capabilities at the convergence of AI and the life sciences and call on governments, industry, the scientific community, and funders to take action to safeguard this technology.
To reduce biosecurity risks that arise at the intersection of AI and the life sciences, NTI | bio convened experts in synthetic biology, machine learning, bioinformatics, and international security policy to outline steps toward establishing an international AI-Bio Forum.
Sign up for regular updates on innovative, real-world solutions to existential threats.
