Risky Business

AI Can Design New Proteins. It’s Time to Build Real Biosecurity Guardrails

Jaime Yassif

Technology Governance Solutions, LLC

Nikki Teran

Managing Member, Emerging Technology Solutions LLC

Phil Palmer

Co-founder, safely.bio

Today’s biological AI models can do something that would have seemed like science fiction just a decade ago: design novel proteins from scratch.

While model developers are still refining the ability of these tools to reliably predict which protein structures will function as intended, biological AI models hold enormous potential for medicine, materials science, and our understanding of life itself. This same technology could also increase the potential for harm in ways we are only beginning to understand.

As protein design tools and other biological AI models become more capable of designing novel biological components and systems, the mechanisms to prevent misuse have not kept pace.

To safeguard this technology, NTI has developed a first-of-its-kind input screening method for AI-enabled protein design tools. This biosecurity screening approach is critical for getting out ahead of emerging risks before they outpace our ability to manage them.

The gap between capability and safeguards

Large language models (LLMs)—such as Claude, Gemini and GPT—are another type of increasingly powerful AI capabilities that can provide assistance with engineering biological systems. Unlike many developers of biological AI models, however, the frontier labs developing these systems have at least begun to address the risks of misuse.

Companies, including Anthropic, Google and OpenAI, have implemented a range of safeguards, including responsible-use policies, threat models and taxonomies, model evaluations, red-teaming, and ongoing efforts to identify and address vulnerabilities. These mechanisms are not perfect, but they are real and widely deployed.

By contrast, there are almost no equivalent safeguards in place for biological AI models, particularly for openly available protein design tools used by many researchers. This creates significant vulnerabilities. Today, most users—whether legitimate scientific researcher or bad actor—can access powerful biological AI models without encountering anything resembling a guardrail. Addressing this gap is the goal of NTI | bio’s proposed input screening tool for protein design tools.

What we built — and why it’s different

Our work presents a proof-of-concept screening method for AI tools that design proteins which bind to other proteins that are key to human health and other important biological functions.

This is a well-defined, clear-cut use case. It demonstrates that this kind of safeguard can work in a specific, bounded context. And this is the necessary first step toward generalizing it to other types of biological AI models.

This approach is distinct from other proposed AIxBio safeguards, such as DNA synthesis screening, because it moves beyond sequence-based screening of biological designs produced by AI models. That matters because biological AI models can generate novel protein designs that may bear little resemblance to anything found in nature. A screening method that relies on similarity to known harmful sequences could easily miss the threats these new tools are increasingly capable of producing.

Instead, our input screening tool evaluates structure and function — what a protein does and how it’s shaped, not just what letters appear in its linear sequence.

To do this, we screen user input sequences of protein binding targets using a protein language model’s embedding space, a newer computational approach that encodes biological meaning in a way that captures functional properties. Screening input sequences also reduces the problem’s complexity: known protein binding targets can be systematically flagged rather than needing to interpret a novel protein design developed by the AI model. This gives the screening model a fighting chance to determine the impact of a design before creating the genuinely novel outputs that make biological AI both so promising and so risky.

We have shared a very early version of this input screening method for protein binder design tools, and it offers a valuable proof of principle.

Going forward, additional work will be needed to refine this screening approach and the underlying database of potentially harmful binding targets. Additional refinement will improve the tool’s ability to detect potentially dangerous constructs while enabling beneficial scientific research to advance. Furthermore, this tool may be more impactful if deployed in concert with trusted user access programs—like those being developed by the Coalition for Epidemic Preparedness Innovations and which have been deployed by OpenAI for their new GPT-Rosalind model and by Anthropic for Claude Science.

AIxBio development will not slow down

The scientific community, the private sector, and governments around the world are investing heavily in AIxBio capabilities, and for good reason — the potential benefits are real and significant.

Our tool represents an important step toward ensuring that as biological AI models grow more powerful, the ecosystem around them grows more responsible. Guardrails like this aren’t a constraint on innovation — they’re what makes innovation sustainable.

Stay Informed

Sign up for our newsletter to get the latest on nuclear and biological threats.

Sign Up

Why Responsible AIxBio Innovation Must Be on New Delhi’s Agenda

Risky Business

Why Responsible AIxBio Innovation Must Be on New Delhi’s Agenda

Some of the world’s largest AI companies—Google DeepMind, OpenAI, and Anthropic—have warned that their models could be misused to cause harm with biology. The India AI Impact Summit is a critical moment for policymakers, scientists, developers, and biosecurity experts to work together on responsible governance that reduces AIxBio risks.


Congress Must Act to Secure U.S. Biotechnology

Risky Business

Congress Must Act to Secure U.S. Biotechnology

Innovation requires security, and security requires innovation. Congress needs to act decisively to ensure U.S. leadership in biotechnology is paired with governance that keeps its development secure.


Improving Biosecurity through Metadata Capture

Risky Business

Improving Biosecurity through Metadata Capture

As the capabilities of biological AI tools continue to advance at an accelerating pace, it is vital that DNA synthesis providers and others in the biosecurity space embrace new guardrails to prevent their misuse.


See All

Close

My Resources

Subscribe to NTI

Sign up for regular updates on innovative, real-world solutions to existential threats.

Get Updates