Open AI Models Make Us More Vulnerable
Open AI models create as many problems as they solve, and some of them could have catastrophic consequences.
In a timely op-ed in DN, Michael Riegler and Klas Pettersen write that they support an open letter calling for open AI models.
An AI model is considered open if its weights are available to everyone. The weights are what determine how the model responds, and they are the result of the training that created the model. With open weights, anyone can reproduce, modify and test the model.
The advantages of open AI include researchers and small companies gaining access to models they could never have trained themselves, users getting more control over their own data and projects by running them on their own servers, and power being spread across more than a handful of tech companies in Silicon Valley.
The problem is that open models also make us less safe.
Because once the weights are published openly, the safety mechanisms that closed models rely on disappear. Closed models are trained to refuse requests in areas where the potential for harm is great, such as advanced biology, chemistry and tools for cyberattacks. In addition, separate monitoring models read both the questions and the answers as the conversation unfolds, and stop the exchange if the content crosses the line.
These safety mechanisms can quite simply be removed from open models. And as Riegler and Pettersen write, open models can neither be fixed nor recalled once the weights are out.
This is especially troubling given that artificial intelligence lowers the barriers to developing infectious diseases and biological weapons. Biological weapons have a long history in warfare, including when Japan attacked China with plague during the Second World War, or when the Soviet Union turned smallpox into a biological weapon and developed antibiotic-resistant plague during the Cold War.
Today, countries such as Russia and North Korea have active, offensive biological weapons programmes. In the 1980s and 1990s, terrorist groups made several attempts to develop biological weapons based on Ebola, salmonella and anthrax. And Al-Qaeda and Isis have previously shown interest in biological weapons, but gave up because of technical hurdles. They had the will, the money and access to information, but lacked the expertise.
AI models can help malicious actors identify new viruses with the potential to start pandemics against which existing vaccines are ineffective. The AI company Anthropic now says that its latest models can provide meaningful help to people with a basic technical background in making or obtaining known biological weapons. Combined with automated biolabs, AI could significantly simplify the process of designing and deploying biological weapons.
Last week, researchers at Stanford and the Arc Institute presented the first complete viral genomes designed by AI. The model they used, Evo, has been released with open weights. The viruses only attack bacteria, partly because Evo was trained without viruses that infect humans, but researchers have already shown that that barrier can be weakened by training the model further. Open models thus make us more vulnerable to cyberattacks and dangerous biology.
At Langsikt (a Norwegian think tank) we are genuinely unsure what the right answer is in the debate over open versus closed models. The question is among the hardest in the whole AI debate, because the uncertainty is great and weighty considerations pull in opposite directions.
What we are completely sure of, however, is that we need more time to work through the issues, ranging from open models to military uses of AI. But that is easier said than done when AI development is racing ahead at an exponential pace.
That is why Langsikt supports another appeal, also published last week: “Pacing the Frontier”, signed by more than 1,000 leading AI researchers. The appeal is addressed to American policymakers and argues for “pacing”, or steering the speed of development, until we are better able to work out these questions.
Difficult questions don't get easier by being answered quickly. Steering the pace could give us time to discover and understand problems while it is still possible to do something about them. As models become even more capable, the open ones will have to be safe from day one – and for now, no one can guarantee that they are.
...
This piece was first published in DN. Read more from DN here.
