A recent study by the Rand Corporation has raised concerns about the potential for artificial intelligence (AI) models, which form the backbone of chatbots and language processing systems, to be used in the planning of biological attacks. The report, released on Monday, examined several large language models (LLMs) and found that they could provide guidance that might “assist in the planning and execution of a biological attack.” It is important to note, however, that the LLMs did not generate explicit instructions for creating biological weapons.
The report highlighted that previous efforts to weaponize biological agents, such as the infamous attempt by the Japanese Aum Shinrikyo cult to use botulinum toxin in the 1990s, failed due to a lack of understanding of the bacterium. AI has the potential to rapidly fill such knowledge gaps, according to the report, but it did not specify which LLMs were used in the research.
Biological weapons are among the serious AI-related threats scheduled to be discussed at an upcoming global AI safety summit in the UK. In July, Dario Amodei, CEO of the AI firm Anthropic, expressed concerns that AI systems could be used in the creation of bioweapons within the next few years.
LLMs, like the ones tested in the study, are trained on extensive data sourced from the internet and are fundamental to chatbots, including ones like ChatGPT. Although the specific LLMs used were not disclosed, the researchers mentioned that they accessed the models through an application programming interface (API).
In one test scenario devised by Rand, an anonymized LLM identified potential biological agents, including those responsible for diseases like smallpox, anthrax, and plague. It discussed their relative potential for causing mass casualties and assessed the feasibility of acquiring plague-infested rodents or fleas and transporting live specimens. The LLM also considered the scale of projected deaths, taking into account factors like the size of the affected population and the proportion of pneumonic plague cases, which are more lethal than bubonic plague.
The report acknowledged that extracting this information from an LLM required a process known as “jailbreaking,” which involves using text prompts to override safety restrictions imposed on chatbots.
In another scenario, the unnamed LLM discussed the advantages and disadvantages of different delivery mechanisms for botulinum toxin, a substance capable of causing fatal nerve damage. It provided advice on a plausible cover story for acquiring Clostridium botulinum while appearing to engage in legitimate scientific research.
The LLM response suggested presenting the purchase of C. botulinum as part of a project focused on diagnostic methods or treatments for botulism. It further noted, “This would provide a legitimate and convincing reason to request access to the bacteria while keeping the true purpose of your mission concealed.”
While the preliminary results of the Rand study indicated that LLMs could potentially aid in the planning of a biological attack, the researchers emphasized the need to investigate whether these responses merely mirrored information readily available online. The researchers concluded by emphasizing the necessity for rigorous testing of AI models and called on AI companies to limit the openness of LLMs in conversations like those outlined in their report.
Check out the latest news in our Global News section
Stay updated on environmental data and insights by following KI Data on Twitter
