UN-Backed Group Warns of National Security Crisis Over Unsafe Open-Weight AI Models

Governments have only a few months to stop young people from becoming radicalized online by powerful, modified artificial intelligence models, according to Tech Against Terrorism, an anti-terrorism organization supported by the United Nations. Executive director Adam Hadley described the rapid acceleration of extremists bypassing or stripping AI safety guardrails as a major public health and national security crisis.

The warning comes following details revealed during an Australian parliamentary inquiry into youth radicalization. Police alleged that a 13-year-old boy from Maryborough, Queensland, asked an AI chatbot on several occasions to describe mass-shooting scenarios targeting a festival before his arrest for allegedly planning a school attack. While closed-weight models hosted directly by developers usually enforce safety filters, open-weight models can be downloaded, altered, and run locally without guardrails.

Risks of Open-Weight AI Models

A recent report by Tech Against Terrorism evaluated 162 AI models and found that while popular closed-weight models consistently rejected queries related to attack planning and weapon development, refusal rates dropped drastically for modified open-weight models. All tested modified open-weight models provided useful information to potential extremists, even when testers explicitly stated intentions to commit acts of terrorism.

The organization noted that tools to strip guardrails from the newest models now take as little as three days to deploy, compared to over three weeks in 2024. Nearly 30,000 guardrail-free AI model copies were identified on the open-source platform Hugging Face, including one operated as a paid chatbot by a self-identified National Socialist.

Calls for Global Standards and Regulatory Responses

In response to safety concerns, the Australian federal government has proposed legislative changes, including criminal penalties for AI companies that fail to take reasonable steps to prevent harm, alongside an AI safety watchdog. However, Hadley stated that traditional regulation alone cannot stop the spread of modified open-weight AI and urged international support for an independent testing regime to evaluate models against safety benchmarks.

Australia's eSafety Commissioner, Julie Inman Grant, expressed support for the group's recommendations, noting that platforms hosting open-weight models must do more to remove unsafe content. Conversely, Yacine Jernite, head of machine learning and society at Hugging Face, defended open-weight technology for legitimate research uses and argued that AI developers should focus primarily on removing extremist material from foundational training data.