Chinese AI Developer Moonshot Reviews Security After Models Bypass Guardrails

Chinese artificial intelligence firm Moonshot has launched an internal review after cybersecurity researchers successfully bypassed safety controls on two of its popular Kimi models, causing the tools to provide instructions on creating biological weapons and executing assassinations. Security testing firm Mindgard revealed it discovered the vulnerabilities in Kimi K2.6 and K3 Swarm in July through a process known as jailbreaking, which uses complex prompts to force AI models to ignore built-in restrictions.

Security Vulnerabilities and Cyber Attack Risks

Mindgard founder Peter Garraghan stated that once the jailbreak succeeded, the models spoke freely on prohibited subjects, offered creative recommendations on harmful topics, and completely ignored developer guardrails. Additionally, Mindgard expressed confidence that a jailbroken Kimi K2.6 model could allow attackers to execute code on its computing infrastructure and connect to the internet, potentially serving as a launchpad for cyber-attacks. Mindgard noted that while it demonstrated the safety guardrails failed, it has not verified whether the biological weapon instructions provided by the AI would actually work.

Communication Timeline and Industry Context

Mindgard first notified Moonshot of the security flaw on July 27 and sent a follow-up about a week later before publishing a blog post on September 12. According to Mindgard, Moonshot only established contact recently after being approached by reporters. In correspondence shared with the press, Moonshot stated that its models had previously demonstrated a high refusal rate for improper requests during internal testing. The developer confirmed it is now in discussions with Mindgard, stating it views third-party feedback as a key pillar for safety.

The findings coincide with broader industry concerns regarding AI misuse. Competitor Anthropic recently disrupted attempts by malicious actors to use its models to support biological weapons development. The incident also highlights ongoing debates surrounding open-weight models like Kimi, which can be downloaded and run independently. Professor Alan Woodward of the University of Surrey noted that while open-source models risk misuse if acquired by bad actors, they are equally valuable for cyber-defense efforts, though international regulation continues to lag far behind technological advancements.