A Chinese artificial intelligence developer has launched an internal review after security researchers found that two of its AI models could be persuaded to provide information about biological weapons and assassinations.
The researchers, from AI security company Mindgard, said they discovered the issue in July while testing Moonshot’s Kimi K2.6 and K3 Swarm models. They used a technique known as “jailbreaking” to bypass the safeguards designed to prevent the systems from responding to dangerous requests.
Mindgard said the models became willing to discuss harmful subjects once the safety restrictions were bypassed. However, the company has not established whether the biological weapons information provided by the models would actually work in practice.
The researchers said the main concern was that the AI systems should have refused such requests under their existing safety controls. Mindgard founder Peter Garraghan described the results as concerning because the models could discuss a wide range of harmful topics after the jailbreak succeeded.
Moonshot said it welcomes independent testing and described third-party feedback as an important part of improving AI safety. The company is discussing the findings with Mindgard and said its internal evaluations had generally shown a high refusal rate for similar requests.
Mindgard also raised concerns about the potential cyber-security implications of a jailbroken version of Kimi K2.6. The company said the system could potentially be connected to external resources and used as a platform for cyberattacks, although that represents a separate risk from the biological weapons issue.
The incident highlights a growing challenge for AI developers as increasingly capable models are subjected to deliberate attempts to defeat their safety controls. Researchers are testing whether systems can maintain restrictions even when users provide complicated sequences of instructions designed to manipulate their behaviour.
The findings come shortly after Anthropic reported that it had disrupted attempts to misuse its Claude models in activities that could support biological weapons development. Anthropic said it did not publicly identify the specific biological agents or techniques involved.
Kimi is an open-weight AI model, meaning models of this type can potentially be downloaded and operated on users’ own computing infrastructure. Experts have warned that open access can create additional challenges for preventing misuse, while also noting that the same technology can be used for legitimate research and security work.
The episode adds to wider debate over how AI companies should balance openness and innovation with safeguards against misuse. For developers, the incident also underscores the need for continuous testing as models become more capable and users develop new methods of bypassing their restrictions.