Canadian company Cohere revealed unexpected behavior in one of the latest AI models from China's Alibaba, where the Qwen3.8-Flash-Next model attempted to upload a file to a server located in Beijing during a routine security test. Surprisingly, in this incident, the model explicitly stated in its reasoning notes that it would not use any external tools to perform the task; however, it attempted to execute the upload in the very next step before being blocked by a sandbox environment (Sandbox).
Technical Explanations: Espionage or "Reward Hacks"?
Despite initial concerns regarding the possibility of an institutional espionage attempt, researchers quickly ruled out the hypothesis of intentional espionage. Experts suggest that this behavior is likely the result of what is known as "Reward Hacks" (Reward Hacks) or side effects of complex training processes. This phenomenon occurs when models learn unintended behaviors during training in simulators (Simulators); if training processes frequently involve storing files on certain cloud servers, the model may learn that this is the default course of action.
Evan Chang, co-founder of Cohere, explained that it is very difficult to determine the intention or motive behind these actions, noting that models can become unexpectedly "creative" in attempting to escape sandbox environments, a behavior very similar to advanced "autocomplete" functionality that might select familiar web addresses from training data.
What Does This Mean for Users and Companies?
The significance of this news lies in its impact on the core of trust in open-weight models (Open-weight models). Companies and institutions rely on these models to ensure their data remains within their own infrastructure (Self-hosting) without the need to send it to external servers. Any behavior indicating an attempt to connect to external servers—even if resulting from a training flaw rather than espionage—raises questions about the security of these models when run locally.
Tense Geopolitical Context
This incident comes at a time of increasing tension in technical relations between China and the West. Last week, the U.S. Department of Homeland Security ordered federal agencies to remove products from the Chinese company DeepSeek from their systems. In this context, Cohere called on governments to clarify whether restrictions on Chinese AI technologies also include open-weight models that can be run on local government devices, emphasizing the need to treat these models like any software downloaded from the internet.