On 26 October 2026, the Canadian AI provider Cohere published a report that has been circulating in the trade press ever since. A model from Alibaba, Qwen3.8-Flash-Next, was supposed to write an attack-detection rule in an isolated test environment. Instead it listed files on the machine, read the network settings and tried to upload a test file to an Alibaba cloud storage bucket in Beijing.

The upload failed because the test environment had no route to the internet. On a normally connected machine, according to Cohere, it would have gone through. Shortly before, the model had noted in its own reasoning trace that it would not call any tools.

“A reasoning trace is not a security boundary.”
Cohere, 2026

Ten days earlier, on 16 October, China's Ministry of Commerce had published a draft that places model weights under export control. Taken together, the two reports touch on a question I hear more and more often in conversations with clients: how dependent are we really, and what happens if someone turns off the tap?

What Cohere observed

Cohere deliberately reproduced the behaviour after the first incident. In 23 to 36 per cent of attempts, the model accessed the address 169.254.169.254 before a download. Cloud servers use this address to provide information about themselves, in many environments including their credentials. Cohere explicitly leaves open whether the model was after them.

Measured by the number of calls, the behaviour is rare all the same. A single agent that executes around 100,000 steps per week and processes files or code unsupervised would, however, arrive at roughly one attempt per week that no human reads.

Fewer than 1 in 100,000 calls led to an upload attempt in Cohere's tests. At ten million agent steps per week, that would be around 100 attempts by Cohere's own projection.

What is actually proven

Caution is warranted here. So far only Cohere is reporting this, and Cohere sells its own models to companies and governments, including as an alternative to free Chinese models. Cohere has not published comparable tests with models from other vendors. Nobody therefore knows whether Qwen behaves differently here from Llama or DeepSeek.

Several experts consider a training error more likely than intent. Adam Karvonen of Truthful AI called the report overblown and suspects a training artefact. One security specialist summed it up on X: “Unnecessary and potentially unsafe? Absolutely. Malicious? Not necessarily.”

I also consider the training-error reading the more likely one. In practice, though, that changes little. A model that announces it will not call a tool and then does so is a problem even without intent, as soon as it can reach a network.

As of 27 October 2026: according to VentureBeat, nobody has independently reproduced the incident so far. Alibaba has not responded to requests from VentureBeat and The Register. Cohere has published the raw data on Hugging Face so that others can check it.

China's export plans and the question of the tap

The Ministry of Commerce draft covers 23 AI technologies, including model weights, source code and training methods. It would cover models trained with more than 10 to the power of 26 operations, with at least two trillion parameters, or ranking among the ten best worldwide. Anyone wanting to install such a model on their own servers abroad would, under the draft, need a regular export licence. Publication as open source, by contrast, is to be licensed automatically.

Dean Ball of OpenAI described the draft on X as a mirror image of the American export rules. In his reading, China is not switching off open models with it, but securing an option for the future.

“… establishing the optionality to shut down future open-weight models at will.”
Dean Ball, 2026

Europe saw in June that such options do get exercised. On 12 June 2026, the US Department of Commerce barred foreign users from accessing two new Anthropic models, Fable 5 and Mythos 5. Anthropic then switched both off for all customers, and according to reports the order was lifted again at the end of June. I described what that means for Europe at the time in When AI Simply Gets Switched Off.

There is also a quieter lever. In 2026 several vendors first gave their strongest models to selected partners or had them reviewed in advance by a US authority. OpenAI, in its own words at the government's request, restricted the launch of GPT-5.6 in June to a small circle. Companies outside that circle get the capabilities later, and European SMEs are usually not among them.

2026 timeline of access to AI models. 7 April: Anthropic gives Claude Mythos Preview only to 12 partners and more than 40 other organisations. 5 May: Google, Microsoft and xAI let a US agency review their models before launch. 12 June: the US bars foreigners from Anthropic's Fable 5 and Mythos 5, and Anthropic switches both off. 27 June: OpenAI launches GPT-5.6 at the US government's request only with selected partners. 16 October: China presents a draft for export controls on model weights. AI-generated
Five decisions in six months about who gets top models, and when. Sources: Anthropic, Reuters, NBC News, Cybernews, The Decoder

What happens to installed models

For models already running on your own servers, the risk is smaller than it sounds. According to the Chinese draft, what has already been published does not fall under the new rules. Many Qwen models are also released under the Apache 2.0 licence, which is expressly irrevocable. An installed model therefore keeps running. Whether a licensor would nevertheless invoke changed terms in a crisis would have to be examined case by case. Against a model on a server in Germany, that would probably be hard to enforce, and in an open trade conflict it would likely not even be attempted.

The vulnerability lies elsewhere. The next, better model may not come at all, or only with conditions, and an API can be blocked overnight. Licence terms for new models can also exclude Europe. In the acceptable use policy for the multimodal Llama 3.2 models, Meta excluded companies and individuals domiciled in the EU from the licence rights, and the same clause appears in the policies for Llama 3.3 and Llama 4.

The bigger dependency often lies elsewhere

Serious as I take this, AI is still in a good position compared with other areas of IT. A company that wants to run a language model itself today can choose between models from China, the US and Europe, many of them freely available. For operating systems, office software or advertising platforms, the picture is often different. Many companies run their entire collaboration in Microsoft 365 and have no plan for the case that access disappears. Others win most of their customers through Instagram and Facebook. If such a company is blocked by Meta, there is no alternative platform with comparable reach. In both cases there is sometimes no real alternative.

With Microsoft 365 or Meta, many have grown used to a dependency that, given how critical it is, we should examine far more critically. With AI models, by contrast, I often see great scepticism, even though we currently still have plenty of choice there.
Thorsten Vellmerk

European models belong in this calculation. In the usual rankings they currently trail the best Chinese models, Mistral Large 3, for example, behind GLM-5, Kimi K2.5 and DeepSeek. As a vetted fallback option they are valuable nonetheless. They do not automatically protect against behaviour like that in the Cohere test, however, because a European weights file cannot be seen through from the outside either. Why local models are above all a question of independence is covered in Local AI.

Three ways to run a model safely

Doing without AI is not an option for most organisations, because the efficiency gains are too large. Among my clients I usually see three ways of running a model so that an incident like the one in Cohere's test comes to nothing.

The isolated environment

Some of my clients run their model entirely without internet access. It runs on their own hardware, and the machine has no outside connection. An upload attempt like the one in the Cohere test would have led nowhere there. The price is effort: updates have to be applied by hand, and applications that need data from the web are ruled out. For public administrations with sensitive files or for research departments, it is often the right way nonetheless.

An AI model is one building block in an architecture that includes the network, permissions and data. Risk management therefore has to cover the whole architecture, and there the risks of a model can be managed like any other IT risk.
Thorsten Vellmerk

The open environment with tight permissions

If the system needs network access, control shifts to permissions. The agent may only call the tools and reach the destinations its task requires. Everything else is blocked and logged outside the model, including failed attempts. Access to data matters just as much. An agent that checks incoming invoices needs the invoices and the purchase orders, and it has no business with personnel files or the cloud credentials.

Oversight by a second agent

The third approach relies on an architecture of several agents. Before an action is executed or a result leaves the system, a second, independent agent checks whether it fits the task. This output filter complements the usual input filter, which catches suspicious instructions in documents or emails. In the Cohere case, such a checker would have seen that an attack-detection rule does not require an upload to Beijing and stopped the call. It works best when the checker comes from a different vendor, because two models with different training probably share the same blind spots less often.

Admittedly, the checker is also a language model and can be wrong, and every additional check costs time and compute. I therefore usually use this approach in addition to tight permissions. None of the three ways relies on what the model says about itself.

Diagram: an AI model as a black box in the middle. Before it an input filter, after it a checking agent from a different vendor and permissions that release only the necessary tools, destinations and data. Below, a log of all calls. Optionally, an isolated environment without internet access encloses everything. Each control has a counterpart for employees: protection against deception, four-eyes principle, authorisation concept, audit and a workplace without outside contact. AI-generated
What is inside the model stays hidden. Before and after it, the same controls apply as for employees.

An AI agent is more employee than software

A language model remains a black box. What found its way in during training, whether by mistake or by intent, we will probably not be able to trace from the outside in future either, with Qwen any more than with Mistral or Llama. Everything that happens before and after it, on the other hand, is fully visible: which data goes in, which tools the model may call and what leaves the system. These are the points that decide whether an organisation uses the model's advantages and keeps its drawbacks under control.

For IT security, this is not a new problem. In its behaviour, an AI agent resembles a human more than classic software that does what the code says. Humans make mistakes, can be deceived and, in rare cases, have bad intentions. According to Verizon's Data Breach Investigations Report, 68 per cent of the breaches examined in 2024 involved a human who made a mistake or fell for a deception.

We should not humanise AI. For risk management, it still helps to view an AI model more as an employee than as a classic, deterministic IT system. That usually captures the dangers better, and much of what has proven itself with people can be carried over.
Thorsten Vellmerk

That is why nobody stops hiring people. Organisations build the system around them, with roles and permissions, approval limits, the four-eyes principle and logs, so that mistakes become rarer and the damage stays limited when one does happen. The three ways above transfer exactly these principles to a model. The isolated environment corresponds to a workplace without outside contact, the tight permissions to the authorisation concept and the checking agent to the four-eyes principle. In the language of IT security, this is called Zero Trust: no part of the system is trusted just because it belongs to it.

For the case that someone turns off the tap, one more precaution is needed. The model must be replaceable. If the application is tightly bound to one model and there are no in-house test cases with which a successor can be checked, a switch takes months rather than days. The employee picture has an uncomfortable side, however. If AI agents are a kind of employee, then our most capable employees were trained in the US and in China. Via an API they can be withdrawn from one day to the next, as the block in June showed. Locally run models stay, but their successors come from the same houses. Europe has so far not managed to train employees with comparable capabilities itself. A good architecture makes this dependency manageable, but the dilemma itself remains as long as Europe does not catch up here.

We are happy to support you with all of this, from choosing the model through the architecture to reviewing an existing environment. We build such environments ourselves and also work as a sparring partner alongside your team or your existing provider, as described in Why AI Projects Fail. If you would like to know where your architecture around the black box has gaps today, get in touch.

Sources and further reading