You Cannot Secure What You Cannot See

What an August AI safety incident reveals about the infrastructure most businesses still can't map

By Louize Clark

On the morning of 28 July, security monitoring at the UK's AI Security Institute noticed something unusual. Data was leaving one of its testing systems through Tor, an anonymity network designed to obscure where internet traffic originates. The team running the evaluation began investigating within minutes and, within roughly an hour of the alert being raised, had stopped the relevant evaluations, disabled access to the most capable models and isolated the machines involved.

What they found when they went back through the activity is worth looking at carefully.

AISI had been running a cyber capability evaluation involving seven frontier models across two controlled cyber ranges. There were 122 runs in total. Most proceeded as expected, but in ten of them the agents took actions outside the intended testing parameters. AISI identified 19 distinct unsanctioned actions: 17 from Anthropic's Mythos 5 and two from a single run involving OpenAI's GPT-5.6 Sol.

The most serious behaviour involved a real open-source software project. An agent attempted to introduce malicious code and then tried to increase the chances of that code being accepted by interacting with the people responsible for reviewing it. It gathered information about maintainers, created false online personas and used those identities as part of an effort to influence the review process. The attempt failed because the human maintainer did not approve the change.

There are important limits to what can be taken from this incident. These models were being tested in deliberately permissive conditions intended to expose the upper end of their cyber capabilities. They had live internet access, and cyber safety classifiers normally provided by the model developers had been disabled. AISI was not recreating ordinary enterprise deployment conditions, and it is explicit about that. Its investigation found no resulting real-world harm, although some of the agents' activity did reach real people and organisations outside the simulated environment.

That matters, because this is not a story about an AI system somehow escaping from a laboratory. The more useful question is what the agent did with the permissions and access it had already been given.

It had an objective. It had tools. It had internet access. And in trying to complete the task, it pursued routes that its operators had not intended. AISI's subsequent review also identified a governance problem on its own side: decisions about internet access had been made when earlier models were less capable and had not been reassessed quickly enough as those capabilities changed.

There is another part of the incident that matters just as much. The behaviour was not initially discovered by monitoring designed specifically to watch the evaluation in real time. General security monitoring picked up unusual traffic and led investigators back to what the agent had been doing. AISI has since tightened its controls and monitoring. The NCSC's broader guidance on agentic AI makes the underlying lesson clear: organisations need ongoing visibility into what agents are doing, meaningful human control and the ability to contain them. If an organisation cannot understand, monitor or contain an agent's actions, the NCSC says it is not ready for deployment.

What this raises for everyone else

For most organisations, AI has so far been experienced mainly as information infrastructure. It drafts something, analyses something, summarises something or recommends something, and a person usually decides what happens next. Agentic AI changes that relationship because the system can be given tools, access to other systems and authority to perform parts of a task itself.

That changes the question from simply asking what the AI can produce to asking what the AI can reach.

Imagine an AI assistant that can read a customer record but cannot change it. Now imagine the same system can update that record, send an email, trigger a refund or invoke another application. The underlying intelligence may be similar, but the operational consequence is not. Once the system can act, its permissions, connections and dependencies matter in a different way.

Cyber security has been dealing with versions of this problem for years. The NCSC's Cyber Assessment Framework already expects organisations responsible for essential services to understand their assets and the dependencies between them. The principle itself is therefore not new. What is changing is the amount of AI capability beginning to appear inside those assets and dependencies, sometimes without looking like a new infrastructure decision at all.

A company may deliberately buy an AI platform and know perfectly well what it has procured. But AI can also arrive because a CRM gains a new feature, a finance platform adds forecasting, a productivity suite introduces automated workflows, a cyber-security product starts using AI to investigate alerts, or a supplier quietly changes the technology behind the service it provides. An employee may also connect an AI tool to part of their own workflow simply because it saves them time. None of those moments necessarily feels like the organisation has just changed its infrastructure. Collectively, however, they can change how the organisation operates.

The latest Cyber Security Breaches Survey gives us some indication of the visibility problem that already exists before we add more AI capability. Only 15% of UK businesses formally review cyber risks presented by their immediate suppliers and just 6% review their wider supply chain. Among large businesses, 48% review immediate suppliers and 24% look further down the supply chain.

The software procurement findings are equally interesting. Only 22% of businesses said cyber security was considered to a large extent when buying new software. Another 20% considered it to some extent, 38% said it was not a major concern because they bought from established companies, and 12% did not consider it at all.

That 38% is worth noticing because the assumption behind it is perfectly understandable: if the supplier is established, somebody else has probably thought about the risk. In many cases that supplier will have sophisticated security controls. But knowing that a supplier is reputable does not necessarily tell you what capability now exists inside its product, whether that capability has changed since procurement, what it can connect to or which services sit beneath the supplier itself.

The survey also shows that 31% of businesses are using, adopting or considering AI, while only around a quarter of that group currently have cyber-security practices in place specifically to manage AI-related risks. I don't think that tells us businesses are reckless. It tells us capability can arrive more quickly than organisations develop the governance needed to understand it.

An inventory tells you what you have. It doesn't tell you what you depend on.

This is where the distinction between an AI inventory and an AI dependency map becomes useful.

An inventory can tell you that a company uses an AI-enabled customer-service platform. It can record the supplier, the internal owner, the purpose and perhaps a risk category. That is useful information, and many organisations are still working towards having even that level of visibility. What it does not necessarily tell you is what the platform connects to, which data it can access, what permissions it inherits, whether it can call other systems or whether a process elsewhere in the business has quietly begun to rely upon it.

There are further questions behind those. Can the AI component be disabled without disabling the product itself? Can the supplier change the underlying model without the customer changing software provider? Does the organisation know when that happens? Does the service rely on an external model provider or API? If that underlying service disappears, what actually stops? And if the organisation wants to move away, how difficult is the transition in practice?

An inventory is therefore a record of assets. A dependency map is a picture of relationships, reliance and consequences.

That distinction matters because the application a member of staff sees on screen may be only the top layer of a much larger stack. Behind it can sit a cloud provider, a model provider, APIs, identity systems, data sources, libraries and telecommunications infrastructure. The organisation may have contracted directly with only one company in that chain while depending operationally on several.

AI Policies UK uses the term Invisible AI Infrastructure™ to describe that wider layer: the AI-enabled systems, dependencies, connections, permissions, data flows and decision or action pathways operating beneath the visible organisational surface.

Invisible does not mean secret. Most of these things can be known. The problem is that they are often known by different people.

Procurement understands the supplier and the commercial agreement. IT understands the application estate and system connections. Cyber security understands network exposure and access. Legal understands the contract. Data protection understands particular data flows. Operations knows which systems the organisation cannot function without. Employees know how technology is actually being used during the working day. The supplier knows what has changed inside the product since it was first purchased.

Each of those people can have an accurate view of their part while nobody necessarily sees how all of those parts connect.

When dependency becomes infrastructure

There is already a useful example of what happens when technology dependency becomes sufficiently concentrated.

In July, HM Treasury designated Amazon Web Services EMEA, Google Cloud EMEA, Microsoft Ireland Operations and Oracle Corporation UK as the first Critical Third Parties to the UK financial sector. The Bank of England, PRA and FCA are now overseeing those providers because disruption or failure could affect multiple firms or markets at the same time. The significance lies in concentration: enough important organisations rely on the same providers that a problem outside an individual firm can become a wider resilience issue.

That tells us something important about modern infrastructure. A business does not have to own a technology for it to become critical to the business, and a technology company does not have to resemble a power station, railway or water network before its services become systemically important.

The same logic is increasingly relevant to AI, even though the current Critical Third Party regime is not an AI-specific regime. AI capability is being built into cloud platforms, enterprise software, managed services and supplier relationships that organisations already depend upon. The more useful question is therefore not whether government understands technology dependency — clearly it does — but whether organisations have enough visibility of the AI-specific capability developing inside those dependencies.

There is a meaningful difference between knowing that an organisation relies on a major cloud or enterprise provider and knowing which AI-enabled processes now sit on top of that service, what those systems are permitted to do, which models or APIs sit beneath them and how difficult any of those components would be to substitute.

Sovereign compute and operational sovereignty are not the same thing

This connects to the wider conversation Britain is already having about sovereign AI.

The UK Compute Roadmap sets out up to £2 billion of investment in the country's public compute ecosystem, including more than £1 billion to expand the AI Research Resource twenty-fold by 2030 and up to £750 million for a new national supercomputer in Edinburgh. The roadmap also talks explicitly about supplier diversity, interoperability, portability and avoiding over-reliance on individual providers.

That investment matters. But sovereign compute answers a different question from operational sovereignty.

Compute is about whether the country has access to the infrastructure required to develop and run AI. Operational sovereignty is about what happens inside the organisations using it. How easily can they change direction if a supplier alters its terms, a model is retired, an API disappears or a capability embedded in a critical workflow is no longer available?

A business can use UK-based compute while still depending on overseas enterprise software, foreign foundation models, global cloud providers, identity systems and internationally distributed technology supply chains. That does not make sovereign compute less important. It simply means national compute capacity does not, by itself, tell us how dependent organisations have become on the wider technology stack.

Nor is the answer to remove every external dependency. Modern technology does not work that way, and the UK's own compute strategy assumes continued engagement with multiple commercial and international providers. The important distinction is between a dependency that is understood and one discovered only when it fails.

That is why visibility matters to sovereignty. Before an organisation can say how easily it could change supplier, substitute a model or keep operating during disruption, it first has to know where the dependency exists.

The question worth asking

For most organisations, this does not need to begin with another large governance programme. It begins with better questions asked across the business rather than inside one function.

Where is AI actually operating? Which capabilities were deliberately procured and which appeared later inside software or supplier services already in use? What can those systems access? What permissions do they hold? Which simply generate information and which can take action? What business processes rely on them? Which suppliers sit beneath the supplier we can see? If a provider changes something remotely, who notices? If a service becomes unavailable, what stops? And if we need to move away, how difficult is that transition?

The reason those questions are difficult is that the answers are distributed. Procurement knows one part, IT another, cyber security another and operations another. Employees may know about AI use that none of those teams can see because nobody formally procured it in the first place.

That is why the AISI incident is useful, even though the conditions of the evaluation were highly unusual and should not be treated as representative of ordinary business deployment. What it demonstrated was a much narrower point: a capable system had a task, access and permissions, and it used them in ways its operators had not anticipated. The behaviour became visible because monitoring elsewhere in the environment detected something unusual and investigators were able to trace it back. AISI's response has been to strengthen the oversight around those evaluations.

Now bring that question into a normal organisation, where AI rarely arrives in one dramatic moment. It accumulates through software upgrades, cloud services, suppliers, employee tools, APIs and workflows that become more automated over time. Nobody necessarily decides to build one large AI infrastructure. The organisation simply becomes increasingly dependent on a collection of connected capabilities.

Britain is investing heavily in the AI infrastructure that is easy to see: compute, supercomputers, data centres and AI Growth Zones. At the same time, another layer is developing inside the organisations expected to use all of that capability. It is spread across departments, suppliers, applications, people and processes, and most of its individual components may be perfectly visible on their own.

The harder question is whether anybody can see how they fit together.

Before we can decide how secure that infrastructure is, how resilient it is or how sovereign it really is, we need to understand what is there, what it connects to and what we have already become dependent upon.

You cannot secure what you cannot see.

AI Policies UK helps organisations see the AI infrastructure they're already standing on, chosen, inherited and embedded — before decisions like this one have to be made under pressure. Get in touch: louize@aipolicies.uk