Your Business Has a Backup Plan. Does It Still Have the Capability to Use It?

Why continuity plans may be assuming knowledge the organisation no longer exercises

By Louize Clark

There is a sentence in a great many business continuity plans that I have been turning over for a few weeks now: if the system becomes unavailable, revert to the manual process.

It's a reassuring sentence. Somebody considered the possibility that the technology might fail, and there is an answer written down. The organisation knows what it would do.

Could anybody still do it?

Having a manual process documented and having people who can still run it have never been quite the same thing. As AI becomes more deeply woven into how organisations work, the distance between them may become one of the more important gaps nobody is measuring.

We are teaching AI how our organisations work

Something has shifted in enterprise AI over the past month, and the direction of travel matters more than any single announcement.

The ambition has moved on from AI that helps a person with a task to AI that understands the organisation itself. On 25 September, Microsoft extended Work IQ, the intelligence layer behind Copilot, into core business data. It described AI that now understands "the records, the processes, and the way your organisation actually gets things done." Earlier in the month, Google announced third-party connectors for Gemini in Workspace, linking it to Salesforce, HubSpot, QuickBooks, Monday and Asana. They are switched on by default for users who already have Gemini access, with administrators able to manage them. Google also described Gemini as increasingly working across Gmail, Drive, Docs and Chat rather than sitting inside any one of them.

Readers of earlier papers will recognise the pattern I have described as inherited AI. The capability arrives inside tools the organisation already trusts, sometimes switched on by default, without an equivalent decision about how much of the organisation it can now reach

Then, on 2 September, Meta published an account of what it calls an organisational second brain. It was built for a specialised compliance domain, and it is designed to capture expertise that usually lives only in specialists' heads. The aim is to turn one-off expert corrections into permanent institutional memory.

I think that's a genuinely good idea. Organisations have worried about knowledge walking out of the door for as long as there have been organisations. Consider the experienced person who knows why a process was designed the way it was, which exception matters, what happened last time, and where the written procedure stops being the whole story. Losing that person has always been expensive, and AI offers a way to keep more of what they know.

But what happens if, as the AI gets better at remembering how the organisation works, the organisation gets worse at remembering without it?

Knowledge doesn't disappear. It stops being exercised.

We tend to picture skills erosion as someone forgetting how to do their job. I don't think it will usually look like that. It will be much quieter.

A task is automated, and the person who used to do it now checks the output instead. A new starter joins and learns the AI-assisted version, because that is simply how the work is done here. Someone with twenty years' experience retires. An exception that once needed a conversation between three people is now resolved by a system before anyone sees it. The manual process still exists, in a folder somewhere. Nobody decided to remove the knowledge. Fewer people use it, and knowledge that isn't used changes shape.

That matters because organisational knowledge includes know-how as well as information. It means knowing that the procedure says one thing but this particular client needs something slightly different. It means knowing which system has to be checked before another is switched off, and recognising when something is technically correct and operationally wrong. Some of that can be written down, and some of it can probably be captured by AI. Neither is quite the same as having people who can still do the work.

There is early evidence that employees are already feeling this. IBM's global study, published on 21 September, surveyed 1,500 CHROs and 8,800 employees. It found 60% of employees worried AI is eroding their skills, with critical thinking cited most often as declining, and three in four of those say it has already begun. At the same time, 71% of CHROs named the ability to supervise, validate and override AI outputs as the workforce's most essential skill. And 41% of CHROs thought employees might not feel safe challenging or overriding what the AI produces.

Self-reported worry should be read as a signal rather than a diagnosis. Even so, the shape of it is interesting. The skill organisations say they most need is the ability to overrule the system, and that skill depends most on people who still understand the work underneath it. Judgement doesn't arrive with a licence. It grows from doing things: getting some of them wrong, meeting the exceptions, watching someone more experienced handle what you haven't seen before.

The second brain still needs a first one

This is where the Meta work becomes more interesting than it first appears, and I don't mean that as a criticism. It depends on something that is easy to overlook.

A system that learns from expert corrections only keeps learning while there are experts to correct it. Its quality is borrowed, continuously, from people who learned their craft the long way. So picture the loop running for five years. The system gets better. The experts retire or move on. Their replacements learned the work through the system, and the corrections they can offer are increasingly shaped by what the system taught them.

Nothing has failed. Every step was sensible. But the organisation may have converted a renewable capability into a finite one, and nobody would see it happening. The output would look fine right up to the day it needed someone to say, "No, that's wrong, and here's why."

I think of this as the correction problem. It is the point at which skills erosion stops being a people issue and becomes an infrastructure one.

A backup plan is not a backup capability

Go back to that continuity plan.

Imagine a critical process supported by AI. The plan says that if the system goes down, the team reverts to manual processing. On paper, the organisation is resilient. Now look underneath. The manual process was last run three years ago. Two of the people who understood it have left, and their replacements were trained on the current system. The surrounding workflow has changed. Volumes have grown, because automation allowed them to grow. Other systems now assume the automated step is happening.

The procedure still exists. What matters is whether it is still operationally viable: at today's volume, with today's people, under pressure. A documented workaround is not necessarily an executable one. And a continuity plan cannot create organisational knowledge simply by assuming it's there.

For regulated firms, this isn't abstract. The UK operational resilience regime already asks financial services firms to identify their important business services, and to map the people, processes and technology needed to deliver them. It also asks them to test whether they could stay within impact tolerances through severe but plausible disruption. People are already on the map. I would be curious how many of those scenario tests have checked whether the people named in the fallback could still perform it.

Dependency you won't find in a register

I've written before about operational dependency, the infrastructure organisations build without meaning to, one sensible decision at a time. Most of that conversation concerns which technology we have come to rely on, and an AI register can help with that.

A register won't tell you what capability the organisation has stopped maintaining because that technology exists. Neither will a contract or a licence count. That lives in which tasks people still perform, which decisions they still make unaided, and how long it has been since anyone had to.

What this does to sovereign choice

The same pattern reaches beyond any single organisation.

Much of the sovereign AI debate is, rightly, about compute, models and data. In an earlier paper I argued that sovereignty ultimately comes down to the freedom to choose, and that years of habit built around one ecosystem can narrow that freedom without anyone noticing. Capability erosion narrows it further, in a way that's easy to miss.

Switching providers assumes an organisation understands its own work well enough to rebuild it somewhere else. It has to know which rules were deliberate, which exceptions matter and what the old system was actually doing. If a meaningful share of that understanding now lives inside the platform being left, portable data does not make a portable organisation. You can have an alternative available to you while gradually losing the ability to use it.

Sovereignty, in the end, rests on what a country or an organisation can still do, as much as on what it owns.

Perhaps resilience needs another test

None of this is an argument against AI, or against capturing expertise. There are very good reasons to do both. But efficiency and resilience don't measure the same thing, and we should know what we're giving up as well as what we're gaining.

One finding in the IBM study struck me as encouraging. Organisations that clearly define workflows as human-led, AI-assisted or AI-executed report an 18% reduction in risk and a 20% improvement in quality. That isn't a solution, but it points somewhere: you can't protect a capability you haven't named.

Most continuity plans record what should happen if the system fails. Far fewer record when anyone last tested the human capability underneath the plan, meaning whether someone could actually run the procedure rather than simply find it. How long could they keep going, and at what volume? Where would the gaps appear first? Which decisions would suddenly become hard, and which people would suddenly become critical?

We are putting a great deal of effort into teaching AI how our organisations work. Rather less is going into making sure they could still work without it.

AI Policies UK helps organisations see the AI infrastructure they're already standing on, chosen, inherited and embedded — before decisions like this one have to be made under pressure. Get in touch: louize@aipolicies.uk