Agentic AI can automate tasks and make decisions across business systems, but greater autonomy increases cyber and operational risk. This guide summarises the NCSC’s advice on controlling agent permissions, human oversight, monitoring, accountability and emergency shutdown.

Agentic AI Risk: What Businesses Should Control Before Deployment

Agentic AI is moving artificial intelligence beyond answering questions and generating content.

An AI agent can be given a goal, access tools and systems, make decisions and take actions on behalf of an organisation. Depending on how it is configured, it may operate with anything from close human supervision to significant autonomy.

That creates potentially significant productivity opportunities. It also changes the risk.

When AI only recommends an action, a person normally decides whether to proceed. When an AI agent can access production systems, communicate externally, use credentials or change data itself, an incorrect decision can have a much greater impact.

The UK’s National Cyber Security Centre has now published interim advice on this subject in Managing the cyber risk of agentic AI.

Read the full NCSC article: Managing the cyber risk of agentic AI

The NCSC is clear that this is still a rapidly developing area. Its current article is interim practical advice, with more formal guidance expected as understanding and evidence develop.  

For organisations considering agentic AI, however, there are already some important principles worth understanding.

Start with the amount of autonomy you actually need

Not every AI use case needs an autonomous agent.

An AI system might simply provide recommendations for a person to review. At the other end of the spectrum, an agent could be allowed to access systems, execute commands and make decisions without prior human approval.

Those are very different risk profiles.

The NCSC’s underlying principle is straightforward:

The greater the autonomy, the greater the potential impact if something goes wrong — and therefore the stronger the controls need to be.

Before asking what an AI agent can do, organisations should first ask:

What does it actually need to be allowed to do?

Giving an agent access to more systems, data and permissions than its task requires unnecessarily increases what the NCSC describes as its potential blast radius.  

Seven things organisations should consider

The NCSC sets out a number of practical considerations. For business and technology leaders, these can be distilled into seven areas.

1. Define what the agent is — and is not — allowed to do. Before deployment, clearly define the task, the acceptable actions and the boundaries the agent must not cross. The NCSC recommends identifying potential failure scenarios in advance. This includes considering what instructions the agent receives, which systems it can reach and what could happen if it interprets its objective in an unexpected way. This matters because an AI agent does not apply human common sense to an instruction. A goal that sounds obvious to a person may be pursued by an AI system in a way that was never intended.Organisations therefore need explicit red lines, not just a desired outcome.
2. Do not treat the prompt as the control system. Good instructions matter.An agent should be told what it should achieve, what it is permitted to do and when it must stop and request human approval. But prompts alone are not sufficient protection. An instruction saying “do not access this system” is weaker than a technical control that prevents access to that system in the first place. The principle should be defence in depth: clear instructions supported by technical and operational controls.
3. Choose the right level of human oversight. The NCSC describes three broad models: Human-in-the-loop: a person approves actions before they happen. Human-on-the-loop: the agent operates, but people monitor it and can intervene.Human-out-of-the-loop: the agent operates autonomously without human review.There is no single correct model. The question is whether the level of oversight matches the consequences of failure.An agent carrying out a low-risk administrative task is different from one able to alter production data, communicate with customers or interact with critical systems.Where consequences could be significant, the NCSC recommends retaining human oversight alongside technically enforced controls. 
4. Control the environment around the agent. This is one of the most important parts of the guidance. The NCSC recommends operating agents inside a controlled or sandboxed environment. That means thinking about everything the agent can potentially reach: systems, networks, compute resources, data and credentials. Access should be restricted to what is genuinely required. This is particularly important for credentials.  An AI agent with access to powerful API keys, authenticated sessions or privileged identities can potentially do whatever those credentials permit.The NCSC therefore recommends restricting permissions, using short-lived credentials where possible and giving agents their own identifiable identities rather than allowing them to operate invisibly under human accounts.This makes identity and access management an increasingly important part of AI implementation.
5. Make AI activity observable. Organisations need to be able to answer:What did the agent do?And, ideally:What is it doing now?The NCSC recommends logging and monitoring agent activity, together with relevant network, access and sandbox information. This is not simply about keeping records for later investigation. Monitoring should make it possible to identify unexpected behaviour while it is happening and intervene quickly. The NCSC also makes an important operational point: agent activity should be treated similarly to other forms of user activity and incorporated into security monitoring and incident-response processes.  
6. Make actions attributable. If an AI agent interacts with external organisations or services, its actions should be identifiable.This matters when something unexpected occurs. Organisations should be able to establish which agent generated an action, which organisation it belonged to and which systems or identities it used. The wider principle is important:
   

   Autonomous should not mean anonymous.

   The more responsibility organisations delegate to AI agents, the more important traceability becomes.

7. Always retain the ability to stop it. Finally, organisations need an effective emergency stop. The NCSC describes this plainly as maintaining the ability to “pull the plug”. That may require more than terminating a single AI process. If an agent starts behaving unexpectedly, organisations may need to rapidly remove network access, revoke credentials, interrupt access to AI models or isolate the wider environment. That capability should be designed before deployment rather than invented during an incident.  

Built-in AI safeguards are not enough on their own

AI platforms and models increasingly include their own controls intended to prevent unwanted behaviour.

Those controls are useful, but the NCSC advises organisations not to treat them as complete protection.

Controls can have limitations, may not cover a specific business environment and may not be sufficient where the consequences of an incorrect action are significant.

This is an important distinction.

Model safety is only one part of system safety.

The surrounding architecture, identities, integrations, data access, monitoring, operating procedures and human controls all contribute to the risk of an agentic AI implementation. 

Practical question for business leaders

Agentic AI creates an understandable temptation to start with capability:

What could we automate?

A better sequence may be:

What business outcome are we trying to achieve?

How much autonomy does the AI genuinely require to achieve it?

What systems, data and permissions does it need?

What happens if it behaves differently from what we expected?

Can we see what it is doing and stop it quickly?

Those questions should be answered before an agent is given significant access to production environments.

The objective is not to remove autonomy altogether.

Doing that could remove much of the value agentic AI promises.

The objective is to make autonomy deliberate, proportionate and controlled.

AI implementation is becoming an architecture and operating-model question

One of the most important implications of the NCSC guidance is that agentic AI cannot be considered purely as an AI-model decision.

Once an agent starts acting across an organisation, it becomes connected to existing technology architecture.

Identity matters.

Integration matters.

Data quality and access matter.

Monitoring matters.

Operational ownership matters.

Incident management matters.

The design question therefore moves from:

“Which AI model should we use?”

to:

“What are we prepared to allow this AI system to do inside our organisation, and how will we remain in control?”

That is likely to become an increasingly important question as businesses move from AI experimentation towards AI systems that can take real actions.

For organisations starting to explore agentic AI, the NCSC article is worth reading in full. Its recommendations will evolve, but the core principle is unlikely to change:

Give an AI agent only the autonomy, access and authority it actually needs — and make sure you can observe, constrain and stop it when necessary.

Assess your organisation’s AI readiness

Before introducing agentic AI into business-critical processes, it is important to understand whether your organisation has the right foundations in place across data, technology, governance, security and operating processes.

Contact us about our AI Readiness Assessment and we’ll identify gaps, clarify priorities and establish a practical starting point for responsible AI adoption.

Frequently asked questions

What is agentic AI?

Agentic AI refers to AI systems that can pursue a goal by making decisions, using tools and taking actions with varying levels of human supervision.

Why is agentic AI riskier than a standard chatbot?

A chatbot generally provides information or recommendations for a person to review. An agentic system may access business systems, use credentials, change data or communicate externally, increasing the potential impact of an error or misuse.

Does every AI agent need human approval for every action?

No. The appropriate level of oversight depends on the task and the consequences of failure. Low-risk activities may be monitored, while higher-risk actions may require approval before they are completed.

What controls should organisations put around AI agents?

Key controls include least-privilege access, separate identities, controlled environments, human oversight, activity logging, monitoring, clear accountability and an effective emergency shutdown process.

Are the safeguards built into AI platforms enough?

No. Built-in safeguards are useful but should form only one part of a wider control framework covering architecture, identity, access, data, integrations, monitoring and operational processes.

How can an organisation assess whether it is ready for agentic AI?

An AI readiness assessment can help evaluate the organisation’s data, technology, governance, security and operating model, then identify the practical steps needed before deployment.