An AI agent is useful when it can pursue a defined goal across several steps: interpret an instruction, retrieve context, choose a tool, take an action and check the result. That is also what makes it different from a drafting assistant. Once software can send a refund, update a customer record, shortlist a candidate or change a schedule, a plausible answer is no longer enough. The organisation needs to control authority, identity, evidence and recovery.
This article, first published in December 2025, is current through 31 July 2026. It addresses UK data-protection and consumer-law considerations, but employment, equality, health and safety, contract and sector rules can depend on the activity and the nation involved. It is an operating guide, not a claim that one policy makes every use lawful.
Replace “autonomous workforce” with bounded delegation
“Autonomous workforce” is an attractive phrase and a poor control model. People have employment duties, professional judgment and accountability. Software has permissions, instructions, failure modes and an owner. Treat an agent as a delegated system whose scope can be expressed and tested.
A useful delegation record answers eight questions:
- What outcome may the agent pursue?
- Which data may it read, and for what purpose?
- Which tools may it call?
- What may it write, send, buy, approve or delete?
- Which value, volume and time limits apply?
- Which actions always require a named human?
- What evidence must be retained for review?
- How is access withdrawn and work recovered?
The Competition and Markets Authority’s March 2026 report on agentic AI and consumers describes an early market with many bounded uses rather than dependable general autonomy. The CMA highlights risks including opaque steering, conflicts of interest and weak accountability. That supports a narrow design: one process, a small tool set and measurable exit conditions.
For an operational pattern that separates planning from execution, see our guide to end-to-end agentic workflows-uk). The central design choice is not how human-like the agent appears. It is the largest consequence it can create before somebody can detect and reverse it.
Put authority in a machine-readable envelope
A policy in a PDF will not stop an API call. Controls should exist at the point of action. Give each agent a service identity, short-lived credentials, the least privilege required and a deny-by-default tool list. Keep production and test accounts separate.
| Control surface | Minimum implementation | Evidence to retain |
|---|---|---|
| Identity | Unique workload identity; no shared staff login | Authentication and credential-rotation log |
| Data | Purpose-scoped retrieval and field filtering | Data-source, query and record references |
| Tools | Allow-listed functions with typed inputs | Tool name, arguments and result status |
| Money | Per-action and cumulative thresholds | Amount, payee, approval and reconciliation |
| External messages | Draft-only or approved recipient/domain rules | Final content, approver and delivery receipt |
| Destructive changes | Two-person approval or reversible staging | Before/after state and rollback outcome |
| Time | Expiring task lease and maximum run duration | Start, stop and timeout reason |
| Volume | Rate and batch-size limits | Counters, exceptions and throttling events |
Do not rely on a prompt that says “ask before doing anything risky.” Encode what “risky” means. A customer-service agent might draft replies freely, send routine status updates only to an existing verified address, and require approval for refunds, contract changes, vulnerable-customer cases or complaints.
The National Cyber Security Centre’s secure AI system development guidance places secure design, development, deployment and operation across the lifecycle. Its secure development guidance calls for supply-chain security, asset protection and documented incident handling. Apply those principles to prompts, retrieval indexes, model endpoints, connectors and the agent runtime—not only the model.
Keep people responsible for consequential decisions
The Data (Use and Access) Act 2025 changed the UK rules for solely automated decisions with legal or similarly significant effects. The ICO’s summary of the data-protection changes explains that the regime is broader than the old general prohibition, but safeguards remain. People must receive relevant information, be able to make representations, obtain human intervention and contest a significant decision. Special-category data remains more restricted.
“Human in the loop” is not satisfied by a rubber stamp. The reviewer needs:
- enough time and authority to disagree;
- the material inputs and limitations, not just a score;
- a route to gather missing context;
- training on common errors and unequal impacts;
- a record of the review and its reason;
- protection from throughput targets that punish intervention.
The ICO’s March 2026 statement on automated decisions in hiring says automation can be used with the right safeguards, while emphasising transparency and meaningful human involvement. For recruitment, promotion, dismissal, performance, pay and shift allocation, involve employment and equality specialists before a pilot. Test outcomes across relevant groups, but do not assume demographic inference is itself lawful or appropriate.
The ICO’s guidance on data protection and monitoring workers also makes proportionality, transparency and fairness central. An agent that continuously scores activity can become worker monitoring even if the product is marketed as productivity assistance.
Involve affected workers before the operating model is fixed. Explain which signals are collected, which actions the agent may take, how an individual challenges an error and whether data can influence performance or disciplinary processes. Consult recognised representatives where required and give teams a route to report unsafe automation without adverse scoring. Training should cover both use and refusal: staff need to recognise when the workflow is outside scope and know that escalation is expected.
Procurement must preserve those controls. Require advance notice of material model, policy and subprocessor changes; access to logs and deletion evidence; security incident terms; tested service recovery; and an exit that returns records in a usable format. A provider’s “human oversight” feature is not meaningful if the organisation cannot configure approval thresholds or stop queued actions.
Design review around failure, not confidence
Model confidence is not a substitute for consequence. Route work according to what can go wrong and how hard it is to repair.
Use four review lanes:
- Observe: the agent reads and summarises; a person decides and acts.
- Draft: the agent prepares a change; an authorised person approves it.
- Constrained execute: the agent performs low-consequence, reversible actions inside limits.
- Prohibited: the agent cannot make the action, regardless of its confidence.
Move a use case between lanes only on evidence. A payroll explanation may stay in draft mode; a reversible ticket label may reach constrained execution; a dismissal or safety-critical instruction should not be delegated merely because earlier outputs looked accurate.
Consumer-facing agents require another layer. The CMA’s guidance on complying with consumer law when using AI agents states that businesses remain responsible for unlawful agent actions. Product information, total prices, cancellation rights, refunds and complaint routes must be accurate, while regular human review and rapid correction are expected. The agent should identify itself plainly and never invent scarcity, eligibility or policy.
Test the workflow as an adversary and an operator
An agent can fail through bad instructions, poisoned content, stale data, excessive permission, unreliable integrations or deliberate manipulation. A realistic test pack should include:
- instructions hidden in documents or web pages;
- a request to reveal system prompts, secrets or another customer’s data;
- duplicate, delayed and out-of-order events;
- an unavailable tool or partially completed transaction;
- an ambiguous identity or changed bank detail;
- a request outside the agent’s jurisdiction or date range;
- a false claim made confidently by a trusted source;
- a runaway loop that consumes time, messages or money;
- a legitimate exception the policy did not anticipate.
For each case, define the safe outcome before running it. “The model noticed” is weaker evidence than “the gateway denied the call, alerted the owner and preserved the trace.”
Log model and policy versions, retrieved sources, tool calls, approvals and final state. Minimise personal data in logs and set retention by purpose. Keep an incident trace, but do not create a permanent duplicate of every customer record. Our AI agent control-room guide covers the operational view: queues, exceptions, limits, owners and emergency stops.
Run a 90-day controlled introduction
Days 1–30: map and bound. Choose one workflow with a stable baseline, reversible actions and an accountable process owner. Complete data-protection, security, worker, consumer and sector screening. Record authority limits, prohibited actions and the human review lane. Build a test set from representative cases, including rare and adverse ones.
Days 31–60: shadow and simulate. Let the agent propose actions without executing them. Compare its proposals with real outcomes. Red-team its connectors and retrieval sources. Measure correction rate, missed exceptions, reviewer effort, unequal impact indicators, latency and cost. Fix the process or stop; do not tune only for a headline accuracy figure.
Days 61–90: release narrowly. Enable a small population or transaction class with hard value, volume and time caps. Staff the escalation route. Reconcile every executed action at first, then reduce sampling only when evidence supports it. Hold weekly owner reviews and a formal go/no-go decision at day 90.
The release paper should state what improved, what became harder, which risks remain, who accepts them and what would trigger withdrawal.
Define pause gates before launch
Pause new agent actions automatically when any agreed boundary is crossed. Appropriate gates include:
- suspected personal-data disclosure or credential compromise;
- an unauthorised payment, deletion, contract change or external message;
- significant-decision safeguards not being available;
- a material rise in complaints, reversals or human overrides;
- unexplained outcome differences between affected groups;
- missing, stale or untraceable source data;
- tool-loop, spend, latency or volume limits being exceeded;
- loss of a critical vendor, integration or audit trail;
- inability to identify an accountable on-call owner.
Pausing should preserve evidence, prevent further side effects and move queued work to a safe manual path. Recovery needs reconciliation: identify what the agent attempted, what actually completed, what a user saw and what must be corrected.
Review the authority envelope after every material process, model, tool or data-source change. Re-approval is particularly important when a read-only connector gains write access, a pilot reaches a new population or an agent begins selecting among suppliers. Those are scope changes even if the user interface looks the same.
Agentic AI can remove coordination work and shorten routine journeys, but only bounded delegation makes that value governable. Start with a use case whose authority can be expressed in code, whose consequences can be reconciled and whose human owner can stop it. The mature question is not whether an agent looks autonomous; it is whether the organisation remains in control when the system is wrong.
