Human-in-the-Loop: Approvals and Control in AI Agents
AI agents do more than answer questions. They send emails, open tickets, update CRM records and place orders. The more an agent is allowed to do, the more it matters where a person checks its work. Human-in-the-loop (HITL) design answers that question, but only when autonomy levels, engineering, the review experience and regulatory oversight are planned together rather than bolted on one by one.
What does human-in-the-loop mean for AI agents?
Human-in-the-loop means an agent pauses at defined points, presents its proposal to a person, and only continues once that person decides. The reviewer can approve, edit or reject. For a short definition, see the glossary entry on human-in-the-loop; this guide covers how to make it work in production.
Three related terms are often mixed up:
- Human-in-the-loop: a person is part of the flow. Without their decision, the process stops at that point.
- Human-on-the-loop: the agent acts on its own while a person monitors and can step in or stop it.
- Human-out-of-the-loop: the agent runs fully autonomously; people only see logs and reports afterwards.
Real agents combine all three. A search in the knowledge base runs without asking. A draft reply to a customer waits for approval before it is sent. A credit note above a set amount needs a second person on top. HITL is therefore not a property of the whole agent but a decision you make for each action it can take.
Why are guardrails alone not enough?
Guardrails are automated checks on inputs, outputs and actions. They catch known failure patterns quickly, but they cannot judge whether a plausible action is the right one in a particular situation. That takes human judgement. HITL adds to guardrails; it does not replace them, and neither replaces the other.
Our guide to guardrails for AI agents covers the automated layers: input filtering, output validation and limiting what tools can do. The difference here is who does the checking. Guardrails are code that decides in milliseconds. Human-in-the-loop is a deliberate pause where a person with context and accountability decides.
Consider a support agent that proposes sending a customer a free replacement part. A guardrail can check that the part number exists and the value is under a threshold. Whether goodwill is appropriate this time is a different question. Perhaps the customer has complained three times already, or their service contract says otherwise. Someone who knows the account is better placed to decide.
A sensible split: guardrails filter the obvious cases and hard-block forbidden actions. Whatever passes and carries real consequences goes to a person for approval.
How do you set autonomy levels?
Autonomy levels define, per action, whether the agent may act alone, must ask first, or may not act at all. Three questions drive the choice: does the action reach outside the organisation, can it be reversed, and how much damage could a mistake cause?
A workable scale has four levels:
| Level | Meaning | Typical actions |
|---|---|---|
| 0 – blocked | The agent may not perform this action | Release payments, delete master data |
| 1 – propose | Agent drafts, a person executes | Quote to a new customer, contract change |
| 2 – approve | Agent executes after approval | Send customer reply, close ticket |
| 3 – autonomous | Agent acts, people spot-check | Categorise ticket, add internal note |
Avoid assigning levels on instinct. For each action, write down who is affected, what an error costs, how quickly it would be noticed and whether it can be fixed. An internal note on the wrong ticket is annoying but easy to correct. An email with confidential pricing sent to the wrong customer cannot be recalled.
Levels should change over time. Start new agents cautiously, then promote individual actions only when the decision log shows that proposals are rarely corrected over a sustained period. Demote an action immediately after a serious error. In our own operations, anything sent outside the organisation stays at level 2, and only internal, reversible steps run autonomously.
How do approvals work technically?
An approval needs three technical pieces: a defined stopping point in the flow, saved state so the agent can resume hours or days later, and a return channel that feeds the decision back in. The main agent frameworks provide their own building blocks for each of these.
Stopping points and saved state
An agent waiting for approval must not sit in memory as a running process. Approvals can take minutes or days, and during that time you will redeploy and servers will restart. The state has to live in durable storage such as a database.
Here is how three widely used frameworks handle it, as of November 2026:
- LangGraph: a node calls
interrupt()to pause the graph. According to the LangGraph interrupts documentation, this requires a checkpointer to persist state; you resume withCommand(resume=...)using the samethread_id, and the value you pass becomes the return value ofinterrupt(). One catch: the node restarts from the beginning on resume, so any code beforeinterrupt()runs again. - OpenAI Agents SDK: tools are marked with
needs_approval. When an approval is pending, the run stops and lists the open items inresult.interruptions. The Agents SDK documentation shows how to turn the result into aRunState, callapprove()orreject(), serialise the state and resume later. It also notes that serialised state is not authenticated, so load it only from trusted storage. - Claude Agent SDK: each tool call passes through a fixed sequence of hooks, deny rules, ask rules, permission mode and allow rules; anything still undecided reaches your
canUseToolcallback. The permissions documentation warns that tools approved automatically never reach that callback, so approval logic placed only there can be silently bypassed.
What an approval record should hold
Whatever the framework, store each approval request as its own record with:
- a unique ID and a link to the agent run,
- the action and its complete parameters (recipient, amount, text),
- the agent's reasoning and the sources it relied on,
- the autonomy level and the rule that triggered the request,
- the responsible role and a deadline,
- the decision, who made it, when, and any edited parameters.
from dataclasses import dataclass
from datetime import datetime
@dataclass
class ApprovalRequest:
request_id: str
run_id: str # points to the saved agent state
action: str # e.g. "send_email"
parameters: dict # exactly what would be executed
reasoning: str # the agent's explanation
sources: list[str] # evidence behind the proposal
assignee_role: str # a role, not just a person
due: datetime
decision: str | None = None # "approved" | "edited" | "rejected"
decided_by: str | None = None
decided_at: datetime | None = None
Common engineering mistakes
Two mistakes come up repeatedly. First, the reviewer approves a draft, but the agent then executes a freshly generated version. Only exactly what was approved may run. Second, actions run twice on resume because code before the stopping point executes again. Put side effects after the stopping point, or make them idempotent, for example with a request ID that the target system refuses to accept twice.
What makes a good review screen?
A good review screen shows within seconds what the agent wants to do, why, and how this case differs from the usual one. It lets the reviewer approve, edit or reject with a reason. And it lives where that person already works: the ticketing tool, the inbox, Slack or Microsoft Teams.
What a review screen should offer:
- The action in plain language: "Email to purchasing at the customer, subject …, attachment quote.pdf", not a block of JSON.
- Reasoning and evidence: which records the agent read and which documents it relied on, with links.
- What is unusual: a discount outside the normal range, a new recipient, an unusually long message.
- Edit, not just yes or no: reviewers who can only reject tend not to, because rejection means doing the work by hand. Reviewers who can fix the draft actually fix it.
- A required reason for rejection: the reasons feed your analysis and reveal where the agent goes wrong systematically.
An underrated risk is automation bias: the tendency to trust a system's suggestions because they are usually right. The EU AI Act names this explicitly in Article 14(4) as something overseers must remain aware of. Countermeasures include fewer but better-targeted approvals, visible flags on anomalies, and occasional test cases where reviewers are knowingly shown a flawed proposal.
What does the EU AI Act require on human oversight?
For high-risk AI systems, the EU AI Act requires that people can effectively oversee the system while it is in use, and that deployers assign oversight to people with the necessary competence, training and authority. Many everyday business agents are not high-risk, but the requirements are still a useful benchmark. What follows is orientation, not legal advice.
Article 14 sets out what overseers should be able to do, where appropriate and proportionate:
- understand the system's capabilities and limitations and monitor its operation, including anomalies and malfunctions,
- remain aware of the tendency to over-rely on its output,
- interpret the output correctly,
- decide in any situation not to use the system, or to disregard, override or reverse its output,
- intervene or halt the system safely through a "stop" button or a similar procedure.
Article 26 adds obligations for deployers: oversight must go to people with the competence, training and authority to do it, and automatically generated logs under the deployer's control must be kept for at least six months unless other law says otherwise. According to the same source, these articles apply to high-risk systems under Annex III from 2 December 2027 and under Annex I from 2 August 2028.
For your architecture, that translates into three concrete items: a stop function for the whole agent, a complete decision log, and oversight assigned to named roles. Build them in from the start and you will not need to retrofit them later, even if your agent is not classed as high-risk today. Outside the EU, these items are still a reasonable baseline for internal governance and audits.
What do you need before you start?
HITL rarely fails on technology. It fails on organisation: who approves, how fast, who covers for them, and what they need to know. On the technical side you need durable storage for agent state, access to the channels where reviews happen, and permissions the agent cannot widen without approval.
Organisation
- A named role per approval type (for example, sales operations for quotes, team leads for credit notes) and a cover rule for absences.
- Agreed response times and a rule for expired requests: let them lapse or escalate, never approve silently.
- Training for reviewers on what the agent does well and where it typically fails.
- In countries with works councils or strong employee representation, early involvement, because approval logs can reveal how individuals work.
Technology
- Persistent state, using a database-backed checkpointer rather than memory.
- A permission model in which the agent only gets the tools it needs and cannot bypass approval requirements itself.
- Tamper-resistant logging of every request and decision, in a form you can analyse.
- A stop function that halts all running and waiting work for an agent.
Data
- Reviewers must be able to open the sources the agent relied on. Reasoning that points to a document the reviewer cannot access does not help anyone.
How do you introduce human-in-the-loop step by step?
Start by listing every action the agent can take, assign each one a level, build the approval for the riskiest action first, and then measure how often people edit or reject proposals. That rate is the evidence you need to adjust levels later with confidence rather than guesswork.
- List actions. Every tool the agent can call, with its target system and effect: read, write, send externally, delete.
- Assign autonomy levels. Use the scale above and note a short reason per action. When in doubt, go one level more cautious.
- Build stopping points. Add the approval step in your framework before every level 1 and level 2 action; enforce level 0 through permissions, not just the prompt.
- Connect the review screen. Inside the reviewers' existing tool, with plain language, reasoning, anomalies and editing.
- Set up logging and stop. Record every decision, and test the stop function.
- Pilot and measure. Run for a few weeks and review the share of edited and rejected proposals, the rejection reasons and the waiting times.
- Adjust levels. Actions that are almost never corrected can move up a level; actions with frequent errors stay or move down.
For how agents are built around specific workflows such as quoting, service and knowledge management, see the overview of AI agents for business teams.
What are the limits and risks?
Human-in-the-loop protects you only as well as people actually review. Too many approvals lead to reflexive clicking; too few let errors through. Waiting times can eat up the agent's value, and a false sense of safety arises when the approval step can be bypassed in practice.
- Approval fatigue: someone asked to approve a hundred proposals a day will not review any of them properly. Treat approvals as scarce and reserve them for actions with real consequences.
- Shifted accountability: approving makes a person responsible for a decision they may only partly understand. That is fair only if the review screen gives them what they need.
- Latency: an agent that can only send replies hours later is a poor fit for time-critical work. A narrower task with more autonomy is often better than a broad one with many approvals.
- Bypassable controls: if the approval requirement lives only in the prompt, a manipulated input can make the agent ignore it. Approvals must be enforced in code and permissions.
- Partial view: the reviewer sees one proposal but not what follows. Multi-step flows need a preview of the whole plan, not just the next step.
HITL is the wrong tool where decisions must be made in seconds or volumes are too high for any realistic review. There, hard automated limits, sampling and after-the-fact analysis are the more honest approach.
How do I get started?
Start with one agent and one consequential action, such as sending an external email. Build the stopping point, review screen and log properly for that action, then track edit and rejection rates for a month. The same pattern then carries over to other actions with far less effort.
Useful first steps:
- Check whether your planned use could fall into one of the EU AI Act's high-risk categories, and get legal advice if you are unsure.
- Check how your framework handles stopping points and state persistence before you design the flow.
- Decide in advance which rejection rate triggers demoting an action or reworking the agent.
- Combine approvals with automated guardrails, so people only review what machines cannot judge reliably.
Frequently asked questions
Should every agent step need approval?
No. Read-only steps such as searching or summarising usually run freely. Approvals belong in front of actions that leave the company, move money or cannot be undone.
Is an approve button enough for the EU AI Act?
Not by itself. Article 14 also expects overseers to understand the system's limits, interpret its output correctly and be able to halt it. Treat this as orientation, not legal advice.
What if nobody approves in time?
Decide this upfront: the request expires, escalates to a deputy, or the case stays parked. Auto-approving after a timeout is not oversight.
How do I stop reviewers from rubber-stamping?
Ask for fewer, better-targeted approvals, show what is unusual about each proposal, and occasionally insert deliberately flawed test cases to check that reviewers really read them.
Sources
- EU AI Act, Article 14: Human Oversight artificialintelligenceact.eu
- EU AI Act, Article 26: Obligations of Deployers of High-Risk AI Systems artificialintelligenceact.eu
- LangGraph documentation: Interrupts docs.langchain.com
- OpenAI Agents SDK: Human-in-the-loop openai.github.io
- Claude Agent SDK: Configure permissions code.claude.com