Human in the Loop: Keeping People in Charge of AI Agents

AI agents can now answer customers, book appointments, update records and send messages with little or no supervision. That is useful, and it raises a practical question every business owner should answer before switching one on: which decisions stay with people?

"Human in the loop" is the shorthand for keeping people in charge. In practice it comes down to three things you design on purpose: escalation (when the agent hands off), review (when a person checks the agent's work), and takeover (how a person steps in or shuts it off).

This guide explains each one in plain terms and shows how a free, voluntary framework from the National Institute of Standards and Technology (NIST) can help a small team organize the work.

What "human in the loop" really means

There is no single setting. NIST's AI Risk Management Framework (AI RMF 1.0) describes a range: "Human-AI configurations can span from fully autonomous to fully manual." It adds that AI systems "can autonomously make decisions, defer decision making to a human expert, or be used by a human decision maker as an additional opinion."

For a small business, it helps to sort each task an agent performs into one of three modes:

• Approve first. The agent drafts or proposes; a person approves before anything happens. Good for refunds, discounts, quotes outside your normal range, and messages to your whole customer list.
• Act, then review. The agent acts on its own, and people check a sample of its work afterward. Good for routine booking, intake and answering common questions.
• Act on its own. Low-stakes, easily reversed tasks such as sending a confirmation for an appointment the customer just booked.

The mode should match the stakes, not the convenience.

Decide what needs a person

A simple way to sort tasks is to ask two questions: how much harm could a mistake cause, and how easy is it to undo?

The OWASP Top 10 for Large Language Model Applications, a security reference from the OWASP GenAI Security Project, names excessive autonomy as one root cause of AI systems taking damaging actions, and recommends a direct fix: "Utilise human-in-the-loop control to require a human to approve high-impact actions before they are taken."

For example, imagine a small property-management office using an agent to handle tenant requests. Logging a dripping faucet and scheduling a visit is low stakes and easy to fix. Waiving a late fee costs money. Responding to a report of a gas smell is a safety issue. A sensible design lets the agent schedule the faucet repair, routes fee waivers to the office manager for approval, and immediately hands the gas call to a person with instructions to follow the utility's emergency guidance.

Write your list down. A one-page "humans decide" list is more useful than a long policy no one reads.

Escalation: make it easy to reach a person

Escalation rules tell the agent when to stop. Common triggers include:

• The customer asks for a person.
• The agent fails to understand or resolve the request after a couple of attempts.
• The topic is on your humans-only list: complaints, cancellations, billing disputes, legal, medical or safety issues.
• The agent cannot find the answer in the information you approved.

Getting this wrong has a cost. A June 2023 report on chatbots in consumer finance from the Consumer Financial Protection Bureau (CFPB) described "doom loops," where customers get stuck in repetitive responses "without an offramp to a human customer service representative," and warned that "deficient chatbots that prevent access to live, human support can lead to law violations, diminished service, and other harms."

A good handoff carries context. The person who picks up should see a summary of what the customer wanted and what the agent already did, so the customer does not have to start over.

Review: check the work without rubber-stamping it

Review is how you catch problems before customers do. Two patterns work well for small teams:

• Approval queues for high-impact actions. Keep them short and give the reviewer the context needed to decide.
• Regular sampling of completed work, such as reading a set number of call transcripts or chat logs each week and noting errors.

Watch for a quiet risk: people trusting the agent too much. NIST's Generative AI Profile (NIST AI 600-1) describes automation bias as "excessive deference to automated systems," and notes that it can make other risks worse, including confident but false output. A reviewer who approves fifty items in a row without reading them is not really in the loop. Smaller queues, clear checklists and occasional deliberate test cases help keep reviews honest.

Takeover: the off switch and who can use it

Every agent needs a way to stop it quickly, and someone with the authority to do so. The AI RMF puts it directly in its MANAGE function: "Mechanisms are in place and applied, and responsibilities are assigned and understood, to supersede, disengage, or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use."

For a small business, that translates to:

1. A documented way to pause the agent, such as forwarding calls to a person or switching chat to a contact form.
2. Named people who are allowed to pull that switch, including after hours.
3. A fallback that works when the agent is off, so customers still reach you.
4. A quick test of the off switch before launch and after major changes.

Use the NIST AI RMF as a lightweight checklist

The AI RMF was released on January 26, 2023, and NIST says it "is intended for voluntary use." It is organized around four functions: Govern, Map, Measure and Manage. You do not need a compliance team to borrow its structure:

• Govern: decide who owns the agent, who approves changes, and who can shut it off.
• Map: list what the agent does, who it affects, and where human oversight applies. The framework's MAP 3.5 calls for processes for human oversight to be "defined, assessed, and documented."
• Measure: track how the agent performs, including a way for customers to report problems. MEASURE 3.3 calls for feedback processes for end users "to report problems and appeal system outcomes."
• Manage: monitor after launch and act on what you find. MANAGE 4.1 includes "appeal and override" and "incident response" in post-deployment monitoring plans.

Agencies that set up agents for several clients can turn these four headings into a standard onboarding template, so every deployment starts with the same oversight questions answered. The guide "How to Tell if an AI Agent Is Working: Metrics That Matter" covers the measurement side.

Key takeaways

• Match the level of human control to the stakes: approve first, act then review, or act alone.
• Require human approval for high-impact or hard-to-reverse actions such as refunds, discounts and mass messages.
• Write escalation rules and make reaching a person easy; pass context along with the handoff.
• Review samples of the agent's work regularly, and guard against rubber-stamping.
• Keep a tested off switch and name who can use it.
• The voluntary NIST AI RMF offers a simple Govern, Map, Measure, Manage structure for organizing oversight.

Sources

• National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 (January 2023)
• National Institute of Standards and Technology, AI Risk Management Framework
• National Institute of Standards and Technology, NIST AI 600-1, Generative Artificial Intelligence Profile (July 2024)
• OWASP GenAI Security Project, LLM06:2025 Excessive Agency
• Consumer Financial Protection Bureau, Chatbots in consumer finance (June 2023)