How companies govern agentic AI at 250, 1,000 and 10,000+ employees, with an autonomy-to-controls table, the standards that apply and what insurers add.
•
•
11 min read time
Companies govern agentic AI by registering every agent, tiering it by autonomy and blast radius, restricting the actions it may take, keeping a human on consequential decisions and logging every step. What changes with company size is the formality: a named owner and a stop switch at 500 employees, three lines of defense and independent validation at 10,000.
Why does agentic AI break model-era governance?
A model produces a score or a passage of text, and a human decides what to do with it. An agent receives a goal, plans a sequence of steps, calls tools and APIs, reads the results and acts again. The object being governed is a policy for acting.
Five properties do the damage. The plan is non-deterministic, so two runs from the same prompt can take different routes through the same tools. The agent holds credentials, so its permissions matter more than its accuracy. It can drift from the stated goal across a long chain without any single step looking wrong. Agents call other agents, so a failure in one propagates. And the base model can change under a vendor update, invalidating last quarter's tests.
The OWASP Top 10 for Agentic Applications, released December 9, 2025, by the OWASP GenAI Security Project's Agentic Security Initiative, names the resulting failure modes with codes ASI01 to ASI10. It is a separate document from the OWASP Agentic Skills Top 10.
Code | Name |
|---|---|
ASI01 | Agent Goal Hijack |
ASI02 | Tool Misuse |
ASI03 | Identity and Privilege Abuse |
ASI04 | Agentic Supply Chain Vulnerabilities |
ASI05 | Unexpected Code Execution |
ASI06 | Memory and Context Poisoning |
ASI07 | Insecure Inter-Agent Communication |
ASI08 | Cascading Failures |
ASI09 | Human-Agent Trust Exploitation |
ASI10 | Rogue Agents |
Enzai's agentic AI governance guide sets out the five control layers that answer these risks. This article asks how much of that apparatus a company of a given size needs, and in what order.
Which standards apply to agentic AI governance today?
Singapore's IMDA launched the Model AI Governance Framework for Agentic AI on January 22, 2026, at the World Economic Forum. It organizes guidance into four dimensions: assess and bound the risks upfront, make humans meaningfully accountable, implement technical controls and processes, and enable end-user responsibility. Law firm summaries report a May 20, 2026, update adding case studies, multi-agent and third-party agent risk, and guidance on automation bias, including monitoring human override rates. Enzai's glossary entry on the Singapore framework covers its lineage.
The EU AI Act is technology-neutral. Article 3(1) defines an AI system as one "designed to operate with varying levels of autonomy," so an agent is an AI system and its use case decides its risk class. General-purpose AI model obligations under Chapter V have applied since August 2, 2025. The Digital Omnibus on AI, Regulation (EU) 2026/1744, entered into force on July 27, 2026, and moved Annex III high-risk obligations to December 2, 2027, and Annex I obligations to August 2, 2028. Enzai's note on the revised deadlines lists what did not move.
The NIST Generative AI Profile (NIST AI 600-1, July 2024) maps twelve risk categories to the AI RMF's four functions. It mentions autonomous agents once, as a security threat, so treat it as a base and not as an agent framework.
How should autonomy level determine the controls?
Tier the deployment, not the model. The same foundation model can power a drafting assistant and an agent that moves money.
Autonomy level | What the agent may do | Minimum controls | Approval and review |
|---|---|---|---|
Assistive | Read data and propose; a human executes every action | Registered in the inventory with a named owner; prompt and output logging; standard AI acceptable-use policy | Owner sign-off; annual review |
Supervised | Execute low-consequence actions; a human approves anything consequential before it runs | Allow-list of tools and parameters; defined human checkpoints; session-level logging of every tool call; documented stop procedure | Governance function approves; review on any base model change and at least semi-annually |
Bounded autonomous | Act within an allow-list without per-action approval; escalate on exceptions | Impact limits (value, record count, reversibility); escalation triggers; independent testing before release; immutable action log; kill switch tested and reachable in seconds; monitoring of action distributions and override rates | Approver separate from builder; quarterly review; re-approval on scope change |
Fully autonomous | Open-ended tasks with broad tool access | All of the above, plus continuous behavioral monitoring, multi-agent recursion caps, incident response plan rehearsed, and a written case for why a lower tier was insufficient | Risk committee or equivalent; independent validation; board visibility |
How do companies with 250 to 1,000 employees govern agentic AI?
A company this size has no second line of defense, and should not build one for agents. Proportionate governance here is five things done properly.
Keep a central register of every agent, including those built inside SaaS platforms the business already pays for, recording the owner, the purpose, the systems it can write to and its autonomy tier. Run a light intake: a one-page form that tiers by data sensitivity, autonomy and reversibility, reviewed by whoever owns risk or security. Name an owner who can answer for each agent. Require a human to approve any action that spends money, changes a customer record or sends anything externally, until the agent has a track record. And define a kill switch: who can stop the agent, how, and how quickly, tested before go-live.
What is not needed at this size is a three-lines-of-defense structure, an independent validation team or a committee. As an illustrative scenario, a 600-person software company with agents in customer support, finance operations and sales research can run this on a register, a form and a fortnightly review, owned by whoever holds the Head of AI or security role.
How do companies with 1,000 to 10,000 employees govern agentic AI?
Between 1,000 and 10,000 employees the register stops being enough. Agents are being built by several teams the governance owner does not manage, and at least one of them touches a regulated decision or a customer.
Intake becomes mandatory. A use case cannot reach production without a recorded approval that names who reviewed it, what evidence they saw and why they agreed. The intake process guide covers the design; Enzai's use case and initiative workflow tiers submissions by use case type, data sensitivity and autonomy, and logs each approval or rejection with reviewers, evidence and rationale.
Risk tiering ties to two variables: the autonomy level from the table above, and blast radius, meaning what the agent can reach and how reversible its actions are. A read-only research agent and an agent that writes to the billing system are governed differently even when both sit on the same model.
Separation between builders and approvers begins here. The team that builds an agent should not approve it for a Bounded autonomous tier. Evidence retention starts to matter for its own sake: test results, allow-lists and approvals stored against the agent and the base model version it was tested on.
Vendor agents enter the picture. Agents embedded in a CRM, an HR platform or a service desk carry the vendor's permissions inside your environment, and the vendor can change the model without notice. Each one needs an AI inventory entry and a vendor assessment; Enzai's vendor AI risk guide covers the questions to ask.
How do enterprises with 10,000+ employees or in regulated sectors govern agentic AI?
At this scale, or in any regulated sector regardless of headcount, agentic governance becomes a control environment that a supervisor can examine.
Three lines of defense apply. The business owns and operates agents; a second-line risk or compliance function sets policy, tiers systems and approves higher-autonomy deployments; internal audit tests both. Independent validation of Bounded and Fully autonomous agents sits in the second line and reviews the allow-list, the escalation logic and the test evidence, not only the model.
A board or risk committee should see how many agents are in production, at which tiers, how many exceptions escalated, how often humans overrode the agent, and any incident. IMDA's suggestion to monitor human override rates is a practical metric here, because a rate that falls to zero indicates automation bias, not a well-behaved agent.
Evidence is built for a regulator. Under the EU AI Act a deployer of an Annex III high-risk system must assign human oversight to natural persons with the competence, training and authority to exercise it (Article 26), and must keep the logs the system generates. That evidence must exist per agent and per version.
Existing model risk management is where large institutions stumble. Banks under SR 26-2 in the United States find generative and agentic AI expressly out of that guidance's scope; a UK bank with internal model approval finds no such exclusion in PRA SS1/23. The working answer is a parallel agentic framework governed by the same committee as model risk, with a documented boundary and one inventory. Enzai's agentic AI governance module is built for this layer: it tiers each agent by what it can do unsupervised, defines the actions an agent is allowed and disallowed in production, routes out-of-bounds attempts to human review, shows how agents call each other, and maps controls to the OWASP Agentic Top 10.
What changes between the three size segments?
Dimension | 250 to 1,000 employees | 1,000 to 10,000 employees | 10,000+ or regulated |
|---|---|---|---|
Register | Central list with owner, purpose, tier | Inventory with framework mapping and vendor agents | System of record examined by audit and supervisors |
Intake | One-page form, single reviewer | Mandatory workflow, recorded approval, tiered templates | Second-line approval, independent validation for higher tiers |
Tiering basis | Autonomy and reversibility | Autonomy and blast radius | Autonomy, blast radius and regulatory classification |
Human oversight | Human approves consequential actions | Checkpoints defined per tier | Named, trained oversight personnel per Article 26; override rates monitored |
Separation of duties | Owner and reviewer distinct | Builders separate from approvers | Three lines of defense |
Evidence | Logs kept; approvals in the register | Tests and approvals stored per agent and model version | Regulator-facing evidence trail, board reporting |
Stop mechanism | Kill switch defined and tested | Kill switch plus escalation triggers | Rehearsed incident response and notification |
What governance frameworks does agentic AI in insurance need?
An agent that quotes a premium, triages a claim or adjusts an underwriting decision is making a regulated decision, and the framework for it already exists in four layers.
The EU AI Act lists "AI systems intended to be used for risk assessment and pricing in relation to natural persons in the case of life and health insurance" as high-risk under Annex III, point 5(c). An agent doing that work in the EU faces the full high-risk obligations from December 2, 2027. For everything else, EIOPA's Opinion on AI governance and risk management (EIOPA-BoS-25-360, August 6, 2025) sets out how existing insurance law applies to AI systems that are neither prohibited nor high-risk, covering risk management, fairness and ethics, data governance, documentation, transparency, human oversight, and accuracy and cybersecurity, resting on Article 41 of Solvency II and Article 17 of the IDD.
In the UK, the FCA's Consumer Duty, introduced by PS22/9 and in force since July 31, 2023, for products open to sale or renewal, requires firms to deliver good outcomes across products, price and value, understanding and support. An agent that sets prices or handles claims operates inside all four outcomes. Do not rely on PRA SS1/23 here; it applies to banks and excludes insurers.
In the United States, the NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers, adopted December 4, 2023, and since issued by more than twenty states and the District of Columbia, expects a written AIS Program covering governance, risk management, third-party AI oversight and cooperation with market conduct examinations.
The operational point is the interaction with actuarial model governance. Pricing and reserving models already sit under an actuarial function with validation and sign-off. An agent that calls a pricing model as a tool does not take that model out of actuarial governance; it adds a layer above it whose actions need their own tier, allow-list and log. Fairness exposure rises, because an agent can combine a compliant model with a discretionary step the model's validation never covered. Insurers of any size should hold pricing, underwriting and claims agents at Supervised until override data supports a higher tier.
What this means operationally
For a governance team at any of the three sizes, the sequence is the same; the depth differs.
Build the register first. Every agent, including vendor-embedded ones, with owner, purpose, tool permissions and autonomy tier in one AI inventory that records risk tier, autonomy level, mapped frameworks and evidence per system.
Tier by autonomy and blast radius, and attach the controls from the table above. At 250 to 1,000 employees this is the whole program; from 1,000 upward it drives intake and approval routing.
Enforce the allow-list and the stop switch technically. A policy that says an agent may not write to the billing system is not a control until the agent cannot. Enzai's agentic module defines allowed and disallowed actions and routes out-of-bounds attempts to human review.
Retain evidence per agent and per base model version, and track override rates and action distributions as part of ongoing AI monitoring. Reconcile the agentic framework with model risk management, and with actuarial governance in insurance.
Map to the standards a regulator will name, starting with the OWASP Agentic Top 10. If you are comparing dedicated platforms, Enzai's comparison with Credo AI notes where Credo AI is stronger, on technical model evaluation, red-teaming and drift detection, and where Enzai enforces at the action layer.
To see how the register, tiering and enforcement fit together for your size of organization, talk to the Enzai team about agentic AI governance.
Empower your organization to adopt, govern, and monitor AI with enterprise-grade confidence. Built for regulated organizations operating at scale.








