The 10,000-Agent Problem: Who Has the Authority to Let Machines Act?

Summary

Within one year, the question facing an enterprise has moved from how do we instruct a model well to who authorised this actor, what may it legitimately do, how far may it delegate, who answers for it, how is it stopped, and what proves whether it stayed within its mandate. Vendors now ship agents that work while the user is away, hold their own identities and are billed by consumption; standards bodies are drafting agent identity, delegation and authorisation profiles; security organisations rank “excessive agency” among the top risks of the year. This analysis reads the published record of September–October 2026 and finds that three layers are maturing quickly — technical identity, technical permission and runtime approval — while a fourth layer remains largely untreated: the translation of an organisational mission or business instruction into a bounded, machine-enforceable and revocable mandate whose legitimacy can be reconstructed from evidence. That fourth layer is the subject of Mandate Engineering, and the reason the Institute proposes an Agent Fleet Governance Applied Case Lab and the ME-001 teaching case.

1. What changed in September 2026

On 25 September 2026 Microsoft announced a reorganisation of Copilot into three spaces — Home, Code and Autopilot — in a post signed by Jared Spataro, Chief Marketing Officer, AI at Work [1][2]. Home merges chat with delegated “Cowork” tasks; Code lets non-developers describe applications in natural language; Autopilot is a persistent agent that continues working while the user is away [1][2]. Secondary reporting adds that Autopilot is given a name, role and goal, has its own identity, memory and workspace inside the tenant, can be @-mentioned in Teams and Outlook, and operates “within the organisation’s permissions and governance rules”; Home and Code roll out through the Frontier programme, while Autopilot moved to private preview at the end of September [3].

Two features of the announcement matter for authority. First, the agent is no longer a session: it persists, holds state and acts between interactions. Second, Microsoft’s billing model splits a per-user licence from usage-based “Copilot Credits” for agentic work, with admin controls for group spending policies, monthly caps, per-user caps and credit requests that an admin approves [3][4]. A spending ceiling for agents is therefore a published product control. What the announcement does not describe — on the published record we examined — is per-action approval for Autopilot, how a user or admin pauses or revokes a running Autopilot, how its actions are reviewed after the fact, or which systems it may change [2]. We note the absence; we do not infer that no such control exists, since Autopilot is in private preview and documentation may follow.

2. Microsoft’s published permission and approval mechanisms

The agent-governance stack Microsoft has documented is more complete than the launch post suggests, and it is instructive to read it as a layered system.

Identity. Since May 2026 every new Copilot Studio agent receives a Microsoft Entra Agent ID, an identity in Entra with audit logging, lifecycle management and integration with Entra ID Governance; connector permissions are attached to the agent identity as API permissions at publication, are visible to admins, can be targeted by Conditional Access policies (network location, device compliance, risk), and are re-validated at runtime by the connector runtime against data policies [5]. Deleting the agent deletes the identity [5]. The documentation we read does not describe a separate consent workflow, an explicit disable/revoke operation on the agent identity (other than deletion and Conditional Access), or on-behalf-of semantics linking the agent to the user who created it [5].

Runtime approval. A Microsoft 365 Roadmap item published 1 September 2026 (RM570434) lets makers require human approval before an agent calls a specific tool: a per-tool, per-agent toggle pauses the run, surfaces an approval request inline in Teams or Microsoft 365 Copilot describing what the agent plans to do, and lets a reviewer approve once, approve for the session or deny; it is presented as a deterministic control independent of the agent’s instructions, aimed at actions such as sending e-mail, closing tickets or processing payments [6]. Who may approve, how reviewers are assigned and what happens on timeout are not stated [6].

Control plane. Agent 365, described in September 2026 reporting, provides a unified registry listing each agent with its identity, owner, permissions and usage history across Microsoft and third-party platforms; a tools view where admins block risky MCP servers, plugins and connectors and approve or reject new tool requests; policy, access and observability controls enforced through Entra, Purview, Defender and Intune; and isolation of unsanctioned local agents [7]. Audit is covered indirectly through usage history; neither revocation of an agent’s authority nor a kill-switch is described in those terms [7].

Read together, these are serious engineering: an agent has an identity, an owner, enumerable permissions, conditional access, per-tool human approval, a spending cap and a registry. They establish who the workload is, what it may call, who may be asked before it calls and how much it may spend.

3. Standards: identity is converging, mandate is not

The IETF’s WIMSE working group (Workload Identity in Multi-System Environments, chartered 2024) is the main home for agent identity; its architecture draft reached version 07 in March 2026 [8]. An individual submission, WIMSE Applicability for AI Agents (draft-ni-wimse-ai-agent-identity-03, 1 October 2026), proposes that agents hold their own workload credentials bound cryptographically to an owner — the accountable individual or organisation — through dual-identity credentials, with agent-, owner- and server-mediated issuance models, attestation at bootstrap and renewal, and a hardware root of trust [9]. The draft states goals of short-lived credentials and narrow, task-oriented tokens, but defines no scope model; delegation chains, revocation and audit records are not addressed, security considerations are marked TODO, and trust-anchor distribution is explicitly out of scope [9]. A separate informational draft (draft-klrc-aiagent-auth-00) profiles OAuth 2.0 token exchange so that an agent’s own client identity is kept separate from the delegated subject, permissions do not accumulate along agent-to-agent call chains, tokens are short-lived and audience-restricted, revocations propagate through the OpenID Shared Signals Framework and detailed audit logging is baseline [10]. None of these has been adopted as a working-group item or advanced to Proposed Standard; terminology has not converged; a proposed discovery group (DAWN) explicitly excludes stable identifiers and trust management from its first phase [10].

On the management side, ISO/IEC 42001:2023 defines an AI management system (AIMS) — organisational policies, objectives, risk and impact assessment, controls and certification of the management system rather than the product [11]. It is the natural home for accountability structures, but the public descriptions we examined do not address autonomous agents or the delegation of authority to them specifically [11].

On the security side, the OWASP Top 10 for LLM applications 2026 (3 August 2026) moved Excessive Agency — more autonomy, permissions or unsupervised reach than a task needs — from sixth to third place, weighting 6,639 real incidents; OWASP’s separate Top 10 for Agentic Applications (December 2025) names identity and privilege abuse, tool misuse and cascading multi-agent failures; and an Agent Control Standard at version 0.1, donated to the OWASP GenAI Security Project in September 2026, proposes “guardian agent” enforcement points, OpenTelemetry/OCSF tracing and an Agent Bill of Materials — with deny/modify enforcement planned for a later version and no stated requirement for human approval steps or kill switches [12].

4. The untreated layer: from mission to mandate

Place the published mechanisms on one axis from technical to organisational:

Three consequences follow for an enterprise with a large agent population, and each is reproducible as a teaching scenario in ME-001:

1. Individually permitted, collectively unauthorised. A permission and a per-user credit cap bound each agent; they do not bound the enterprise. Three hundred payment agents each within limit can exceed a treasury ceiling that no mechanism in Section 2 evaluates by default (ME-001 S2). Aggregate limits must be designed as mandate limits with an evaluator that fails closed when the aggregate state is unknown.
2. Approved tool, changed plan. Per-tool approval gates which capability is used, not whether the plan the agent now proposes is the plan that was authorised. An agent that cannot ship by carrier A and proposes carrier B at higher cost calls the same tool (ME-001 S3). Change-of-plan re-authorisation is a mandate concept, not a permission concept.
3. Logged is not legitimate. Audit logging in every stack above records that an action occurred under an identity. It does not record that a valid mandate existed for it, that the mandate was within the owner’s authority, or that the owner’s authority was legitimate (ME-001 S6). Evidence sufficiency must be designed: decision records that name the mandate version, the boundaries evaluated, the aggregate state and the approval act, hashed and reproducible.

To these add the fleet-scale questions the drafts leave open: revocation of a parent mandate while delegated tasks are pending (S5), inheritance that narrows rather than creates (S1), and identity compromise handled as suspension and incident rather than retry (S4).

5. What vendors can do today, and what remains a research question

Established on the published record: enterprises can give agents identities and owners, enumerate and condition their permissions, require human approval per tool, cap spend, register and block tools, and isolate unsanctioned agents [4][5][6][7]. Drafted but not standardised: dual-identity credentials bound to owners, token-exchange delegation without permission accumulation, propagated revocation signals [9][10]. Research questions on which the Institute takes a position: how an organisational objective is decomposed into mandates with testable boundaries and reserved decisions; how aggregate and portfolio limits are specified and enforced across populations; what a sufficient evidence record for authority (not merely for action) contains; how revocation cascades are verified; and how any of this is assessed as a professional competence rather than a configuration task. None of these is answered by a product feature, and the Institute does not claim that any vendor or standards body has failed — only that the published work we examined addresses other layers.

6. Implications for professional formation

If the organisational-mandate layer is where enterprise exposure concentrates, then the capability the economy lacks is not another tool operator but a practitioner who can write a mandate another person could operate, test and revoke; design limits a fleet cannot bypass; make and record runtime authorisation decisions; execute and verify revocation; and reconstruct, from evidence alone, which authority an action rested on and where proof ends. The Institute’s Foundation programme (specified, development-draft level) prepares the single-actor case; the proposed Applied Case Lab and ME-001 extend it to populations. Both remain unvalidated until a cohort has run; the deterministic demonstration that accompanies ME-001 establishes only that a stated rule set can be enforced — it establishes nothing about any real product, enterprise or legal position.

7. Uncertainty statement

This analysis relies on vendor posts, documentation, roadmap items, IETF drafts and trade and research-note reporting as of 12 October 2026. Private-preview products may carry controls not yet documented. Draft status, roadmap dates and standard adoption change frequently; readers should verify current status at the sources. Secondary sources ([3], [7], [10], [12]) summarise primary material we could not fully access; where their claims are not corroborated by a primary source they are reported as such.

References

[1] Microsoft, “Introducing the new Copilot with Home, Code and Autopilot”, Official Microsoft Blog, 25 September 2026, J. Spataro. https://blogs.microsoft.com/blog/2026/09/25/introducing-the-new-copilot-with-home-code-and-autopilot/ (accessed 12 October 2026 via secondary summaries; the primary page did not load at the time of access). [2] Microsoft Source EMEA, “New Microsoft Copilot brings Home, Code and Autopilot together”, 25 September 2026, K. Thurnher. https://news.microsoft.com/source/emea/2026/09/new-microsoft-copilot-brings-home-code-and-autopilot-together/ (accessed 12 October 2026). [3] DEV Community, “The new Microsoft Copilot: Home, Code and Autopilot explained”, 28 September 2026 (secondary; cites Microsoft Learn billing pages updated 25 September 2026). https://dev.to/getpack/the-new-microsoft-copilot-home-code-and-autopilot-explained-5f7i (accessed 12 October 2026). [4] Microsoft Learn, “Usage-based billing overview — Copilot Credits” and “Manage Copilot Credits”, updated 25 September 2026 (as cited in [3]). https://learn.microsoft.com/en-us/microsoft-365/copilot/usage-based-billing-overview-copilot-credits [5] Microsoft Learn, “Microsoft Entra Agent IDs for Copilot Studio agents”, last updated 15 September 2026. https://learn.microsoft.com/microsoft-copilot-studio/admin-use-entra-agent-identities (accessed 12 October 2026). [6] Microsoft 365 Roadmap item RM570434, “Microsoft Copilot Studio: Enabling makers to require human approval for tool calls”, published 1 September 2026, GA target September 2026. https://mc.merill.net/message/RM570434 (accessed 12 October 2026). [7] Redmond Magazine, “Microsoft Agent 365 adds centralized controls”, 16 September 2026 (secondary; summarises a Microsoft Mechanics post). https://redmondmag.com/blogs/redmond-dispatch/2026/09/microsoft-agent-365-adds-centralized-controls.aspx (accessed 12 October 2026). [8] IETF, draft-ietf-wimse-arch-07, “Workload Identity in Multi System Environments (WIMSE) Architecture”, March 2026. https://datatracker.ietf.org/doc/html/draft-ietf-wimse-arch-07 [9] Y. Ni, C. P. Liu, M. Richardson, draft-ni-wimse-ai-agent-identity-03, “WIMSE Applicability for AI Agents”, individual submission, 1 October 2026 (expires 4 April 2027). https://www.ietf.org/archive/id/draft-ni-wimse-ai-agent-identity-03.html (accessed 12 October 2026). [10] Cloud Security Alliance, “IETF’s Race to Standardize AI Agent Identity”, research note, 2 September 2026 (secondary; covers draft-klrc-aiagent-auth-00, DAWN, draft-sharif-agent-identity-framework-01, draft-ihsanullah-dnsid-01, MCP authorization). https://labs.cloudsecurityalliance.org/research/csa-research-note-ietf-agent-identity-standards-20260902-csa/ (accessed 12 October 2026). [11] ISO/IEC 42001:2023, “Information technology — Artificial intelligence — Management system”, December 2023; ANAB/ANSI blog summary, 23 August 2024 (modified 21 May 2025). https://blog.ansi.org/anab/iso-iec-42001-ai-management-systems/ (accessed 12 October 2026). [12] Cloud Security Alliance, “OWASP’s 2026 LLM Top 10 and New Agent Control Standard”, research note, 4 September 2026 (secondary; covers OWASP Top 10 for LLM Applications 2026 of 3 August 2026, OWASP Top 10 for Agentic Applications of December 2025, and the Agent Control Standard v0.1). https://labs.cloudsecurityalliance.org/research/csa-research-note-owasp-genai-top10-2026-agent-control-stand/ (accessed 12 October 2026). [13] IETF RFC 8693, “OAuth 2.0 Token Exchange”, January 2020; RFC 9728, “OAuth 2.0 Protected Resource Metadata”, April 2025 — background standards referenced by [10].