An AI agent becomes materially more powerful when it can call tools. Search, databases, email, ticketing systems, code execution, APIs, and Model Context Protocol servers can turn a language response into a real business action. The same capability changes the security model: a manipulated or mistaken agent can now attempt operations against systems that previously required explicit application logic or human interaction.
The current AB-100 scope addresses agent security, governance, prompt manipulation, access controls, audit, extensibility, and tool selection. Those concerns are inseparable. Tool security is not a prompt-engineering problem. It is an identity, authorization, interface, validation, network, and monitoring problem in which the language model is an untrusted decision-making component.
The safest architecture assumes the agent may eventually request the wrong action. The surrounding system must make harmful requests hard or impossible to execute.
This is a useful zero-trust mindset for agentic systems: do not grant authority because the request was generated by an approved model or arrived from an internal runtime. Authenticate every workload, authorize every sensitive operation, validate every parameter, and log the result. The agent participates in the trust model; it does not sit above it.
Security teams should inventory tools just as they inventory applications and service accounts. Record owner, purpose, identities, permissions, target systems, data classification, network route, and retirement date. An agent platform can otherwise accumulate connectors faster than ordinary access-review processes notice them.
Build a threat model around actions, not only prompts
List every tool and the consequences of each operation it exposes. Reading a public status page is different from retrieving customer records. Creating a draft ticket is different from deleting an account. The threat model should consider confidentiality, integrity, availability, financial impact, external communication, and whether the action can be reversed.
Then identify how the agent could be induced to misuse the tool. A malicious user might ask directly. A retrieved document might contain hidden instructions. A compromised downstream service might return adversarial text. The model might simply misunderstand a legitimate request. These causes differ, but the control objective is the same: the agent should not possess more authority than the workflow needs.
The general AI security risk discussion becomes concrete at the tool boundary because model behavior can create side effects. Treat every tool call as a security-sensitive request that needs ordinary application controls.
Expose the smallest useful tool contract
A tool should represent a business capability, not a raw administrative surface. “Get order status” is safer than unrestricted database query access. “Create refund request” is safer than a generic payment API credential that can move money directly. Narrow contracts reduce the number of harmful actions the model can even express.
Keep schemas explicit. Enumerate required fields, accepted values, limits, identifiers, and operation types. Strongly typed inputs make validation easier and reduce ambiguity. Descriptions should help the model choose the right tool, but security must not depend on the model interpreting the description correctly.
Avoid combining unrelated privileges in one tool for convenience. Read, create, approve, and delete operations may deserve separate interfaces and separate authorization. This mirrors established application security practices: reduce attack surface and validate at trust boundaries.
Tool descriptions are security-relevant configuration because they influence when the model chooses an operation. Keep descriptions precise, test them with ambiguous requests, and version changes. A seemingly harmless rewrite can increase unintended tool selection even though the underlying API permissions are unchanged.
For MCP and other extensible tool ecosystems, validate server provenance and configuration before connection. A general protocol makes integration easier, but it does not make every server trustworthy. Apply the same vendor, code, network, identity, and change-management controls used for other privileged integrations.
Capability boundaries should be reflected in the API surface. A generic database query tool, shell tool, or “call arbitrary URL” connector makes authorization difficult because the model can construct operations the designer never reviewed. Prefer business-level functions such as `get_invoice_status`, `create_case_note`, or `submit_refund_request`, each with constrained schemas and server-side policy checks. This reduces the action space the model can explore and produces audit records that map directly to business intent.
Separate read and write capabilities even when the downstream platform uses one API. Read access may be safe for routine automation, while write access may require a narrower identity, an approval, or additional validation. A model should not gain mutation authority simply because it needs to inspect the same resource. This separation also makes incident containment easier because operators can disable writes without making the agent completely blind.
Use distinct workload identities and least privilege
Do not hide all agent activity behind one shared master credential. Where the platform and target system support it, use workload or agent identities that can be assigned explicit roles. This improves both enforcement and auditability because the downstream service can recognize which workload performed the operation.
Permissions should be scoped to resources and actions. An agent that summarizes tickets may need read access to one queue, not tenant-wide service administration. A deployment agent may need to update one application in a test environment, not every production resource. Separate identities by environment so a development agent cannot accidentally reach production.
The identity and access principles represented by SC-300 are central here: authenticate the workload, assign least privilege, control privileged operations, review access, and revoke it when the system changes. Prompt instructions can guide behavior, but RBAC and target-system policy determine what is actually allowed.
Preserve user authority in delegated scenarios
Some agents act on behalf of a signed-in user. The agent’s service identity should not silently become a privilege escalation path. If the user cannot access a record directly, asking the agent for it should not make the record available unless the business process explicitly authorizes a different service scope.
Use delegated identity or equivalent authorization context where appropriate so downstream systems can enforce the user’s effective permissions. Where a service identity is necessary, document why the broader authority is required and put additional workflow controls around sensitive operations.
Segregation of duties still applies. A user allowed to create a transaction may not be allowed to approve it. An agent should not merge those roles merely because it has tools capable of both steps. Keep approval authority independent of the requester and the drafting agent.
Delegation should be bound to a user, session, purpose, and duration. If a user authorizes an agent to update one support case, that consent should not become a reusable credential for unrelated cases. Pass only the scopes needed for the requested action and re-evaluate authorization when the target or operation changes. Where the platform supports on-behalf-of flows, preserve the originating identity in downstream audit records so the action is attributable to both the user and the automated agent.
Validate every tool request outside the model
The application or target service should validate parameters before execution. Check resource ownership, amount limits, permitted destinations, identifier formats, state transitions, and business rules. Reject unexpected fields rather than passing them through. The model’s apparent confidence is not a substitute for server-side validation.
Use idempotency for operations that may be retried. Agent workflows can repeat calls after timeouts or ambiguous responses. A duplicate request should not create two payments, two tickets, or two customer records. Carry a stable operation identifier through retries and make the target system recognize duplicates.
Enforce state transitions in the business service. An agent should not be able to approve a request that is already canceled, refund an order that has not settled, or close an incident that still has mandatory tasks open. The target application knows the authoritative state and should reject transitions that violate it regardless of the model’s explanation.
For financial or quota-sensitive operations, add aggregate limits. Per-call validation might allow a small payment, but hundreds of valid small payments can still exceed the intended authority. Daily totals, velocity checks, and anomaly rules protect against repeated misuse that stays below individual thresholds.
For destructive actions, consider a prepare-and-commit design. The agent can create a proposed change, but a deterministic service or human approval performs the final commit only after validation. This narrows the authority of the model while preserving automation in the earlier steps.
Validation should include semantic preconditions, not only schema checks. A refund amount may be a valid number yet exceed the user’s entitlement; a deployment target may be a valid string yet refer to production during a freeze window. Check business rules, resource state, authorization, rate limits, and replay protection in deterministic code before execution. The model can propose an operation, but it should never be the final enforcement point for the policy governing that operation.
Protect secrets, connections, and network paths
API keys and connection secrets should never be placed in prompts or returned to the model. Store them in managed secret or connection systems and let the runtime use them without exposing raw values. Rotate credentials and make ownership visible so abandoned integrations do not remain active indefinitely.
Network controls can limit blast radius. Restrict outbound destinations where practical, use private connectivity for sensitive services, and avoid allowing an agent runtime to connect arbitrarily to any host on the internet or internal network. If a tool connection is compromised, network policy can still prevent it from reaching unrelated systems.
Tool outputs can contain secrets or sensitive records too. Limit returned fields to what the next step needs. The practices in secure AI handling of PII apply to both tool inputs and outputs because either can become part of conversation state, traces, or subsequent prompts.
Put human approval around high-impact operations
Some actions remain too consequential for autonomous execution even when the tool is technically secure. Transfers, account closure, access grants, legal notices, and irreversible data changes may require an authorized reviewer. The agent can prepare the action and evidence, but the workflow should block execution until approval is recorded.
Approval should bind to the exact operation. If the agent changes the recipient, amount, resource, or requested permission after approval, the approval should no longer be valid. This prevents a benign proposal from becoming a more privileged action during a later step.
Do not use human review to compensate for a dangerously broad tool. Reviewers are fallible and can experience fatigue. Narrow interfaces, least privilege, validation, and approval reinforce one another rather than substitute for one another.
Log, monitor, and recertify tool permissions
Record tool selection, operation type, agent identity, user context where appropriate, target resource, result, and authorization outcome. Sensitive argument values can be redacted or referenced by secure identifiers. The log should support investigation without becoming a second unrestricted copy of the underlying data.
Monitor for unusual behavior such as repeated authorization failures, new write patterns, unexpected destinations, excessive retries, or a sudden increase in high-risk operations after a release. These signals can indicate prompt manipulation, a routing defect, or a configuration change that expanded the agent’s effective authority.
Link security telemetry to the deployed agent and tool version. If a new prompt or toolbox release causes a rise in denied operations, rollback may be safer than tuning in place. Without version correlation, teams can see suspicious activity without knowing which change caused it.
Recertification should involve the business owner as well as the platform team. Technical logs may show that a permission is used, but only the business owner can confirm that the use is still necessary and within the intended process. Remove stale privilege rather than keeping it because no incident has happened yet.
Access also needs periodic recertification. Remove tools the agent no longer uses, narrow roles when processes change, rotate connections, and verify ownership. Tool access tends to accumulate as prototypes evolve; governance should actively reduce stale privilege.
Maintain an inventory of which agents can call which tools under which identities and scopes. Review that inventory when owners change, workflows retire, or tools add new operations. Recertification is especially important for long-lived agents because permissions tend to accumulate as features are added. A quarterly or release-triggered review can remove dormant access before it becomes an invisible privilege path.
Permission reviews should consider behavior as well as configuration. An agent may technically need a capability but use it only in one rare workflow; telemetry can reveal whether the granted scope is broader than the observed requirement. Combine entitlement inventories with tool-call history to identify dormant privileges, unexpectedly frequent sensitive operations, and agents that have drifted into responsibilities outside their original design.
Test security boundaries with adversarial scenarios
Pre-production testing should actively try to make the agent misuse tools. Include direct malicious prompts, instructions embedded in retrieved documents, conflicting user roles, invalid identifiers, attempts to exceed amount limits, duplicate events, and requests to reveal connection secrets. Verify both the model response and the actual target-system behavior.
Security regression tests should run after prompt, model, tool, identity, or policy changes. A new model may interpret a tool description differently. A connector update may add fields or operations. A revised prompt may make the agent more willing to act. Release control should treat these as changes to the effective security posture.
For organizations using the broader Microsoft agent stack, secure tool access comes from assuming the model can be wrong and building controls that remain correct anyway. Narrow tools, scoped identities, delegated authorization, server-side validation, protected connections, human gates, monitoring, and adversarial testing turn agentic action from an implicit privilege into a governed interface.