The use of autonomous AI agents in security operations has moved well past the pilot stage. Increasingly, enterprises have at least one end-to-end automated agentic AI workflow. Few enterprises ever reached even that level of automation through their security orchestration, automation, and response efforts. And that gap speaks to something real about what agentic AI is doing for enterprise security programs that rigid playbook automation never could.
It also hints at the risks for enterprises if their governance frameworks have fallen behind. The challenge isn’t inherently with the automation itself. Rather, it’s the absence of the explicit controls needed to define what data an agent can access, what actions it can take without human approval, and what the audit record must look like when something needs review. An agent operating in a mature secure operations center (SOC) with connected identity systems, endpoint tools, HR databases, network controls, threat intelligence and hunting, and more carries broad permission levels.
Those identities and other security controls, if not guarded by proper boundaries, pose significant risks and liabilities.
Why Guardrails Must Be Explicit
At a recent AI security panel, Mrinal Anand, a cybersecurity architect at SoFi Bank, described how his team approaches this problem. Every agent in SoFi's environment is scoped to a persona: a defined profile specifying what that agent is permitted to touch, what data it can access, and what actions fall within its authorized range.
If anything deviates from that persona or its boundaries, an alert is triggered. What data will this application touch? What happens if something goes wrong? Who is accountable? These are the questions Anand says must be answered before an agent goes live.
In an interview with CYBR.SEC.Media, Wim Remes, principal consultant at Toreon spoke to the importance of the “auditability” of agents and tracking whether they will act within their guardrails. “It remains a significant concern,” said Remes. Much of the difficulty traces back to foundational data quality problems. Most organizations do not have a clean asset inventory or reliable change management database. Without that, defining what an agent is sanctioned to do becomes a guessing exercise without a grounding to validate against.
Allen Badeau, chief AI officer at DigitalNet.ai, agrees. His team treats every agent as an insider threat from day one and monitors every action, every tool call, and every behavior for signs of drift. Because the agent's built-in motive is to produce an answer that the human will accept, which can be dangerous. Badeau’s policy is to ensure each agent operates under its own "constitution": a defined set of rules, skills, permitted actions, and confidence thresholds that must be met before the agent takes any consequential step.
Human-in-the-Loop and Approval Gates
How are organizations “keeping humans in the loop?” Stephen Morrow, chief solutions officer at AIR MDR, places the intervention point after the AI has completed triage and enrichment, after what he calls “the monotony.” At that point, an analyst reviews what the AI determined and decides whether they agree with the agent’s “decision.” For higher-consequence actions, containment among them, the gate is most explicit: a human sign-off is required before the agent proceeds rather than acting autonomously.
Terry O'Daniel, a security leader with extensive experience, describes a tiered model. There are low-risk, well-understood actions, such as closing an open port, that sit at the base and are reasonable candidates for autonomous execution. Higher up the pyramid, where context, blast-radius assessment, and business impact judgment matter, humans stay in the decision chain. Those tier assignments are not permanent. They shift as red team exercises validate the system's accuracy and as organizational confidence in the agent's judgment accumulates over time.
That’s why most organizations aren’t starting with fully automated end-to-end decisions on day one. The pattern is to enable an agent to make consistent, accurate decisions in a tight domain over weeks, then incrementally expand the scope. Trust is earned from demonstrated performance, not granted automatically from a planning diagram.
Data Privacy and Least-Privilege for AI
Agents create a class of data exposure that conventional least-privilege frameworks were not designed to address. They can query multiple systems simultaneously, and everything pulled into a prompt is what the model processes and may surface in its output, its logs, or a downstream tool call. Anand shared an example from model testing at SoFi, where a coding agent tasked with converting a document to PDF began uploading images to unknown third parties, then to Imgur after the egress proxy blocked the initial attempts, and it never stopped trying. The behavior was caught because observability was in place across pre-execution, execution, and post-execution layers. In production, without that telemetry, it would have gone undetected.
A reasonable response to such issues is prompt hygiene and tight data scoping: constraining what telemetry gets passed to the model, masking fields containing regulated identifiers unless the investigative context specifically requires them, and segmenting what reaches the LLM. Morrow's team at AIR MDR built a chain-of-custody system as a separate control the AI can’t change. Every agent action, including the reasoning behind it, is immutably logged. If a case reaches litigation, the first question from the opposing side will be how you trust what the AI did. The answer must already be in the record.
Change Control for Prompts, Tools, and Runbooks
Prompts and playbooks in an agentic SOC are typically operational logic. They determine how an agent reasons about an alert, what enrichments it pulls, and what actions it considers. Treating them informally, such as ad hoc editing, rapid deployment without a review, and running without version history, introduces the same risks as unreviewed detection rule changes: silent regressions, unexpected behavior, and no clean rollback path when something goes wrong.
Andrew Storms, director of engineering for AI security and guardrails at Anaconda, describes a workflow at his organization in which every AI agent configuration undergoes structured proposal review, peer sign-off, and a written specification before it reaches production. The agent's operating instructions, including what it must always check, what it must never do, and what classes of action require human escalation, are reviewed, versioned, and distributed centrally. The discipline should be the same as that applied to a mature code review process, because, operationally, that is what prompts and playbooks are. Badeau makes the same point regarding infrastructure: agents that behave within the constitution are observable and controllable; agents whose instructions were informally modified and never tracked are not.
Measuring Guardrail Effectiveness
Guardrails degrade silently. A threshold set for last quarter's alert volume and analyst capacity may be wrong today, and there is no automatic signal to alert to changes. Chris Steffen, vice president of research at Enterprise Management Associates, noted at a recent SOC AI panel that AI alone should not be making final decisions, but as accuracy and organizational confidence improve, the scope of what can be safely automated will expand. Tracking that expansion requires deliberate measurement: how often analysts approve versus override agent recommendations, how often they reverse autonomous actions the agent took without asking, and where near-misses occurred, such as cases where the agent approached an action that should have required human judgment.
Those patterns surface in two distinct ways: guardrails set too conservatively push unnecessary decisions to analysts and recreate the exact fatigue the agentic AI was supposed to relieve. While guardrails set too permissively let the agent act where human judgment should have remained in the chain. Here, significant mistakes are bound to occur rapidly.
The organizations getting this right treat guardrail design as ongoing operational discipline and a discipline that’s never finished.
