Why the “self-defending enterprise” may be the most dangerous agent in the building

Beyond Zero: Enterprise Security for the AI Era is an intelligent paper.

That is what makes it dangerous.

Joseph Valente and Michal Zalewski correctly identify a profound shift in enterprise computing: the application is no longer a sufficiently precise security boundary. An AI agent given access to email, file storage, customer records, source code, or payment systems can perform thousands of actions at machine speed, often using the broad permissions inherited from a human operator. Their proposed response is to move authorization down to each action on each resource, combine static policies with dynamic AI reasoning, maintain a continuously updated model of the enterprise, and allow the resulting system to challenge or interrupt activity in real time.

It sounds sensible.

It may even be inevitable.

It is also an excellent recipe for recreating, at a vastly greater scale, the problem it claims to solve.

The paper calls its destination a self-defending enterprise. I see the outline of a machine that observes everyone, interprets their purposes, assigns meaning to their behaviour, and decides - at machine speed - which actions humans and other machines should be permitted to take.

That is not merely a security control.

It is an institutional authority.

And the paper never fully explains who authorized that machine.

The diagnosis is right

The familiar enterprise model is broken.

We assign a person a role. We grant the role access to an application. The application exposes dozens or thousands of functions. Inside those functions are countless data objects, workflows, communications, and potential consequences.

That was already a rough approximation when humans operated the software manually.

It becomes absurd when an agent can:

• read every document available to its operator;
• connect information across systems;
• send messages;
• modify records;
• invoke tools;
• create accounts;
• deploy code;
• initiate payments;
• delegate work to other agents.

The difference between “may access the application” and “may take this action for this purpose” becomes existential.

The authors are also right about ambient authority. An agent should not inherit every permission accumulated by a human over ten years of changing jobs, projects, and exceptional assignments. The fact that a credential permits an operation does not mean that every automated process acting through that credential should perform it.

Their chosen enforcement point - the individual action on a particular resource - is therefore far better than the coarse application boundary.

But a correct enforcement point does not guarantee a correct source of authority.

That is where the paper turns.

The new trust center is an AI-generated picture of the enterprise

Beyond Zero proposes an Enterprise Security World Model assembled from organizational charts, human-resources systems, project records, identities, relationships, resource classifications, historical behavior, and other enterprise data. AI would help convert this material into attributes that could be consulted quickly when access decisions are made. (ResearchGate)

The idea is to determine not only:

Does Chris possess a credential that can access this file?

but:

Given what the enterprise knows about Chris, his role, his current projects, this file, its sensitivity, and recent behavior, does this request make sense?

That is an appealing security question.

It is also an invitation to build a corporate epistemology engine.

The world model will contain some authoritative records:

• formal job assignments;
• active credentials;
• approved projects;
• data classifications.

It will also contain interpretations:

• inferred relationships;
• predicted relevance;
• normal behavior;
• unusual behavior;
• likely purpose;
• expected activity;
• estimates of risk.

Those are not the same kind of information.

A payroll record is not an inference.

An assignment is not a behavioral norm.

An anomaly is not misconduct.

An organizational chart is not a complete account of who is entitled to help whom.

An AI-generated attribute extracted from unstructured corporate material is not an institutional fact merely because it is available at low latency.

Yet to achieve the paper’s proposed machine-speed decisions, these distinctions must be compressed into attributes the policy engine can consume.

That compression is where judgment disappears while the appearance of precision increases.

Intent is not telemetry

The paper repeatedly invokes context and intent. Its dynamic system is expected to notice when an action departs from established patterns or appears inconsistent with the user’s task. (ResearchGate)

But intent is not a sensor reading.

We can record:

• what a person requested;
• what task was assigned;
• what an agent proposed;
• what policy applied;
• what action was attempted;
• what result followed.

We may infer that these things are inconsistent.

That inference may justify a challenge or a pause.

It does not establish what the person truly intended, and it certainly does not establish what they were institutionally authorized to do.

Consider four cases:

1. A person behaves unusually while performing legitimate emergency work.
2. A person behaves normally while carrying out an illegitimate instruction.
3. An agent performs a low-risk operation for an unauthorized purpose.
4. An agent follows a familiar pattern after its delegated authority has expired.

Behavioral reasoning will be useful in all four cases.

It will not answer the governing question in any of them.

Authority is not a probability inferred from similarity to past behavior.

Authority must come from an attributable delegation made by a person or institution entitled to make it.

Dynamic risk analysis may narrow that authority.

It may require additional confirmation.

It may suspend activity when something appears wrong.

It must not manufacture authority merely because an action appears normal.

Risk reasoning may reduce authority. It must never create it.

Beyond Zero does not state that boundary strongly enough.

The paper puts AI in judgment over AI—and everyone else

The proposed architecture adds AI reasoning to the enterprise’s policy-decision machinery. It is meant to interpret the world model, understand the request, assess contextual risk, and decide whether to allow, deny, challenge, or contain activity. (arXiv)

This creates a peculiar answer to the agent problem:

We cannot safely allow AI systems to act rapidly across the enterprise, so we will place another AI system in a position to judge their actions rapidly across the enterprise.

The supervisory machine may be more constrained than the agents it monitors. It may be surrounded by static rules. It may be carefully evaluated.

It nevertheless becomes one of the most powerful actors in the organization.

It can potentially influence:

• who accesses information;
• whose work appears anomalous;
• which projects are treated as legitimate;
• when a person must justify an action;
• which agents may operate;
• which communications are interrupted;
• which accounts are contained;
• which novel behaviors are discouraged.

The paper calls static policy the floor and dynamic reasoning the ceiling. But ceilings can become roofs, and roofs can become cages.

Once the system is embedded deeply enough, its judgments will no longer feel like recommendations. They will become the environment in which work is possible.

People will adapt their behavior to satisfy the model.

Managers will teach employees how not to trigger it.

Teams will route legitimate work around it.

Vendors will offer “optimization” services for it.

Eventually, compliance with the enterprise world model may become more operationally important than compliance with the enterprise’s actual human purpose.

That is not an exotic failure mode.

That is how bureaucracy works.

And we are proposing to run it at machine speed?

Per-action control is not the same as per-action accountability

Beyond Zero is highly interested in the moment when an action is requested.

Should this accessor perform this method on this resource right now?

That is useful.

But the architecture is still centerd on the decision to permit access rather than the full institutional meaning of the action.

Suppose an agent reads a date from a legal intake form and uses it to set a checkbox representing a legally significant assertion.

Every access may be valid.

• The agent may have permission to read the date.
• It may have permission to generate the document.
• It may be operating inside the correct application.
• Its behavior may look entirely normal.

The real problem is semantic and institutional:

Was the system authorized to convert that date into an affirmative legal claim on behalf of the person?

No resource-level access rule answers that by itself.

The relevant object is not simply:

agent + method + resource

It is:

principal + delegated purpose + agent + evidence + transformation + consequential assertion + affected party + resulting state.

The system must know the difference between:

• retrieving a fact;
• deriving a value;
• recommending an interpretation;
• making an institutional assertion;
• executing an authorized action.

Those differences sit above the packet, above the API call, and often above the data object.

They are distinctions of standing, meaning, and responsibility.

False precision will spread faster than explicit uncertainty

Machine-speed authorization requires precomputed attributes and rapid decisions. That creates strong pressure to convert messy institutional reality into clean categories:

• assigned or unassigned;
• normal or anomalous;
• sensitive or ordinary;
• relevant or irrelevant;
• allowed or denied.

But real organizations run on partially resolved states:

• temporary authority;
• informal cooperation;
• emergency judgment;
• contested ownership;
• incomplete classification;
• unclear jurisdiction;
• contradictory instructions;
• obsolete records;
• work that has never been done before.

A good governance system preserves those uncertainties.

A high-frequency enforcement system wants them resolved before the request arrives.

The result will not necessarily be better knowledge. It may be merely faster certainty.

The more authoritative the world model becomes, the more dangerous its classification errors become. A wrong label in a conventional inventory is inconvenient. A wrong label in a machine-speed authorization system determines what can happen.

A person misclassified by the system may have no opportunity to correct the record before the action is blocked.

A novel response to an emergency may appear anomalous by definition.

An old organizational relationship may continue to influence access long after it ceased to be valid.

The system will look precise because it produces discrete decisions.

That does not mean the world underneath those decisions was ever precise.

The immune system will have autoimmune disease

The paper’s concluding metaphor is the enterprise as an adaptive immune system: continually sensing, reasoning, and interrupting threats. (ResearchGate)

It is a good metaphor, but perhaps not in the way intended.

Immune systems do not merely defend.

They also:

• attack healthy tissue;
• overreact to harmless stimuli;
• fail to recognize novel threats;
• become chronically inflammatory;
• cause severe damage while attempting protection.

An enterprise immune system that watches every actor, models every relationship, evaluates every action, and interrupts activity based on inferred risk will have similar failure modes.

Its false positives will not be random inconveniences. They will cluster around:

• people with unusual responsibilities;
• cross-functional teams;
• investigators;
• security responders;
• whistleblowers;
• new employees;
• marginalized workers whose behavior differs from the trained norm;
• anyone doing work the historical data does not describe well.

The system’s very success at modelling ordinary enterprise behavior may make it hostile to legitimate change.

The self-defending enterprise may become exceptionally good at defending yesterday’s organization from tomorrow’s work.

This is how we got ants

We did not get agent proliferation because enterprises lacked one sufficiently intelligent central decision-maker.

We got ants because organizations repeatedly converted human purposes into crude technical permissions.

• A human needed help with one task, so we created a service account.
• The service account needed a tool, so we gave it application access.
• The application needed integration, so we created a token.
• The token was difficult to constrain, so we wrapped it in workflow code.
• The workflow was too rigid, so we added an agent.
• The agent could not understand the organization, so we gave it more context.
• The context was fragmented, so we created another agent to assemble it.

Eventually there were ants everywhere: small automations carrying fragments of institutional purpose through tunnels of credentials, APIs, data stores, and undocumented assumptions.

Beyond Zero responds by proposing a highly capable colony regulator: a system that observes the ants, reconstructs the colony, infers what each ant should be doing, and blocks those whose behavior appears wrong.

But the regulator is also an ant. It is simply the largest ant in the colony.

And because it is charged with understanding the whole enterprise, it will receive more data, more privilege, more interpretive power, and more operational authority than any of the agents it governs.

This does not remove ambient authority.

It concentrates ambient authority in the security plane.

Zero trust already had a policy engine

There is another reason to be skeptical of the “Beyond Zero” framing.

Zero trust was never simply “authenticate once, then trust an application.” NIST’s Zero Trust Architecture already centers resource protection and includes policy engines, policy administrators, enforcement points, contextual information, and decisions to grant, deny, or revoke access. It explicitly moved security away from trust based on network location.

The new problem is not that zero trust failed to imagine granular policy decisions.

The new problem is that autonomous systems make institutional meaning and delegated purpose operational.

That cannot be solved merely by making the policy engine faster, more granular, and more intelligent.

It requires additional objects that conventional access control does not adequately represent:

• a declared purpose;
• an accountable principal;
• a bounded delegation;
• the limits of onward delegation;
• the source and status of evidence;
• the distinction between observation and inference;
• the consequence of the action;
• the right to challenge;
• the ability to revoke;
• a durable receipt.

Without these, Beyond Zero risks becoming extraordinarily sophisticated access control wrapped around an unresolved authority model.

A safer architecture would know less, and prove more

The alternative is not to return to application-level permissions or human-speed review of every machine action.

The action boundary is the right place to operate.

But the system at that boundary should be less omniscient and more explicit.

• It should not attempt to infer the full meaning of the enterprise.
• It should require machines and institutions to carry bounded claims.

For consequential actions, the system should be able to establish:

Who is acting?

A stable identity for the agent, service, organization, and accountable human or institutional principal.

Under what delegation?

A machine-readable authority envelope specifying purpose, scope, limits, duration, required approvals, and whether onward delegation is allowed.

On the basis of what evidence?

Attributable records with source, time, version, status, limitations, and revocation information.

What kind of act is this?

A distinction among observation, analysis, recommendation, assertion, decision, communication, and execution.

What will change?

The external state, obligation, communication, record, payment, access, or representation produced by the action.

What remains afterward?

An action receipt binding the agent, authority, relevant evidence, policy decision, tool invocation, result, and resulting state.

Who can challenge it?

A process for human review, correction, appeal, rollback, containment, and remedy.

This architecture does not require one system to possess a single living theory of everyone and everything.

• It allows local systems to verify the specific claims necessary for a specific action.
• It supports federation rather than omniscience.
• It accepts that two institutions may understand the same evidence and reach different legitimate decisions.
• It records uncertainty rather than requiring every ambiguity to become an authorization attribute.

The enterprise does not need a brain

Organizations have spent decades trying to create universal repositories, master data systems, unified policy engines, authoritative dashboards, and enterprise knowledge graphs.

They always begin as efforts to create coherence.

They frequently end as additional layers that people must work around.

The enterprise is not an organism with one mind.

It is a negotiated system of people, duties, authorities, exceptions, communities, technologies, and partially overlapping truths.

Security architecture should respect that reality.

The goal is not to give the enterprise a brain capable of interpreting every action.

The goal is to ensure that every consequential machine action has:

• an identifiable source;
• an explicit purpose;
• bounded authority;
• sufficient evidence;
• enforceable limits;
• a visible consequence;
• a reviewable receipt;
• a path to revocation and remedy.

That is less glamorous than a self-defending enterprise.

It is also less likely to become the most powerful unaudited agent in the building.

Beyond Beyond Zero

The authors deserve credit for identifying the correct point of intervention.

The application boundary is too coarse.

The individual action matters.

But the proposed solution is wrong in the particularly consequential way that sophisticated security architectures can be wrong: it seeks to resolve a deficit of explicit institutional authority by increasing centralized machine intelligence.

We should not ask an AI security engine to infer whether the machine was entitled to act.

• We should require the authority to exist before the action and require evidence of it afterward.

We should not build an enterprise world model and then treat its output as reality.

• We should preserve the sources, disagreements, uncertainties, jurisdictions, and limits that make institutional facts what they are.

We should not confuse a machine’s ability to detect abnormality with society’s ability to determine legitimacy. And we should not solve the problem of agents carrying ambient human authority by creating a larger agent with ambient authority over everyone.

The question after zero trust is not:

Can an intelligent security system decide whether this action looks safe?

It is:

Who authorized the machine, what exactly was it authorized to do, what did it do, and what evidence allows the rest of us to hold that action accountable?

Beyond Zero gives us the reflex. Before we wire it into every nerve of the enterprise, we should ask whether we have also built the memory, authority, restraint, and conscience required to control it.

Otherwise, we will not have eliminated the ants.

We will simply have crowned one.