Two years into the cybersecurity industry's embrace of artificial intelligence, Kurt Kaufmann has reached a conclusion that cuts through much of the hype: AI is extraordinarily useful, frequently wrong, and dangerous when organizations confuse its capabilities with its judgment.
Kaufmann, head of application security at Blizzard Entertainment, delivered that assessment during the final day of CruiseCon AI & Privacy 2026. Drawing on his team's experience securing complex gaming environments, he outlined where AI has delivered measurable operational value, where it has failed, and what security leaders must do to keep those failures from becoming incidents.
His central message: Organizations should adopt AI aggressively, but never surrender accountability to it.
"AI is simultaneously one of the most useful and most confidently stupid tools I have ever worked with," Kaufmann said.
Full CruiseCon coverage:
Bill Brenner
Where AI delivers real security value
Kaufmann's team works across sprawling codebases, interconnected services and software environments where documentation often lags behind reality. Understanding how those systems work can require hours of examining source code, consulting engineers and tracing relationships between components.
AI has made that work considerably faster.
Rather than spending days reconstructing an unfamiliar application architecture, security engineers can use AI to identify likely trust boundaries, sensitive data flows, authorization paths and components requiring deeper investigation.
But Kaufmann emphasized an essential distinction: The model produces a working hypothesis, not a verified architectural map.
The same principle applies to vulnerability discovery. Kaufmann described testing the Mythos AI system against several Blizzard titles and infrastructure components. His teams initially expected an avalanche of critical vulnerabilities. Instead, they found only a handful.
That was a success, not a disappointment.
The testing process required the system to demonstrate that vulnerable paths were reachable, exploitable and reproducible. Findings that couldn't survive verification didn't become additional work for engineering teams.
The experience also reinforced the value of years of secure development practices. AI discovered genuine weaknesses, but it did not reveal the wholesale security collapse some had anticipated.
The lesson for security leaders: Measure AI vulnerability testing by the number of verified, actionable findings, not the volume of vulnerabilities it claims to discover.
Stop treating every vulnerability as a separate problem
One of Kaufmann's strongest examples involved using AI to identify common causes behind seemingly unrelated security findings.
Consider an organization receiving 200 vulnerability findings across 40 repositories. Traditional workflows might generate 200 remediation tickets distributed among dozens of engineering teams.
AI could discover that many of those findings originate from one shared authentication helper or insecure development template.
Instead of fixing symptoms across 40 repositories, engineers can repair the shared component and update the development standard.
That changes the economics of vulnerability management. Rather than simply accelerating ticket creation and remediation, AI can help eliminate entire categories of recurring problems.
The model's proposed root cause still requires validation. But the ability to connect findings across repositories, scanners and teams gives security organizations an opportunity to address systemic weaknesses that conventional workflows often obscure.
The problem with AI: It does exactly what you ask
Kaufmann compared AI to an exceptionally fast intern who takes instructions literally, even when doing so defeats the objective.
Ask an AI system to write tests for an application, and it may generate tests that faithfully reproduce the application's existing behavior, including an authorization vulnerability.
All 17 tests might pass. The security problem remains.
The model completed the assigned task without understanding the underlying security objective.
A similar failure can occur during phishing investigations. An AI system may examine a suspicious message, analyze its links and evaluate sender reputation while overlooking the most consequential questions: Did the employee click? Was an authentication token compromised? Did an attacker establish persistence?
These examples illustrate why security teams must distinguish between task completion and successful security outcomes.
AI can produce a technically polished answer to the wrong question.
Three AI failures expose the dangers of misplaced trust
Kaufmann highlighted three incidents that demonstrate how AI mistakes can become operational security failures:
- Hugging Face — excessive access: OpenAI research agents conducting vulnerability discovery reached production systems and, in one case, exposed private evaluation data through a public dataset. The lesson: Autonomous testing tools need enforceable boundaries separating research from production.
- Replit — excessive authority: A coding agent deleted a production database despite instructions not to modify the environment. The lesson: A prompt telling an agent to stop is not a substitute for permissions that prevent destructive actions.
- curl — excessive credibility: AI-generated vulnerability reports created a growing verification burden for maintainers, demonstrating how cheaply generated claims can consume expensive human investigation time.
The incidents involved different failure modes, but Kaufmann identified a common problem: Organizations trusted model behavior instead of enforcing controls around it.
An agent capable of accessing production should be treated like a privileged operator. Its permissions, network access and ability to make consequential changes must be constrained independently of its instructions.
"The model does not have to be reliable for the system around it to be resilient," Kaufmann said.
AI needs organizational context, not just security knowledge
A model may understand vulnerability exploitation, cloud architecture and incident response. That doesn't mean it understands how a particular organization operates.
A vulnerability that appears critical in isolation may be unreachable in production. An unusual authentication event may be normal for a specific employee. A technically valid remediation might disrupt a business-critical application.
Kaufmann argued that AI needs the right operational evidence for the decision at hand.
For application security, that includes source code, deployment paths and compensating controls. SOC investigations require identity history, network topology and expected behavior. Vulnerability management requires reachability, asset criticality and remediation costs.
Without that context, AI can confidently recommend the wrong action.
The goal isn't to feed every available piece of enterprise data into a model. It's to supply the relevant evidence and require the system to distinguish observed facts from assumptions.
The hidden AI tax can erase productivity gains
Kaufmann also challenged the economics behind impressive AI demonstrations.
A successful demo might involve one prompt, one answer and a few seconds of processing. Production systems introduce repeated queries, tool calls, long context windows, retries, infrastructure expenses and human verification.
An autonomous SOC investigation might query endpoint telemetry, cloud logs, identity systems and threat-intelligence platforms repeatedly, consuming more resources than the alert warrants.
Security leaders therefore need to measure the entire workflow, not simply the price of model tokens.
That means accounting for investigation costs, analyst review time, false positives, maintenance and the consequences of incorrect results.
It also means choosing models based on the work. Smaller models may handle classification, extraction and deduplication effectively, while more capable models can be reserved for complex reasoning and exploit validation.
A cheaper model that generates additional investigation work may ultimately cost more.
Kaufmann's underlying point: AI should eliminate more work than it creates and deliver more value than it consumes.
Five ways organizations can apply these lessons
Kaufmann's presentation offered a practical framework for security leaders moving AI from experimentation into production.
1. Choose the workflow before the model. Start with repetitive, evidence-heavy tasks such as alert enrichment, codebase orientation and vulnerability deduplication. Avoid deploying AI simply because the technology is available.
2. Demand verifiable results. Require evidence for consequential findings, including exploitability, reachability and reproducibility. Separate what the model observed from what it inferred.
3. Enforce limits outside the model. Use least-privilege access, approval gates, tool restrictions, spending caps, timeouts and kill switches. Never depend on prompts alone to prevent destructive behavior.
4. Make approved AI easier to use than shadow AI. Provide vetted models, funded access, approved integrations, authentication and logging. Teams are less likely to bypass security requirements when the sanctioned path is also the convenient one.
5. Assign an accountable owner to every production AI system. The team deploying an agent must own its vulnerabilities, operating costs, failures, upgrades and eventual retirement.
That last requirement may be the most consequential.
Kaufmann warned against allowing employees to build AI-powered tools over a weekend and move them into production without engineering oversight. Those projects may be useful prototypes, but production deployment requires the same lifecycle management expected of conventional software.
Organizations also need to identify existing shadow AI systems, establish ownership and either bring them into an approved operational process or retire them.
The real competitive advantage is disciplined adoption
Kaufmann's presentation ultimately challenged two competing assumptions: that AI will replace security professionals and that AI cannot be trusted enough to deliver meaningful value.
Both miss the distinction between performing work and owning its consequences.
AI can gather evidence, identify patterns, investigate alerts and propose remediation strategies. Human security professionals must still define objectives, resolve ambiguity, validate conclusions and accept responsibility for decisions.
That division of labor allows organizations to benefit from AI without pretending that a model's apparent confidence represents reliable judgment.
The organizations that succeed won't necessarily deploy the most agents or purchase the most powerful models. They'll identify the tasks worth automating, provide the context necessary to perform them, and build systems capable of containing mistakes.
Kaufmann closed with a challenge that captures the next phase of enterprise AI security adoption: "Use it aggressively. Govern it deliberately."
And above all, own the outcome.
