AI security is entering a new phase – four things leaders need to know

Today’s landscape

At this year’s World Economic Forum (WEF), discussions about secure AI took on new urgency. Leaders across industries acknowledged that AI is no longer limited to experimentation or decision support — it is increasingly embedded in core business operations.

AI systems are now operating with greater autonomy: negotiating contracts, enhancing supply chains, managing critical infrastructure and interacting directly with customers. Domain‑specific, proprietary agents are handling more sensitive data and operating across distributed environments. At the same time, geopolitical tensions, data sovereignty requirements, and rapidly evolving regulation are fragmenting the global landscape, making secure deployment significantly more complex.

Why it matters

As autonomy and interconnectedness increase, so does exposure. When AI agents are embedded in everything from self-driving cars to electricity grids, system failure can disrupt critical infrastructure, threaten human safety and even trigger national security crises.

As AI systems become more autonomous, security failures are no longer confined to data breaches or system outages. Failures can be subtle, silent and difficult to trace — producing outputs that appear reasonable while driving harmful or sub optimal outcomes.

Leaders were clear on one point at WEF: Slowing AI adoption is not a viable response.

Organizations that hesitate risk losing ground to competitors that embed security into AI by design. The challenge now is to balance speed with trust — and innovation with resilience.

Strategies for success

Avoiding silent failure requires a new approach — one that balances rapid innovation with safety across jurisdictions and use cases.

1. Align AI security with business value

The more directly an AI agent is tied to value creation, the higher the stakes when something goes wrong. Not all use cases carry the same risk: an agent that helps an internal team collaborate, and a customer-facing booking agent will require very different security postures.

Mapping where AI plays a critical role in workflows, and how it underpins core business outcomes, helps design agents with appropriate security in mind. For higher security cases, investing early in robust architecture, clear design rules, embedding guardrails and controls – including ontologies that govern how and when AI uses data – will keep agents on track as they mature and threats evolve.

2. Test AI systems in safe, realistic environments

Understanding how AI fails is becoming just as important as understanding how it performs.

Synthetic data and sandbox environments give teams the ability to safely simulate everything from adversarial prompts to model drift and unexpected interactions across complex architectures, without exposing sensitive data. “Red teaming” AI agents before release – deliberately trying to manipulate and override prompts – can spot potential vulnerabilities and test whether guardrails effectively intercept risky inputs and outputs.

Understanding how AI systems fail, and how the failure of one agent can cascade across an agentic AI workforce, gives leaders the confidence to scale AI innovation, knowing security has been tested under realistic conditions, not just ideal ones.

3. Shift from static controls to real-time, explainable defense

Autonomous AI agents operate continuously across a complicated, interconnected infrastructure. As they interact with other agents and people, risks emerge dynamically, in unexpected ways and often leave no clues as to their source.

To keep pace, data sovereignty policy, real-time monitoring and detection capabilities should be built into AI systems from the outset. This includes applying zero trust principles – narrowing the scope of AI agents, so they can access only the data required to perform intended tasks.

The ability to understand how AI agents reason and, therefore, explain behavior and decisions remains elusive. But progress is ongoing, including at EY, to “get inside AI’s brain” - monitoring how agents think, not just what they output. Over time, this will help identify when and why agents veer off course.

4. Reframe security as an enabler, not a constraint

Global uncertainty and the pace of AI innovation make it difficult to predict how security risks will evolve. What is clear is that leaders must reframe AI security from a compliance exercise to a source of competitive advantage. Organizations that proactively embed security into how responsible AI is designed, deployed and scaled will be better positioned to innovate at speed and with confidence, while protecting trust.

What this means for leaders

The WEF conversations made one thing clear: secure AI is entering a new phase. As AI systems become more autonomous and interconnected, security can no longer be bolted on after deployment.

Leaders who embed security, governance and explainability into how AI is designed and scaled will be better positioned to innovate with confidence — even in an uncertain and fast-moving environment.