The Agentic Security Butterfly Effect

The not-so-fun part of agentic systems is that the more you plug into it, the more complex securing it becomes. It’s the butterfly effect.

If you’re not familiar with the term, meteorologist Edward Lorenz used it to describe how one tiny change in a complex system’s initial conditions can lead to unpredictable outcomes over time. He used the analogy of the flap of a butterfly’s wing causing a distant typhoon.

In agentic systems, the butterfly’s wing comes in the form of the different tools and data given to agents to operate. The challenge comes in that these systems aren’t static. New data comes into play, new tools are added, and always those unpredictable inputs along the way. It’s an ecosystem in constant flux.

What’s one to do to understand the risk?

Hello, attack path analysis of agentic systems! To secure agentic systems, start with understanding the system. This means creating an inventory of its components and mapping its flow. Understand how each component connects and how each one impacts the other, regardless of how many steps are in the workflow. It’s easy to see that this can get complex real fast, and that’s before we take into consideration agentic systems talking with other agentic systems.

Let’s see an example. Take this basic customer support agent agentic workflow. Seems innocent enough.

But the risks lurk right at the surface level. Without the proper protections in place, that agent can cause a lot of damage. It’s one rogue action away from giving away sensitive and confidential information.

An attack path analysis shows a basic risk path, or toxic flow, that can lead to data exfiltration. An attacker can use prompt injection through ticket requests to probe the inner workings of the agentic system and extract protected information, sending it back through the ticket system.

Agentic Attack Path Patterns

Two key attack path patterns show up in agentic systems.

1. The lethal trifecta

2. The lethal duet

Let’s explore them both.

The Lethal Trifecta

Simon Willison, the man who brought us the term prompt injection, also brought us the concept of the Lethal Trifecta. It’s an easy way to understand the different components of an attack path that are necessary to steal data. Those components are:

1. Exposure to untrusted content

2. Access to private data

3. Ability to communicate outbound

Let’s break each component down.

The Input: Exposure to Untrusted Content

The attack path starts with an input that an attacker can use to send a malicious prompt. The easiest way to conceptualize this is through prompt injection via a chat interface. But we must think more broadly than that.

With agentic workflows, data can originate from multiple sources, even ones that may seem innocent or benign. Think of the customer service portal that tracks requests with a ticket system like JIRA or ServiceNow. Or a website contact form that links into a Customer Relationship Management (CRM) tool like HubSpot. These are all sources of information that have a high likelihood of getting hooked into an agentic workflow.

The untrusted input, a malicious prompt, is the first step in stealing data. An attacker must craft the prompt in a way that tricks the LLM, the “brains” of the agentic workflow, into taking action with the next two components.

The Target: Access to Private Data

The agentic workflow must have access to a data store that has sensitive information. Data segmentation is a difficult thing to accomplish, as agents are only as effective as the data they have access to. So, companies must balance what data the agents should access to but that also doesn’t open access to confidential or sensitive data.

Following the example of a customer service portal, the agent may have access to other client information or internal company data to help automate replies. An attacker may try to steal a list of the company’s clients and any information about them.

The Getaway: Ability to Communicate Externally

A theft is only successful if you can get away with the stolen goods. The last component is critical because remote attackers must find a way to send the stolen information to them. In the customer service portal example, the agent likely has some means to respond to the attacker, whether through email or a chat response. That’s the attacker’s way of getting the stolen information.

The method or tool that is used to steal data is known as a sink or data sink.

There are other, more sophisticated, methods that attackers can use to secretly send information to their servers, a fun post for another time.

While the lethal trifecta focuses on data theft, another attack path lurks nearby. It’s something I call the Lethal Duet.

The Lethal Duet

Using the same concept of the lethal trifecta, the lethal duet has the following components:

1. Exposure to untrusted content

2. Access to privileged actions

The first component remains the same as the lethal trifecta, so we won’t cover that again.

The Action: Access to privileged actions

For agents to be effective, they must have access to different tools and functions that humans do. This can range widely, but common use cases include executing SQL queries against databases, writing code, or using IT administrator tools. Just think about this in the context of MCP servers.

With these tools come the permissions that agents have to execute them. Agents that are running with excessive permissions or tooling configured with write permissions leave the door open for agents to be manipulated to take actions that they shouldn’t be.

It’s situations like these that lead to coding agents deleting production databases, which I wrote about previously here.

The first step to securing agentic systems is to map your systems, search for risky configurations, and identify attack paths. Ideally, this is happening during the design phase, but that can be wishful thinking. In complex systems, it’s difficult to map all of these things out. And these systems change. So a one-time exercise is not enough. You have to continuously monitor the environment for changes to understand how the risk evolves.

If you’re interested in discovering attack paths in your agentic systems, we’re building something that does just that. Let’s chat.