Safety vs Security

A common problem in AI today is that everyone is speaking a slightly different language. What seem like basic terms, like “agent”, or simple questions like “what are you building with AI,”= can have drastic interpretations. So many of my conversations start with a level set on what we mean when we say a specific term.

One of those conversations this week covered the difference between “safety” and “security.” And serendipitously, I randomly found a new paper from researchers at The Ohio University that covered that exact distinction.

AI Safety is all about avoiding Skynet. The researchers stated, “Safety is concerned with mitigating risks from non-malicious failure modes, such as software bugs, specification errors, or misaligned objectives.” Like slipping on ice, these are accidental or unintended risks that can come up, often from “complexity, uncertainty, or an incomplete understanding of emergent behaviors.”

The best example here is how ChatGPT refuses to give you information on how to cook meth or make a bomb. Both questions, if answered freely, would probably not be a net positive for the community. And, yes, you can save your argument about how that information is already available on the web for another day.

It also addresses the issues where people (especially kids) become addicted to Chatbots and those chatbots start to give bad advice, which has led to terrible consequences.

AI safety becomes increasingly important as AI systems begin to operate with greater autonomy.

AI Security is about stopping bad actors from doing bad things. The researchers stated, “Security addresses adversarial threats arising from deliberate attempts to subvert, manipulate, or exfiltrate from AI systems.” Like Liam Neeson on a focused mission, these are intentionally malicious actions taken against AI systems.

The paper highlights that the core challenge with AI security involves three elements:

1. Asset: the AI system (e.g. the model, infrastructure, and the functions the AI system performs)

2. Adversary: the bad actor who is putting bad things into motion

3. Vulnerability: an exploitable weakness in the system (prompt injection, misconfigurations, etc. that will lead to a security issue)

Like traditional security, this comes down to preventing bad actors from harming your systems and data.

AI Security and AI Safety play off each other. Safety issues can lead to security issues and vice versa.

A prompt injection (AI security) can modify the behavior of the model, causing the model to return harmful information or leak sensitive data. Models with misaligned safety objectives can be too trusting of users and be more susceptible to prompt injection. This all plays in the same sandbox.

This quote sums it up nicely:

If you have questions on whether you’re securely deploying AI, let’s chat.