Making Threat Modeling Work Better

Adam Shostack here, with part four of my threat modeling series. This post is a little less philosophical and a lot more prescriptive than the one about flow. It explains exactly how and why I changed a couple of elements of the process. The first is the brainstorming meeting, and the second is the way trust boundaries may be placed.

The brainstorming meeting is a mainstay of expert threat modeling. It’s pretty simple: you put your security experts in a room with system diagrams and a whiteboard. Usually, you put your system designers in there, and make them promise not to strangle your experts. Optionally, you can add beer or scotch. Sometime later, you get a list of threats. How long depends on how big the system is, how well its requirements are documented, and how well your experts work together.

We like having our developers threat model. There are a lot of reasons for this. Not only do they know the system better than anyone else, but getting people involved in a process helps ensure that they buy into it.

Now this desire is great, but it leads to some issues, first and foremost is that many of the people who are now involved aren’t security experts. This means that they lack both direct experience of the process and the background that informs it. This isn’t a slam on them. I lack experience in the database design process, and I don’t have years of experience to help orient me. So I’d make mistakes designing a database, and someone who isn’t a security expert may make mistakes in security. For example, someone might try to use encryption to mitigate tampering threats. (The SDL crypto requirements cover this, and I try to gently correct them to integrity mechanisms like MACs or signatures.) This is a reality that we have to account for at the process design level.

Adding Structure to Chaos

So how does this relate to the brainstorming meeting? It’s a dramatic increase in the need for structure. Where experts may think they do better threat modeling with scotch in hand, , it certainly doesn’t lead to beginners having a flow experience. So we need a structure, and we need to provide it.

We encourage people to get started by drawing a whiteboard diagram. Almost everyone in software draws on whiteboards regularly, and this makes it an ideal first step. It’s an ideal first step because everyone can do it, see that they’ve done it, and feel like they’re making progress.

The core mechanism we’ve used to provide it is the STRIDE/element chart. (I’ll talk a lot more about its origins and limits in a few posts, but for now, let’s pretend it’s gospel, and enumerates all possible threats.) Given this gospel, it becomes possible to step through the threat modeling diagram, “turn the crank,” and have threats come out. “Item 7 is a data flow? Let’s look for T,I and D.” (Tampering, Information disclosure, and Denial of service.)

Similarly, we have four ways of addressing threats – redesign, standard mitigations, new mitigations, and risk acceptance. We have training on mitigating threats, we have explanation of why and when to use each (and they’re presented in a preferred order).

Lastly, we provide advice about how to validate the threat model and it’s relation to reality.

Between these four steps and the hamster wheel which ties them together, we give people the structure in which they can take on the process. The other thing I wanted to address is how we respond to consistent “errors” that we see.

Where Trust Boundaries Show Up

We used to give people clear guidance that trust boundaries should only intersect with data flows. After all, you can’t really have a process that’s half-running as admin, and half as a normal user. Logically, you have two entities. And people kept drawing trust boundaries across processes and data stores. It drove me up the wall. It was wrong.

As people kept doing it, I decided to swallow my pride and accept it. I now tell people to put their trust boundaries wherever they believe one exists. And they’ve continued exactly as before, but I’m a lot happier, because I’ve found a way to help them draw more detailed diagrams where they need them. Which includes anywhere a trust boundary crosses a process or data store. They’re happier too. No one is telling them that they’re wrong.

I was going to title this post “Lord grant me the strength to change the things I can, the courage to accept what I can’t, and the wisdom to know the difference,” but, first, it’s too long, and second, if we started that way, it would be wrong to add beer or scotch.