Getting into the Flow With Threat Modeling

Adam Shostack again, with the third in our series on threat modeling. In this post, I want to explain one of the ‘lenses’ that seemed to help us focus threat modeling, and how I’ve applied it.

The concept of flow originated with Mihaly Csikszentmihalyi. It refers to a state where people are energetically involved with what they’re doing. Seeing this a few times during threat modeling sessions made it obvious when it was missing, and it was missing often. I set out to address some of the elements that seemed to make threat modeling harder. The Wikipedia article (currently) has a good list, so I’ll focus in on a few of them:

  1. Clear goals
  2. Direct and immediate feedback
  3. Balance between ability and challenge
  4. Focused attention

Let’s take these one at a time.

Clear Goals

Giving people clear goals is important because it helps take them from worrying about what your goals mean to worrying about how to achieve them. Without clear goals, it’s very challenging to get into the spirit of anything, whether playing a game or shipping an operating system. As goals go, “think like an attacker” and “brainstorm” aren’t up there with “A PC on every desktop.” The lack of a clear answer to “how do I know I’m done with my diagrams” or “how many threats should I have,” or “what is a good threat model” made it hard for people to do well, and just as importantly, even people who were doing well weren’t sure that they were doing well. Which brings us directly to the next point:

Direct and Immediate Feedback

When people have direct and immediate feedback, they can adjust their activities relatively easily.  If you’re cooking from a well written recipe, at each stage, you have some indicators that things are done.  “Are the onions brown?”  “Is the water boiling?”  “Does the pasta stick to the wall?”  This is true of any skill we’re learning.  With threat modeling, when there is no expert in the room, feedback can take weeks or months to come back. As we rolled threat modeling out at Microsoft, it was possible for an entire threat model to be cooked without any course correction. We’re doing better now – there are more experienced people around, better documentation, and more focus on how to self-check. But at one time we asked people to engage in really complex tasks from the get go, without trying to achieve a…

Balance Between Ability and Challenge

Almost by definition, when you’re new at something, you don’t have a lot of skill around it. If the new activity is something highly complex, like speaking a foreign language, you’re likely to be anxious. And once you are skilled, simple tasks, like talking to a beginner, are often boring. Which brings us to the chart, which I’ve redrawn from Csikszentmihalyi’s book, Flow.    image

The ideal learning experiences involve a balance of challenge and reward, each of which grows as you learn. For example, when you buy Halo, there’s a training mission that allows you to run around doing simple tasks, learning the controls. If we dropped people into the live online game without any practice, they’d get slaughtered, and that’s bad for everyone. The newb is frustrated, and the experienced player is bored. (In fact, there’s pretty explicit acknowledgement of this in the Wired article on Halo.)

So looking at the chart of skills versus challenge, there’s a channel in the middle—it’s the flow channel. We’d like to guide people there, by giving them challenges that are in line with their skills. Once people are there, we want to prod them from point A to point D, where there are larger challenges, and skills that allow people to address those challenges. Sometimes, we want them to leave their comfort zone and learn new things. To take on new challenges (going to B) and learn from them, going to D. If we don’t challenge people, they get bored, and it’s less likely that they’ll learn. (This obviously isn’t a complete model of learning, but it flows out of the discussion.)

We want to do that for Halo, and we want to do that for threat modeling. To date, this has been implicit. We’ve simplified the process, in part in the interest of getting people out of the anxiety zone, and into a place where they can feel comfortable and then learn. Over time, we’ll be challenging and encouraging people to do more complex tasks as part of SDL threat modeling.

This assumes that threat modeling is a frequent enough activity that people develop skills around it. I think that’s an ok assumption, as long as we design good refresher and update training and documentation.

Focused Attention

If you’re reading this blog post while talking on the phone and petting your dog, you’re going to get less out of it than if you do nothing else while reading it. Similarly, when you’re learning a new skill or performing a hard task, you need to focus your attention. Related to this is moving around. Getting a little blood flow helps you be energetic and engaged. This has been one of the hardest elements of flow to add to threat modeling. Many people here multi-task intensively, with laptops or blackberries always at hand. Challenging that habit seems like a challenge to the way people work, and I’ve shied away from issuing that challenge.

Why flow?

One final point I’d like to address is that flow is one of many frameworks we could use to address problems in threat modeling. My choice to use it was a driven by its striking absence. Flow isn’t the only framework for thinking about these things, and there are some criticisms associated with trying to shoehorn everything into flow. For threat modeling, it just seemed to well…flow.