Cory at Matasano has a new blog post explaining “Ninja threat modeling.” Ninja threat modeling is Matasano’s approach to threat modeling as part of a penetration test. I’m really happy that they’ve given their approach a name. A few years back, we would just talk about “threat modeling” and it got confusing. With that said, Adam here, and I wanted to offer up our perspective. I’ll do that by first comparing and contrasting the SDL and ninja approaches, and then respond to on some Cory’s impressions of the STRIDE-per-Element approach to threat modeling which we’re using in the SDL.
Pirates are Way Cooler than Ninjas, but Engineering Got us to the Moon
There’s a lot to be said for giving your approach a cool name, and we love cool names too, like “The SDL Threat Modeling Tool.” How cool is that? Ok, ninja is much cooler. It seems from Cory’s post that Matasano’s customers are coming to them for security at the end of their process, rather than at the start. I think we all agree that threat modeling late produces less value. Here at Microsoft, we’ve invested in making it possible for any software engineer to threat model at the start of development. We’ve made enough progress in this that Forrester has said “Many application architects and developers don’t know enough about developing secure applications… Microsoft’s SDL Threat Modeling Tool is a unique new tool that helps developers identify and mitigate security risks to make applications more secure from the get-go.” (“Use Threat Modeling To Develop More-Secure Applications,” March, 2009.)
I do think that we can map between the current SDL approach and the Ninja approach:
|Model||DFD||App overview, data flow|
|Identify Threats||STRIDE/Element||Assumptions, deadly sins|
|Validate||Check model, all threats have bugs||Test plan|
For a summary of their process, I looked at the boxed text “Ninja threat modeling at a glance.” I wish Cory had explained the approach a bit more: what’s the difference between an app overview and a data flow? Why are there 2 threat enumeration checklists (assumptions, deadly sins)? I think it might be interesting to combine the two threat enumerations. I also think that the risk management step could be formalized a bit more.
So I’m glad that Matasano has a way to help you if you haven’t threat modeled. Our experience and observations over many, many years has shown that most people don’t want (or haven’t budgeted for) ninjas to drop into their process and slice up their design at the last minute. That’s why we’ve been sharing the SDL optimization model, building out the SDL Pro Network and sharing our approaches. We think that most people want to engineer a good and secure product from the start. We all need to work to make that easier, more predictable, and more effective. I also recognize that many organizations are not building security into their development processes yet. So it’s great to see Matasano think through what a threat model at the end of the dev process should look like, and share that thinking.
I wanted to reply to one thing that Cory said:
“It has spawned not just one, but two, Visio-driven toolsets from Microsoft and countless data-flow diagrams, attack trees, consulting engagements, and perplexed developers. When performed by a skilled and experienced team member, the model can be used to identify architectural weaknesses, guide default application behavior, and outline functional requirements for the product.”
Cory’s right. We have two tools, and it’s confusing. We’ll be making that much clearer soon. Additionally, we’ve presented a lot of information about our many approaches over the years.Today, we have one authoritative site at microsoft.com/sdl which presents the most current guidance. We no longer use attack trees. We’re working hard to speak clearly. Is it working for you? Let us know what’s not clear. Yes, there are a lot of books and what-have-you that can’t be updated, but we aim to publish and maintain guidance on the SDL portal that is authoritative, current, and understandable. Kicking attack trees is sort of like commenting on the security of Win98: we’ve learned a lot since then.
One of the most important things we’ve learned is that we needed to simplify the model, the approach, and the training, and we’ve done all three of those things. Having done those things, we’ve seen non-experts pick up the tool and create good threat models. We’ve heard from partners who are using the tool successfully, and we’ve received great feedback from analysts about efforts. None of which means we’re perfect. We’re still continuing to innovate with the aim of making the process better, and seeking the feedback from anyone who’s downloaded and applied our free tools and guidance. We’ve got some tricks up our sleeve, and while we don’t want to play them too close to the chest, we’re going to continue to innovate, and are glad to see a profusion of ideas for making things better. Finally, we work to share our experience.
Ninjas and Engineers Agree: Threat Model
We’ve seen the STRIDE-per-element approach work for non-experts. We suggest you give it a try. But far more important than which approach you try is when you try it. Start early. Take a look at the optimization model. If you want some consulting help, go to one of our Pro Network partners or even to Matasano. If you have a few hours, experiment with both approaches and see which fits. But start early and find a threat modeling approach that helps you deliver more secure software.
Pirates and script kiddies would prefer you just fuzzed.