Why the SDL Works: Counterpoint

Eric Bidstrup here.

As James wrote up the previous posting on “Why the SDL works”, it generated some interesting discussion. It was fascinating for me to see the perspective from James’ point of view as an experienced security professional that semi-recently joined Microsoft, and compare it to mine as an “old time” Microsoft person who has been driving SDL policy for the last several years. I have to concede that James is correct in that there has been a culture shift and it’s a lot harder for those of us who lived through it to fully appreciate how much change has indeed occurred. However, I have a different perspective and hence the reason for this posting. I’d summarize “why the SDL works” in four key reasons:

1) Management support and commitment in allocating resources

Regardless of what you build (software or otherwise); deciding what to build, how to build it, actually building it, testing it, delivering it, and supporting it all takes time, energy, and resources. The longer the list of requirements “it” is expected to satisfy – the larger the amount of time, energy, and resources that will be needed. So, you also want it to be secure? Better make time, energy, and resources available for that. It helps to have the boss “sign the check” and support how you spend money. Not very technical to be sure, but also critically important to be successful. There are no free lunches…

2) Provide explicit directions and as much automation as you can  If you want people to do something, it’s a good idea to tell them what you want them to do… …and telling them how they can best do it doesn’t hurt either. Tools help too!

Most people with a given job to do are pretty receptive to receiving instructions on how they can most successfully do it. Getting the benefit of previous learning experiences on good and not so good ways of meeting a need never hurt anyone. If you give them tools that can help optimize their efforts even more… You get the idea. Related points: Unless the problem you are solving is extremely well understood and there is a widely accepted solution that addresses the problem (security isn’t quite there yet) – remain receptive to feedback and be willing to continue to invest in innovation, improvements, and opportunities for greater efficiency. Similarly, continue to critically evaluate how effective the tools and processes are in actually solving the problem you care about. Measuring the “effectiveness” of a given tool in identifying vulnerabilities is actually an extremely hard problem in optimizing for a minimum number of false positives (that engineers have to spend time on to verify as false positives) vs. avoiding “false negatives” (failing to identify actual vulnerabilities). Perhaps I’ll discuss this more in a subsequent blog post…

3) Use measurements and metrics to communicate about successes and issues

SDL covers a variety of development activities spanning all aspects of the software development process. Measuring some of those activities is harder than others. Validating some elements of SDL is relatively straightforward (ex: ensuring that C/C++ code was compiled with a specific version of a compiler, and validating that the /GS option was used). Validating other elements of SDL can be very challenging and/or labor intensive to verify (ex: ensuring that threat models reflect software as built, have considered all known/meaningful risks, and appropriate mitigations exist). It’s important to both consider how the development team validates that they are doing what is expected of them as well as considering how validation can be done outside of the development team. Measuring the security of software and the effectiveness of development practices intended to deliver more secure software are both really, really hard. Don’t expect perfection anytime soon, but it’s important to think about how each tool or process can be monitored to ensure that it is being successfully applied. If you can’t measure it, how do you tell if you’ve done it?

4) Stay current with changes in security. Attacks and methods change, so should you.

It’s interesting to consider that when Michael Howard and David LeBlanc wrote the first edition of “Writing Secure Code”, it didn’t contain references to either SQL injection attacks or integer overflows. The reason for that is no oversight on their part, but rather is a reflection on the fact that such attacks were unknown when the first version was written (and those familiar with WSC will see that both were added for the second edition). Time passes and stuff happens. Count on it. There is a lot of very impressive research and development done by an ever increasing number of security researchers, so the security landscape will always be changing. It’s important to stay current and pay attention; this is a dynamic and rapidly evolving field. You snooze, you lose (or more importantly, your customers could lose…).

In late 2005 I gave a talk on SDL at a university symposium in Germany. When I was walking out of the auditorium, I was walking behind a few students who were discussing my talk. (I’m pretty certain they didn’t notice me in the crowd behind them). One student said “I was really expecting to hear about new innovation Microsoft is doing in security, but his talk just described a lot of focus on engineering fundamentals and learning from security researchers”. While there are some interesting innovations in the specifics of SDL tools and processes, at the conceptual level that summary is not an unfair characterization. There is an amazing community of talented security researchers who are constantly advancing the state of the art in security, and it’s important to pay attention to research and think about how to apply that knowledge in order to protect our customers who purchase our products. Those who do not learn from the past (and present!) are doomed to repeat it.

…and keep patching.

About the Author
SDL Team

Trustworthy Computing, Microsoft