Hi everyone, this is Eleanor Saitta with iSEC Partners, with a brief post about return on investment and structured security.
A few weeks ago, Microsoft and iSEC Partners published a joint whitepaper titled, “Microsoft SDL: Return On Investment”, and I’d like to highlight a contradiction the paper discusses between what return on investment numbers show and common industry practice. In many cases, we see companies spending most of their security budget on gatekeeper-style security projects — right before the product is released, a security team gets called in to try to find vulnerabilities. This is more expensive and less effective than building security in from the start of a project and throughout the project’s development. Vulnerabilities are missed with the gatekeeper approach because in any large and complex system there’s rarely time to look at every line of code and every function. That’s not the worst of it, though — the cost of fixing the vulnerabilities found by the team can end up being huge because fixing security problems found late in the game may require a product be pushed back several stages in the development cycle and then retested to make sure no regressions are introduced and that the intended functionality didn’t change.
This type of late-development churn is inefficient. In fact, the difference in cost between finding and fixing vulnerabilities early and fixing them once an application is about to deploy can be a factor of 30 or more (per a 2002 NIST study — see the whitepaper for more details) For example, if you’re writing a web application and you perform an architectural security review, protecting against Cross-Site Scripting can be built-into the software as a functional requirement, and you can ensure that the application is designed so all output is correctly encoded. Putting in a point-fix at the right place in the framework and verifying that developers used the routine correctly is much easier and cheaper than trying to hunt down widely scattered cross-site scripting issues just as your ship deadline is approaching. Similar platform-level mitigations can solve a wide range of what are commonly considered low-level issues. Application platform vulnerabilities like Cross-Site Scripting or Cross-Site Request Forgery are what penetration testing is best at finding, but solving them up front with architectural review and secure design is still easier, cheaper, and more reliable than finding them in a gatekeeper-style penetration test and patching them.
In comparison, higher-level vulnerabilities in business rules, authentication, authorization or similar design issues can be both difficult to find and extremely time-consuming to fix if you only look for them once development has finished. Developers may have to change the core architecture of the system, leading to cascading code changes and regressions. On the other hand, a high-level security analysis (via threat modeling, security design review and related techniques) can be very effective at finding these types of issues. You can do this analysis even before development starts, preventing expensive architecture changes.
For both design and implementation vulnerabilities, looking at the return on investment shows that engaging with security early in a structured fashion is more effective than common gatekeeper-style security practices. When you’re making strategic decisions like this about how to approach security, good metrics that show what you’re getting for your investment make choices easier and your organization more efficient. Take a look at the whitepaper here for more information.