I am a couple of articles into my series:
- Can Mozilla Support Claims of Firefox Being the Most Secure Web Browser?, and
- Can Mozilla Support Claims of Firefox Being the Most Secure Web Browser? (Part 2)
In part 2, I probed Mozilla’s usage of an ‘at risk’ chart to claim that their customers were only exposed to unpatched vulnerabilities for nine days in 2006. With some quick research, I came up with enough vulnerabilities to show that Firefox users were vulnerable to unpatched security flaws for at least 285 days.
Earlier this week, Brian Krebs over at the Washington Post contacted me about part 2. His original article, published at the beginning of 2007, formed the basis for Mozilla’s security marketing claims on the Firefox security page and he asked if I realized he had only charted Critical or High severity issues? I had not realized that, though I had read the article very closely and noted that he talked about severity on the IE issues and provided a table, he didn’t mention severity ratings when discussing the Firefox issues. (In fact, he only provided a pointer to the one Firefox issue, which the NVD rated as Medium severity.) I had the one example to go on and I definitely didn’t map the methodology to just high severity issues.
However, I want to respect Brian’s feedback, so I went back and checked severity ratings and I found that the issues I did list were:
- 1 vulnerability rated High in the NVD (http://nvd.nist.gov)
- 6 vulnerabilities rated Medium in the NVD
As noted above, CVE-2006-1993, the one vulnerability which I believe Brian counted against Firefox in the original 2006 report was one of the Medium severity vulnerabilities in the NVD, though rated Critical by Mozilla.
I also pointed out to Brian that I didn’t try to be exhaustive in my revised ‘at risk’ chart, as I was looking for just enough examples to show the large disparity between the Mozilla security marketing claims and reality. However, given the deeper scrutiny and Brian’s blog posting, I think I should take another look.
So, back to the drawing board to dig a bit deeper and come up with a revised list:
CVE-2006-0748 High/Critical Apr 13, 2006 – Apr 21, 2006 CVE-2006-1993 Medium/Critical Apr 23, 2006 – May 2, 2006 CVE-2006-2788 High/ Critical –
not fixed in FF1.0
<Jan 1, 2006* – Jun 1, 2006 CVE-2006-4253 High/Critical Aug 12, 2006 – Sep 14, 2006 CVE-2006-4561 High/ (no MFSA) Aug 14, 2006 – CVE-2006-5462 Medium/Critical Sep 18, 2006 – Nov 7, 2006
* NVD entry associates CVE-2006-2788 with bugzilla entry 321598, disclosed on 12/27/05. However, 220816 is marked as a solved duplicate, which was first opened as an issue 9/30/03. Adding to the mix, MFSA 2006-38 appears to associate bug 330897 with the vulnerability. This may actually be more than a single issue, but I’m counting them as one under CVE-2006-2788.
It turns out that this was a very interesting exercise for me. Two of the issues included above that were rated Critical by Mozilla were only listed as Medium in the NVD, including the one covered by Brian. With that insight, my original chart which included only NVD-rated High and Medium severity issues doesn’t seem unreasonable as a methodology.
In fact, I found nine other vulnerabilities that have:
- been rated Medium in the NVD
- don’t have a rating from Mozilla, because there has been no MFSA issued
- each of them have been public for more than a year
Now, given that Mozilla has not acted on any of these for such a long time, it is probably safe to assume that they would not rate any of these NVD-Medium issues as Critical and I will exclude them from my chart revision. But, for completeness, they are: CVE-2005-2114, CVE-2005-2395, CVE-2005-4685, CVE-2005-4809, CVE-2006-0496, CVE-2006-2613, CVE-2006-4310, CVE-2006-6585. Discuss amongst yourselves.
Here is the updated ‘at risk’ chart using only High or Critical severity rated issues:
That seems like a bit more than nine days to me. Pick any two and remove them (the lighter red areas are the two longest, to help you visualize it) and it is still quite a few more than nine days.
Since I had originally only planned the 2006 ‘at risk’ as the first step in the discussion, I think it is probably time to move on to the next Mozilla claim I want to dig into. In Part 3 in the series, the statements I want to probe are:
“We count every defect distinctly. We count the ones that Mozilla developers find in-house. We count the things we do to mitigate defects in other pieces of software, including Windows itself and other third-party plugins. We count memory behaviour that we think might be exploitable, even if no exploit has ever been demonstrated and the issue in question was found in-house.”
Look for discussion in Part 3 early next week.
Best regards ~ Jeff