Forrester report for Rapid7: number juggling and an excellent overview of Vulnerability Management problems. I recently read Forrester’s 20-page report “The Total Economic Impact™ Of Rapid7 InsightVM“. It is about the Cost Savings And Business Benefits that Vulnerability Management solution can bring to the organizations.

In short, I didn’t like everything related to money. It seems like juggling with numbers, useless and boring. But I really liked the quotes from customers who criticized existing Vulnerability Management solutions, especially the low quality of the remediation data. These are the real pain points of Vulnerability Management process.

How did Forrester count money?

Forrester interviewed five existing customers of Rapid7 and created a “composite organization”.

This “composite organization” has 12,000 IT assets and spends $223,374 per year on Rapid7 InsightVM ($670,123 for 3 years) including integrations and trainings costs. That means $18 per host. Well, quite a lot, especially when compared to unlimited Nessus Professional for just $2,390 per year. A wonderland of Enterprise Vulnerability Management. 🙂

All the Cost Savings And Business Benefits are based on the assumption that this hypothetical “composite organization” already had a VM process for some time. And they spent a lot of money and effort on working with vulnerabilities. But when they switched to the new shiny VM solution by Rapid7, they saved money on the salaries of security and IT specialists, and there were no major incidents and no need to investigate them. So, they received $3 million (a million per year) as Cost Savings And Business Benefits.

Of course, in real life, most organizations do not have active VM process. And for them such a purchase won’t bring any Business Benefit at all. It will be just new additional costs for the Vulnerability Management solution itself and for the endless patching process. If the organization does not have an understanding of how these vulnerabilities will be exploited by the attackers, if the organization was never hacked or was hacked a long time ago, it will be difficult to justify these additional costs. So, that’s why I don’t like everything related to money in this report. But I liked the other part.

Criticism of existing VM solutions

I really liked quotes from real customers about Vulnerability Management solutions and related problems. Of course, it was presented as if all these problems were magically resolved by Rapid7 InsightVM. But any Vulnerability Management vendor can be promoted this way.

So, what are the problems?

1. Problems with hosts scanning

Network scanning is the most inconvenient way of collecting host data.

The hosts may be switched off or may be out of the network: “We had machines off the network for extended periods of time so a quick scan wouldn’t work. We needed an agent that would report back as soon as the machine hit our network.” “Off-network devices are not always accounted for, nor are they regularly scanned with existing systems.”

Vulnerability Scanner may skip the hosts. For example, when hosts do not respond to ping requests or respond slowly to port scans: “the existing solution missed a number of nodes and endpoints on scheduled scans, manual scans were often required.”

2. Problems with vulnerability detection methods

I really like to scan the same host with multiple vulnerability scanners, compare the results and ask the VM vendors about the differences. And naturally I’m not the only one: “some vulnerabilities were missed when doing head-to-head tests”.

A classic problem of credentialed scans with root/administrator access. They can potentially break the host, and no one wants to be responsible for this: “deep-level credential access was a non-starter, as it would present a new risk for his organization”.

3. Problems with prioritization of detected vulnerabilities

Ok, we got the scan results, but what’s next? It seems silly to patch everything or patch only vulnerabilities with high CVSS. “CVSS scoring system is a broad guideline that does not always lead security professionals to the vulnerability that needs addressing most urgently.”

It’s important to understand how these particular vulnerabilities can be exploited on these particular hosts and why these hosts are important. “…prioritization of vulnerabilities on the network map that are critical to that specific business”. “Assets like customer records are always important to protect, but carry different significance between organizations such as healthcare and professional services.”

4. Problems with vulnerability remediation

And I really like that the emphasis was placed on that!

The problem is that remediation process in not only manual, but also requires some research. “The question was not whether the organization wanted to remediate the vulnerabilities, but rather a matter of people resources to analyze and tackle the problems.”

What does the client want? “An actionable advice”.

What does the client get? “The other problem with the competitor was the remediation instructions not being specific”. They “…describe the resolution broadly and send us to articles on the vulnerabilities.” And this articles may contain detailed instructions or not. The solution “needed a lot of massaging of the data and essentially made for a very manual remediation process”.

And it’s important to understand that the reason is not always clear. Just to remove some bug that has never been exploited in the wild? Even if there is some information that vulnerability is exploitable, how reliable this information is?

Of course, it would be great to see the automated patching (at least trough integration with the systems like SCCM or BigFix), but who will be responsible if (or when) this system will break something important. Who will conduct testing after each and every update. But, at a minimum, it would great to see some features to make “many of the repetitive tasks such as information collection, requests to system administrators, and validation of patching were reduced or consolidated”.

5. Problems with reporting

Since I prepair most of the reports and dashboards using my own scripts, I don’t quite understand when VM customers spend a lot of time discussing integrated dashboards. But for some people it is really important and I agree that “easily deciphered dashboarding” is important.

In conclusion

Despite the fact that I don’t like the part about money, this report is much more interesting than most of marketing reports on Vulnerability Management. I can recommend you to read it.