Today I would like to write about a popular type of “security research” that really drives me crazy: when author takes public Vulnerability Base and, by analyzing it, makes different conclusions about software products or operating systems.
The latest research of such type, was recently published in CNews – a popular Russian Internet portal about IT technologies. It is titled ““The brutal reality” of Information Security market: security software leads in the number of holes“.
The article is based on Flexera/Secunia whitepaper. The main idea is that various security software products are insecure, because of amount of vulnerability IDs related to this software existing in Flexera Vulnerability Database. In fact, the whole article is just a listing of such “unsafe” products and vendors (IBM Security, AlienVault USM and OSSIM, Palo Alto, McAfee, Juniper, etc.) and the expert commentary: cybercriminals may use vulnerabilities in security products and avoid blocking their IP-address; customers should focus on the security of their proprietary code first of all, and then include security products in the protection scheme.
What can I say about these opuses of this kind?
They provide “good” practices for software vendors:
- Hide information about vulnerabilities in your products
- Don’t release any security bulletins
- Don’t request CVE-numbers from MITRE for known vulnerabilities in your products
And then analysts and journalists won’t write that your product is “a leader in the number of security holes”. Profit! 😉
A large number of known vulnerabilities in Vulnerability Base, means that some researcher is motivated enough to search and report them. This means the product is interesting for someone. And that’s all.
Vulnerabilities that have been found, described and fixed are better than vulnerabilities that are unknown and secretly exploited. To protect against the first type of vulnerabilities scan and update your systems regularly and all will be fine.
When you analyses amount of CVEs, you can falsely decide that long time ago abandoned software developed by small local vendor is much safer than software developed by respectable international vendor, who regularly publish security bulletins and provide new CVE IDs. Well, this situation is possible, but is not typical.
Let’s say we have some road repairs list, that says how many holes in the road were repaired for the year. If a lot of holes were fixed, does it say that the road is bad?
Not necessary. Of course, the road could be built with some technological violations and that’s why it breaks easily. However, it’s also possible, that people complain on this road more often. Perhaps the governor use it regularly and it is actually the best road in the county. From the road repairs list it is not clear.
The fact, that a lot of holes were fixed tells us is that somebody cares about this road. Theoretically, there may be roads, for which repair is not necessary at all. But usually empty list of holes indicates that no one look after it, and perhaps no one use it.
Once again, the software with fewer security holes is, of course, better than software with a lot of holes. But the simple amount of CVEs is not the good indicator. Just because not every “hole” has the corresponding CVE id. Known vulnerabilities don’t fall from the sky, the developers and researchers report them. For some software there are no CVEs (like for SAP there are only SAP Notes) and some vendor may even hide vulnerabilities in their products.
If you still want to compare software product security using some Vulnerability Database, these metric seems to be more adequate:
- The number of critical non-patched vulnerabilities with public exploit available for the latest version of the software
- Average latency between CVE publication and release of the vendor’s patch
The first metric will give you the number of vulnerabilities that an attacker can exploit right now. The second metric shows how seriously the software vendor takes security issues.
This values may be harder to get than number of CVE IDs in NVD. However, with Vulners.com it become much more easier. This vulnerability search system contains information on vulnerabilities and exploits and patches. There are timestamps available for all entities.
Here I would like to recommend Kirill Ermakov “Vulnerability Funalitics with vulners.com” presentation.
For example, on this slide you can see how quickly various Linux distributions manufacturers release security updates. FreeBSD makes it much faster than enterprise-ready RedHat Linux.
Hi! My name is Alexander and I am a Vulnerability Management specialist. You can read more about me here. Currently, the best way to follow me is my Telegram channel @avleonovcom. I update it more often than this site. If you haven’t used Telegram yet, give it a try. It’s great. You can discuss my posts or ask questions at @avleonovchat.
А всех русскоязычных я приглашаю в ещё один телеграмм канал @avleonovrus, первым делом теперь пишу туда.