Vulners – Google for hacker. How the best vulnerability search engine works and how to use it. Original article was published in Xakep Magazine #06/2016 (in Russian)

The common task. Уou need to find all information about some vulnerability: how critical the bug is, whether there is a public exploit, which vendors already released patches, which vulnerability scanner can detect this bug in the system. Previously, you had to search it all manually in dozens of sources (CVEDetails, SecurityFocus, Rapid7 DB, Exploit-DB, CVEs from MITRE / NIST, vendor newsletters, etc.) and analyze the collected data. Today, this routine can be (and should be!) automated with specialized services. One of these services – Vulners.com, the coolest search engine for bugs. And what is the most important – it’s free and has an open API. Let’s see how it can be useful for us.

What is it?

Vulners is a very large constantly updating database of Information Security content. This site lets you search for vulnerabilities, exploits, patches, bug bounty programs the same way a web search engine lets you search for websites. Vulners aggregates and presents in convenient form seven major types of data:

• Popular vulnerability databases, containing general descriptions of vulnerabilities and links. For example, well-known NVD CVEs of MITRE US agency and NIST Institute. In addition to this, Vulners supports vulnerability descriptions from various research centers and response teams: Vulnerability Lab, XSSed, CERT, ICS, Zero Day Initiative, Positive Technologies, ERPScan.
• Vendor’s security bulletins. This bug-reports are published by software vendors and contain information about vulnerabilities in their own products. At current moment Vulners supports various Linux distributions (Red Hat, CentOS, Oracle Linux, Arch Linux, Debian, Ubuntu, SUSE), FreeBSD, network devices (F5 Networks, Cisco, Huawei, Palo Alto Networks), popular and critical software (OpenSSL, Samba, nginx, Mozilla, Opera), including CMS (WordPress, Drupal).
• Exploits from Exploit-DB, Metasploit and 0day.today. Exploits are parsed and stored in full-text form and you can read the sources in a convenient text editor.
• Nessus plugins for vulnerability detection. It makes easy to find out whether a particular vulnerability can be detected using this popular network scanner. Why is it important? Read in my article “When a free scanning service detects vulnerabilities better“.
• Bug disclousers for bug bounty programs. At current moment Vulners supports HackerOne and Open Bug Bounty.
• Potential vulnerabilities of mobile applications and CMS. It is possible in cooperation with the static application security testing (SAST) vendors Hackapp and InfoWatch APPERCUT.
• Posts from hacking resources. Vulners collects Threatpost and rdot.org publications, which often cover vulnerability related topics.

All this information is handled, cataloged, structured and is always available for the search.

Full list of vendors, articles and databases supported by Vulners you can see at https://vulners.com/#stats page.

Unlike other security databases in which information is stored in a highly formalized form (for example in OVAL-based CIS, SecPod and Altx-Soft databases), Vulners data format is much more flexible. This makes it easier to add different kinds of sources and establish connections between all entities in the database automatically. Vulners provides fast search mechanism and presents search results in a nice form. What to do next with this information depends entirely on end-user’s fantasy.

Who makes Vulners and what is under the hood?

Vulners is developing by a small group of security enthusiasts in their spare time:

• Kirill «isox» Ermakov codes kernel and makes system administration tasks;
• Igor «Videns» codes the search engine;
• Vanya «Vankyv3r» codes the front-end;
• Sasha «Plex» codes data collecting robots;
• Alexander Leonov writes articles and makes analytics.

The first version of Vulners was rolled out in a couple of months after beginning of the development and was presented at Black Hat USA 2015 conference in Las Vegas. In June, the project celebrated it’s first anniversary.

Vulners engine is written in Python + Django and uses MongoDB + Elasticsearch databases. MongoDB is used only by data collecting robots, Elasticsearch is used only by front-end. Deploy is made with Bitbucket script. Scaling is released directly in the kernel through MongoDB and Elasticsearch sharding. Robot factory does not depend on host and may work away of the project. One of the coolest pieces – project uses Python 3.5+ and asyncio. That’s why search operations always work very fast :).

Vulners contains 319,557 bulletins and 144,684 exploits. Database size is less than 2 GB. This compactness is achieved by deduplication and packaging. All data is stored in RAM and this  increases search speed greatly. It is worth mentioning that Vulners is protected by Wallarm WAF operating in blocking mode.

Vulners Architecture

But enough words.

Let’s try to search something

The first thing you see when you open Vulners.com is, of course, the search string. Just enter the name of the application, website URL or vulnerability CVE number and Vulners will give you all latest publicly known bugs of the product with links to exploits, detection plugins and various publications.

WordPress bugs found by Vulners. Please note: The data is updated continuously and automatically

Of course, it’s boring to search something simple like «wordpress» or «xakep.ru». Let’s see what interesting things that Vulners can do.

Task: find critical CentOS bugs with public exploits

Query: type:centos order:published

Vulners allows you to filter search results and/or sort it by any field in bug description:

• by type of the bulletin;
• by CVSS Score;
• by date;
• by detection plugin number;
• by researcher name;

And so on.

That’s why we can form a complex queries like “type:centos cvss.score:[8 TO 10] order:published“, which means “find all new critical CentOS bugs, with CVSS Base Score from 8 to 10”. Since Vulners automatically add links to all collected data, you will see all related CVEs, detecting plugins, and exploits on every CentOS CESA bulletin page in search results.

Search results for this query can be obtained via Vulners API – it is may be useful for scripting. You just need to make a GET-request https://vulners.com/api/v3/search/lucene/?query=type:centos%20cvss.score:[8%20TO%2010]%20order:published. The answer will be in JSON.

Another useful API-request option – references=true, which allows you to get not only security objects (CentOS bulletins) in the query results, but all of related linked objects (detection plugins, exploits, etc.). For example,
https://vulners.com/api/v3/search/lucene/?references=True&query=type:centos%20cvss.score:[8%20TO%2010]%20order:published

GUI master for search requests

Getting more than 20 objects from Vulners

By default Vulners returns only first twenty objects in the search results. If you want more, you need to set the parameter size. So you can get up to 500 objects. And if that’s not enough, you can request several times by 500 using parameter skip.

Task: explain IT-department why we need to patch this vulnerabilities (or just to find all exploits for a particular bug :-))

Query: cvelist:CVE-2014-0160 type:exploitdb

With Vulners it is relatively easy to explain IT-department why vulnerabilities detected by the scanner are really dangerous and should be patched. To do this, you can display a list of exploits found by the CVE number or by another identifier. You can search in Exploit-DB or Metasploit. On the exploit page, full description and the source code of the exploit will be displayed.

Looking for CVE-2014-0160 exploits

You can see full text of the exploit in a convenient web editor.

Task: find out how much money some hacker gained with bug bounty programs

Query: isox order:bounty

Bug bounty search is an unique Vulners feature. You can find out which vulnerabilities were reported by the researcher and see his achievements in the bug bounty programs. Results can be sorted by company, researcher, price and so on.

For example, we are searching for a nick, sort on by bounty size:

Bounty search example

Vulners found reported bug in Mail.Ru, $400 was paid

You can find out how much money people earned on bug bounty:

$ curl "https://vulners.com/api/v3/search/lucene/?query=type%3Ahackerone+order%3Alastseen+reporter%3Aisox" 2>/dev/null | awk '{if($0~"\"bounty\""){gsub(",","",$2)}; earn+=$2 }END{print earn}'

The answer (in US $):  2762

You can also look for real SQL-injection vulnerabilities or vulnerabilities on a particular web-service, such as Vimeo: type:hackerone Vimeo.

Reported Vimeo bugs at HackerOne

Task: find bugs with Nessus detection plugins

Query: type:nessus order:published

Nessus plugin search is also a unique feature Vulners. Query will display a list of recently added plugins.

Example of Nessus plugin search

Some vulnerabilities in GNU C Library

Task: find potential vulnerabilities in mobile applications

Query: type:hackapp

Another cool Vulners feature – the ability to search for vulnerabilities in more than 13,000 free Android apps from US Google Play! Store through HackApp base. HackApp – is shareware toolkit and service for analyzing mobile applications.

The search results contain bulletin title, number of vulnerabilities by severity (red circle – critical, yellow circle – medium, gray circle – notice), information about the application (icon, current version, vendor name and release date).

Example of HackApp reports search

The bulletin contains brief description of vulnerabilities, vulnerable version of the application and a link to full version of the report on hackapp.com.

HACKAPP:COM.TIR.SIMULASYONU.APK vulnerabilities

Task: find potential vulnerabilities in popular CMS

Query: type:appercut

With Vulners.com you can search for potential vulnerabilities in the popular CMS and plugins. Application source codes are checked by InfoWatch APPERCUT static source code analyzer. It is generally known that the most exploited vulnerabilities are not in CMS engines, but in thousands of third-party plugins. Developers rarely fix this vulnerabilities quickly or even don’t fix them at all. You can find examples of such vulnerabilities and exploits with “wordpress plugin bulletinFamily:exploit” request.

Appercut is well suited for CMS analysis. Appercut® Custom Code Scanner supports a wide range of programming languages: 1C 8x, Delphi, Java, JavaScript, LotusScript, PHP, C#, PLSQL, SAP Abap4, T-SQL. One of the main Appercut features is concentration on developer’s undocumented features (backdoors) detection. It is very important in the case of open source software.

Appercut reports

Appercut bulletin contains all information about found vulnerabilities, including vulnerability description, criticality and a piece of code where the vulnerability was detected. Vulnerable version of the application is also indicated, e.g. “WordPress CMS <= 4.5.2”.

Appercut bulletin

Using the API

Since Vulners uses Elasticsearch, Vulners supports standard Apache Lucene queries. You can find Field names for the search with API helper (https://vulners.com/api/v3/search/stats/). Any key from “schemes” may be used as collector “key” in the Lucene query, for example:

• title
• description
• affectedPackage
• sourceData
• cvelist

Example of API search/lucene request for CVE-2014-0160:  curl https://vulners.com/api/v3/search/lucene/?query=type:cve%20id:CVE-2014-0160

Answer in JSON:

{
  "data": {
    "exactMatch": null,
    "search": [
      {
        "_index": "bulletins",
        "_score": 9.942732,
        "_source": {
          "type": "cve",
          "title": "CVE-2014-0160: OpenSSL heartbeat information disclosure",
          "published": "2014-04-07T18:55:03",
          "objectVersion": "1.0",
          "href": "https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-0160",
          "reporter": "NVD",
          "modified": "2015-10-22T10:19:38",
          "references": [
            "http://www.securitytracker.com/id/1030081",
            "http://public.support.unisys.com/common/public/vulnerability/NVD_Detail_Rpt.aspx?ID=1",
            "http://advisories.mageia.org/MGASA-2014-0165.html",
          ],
          "description": "OpenSSL could allow a remote attacker to obtain sensitive information, caused by an error in the TLS/DTLS heartbeat functionality. An attacker could exploit this vulnerability to remotely read system memory contents without needing to log on to the server. Successful exploitation could allow an attacker to retrieve private keys, passwords or other sensitive information.\r\n\r\nThis vulnerability is commonly referred to as \"Heartbleed\".",
          "lastseen": "2016-03-19T07:17:51",
          "cvss": {
            "vector": "AV:NETWORK/AC:LOW/Au:NONE/C:PARTIAL/I:NONE/A:NONE/",
            "score": 5.0
          },
          "id": "CVE-2014-0160",
          "scanner": [],
          "bulletinFamily": "NVD"
        },
        "_id": "CVE-2014-0160",
        "_type": "bulletin"
      }
    ]
  },
  "result": "OK"
}

Vulners archive/collection call provides an easy way to export whole collections of security bulletins.

For example, to download all CVEs you need to

wget “https://vulners.com/api/v3/archive/collection/?type=cve” -O cve.zip

The result will be cve.zip with cve.json inside.

The same file can be downloaded with GUI at Stats tab

Moreover, with this API call you can download security bulletins for a particular OS version:

wget “https://vulners.com/api/v3/archive/distributive/?os=centos&version=6” -O centos.zip

It makes possible to get the data, which you won’t find anywhere else: the archives of exploits, hackerone history, all CentOS vulnerabilities, etc. This functionality might be useful if you want to make your own tools and data synchronization, if you already use some knowledge base.

No problem if you forgot “type” values. Just enter a nonexistent type, and you get a full list of available values:

https://vulners.com/api/v3/archive/collection/?type=FAKE

“Error”: “There is no type ‘FAKE’ Available collection types:. [ ‘Nessus’, ‘cve’, ‘exploitdb’, ‘xssed’, ‘zdt’, ‘hackapp’, ‘threatpost’, ‘redhat’, ‘debian’, ‘ubuntu’, ‘cert’, ‘metasploit’, ‘freebsd’, ‘zdi’, ‘oraclelinux’, ‘suse’, ‘centos’,’ cisco ‘,’ hackerone ‘,’ vulnerlab ‘,’ f5 ‘,’ mozilla ‘,’ ics’, ‘archlinux’, ‘ptsecurity’, ‘rdot’, ‘erpscan’, ‘huawei’, ‘xen’, ‘openssl’, ‘opera’, ‘vmware’, ‘wpvulndb’, ‘samba’, ‘postgresql’, ‘drupal’, ‘lenovo’, ‘msvr’, ‘paloalto’, ‘nginx’] “

Telegram Bot with subscriptions to query results

In April Vulners launched a bot for Telegram messenger. It is very simple in use. Send “/subscribe your_search_query” message to a bot and get new search results as they would appear on Vulners. Bot understand the same queries, as the web search.

This service can help security expert to stay informed:

• Operation guys can track vulnerabilities in the software they use.
• Penetration testers can receive information practical use of vulnerabilities.

Do you want to view the latest CVEs? No problems:

/Subscribe type:cve

Do you want to see updates on the exploits?

/Subscribe bulletinFamily:exploit

Do you use Debian? Latest debian updates:

/Subscribe type:debian

Vulners RSS feeds

Let’s say you want to track HackerOne updates (query “type:hackerone”) with your favourite RSS reader. It’s easy. RSS feed will have an URL: https://vulners.com/rss.xml?query=type:hackerone

Add this link to the rss-reader, and receive alerts on a new results.

What about alternatives to Vulners?

Vulners is not the only vulnerability aggregator. We can mention, for example, OSVDB and Secunia databases. But one OSVDB is closed since April 5 and the Secunia became a paid one. It can be said that unfortunately there are no yet alternatives comparable by amount of sources, data formalization and automation capabilities.

We can also look at the problem from the Vulnerability Intelligence perspective. I recently wrote a post about this class of solutions “PCI DSS 3.2 and Vulnerability Intelligence“, including Vulners. So, if you are interested please read it.

Conclusion

Vulners is a unique and indispensable tool for any hacker and security expert. It is very time-saving in exploration and exploitation of the complex attack vectors. Of course, the tool is only in developing stage, but even now it is quite usable. And more importantly, Vulners open and free for the end user and will always be.

By the way, vulners.com vulnerabilities, can be submitted on https://hackerone.com/vulnerscom. Since the project is free, there are no rewards, but Vulners developers guarantee public disclosure. Email for communication support@vulners.com, other contacts here https://vulners.com/#contacts.

Good luck!