Processing Vulners collections using Python. Vulners collection is a zip archive containing all available objects of some type (e.g. CentOS security bulletins or OpenVAS detection plugins) from the Vulners Knowledge Base. Let’s see how to work with this data using powerful Python scripting language. You can read more about Vulners itself at “Vulners – Google for hacker“.

All collections are listed at https://vulners.com/#stats:

Note a gray icon with black arrow. Press it to download particular vulners collection.

OpenVAS collection link: https://vulners.com/api/v3/archive/collection/?type=openvas

If you need to get all objects for further analysis, you don’t need to make huge amount simmilar Search API requests. You just need to download one file. It’s takes less time and efforts and makes less load on Vulners service.

To download file in Python you may use:

import urllib
urllib.urlretrieve("https://vulners.com/api/v3/archive/collection/?type=openvas", "openvas.zip")

for Python 3+ use ‘import urllib.request’ and urllib.request.urlretrieve

In Python you can read directly from the zip archive:

from zipfile import ZipFile
archive = ZipFile("openvas.zip")
archived_file = archive.open(archive.namelist()[0])
archive_content = archived_file.read()
archived_file.close()

Vulners collection files contains data in JSON. If you do “print(archive_content)“, you will see:

...
  {
      "_index": "bulletins",
      "_type": "bulletin",
      "_id": "OPENVAS:892605",
      "_score": 0.0,
      "_source": {
      "sourceData": "# OpenVAS Vulnerability Test\n# $Id:[...] {\n    security_message(data:report);\n} else if (__pkg_match) {\n    exit(99); # Not vulnerable.\n}\n",
      "naslFamily": "Debian Local Security Checks",
      "title": "Debian Security Advisory DSA 2605-2 (asterisk - several issues)",
      "bulletinFamily": "exploit",
      "reporter": "Copyright (c) 2013 Greenbone Networks GmbH http://greenbone.net",
      "description": "Several vulnerabilities were discovered in Asterisk, a PBX and telephony\ntoolkit, that allow remote attackers to perform denial of service\nattacks.",
      "lastseen": "2016-09-26T20:40:29",
      "references": [
        "http://www.debian.org/security/2013/dsa-2605.html"
      ],
      "type": "openvas",
      "published": "2013-01-19T00:00:00",
      "hashmap": [
        {
          "hash": "708697c63f7eb369319c6523380bdf7a",
          "key": "bulletinFamily"
        },
        {
          "hash": "7f867f04ef59aa5cade6911e81c14a6f",
          "key": "cvelist"
        },
        {
          "hash": "84813b1457b92d6ba1174abffbb83a2f",
          "key": "cvss"
        },
        {
          "hash": "6767bcda3f63a278d6ce365bec2419bf",
          "key": "description"
        },
        {
          "hash": "805b7b8dfbbd4d9d41ec7f38ec45197d",
          "key": "href"
        },
        {
          "hash": "e29ab3248acfd6527d1025d945e184c6",
          "key": "modified"
        },
        {
          "hash": "74562d71b087df9eabd0c21f99b132cc",
          "key": "naslFamily"
        },
        {
          "hash": "56765472680401499c79732468ba4340",
          "key": "objectVersion"
        },
        {
          "hash": "980e8fbb7a34d92c9977698d6b4a6e38",
          "key": "pluginID"
        },
        {
          "hash": "81ce5c6faae6a195a646ae91c15b0cc8",
          "key": "published"
        },
        {
          "hash": "2d83a0110462ad2c6311bad5832c72fd",
          "key": "references"
        },
        {
          "hash": "5474798d22021d679c1738be31fc4947",
          "key": "reporter"
        },
        {
          "hash": "ecdde94497f1901046a45666767ead49",
          "key": "sourceData"
        },
        {
          "hash": "2972d92c6acaa39b8d3b96678f4423db",
          "key": "title"
        },
        {
          "hash": "47c1f692ea47a21f716dad07043ade01",
          "key": "type"
        },
        {
          "hash": "cfcd208495d565ef66e7dff9f98764da",
          "key": "viewCount"
        }
      ],

      ],
      "href": "http://plugins.openvas.org/nasl.php?oid=103593",
      "cvelist": [
        "CVE-2012-5192",
        "CVE-2012-5193"
      ],
      "cvss": {
        "vector": "AV:NETWORK/AC:LOW/Au:NONE/C:PARTIAL/I:NONE/A:NONE/",
        "score": 5.0
      },
      "objectVersion": "1.2",
      "pluginID": "103593",
      "modified": "2016-04-14T00:00:00",
      "hash": "ae028cee0fc54c8eef6b544331442af742787cdf75cb67a8e815179968e96495",
      "id": "OPENVAS:103593",
      "edition": 1
    }
  },
...
]

We can trivially present this json file as a list of dictionaries:

import json
vulners_objects = json.loads(archive_content)
print(vulners_objects[10]["_id"])
print(vulners_objects[10]["_source"]["title"])
print(vulners_objects[10]["_source"]["cvelist"])

Output:

OPENVAS:71963
Slackware Advisory SSA:2011-224-01 bind 
[u'CVE-2011-2464', u'CVE-2011-1910']
AV:NETWORK/AC:LOW/Au:NONE/C:NONE/I:NONE/A:PARTIAL/

If you need, for example, a dict, from which you can easily get OpenVAS Plugin data by it’s ID (openvas_objects["OPENVAS:71963"]) and you want to know all CVEs that are mentioned in OpenVAS Plugin references (openvas_cves), you can run:

openvas_cves = set()
openvas_objects = dict()
for obj in vulners_objects:
    openvas_objects[obj["_id"]] = obj
    for cve in set(obj["_source"]["cvelist"]):
        openvas_cves.add(cve)

Alexander Leonov

Hi! My name is Alexander and I am a Vulnerability Management specialist. You can read more about me here. Currently, the best way to follow me is my Telegram channel @avleonovcom. I update it more often than this site. If you haven’t used Telegram yet, give it a try. It’s great. You can discuss my posts or ask questions at @avleonovchat.

А всех русскоязычных я приглашаю в ещё один телеграмм канал @avleonovrus, первым делом теперь пишу туда.