Continuing the topic about perimeter services. As I mentioned earlier, I don’t think that the external perimeter services should be considered as a fully functional replacement for custom Vulnerability Management processes. I would rather see their results as an additional feed showing the problems your current VM process has. Recently I tested the Detectify’s Asset Inventory (Monitoring) solution, which provides such feed by automatically detecting the issues with your second, third (and more) leveled domains and related web services.

Let say your organization has several second level web domains, over9000 third (and more) level domains, and you even don’t know for what services they are used. This is a normal situation for a large organization. So, you simply add yourorganization.com to Detectify, activate Asset Monitoring, and Detectify automatically discovers third (and more) level domains and related technologies: web services, CMS, JavaScript frameworks and libraries. “It provides thousands of fingerprints and hundreds of tests for stateless vulnerabilities such as code repository exposure for SVN or Git.” This is called fingerprinting.

Using this data Detectify can detect anomalies – findings: known vulnerabilities, misconfiguration, like the redirects to non-existing server, data disclosures, path traversal, etc.

Everything is practical, without any unexploitable trash, and can be exported using the API .

They have some python examples on github, but non-functional or for old python 2.7, so I wrote my own small python 3 scripts to export the data (fingerprints and findings) to the JSON file and read it.

Exporting Detectify Asset Inventory data to JSON file

#!/usr/bin/python

import requests
import json
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

headers = {"X-Detectify-Key":"a2341cb9c891436eb46cfdd41b255e18"}
url = "https://api.detectify.com"

results_fingerprints = dict()
results_findings = dict()

for domain in requests.get(url + "/rest/v2/domains/", headers=headers, verify=False).json():
    subdomains = requests.get(url + "/rest/v2/domains/" + domain['token'] + "/subdomains/", headers=headers, verify=False).json()
    print(domain['name'])
    results_findings[domain['name']]  = requests.get(url + "/rest/v2/domains/" + domain['token'] + "/findings/", headers=headers, verify=False).json()

    results_fingerprints[domain['name']] = dict()
    results_fingerprints[domain['name']]['params'] = domain
    results_fingerprints[domain['name']]['fingerprint'] = requests.get(url + "/rest/v2/fingerprints/" + domain['token'] + "/?true", headers=headers, verify=False).json()

    n= 1
    for subdomain in subdomains:
        print(str(n) + "/" + str(len(subdomains)) + ". " + subdomain['name'])
        results_fingerprints[subdomain['name']] = dict()
        results_fingerprints[subdomain['name']]['params'] = subdomain
        results_fingerprints[subdomain['name']]['fingerprint'] = requests.get(url + "/rest/v2/fingerprints/" + subdomain['uuid'] + "/?true", headers=headers, verify=False).json()
        n+=1

f = open("results.json","w")
f.write(json.dumps({"results_fingerprints":results_fingerprints,"results_findings":results_findings}, sort_keys=True, indent=4,))
f.close()

Reading the exported Detectify Asset Inventory data file

And here is how to work with such file:

import json

f = open("results.json","r")
results = json.loads(f.read())
f.close()

print(len(results['results_fingerprints'].keys()))

for domainname in results['results_fingerprints']:
    print(domainname)
    for fingerprint in results['results_fingerprints'][domainname]['fingerprint']:
        print( "  " + fingerprint['software_name'] + " - " + fingerprint['software_version'])

findings_all = dict()
for domain in results['results_findings']:
    for finding in results['results_findings'][domain]:
        if not finding['title'] in findings_all:
            findings_all[finding['title']] = set()
        findings_all[finding['title']].add(finding['found_at'])

findings_all_keys = list(findings_all.keys())
findings_all_keys.sort()
for title in findings_all_keys:
    print(title)
    print(" - " + str(findings_all[title]))

Detectify also provides a traditional Web Application Scanner, but I haven’t tested it yet.