Tag Archives: xml

Retrieving data from Splunk Dashboard Panels via API

Fist of all, why might someone want to get data from the panels of a dashboard in Splunk? Why it might be useful? Well, if the script can process everything that human analyst sees on a Splunk dashboard, all the automation comes very natural. You just figure out what routine operations the analyst usually does using the dashboard and repeat his actions in the script as is. It may be the anomaly detection, remediation task creation, reaction on various events, whatever. It really opens endless possibilities without alerts, reports and all this stuff. I’m very excited about this. ūüôā

Exporting data from Splunk dashboard

Let’s say we have a Splunk dashboard and want to get data from the table panel using a python script. The problem is that the content of the table that we see is not actually stored anywhere. In fact it is the results of some search query, from the XML representation of the dashboard, executed by Splunk web GUI. To get this data we should execute the same search request.

That’s why we should:

  1. Get XML code of the dashboard
  2. Get the search query for each panel
  3. Process searches based on other searches and get complete search query for each panel
  4. Launch the search request and get the results

First of all, we need to create a special account that will be used for getting data from Splunk. In Web GUI “Access controls -> Users”.

Continue reading

Converting Nmap xml scan reports to json

Unfortunately, Nmap can not save the results in json. All available output options:

-oN <filespec> (normal output)
-oX <filespec> (XML output)
-oS <filespec> (ScRipT KIdd|3 oUTpuT)
-oG <filespec> (grepable output)
-oA <basename> (Output to all formats)

And processing xml results may not be easy an easy task. Just look how I analyze the contents of the Nessus report in “Parsing Nessus v2 XML reports with python“. Not the most readable code, right? And what alternatives do we have?

Nmap json scan report

Formal XML to json conversion is impossible. Formats are very different. However, there are python modules, for example xmltodict, that can reliably convert XML into Python structures of dictionaries, lists and strings. However, they have to change some names of parameters to avoid collisions. In my opinion this is not a big price for convenience.

So, let’s see how this will work for Nmap command:

nmap -sV -oX nmap_output.xml avleonov.com 1>/dev/null 2>/dev/null

Continue reading

Masking Vulnerability Scan reports

Continuing the series of posts about Kenna (“Analyzing Vulnerability Scan data“,¬†“Connectors and REST API“) and similar services. Is it actually safe to send your vulnerability data to some external cloud service for analysis?¬†Leakage of such information can potentially cause great damage to your organization, right?

Masking Vulnerability Scans

It’s once again a problem of trust to vendor. IMHO, in some cases it may make sense to hide the real hostnames and ip-addresses of the target hosts in scan reports. So, it would be clear¬†for analysis vendor that some critical vulnerability exists somewhere, but it would not be clear where exactly.

To do this, each hostname/ip-address should be replaced to some values of similar type and should be replaced on the same value each time. So the algorithms of Kenna-like service could work with this masked reports. This mean that we need to create a replacement dictionary.

Continue reading

Kenna Security: Connectors and REST API

In the last post about Kenna Security cloud service I mentioned their¬†main features for analyzing data from different vulnerability scanners. Now let’s see how to import Tenable Nessus scan results in Kenna.¬†Here you can see the list of connectors for all supported products:

Kenna connectors

Three connectors for Nessus are available:

  • Nessus Importer retrieves existing scan results from your Nessus server.
  • Nessus Scanner can schedule scans on your Nessus server.
  • Nessus XML imports xml (.Nessus2) files.

First two connectors work with Nessus server directly. And they probably won’t work anymore with Nessus Professional 7, because of API removing (see “New Nessus 7 Professional and the end of cost-effective Vulnerability Management (as we knew it)“). If Nessus server is deployed on-premise you should use special¬†Kenna¬†Virtual Tunnel.

Last “Nessus XML” connector is the most flexible. No matter how you got your scan results, it will be possible to import them to Kenna.¬†See how to get XML reports from from Nessus server in a post “Retrieving scan results through Nessus API“.¬† You can upload XML scan results using Kenna web GUI (not very efficient way, but for testing – why not?) or REST API.

To use Kenna REST API you will need an Application Token. Go to the the Settings menu -> Applications:

Kenna settings

Continue reading

Parsing Nessus v2 XML reports with python

Previous post about Nessus v2 reports I was writing mainly about the format itself. Now let’s see how you can parse them with Python.

Please don’t work with XML documents the same way you process text files. I adore bash scripting and awk, but that’s an awful idea to use it for XML parsing. In Python you can do it much easier and the script will work much faster. I will use lxml library for this.

So, let’s assume that we have Nessus xml report. We could get it using Nessus API or SecurityCenter API. First of all, we need to read content of the file.

Continue reading

F-Secure Radar basic reporting

In previous post about Radar (“F-Secure Radar Vulnerability Management solution“) I was describing how to use it for authenticated and unauthenticated scanning both inside and outside of your network.

But what about the vulnerability reports?

To get vulnerability report you should open Reporting Tab. As you can see, Radar supports reports for single scan results and summary reports. I don’t actually a big fan of standard vulnerability summary reports, because in practice you will always need to change something in them, and it’s impossible in most cases.

F-Secure Radar reports

I have filtered only Linux OS scans using filter. You can also filter by friendly name (some id, that you can set manually), host name/ip , time of scanning, responsible person, severity level, scan group or even by scan tags.

Radar Filters

Continue reading

Nessus v2 xml report format

Knowing the structure of Nessus v2 (xml) report may be useful for those who want to analyze scan results in SIEM solution or with own scripts (in this case see also “Retrieving scan results through Nessus API” and “VM Remediation using external task tracking systems“).

Upd. Read this post about practical processing of such reports: “Parsing Nessus v2 XML reports with python

nessus_v2_report_logo

There is a good official Tenable manual: Nessus v2 File Format. If you want to get a detailed description of this format i recommend you to read it.

nessus v2 xml format

Here I would like to share my impressions, explain how to retrieve useful information from the scan report easily and note some dangers during the processing that may lead to incorrect prioritisation of vulnerabilities.

Nessus report contains information about the actual scanning results (Report) and the scan settings (Policy). Sometimes it is very useful to look at a scanning policy for debugging. But in most cases we just need proper information about detected vulnerabilities. Therefore, we examine Report section.

Nessus Client Data

Continue reading