Converting Nmap xml scan reports to json. Unfortunately, Nmap can not save the results in json. All available output options:
-oN <filespec> (normal output)
-oX <filespec> (XML output)
-oS <filespec> (ScRipT KIdd|3 oUTpuT)
-oG <filespec> (grepable output)
-oA <basename> (Output to all formats)
And processing xml results may not be easy an easy task. Just look how I analyze the contents of the Nessus report in “Parsing Nessus v2 XML reports with python“. Not the most readable code, right? And what alternatives do we have?
Formal XML to json conversion is impossible. Formats are very different. However, there are python modules, for example xmltodict, that can reliably convert XML into Python structures of dictionaries, lists and strings. However, they have to change some names of parameters to avoid collisions. In my opinion this is not a big price for convenience.
So, let’s see how this will work for Nmap command:
nmap -sV -oX nmap_output.xml avleonov.com 1>/dev/null 2>/dev/null
Content of nmap_output.xml:
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE nmaprun> <?xml-stylesheet href="file:///usr/bin/../share/nmap/nmap.xsl" type="text/xsl"?> <!-- Nmap 7.60 scan initiated Sun Mar 11 12:21:30 2018 as: nmap -sV -oX output.xml avleonov.com --> <nmaprun scanner="nmap" args="nmap -sV -oX output.xml avleonov.com" start="1520760090" startstr="Sun Mar 11 12:21:30 2018" version="7.60" xmloutputversion="1.04"> <scaninfo type="connect" protocol="tcp" numservices="1000" services="1,3-4,6-7,9,13,17,19-26,30,32-33,37,42-43,49,53,70,79-85,88-90,99-100,106,109-111,113,119,125,135,139,143-144,146,161,163,179,199,211-212,222,254-256,259,264,280,301,306,311,340,366,389,406-407,416-417,425,427,443-445,458,464-465,481,497,500,512-515,524,541,543-545,548,554-555,563,587,593,616-617,625,631,636,646,648,666-668,683,687,691,700,705,711,714,720,722,726,749,765,777,783,787,800-801,808,843,873,880,888,898,900-903,911-912,981,987,990,992-993,995,999-1002,1007,1009-1011,1021-1100,1102,1104-1108,1110-1114,1117,1119,1121-1124,1126,1130-1132,1137-1138,1141,1145,1147-1149,1151-1152,1154,1163-1166,1169,1174-1175,1183,1185-1187,1192,1198-1199,1201,1213,1216-1218,1233-1234,1236,1244,1247-1248,1259,1271-1272,1277,1287,1296,1300-1301,1309-1311,1322,1328,1334,1352,1417,1433-1434,1443,1455,1461,1494,1500-1501,1503,1521,1524,1533,1556,1580,1583,1594,1600,1641,1658,1666,1687-1688,1700,1717-1721,1723,1755,1761,1782-1783,1801,1805,1812,1839-1840,1862-1864,1875,1900,1914,1935,1947,1971-1972,1974,1984,1998-2010,2013,2020-2022,2030,2033-2035,2038,2040-2043,2045-2049,2065,2068,2099-2100,2103,2105-2107,2111,2119,2121,2126,2135,2144,2160-2161,2170,2179,2190-2191,2196,2200,2222,2251,2260,2288,2301,2323,2366,2381-2383,2393-2394,2399,2401,2492,2500,2522,2525,2557,2601-2602,2604-2605,2607-2608,2638,2701-2702,2710,2717-2718,2725,2800,2809,2811,2869,2875,2909-2910,2920,2967-2968,2998,3000-3001,3003,3005-3007,3011,3013,3017,3030-3031,3052,3071,3077,3128,3168,3211,3221,3260-3261,3268-3269,3283,3300-3301,3306,3322-3325,3333,3351,3367,3369-3372,3389-3390,3404,3476,3493,3517,3527,3546,3551,3580,3659,3689-3690,3703,3737,3766,3784,3800-3801,3809,3814,3826-3828,3851,3869,3871,3878,3880,3889,3905,3914,3918,3920,3945,3971,3986,3995,3998,4000-4006,4045,4111,4125-4126,4129,4224,4242,4279,4321,4343,4443-4446,4449,4550,4567,4662,4848,4899-4900,4998,5000-5004,5009,5030,5033,5050-5051,5054,5060-5061,5080,5087,5100-5102,5120,5190,5200,5214,5221-5222,5225-5226,5269,5280,5298,5357,5405,5414,5431-5432,5440,5500,5510,5544,5550,5555,5560,5566,5631,5633,5666,5678-5679,5718,5730,5800-5802,5810-5811,5815,5822,5825,5850,5859,5862,5877,5900-5904,5906-5907,5910-5911,5915,5922,5925,5950,5952,5959-5963,5987-5989,5998-6007,6009,6025,6059,6100-6101,6106,6112,6123,6129,6156,6346,6389,6502,6510,6543,6547,6565-6567,6580,6646,6666-6669,6689,6692,6699,6779,6788-6789,6792,6839,6881,6901,6969,7000-7002,7004,7007,7019,7025,7070,7100,7103,7106,7200-7201,7402,7435,7443,7496,7512,7625,7627,7676,7741,7777-7778,7800,7911,7920-7921,7937-7938,7999-8002,8007-8011,8021-8022,8031,8042,8045,8080-8090,8093,8099-8100,8180-8181,8192-8194,8200,8222,8254,8290-8292,8300,8333,8383,8400,8402,8443,8500,8600,8649,8651-8652,8654,8701,8800,8873,8888,8899,8994,9000-9003,9009-9011,9040,9050,9071,9080-9081,9090-9091,9099-9103,9110-9111,9200,9207,9220,9290,9415,9418,9485,9500,9502-9503,9535,9575,9593-9595,9618,9666,9876-9878,9898,9900,9917,9929,9943-9944,9968,9998-10004,10009-10010,10012,10024-10025,10082,10180,10215,10243,10566,10616-10617,10621,10626,10628-10629,10778,11110-11111,11967,12000,12174,12265,12345,13456,13722,13782-13783,14000,14238,14441-14442,15000,15002-15004,15660,15742,16000-16001,16012,16016,16018,16080,16113,16992-16993,17877,17988,18040,18101,18988,19101,19283,19315,19350,19780,19801,19842,20000,20005,20031,20221-20222,20828,21571,22939,23502,24444,24800,25734-25735,26214,27000,27352-27353,27355-27356,27715,28201,30000,30718,30951,31038,31337,32768-32785,33354,33899,34571-34573,35500,38292,40193,40911,41511,42510,44176,44442-44443,44501,45100,48080,49152-49161,49163,49165,49167,49175-49176,49400,49999-50003,50006,50300,50389,50500,50636,50800,51103,51493,52673,52822,52848,52869,54045,54328,55055-55056,55555,55600,56737-56738,57294,57797,58080,60020,60443,61532,61900,62078,63331,64623,64680,65000,65129,65389"/> <verbose level="0"/> <debugging level="0"/> <host starttime="1520760091" endtime="1520760120"><status state="up" reason="syn-ack" reason_ttl="0"/> <address addr="81.177.6.213" addrtype="ipv4"/> <hostnames> <hostname name="avleonov.com" type="user"/> <hostname name="srv36-h-st.jino.ru" type="PTR"/> </hostnames> <ports><extraports state="filtered" count="936"> <extrareasons reason="no-responses" count="916"/> <extrareasons reason="host-unreaches" count="20"/> </extraports> <extraports state="closed" count="56"> <extrareasons reason="conn-refused" count="56"/> </extraports> <port protocol="tcp" portid="21"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="ftp" product="ProFTPD" method="probed" conf="10"><cpe>cpe:/a:proftpd:proftpd</cpe></service></port> <port protocol="tcp" portid="22"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="ssh" product="OpenSSH" version="5.3" extrainfo="protocol 2.0" method="probed" conf="10"><cpe>cpe:/a:openbsd:openssh:5.3</cpe></service></port> <port protocol="tcp" portid="80"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="http" product="nginx" method="probed" conf="10"><cpe>cpe:/a:igor_sysoev:nginx</cpe></service></port> <port protocol="tcp" portid="443"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="http" product="nginx" tunnel="ssl" method="probed" conf="10"><cpe>cpe:/a:igor_sysoev:nginx</cpe></service></port> <port protocol="tcp" portid="2222"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="ssh" product="ProFTPD mod_sftp" version="0.9.7" extrainfo="protocol 2.0" method="probed" conf="10"><cpe>cpe:/a:proftpd:proftpd:0.9.7</cpe></service></port> <port protocol="tcp" portid="3306"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="mysql" product="MySQL" version="5.5.59-MariaDB" method="probed" conf="10"><cpe>cpe:/a:mysql:mysql:5.5.59-mariadb</cpe></service></port> <port protocol="tcp" portid="8080"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="http" product="Apache httpd" method="probed" conf="10"><cpe>cpe:/a:apache:http_server</cpe></service></port> <port protocol="tcp" portid="8443"><state state="open" reason="syn-ack" reason_ttl="0"/><service name="ssl" product="Apache httpd" extrainfo="SSL-only mode" tunnel="ssl" method="probed" conf="10"><cpe>cpe:/a:apache:http_server</cpe></service></port> </ports> <times srtt="6937" rttvar="1642" to="100000"/> </host> <runstats><finished time="1520760120" timestr="Sun Mar 11 12:22:00 2018" elapsed="29.83" summary="Nmap done at Sun Mar 11 12:22:00 2018; 1 IP address (1 host up) scanned in 29.83 seconds" exit="success"/><hosts up="1" down="0" total="1"/> </runstats> </nmaprun>
Installing xmltodict:
$ pip install xmltodict
Collecting xmltodict
Downloading xmltodict-0.11.0-py2.py3-none-any.whl
Installing collected packages: xmltodict
Successfully installed xmltodict-0.11.0
Processing nmap_output.xml:
import json
import xmltodict
f = open("nmap_output.xml")
xml_content = f.read()
f.close()
print(json.dumps(xmltodict.parse(xml_content), indent=4, sort_keys=True))
Output:
{
"nmaprun": {
"@args": "nmap -sV -oX output.xml avleonov.com",
"@scanner": "nmap",
"@start": "1520760090",
"@startstr": "Sun Mar 11 12:21:30 2018",
"@version": "7.60",
"@xmloutputversion": "1.04",
"debugging": {
"@level": "0"
},
"host": {
"@endtime": "1520760120",
"@starttime": "1520760091",
"address": {
"@addr": "81.177.6.213",
"@addrtype": "ipv4"
},
"hostnames": {
"hostname": [
{
"@name": "avleonov.com",
"@type": "user"
},
{
"@name": "srv36-h-st.jino.ru",
"@type": "PTR"
}
]
},
"ports": {
"extraports": [
{
"@count": "936",
"@state": "filtered",
"extrareasons": [
{
"@count": "916",
"@reason": "no-responses"
},
{
"@count": "20",
"@reason": "host-unreaches"
}
]
},
{
"@count": "56",
"@state": "closed",
"extrareasons": {
"@count": "56",
"@reason": "conn-refused"
}
}
],
"port": [
{
"@portid": "21",
"@protocol": "tcp",
"service": {
"@conf": "10",
"@method": "probed",
"@name": "ftp",
"@product": "ProFTPD",
"cpe": "cpe:/a:proftpd:proftpd"
},
"state": {
"@reason": "syn-ack",
"@reason_ttl": "0",
"@state": "open"
}
},
{
"@portid": "22",
"@protocol": "tcp",
"service": {
"@conf": "10",
"@extrainfo": "protocol 2.0",
"@method": "probed",
"@name": "ssh",
"@product": "OpenSSH",
"@version": "5.3",
"cpe": "cpe:/a:openbsd:openssh:5.3"
},
"state": {
"@reason": "syn-ack",
"@reason_ttl": "0",
"@state": "open"
}
},
{
"@portid": "80",
"@protocol": "tcp",
"service": {
"@conf": "10",
"@method": "probed",
"@name": "http",
"@product": "nginx",
"cpe": "cpe:/a:igor_sysoev:nginx"
},
"state": {
"@reason": "syn-ack",
"@reason_ttl": "0",
"@state": "open"
}
},
{
"@portid": "443",
"@protocol": "tcp",
"service": {
"@conf": "10",
"@method": "probed",
"@name": "http",
"@product": "nginx",
"@tunnel": "ssl",
"cpe": "cpe:/a:igor_sysoev:nginx"
},
"state": {
"@reason": "syn-ack",
"@reason_ttl": "0",
"@state": "open"
}
},
{
"@portid": "2222",
"@protocol": "tcp",
"service": {
"@conf": "10",
"@extrainfo": "protocol 2.0",
"@method": "probed",
"@name": "ssh",
"@product": "ProFTPD mod_sftp",
"@version": "0.9.7",
"cpe": "cpe:/a:proftpd:proftpd:0.9.7"
},
"state": {
"@reason": "syn-ack",
"@reason_ttl": "0",
"@state": "open"
}
},
{
"@portid": "3306",
"@protocol": "tcp",
"service": {
"@conf": "10",
"@method": "probed",
"@name": "mysql",
"@product": "MySQL",
"@version": "5.5.59-MariaDB",
"cpe": "cpe:/a:mysql:mysql:5.5.59-mariadb"
},
"state": {
"@reason": "syn-ack",
"@reason_ttl": "0",
"@state": "open"
}
},
{
"@portid": "8080",
"@protocol": "tcp",
"service": {
"@conf": "10",
"@method": "probed",
"@name": "http",
"@product": "Apache httpd",
"cpe": "cpe:/a:apache:http_server"
},
"state": {
"@reason": "syn-ack",
"@reason_ttl": "0",
"@state": "open"
}
},
{
"@portid": "8443",
"@protocol": "tcp",
"service": {
"@conf": "10",
"@extrainfo": "SSL-only mode",
"@method": "probed",
"@name": "ssl",
"@product": "Apache httpd",
"@tunnel": "ssl",
"cpe": "cpe:/a:apache:http_server"
},
"state": {
"@reason": "syn-ack",
"@reason_ttl": "0",
"@state": "open"
}
}
]
},
"status": {
"@reason": "syn-ack",
"@reason_ttl": "0",
"@state": "up"
},
"times": {
"@rttvar": "1642",
"@srtt": "6937",
"@to": "100000"
}
},
"runstats": {
"finished": {
"@elapsed": "29.83",
"@exit": "success",
"@summary": "Nmap done at Sun Mar 11 12:22:00 2018; 1 IP address (1 host up) scanned in 29.83 seconds",
"@time": "1520760120",
"@timestr": "Sun Mar 11 12:22:00 2018"
},
"hosts": {
"@down": "0",
"@total": "1",
"@up": "1"
}
},
"scaninfo": {
"@numservices": "1000",
"@protocol": "tcp",
"@services": "1,3-4,6-7,9,13,17,19-26,30,32-33,37,42-43,49,53,70,79-85,88-90,99-100,106,109-111,113,119,125,135,139,143-144,146,161,163,179,199,211-212,222,254-256,259,264,280,301,306,311,340,366,389,406-407,416-417,425,427,443-445,458,464-465,481,497,500,512-515,524,541,543-545,548,554-555,563,587,593,616-617,625,631,636,646,648,666-668,683,687,691,700,705,711,714,720,722,726,749,765,777,783,787,800-801,808,843,873,880,888,898,900-903,911-912,981,987,990,992-993,995,999-1002,1007,1009-1011,1021-1100,1102,1104-1108,1110-1114,1117,1119,1121-1124,1126,1130-1132,1137-1138,1141,1145,1147-1149,1151-1152,1154,1163-1166,1169,1174-1175,1183,1185-1187,1192,1198-1199,1201,1213,1216-1218,1233-1234,1236,1244,1247-1248,1259,1271-1272,1277,1287,1296,1300-1301,1309-1311,1322,1328,1334,1352,1417,1433-1434,1443,1455,1461,1494,1500-1501,1503,1521,1524,1533,1556,1580,1583,1594,1600,1641,1658,1666,1687-1688,1700,1717-1721,1723,1755,1761,1782-1783,1801,1805,1812,1839-1840,1862-1864,1875,1900,1914,1935,1947,1971-1972,1974,1984,1998-2010,2013,2020-2022,2030,2033-2035,2038,2040-2043,2045-2049,2065,2068,2099-2100,2103,2105-2107,2111,2119,2121,2126,2135,2144,2160-2161,2170,2179,2190-2191,2196,2200,2222,2251,2260,2288,2301,2323,2366,2381-2383,2393-2394,2399,2401,2492,2500,2522,2525,2557,2601-2602,2604-2605,2607-2608,2638,2701-2702,2710,2717-2718,2725,2800,2809,2811,2869,2875,2909-2910,2920,2967-2968,2998,3000-3001,3003,3005-3007,3011,3013,3017,3030-3031,3052,3071,3077,3128,3168,3211,3221,3260-3261,3268-3269,3283,3300-3301,3306,3322-3325,3333,3351,3367,3369-3372,3389-3390,3404,3476,3493,3517,3527,3546,3551,3580,3659,3689-3690,3703,3737,3766,3784,3800-3801,3809,3814,3826-3828,3851,3869,3871,3878,3880,3889,3905,3914,3918,3920,3945,3971,3986,3995,3998,4000-4006,4045,4111,4125-4126,4129,4224,4242,4279,4321,4343,4443-4446,4449,4550,4567,4662,4848,4899-4900,4998,5000-5004,5009,5030,5033,5050-5051,5054,5060-5061,5080,5087,5100-5102,5120,5190,5200,5214,5221-5222,5225-5226,5269,5280,5298,5357,5405,5414,5431-5432,5440,5500,5510,5544,5550,5555,5560,5566,5631,5633,5666,5678-5679,5718,5730,5800-5802,5810-5811,5815,5822,5825,5850,5859,5862,5877,5900-5904,5906-5907,5910-5911,5915,5922,5925,5950,5952,5959-5963,5987-5989,5998-6007,6009,6025,6059,6100-6101,6106,6112,6123,6129,6156,6346,6389,6502,6510,6543,6547,6565-6567,6580,6646,6666-6669,6689,6692,6699,6779,6788-6789,6792,6839,6881,6901,6969,7000-7002,7004,7007,7019,7025,7070,7100,7103,7106,7200-7201,7402,7435,7443,7496,7512,7625,7627,7676,7741,7777-7778,7800,7911,7920-7921,7937-7938,7999-8002,8007-8011,8021-8022,8031,8042,8045,8080-8090,8093,8099-8100,8180-8181,8192-8194,8200,8222,8254,8290-8292,8300,8333,8383,8400,8402,8443,8500,8600,8649,8651-8652,8654,8701,8800,8873,8888,8899,8994,9000-9003,9009-9011,9040,9050,9071,9080-9081,9090-9091,9099-9103,9110-9111,9200,9207,9220,9290,9415,9418,9485,9500,9502-9503,9535,9575,9593-9595,9618,9666,9876-9878,9898,9900,9917,9929,9943-9944,9968,9998-10004,10009-10010,10012,10024-10025,10082,10180,10215,10243,10566,10616-10617,10621,10626,10628-10629,10778,11110-11111,11967,12000,12174,12265,12345,13456,13722,13782-13783,14000,14238,14441-14442,15000,15002-15004,15660,15742,16000-16001,16012,16016,16018,16080,16113,16992-16993,17877,17988,18040,18101,18988,19101,19283,19315,19350,19780,19801,19842,20000,20005,20031,20221-20222,20828,21571,22939,23502,24444,24800,25734-25735,26214,27000,27352-27353,27355-27356,27715,28201,30000,30718,30951,31038,31337,32768-32785,33354,33899,34571-34573,35500,38292,40193,40911,41511,42510,44176,44442-44443,44501,45100,48080,49152-49161,49163,49165,49167,49175-49176,49400,49999-50003,50006,50300,50389,50500,50636,50800,51103,51493,52673,52822,52848,52869,54045,54328,55055-55056,55555,55600,56737-56738,57294,57797,58080,60020,60443,61532,61900,62078,63331,64623,64680,65000,65129,65389",
"@type": "connect"
},
"verbose": {
"@level": "0"
}
}
}
Not bad, right?
Let’s get open ports and CPEs:
nmap_results = xmltodict.parse(xml_content)
for port in nmap_results['nmaprun']['host']['ports']['port']:
if 'cpe' in port['service']:
cpe = port['service']['cpe']
else:
cpe = "unknown"
print(port['@portid'] + " - " + cpe)Output:
21 – cpe:/a:proftpd:proftpd
22 – cpe:/a:openbsd:openssh:5.3
80 – cpe:/a:igor_sysoev:nginx
443 – cpe:/a:igor_sysoev:nginx
2222 – cpe:/a:proftpd:proftpd:0.9.7
3306 – cpe:/a:mysql:mysql:5.5.59-mariadb
8080 – cpe:/a:apache:http_server
8443 – cpe:/a:apache:http_server
Hi! My name is Alexander and I am a Vulnerability Management specialist. You can read more about me here. Currently, the best way to follow me is my Telegram channel @avleonovcom. I update it more often than this site. If you haven’t used Telegram yet, give it a try. It’s great. You can discuss my posts or ask questions at @avleonovchat.
А всех русскоязычных я приглашаю в ещё один телеграмм канал @avleonovrus, первым делом теперь пишу туда.
Pingback: Vulchain scan workflow and search queries | Alexander V. Leonov
Very useful! Thanks a lot!