Category Archives: Projects

The first Zbrunk dashboard and other news

The long New Year holiday season in Russia was not in vain. I had time to work on Zbrunk. πŸ˜‰ As you can see, I made my first dashboard and added other features.

The first Zbrunk dashboard

No more timestamps in code

I added functions to get Unix timestamps from lines in human-readable time format, e.g. “2019.12.10 13:00:00”.

Instead of a date, you can use words:

  • Today
  • Yesterday
  • N days ago
  • Beginning of Time
  • End of Time

API requests will continue to support only Unix timestamps.

Continue reading

CISO Forum 2019: Vulnerability Management, Red Teaming and a career in Information Security abroad

Today, at the very end of 2019, I want to write about the event I attended in April. Sorry for the delay πŸ˜…. This doesn’t mean that CISO Forum 2019 was not Interesting or I had nothing to share. Not at all! In fact, it was the most inspiring event of the year, and I wanted to make a truly monumental report about it. And I began to write it, but, as it usually happens, more urgent tasks and topics appeared, so the work eventually stopped until now.

The first discussion was about Offensive Security and Red Teams in particular

At CISO Forum 2019 I participated in two panel discussions. The first one was about Offensive Security and Red Teams in particular.

Continue reading

Zbrunk search launcher and event types statistics

I also changed the priorities. Now I think it would be better not to integrate with Grafana, but to create own dashboards and GUI. And to begin with, I created a simple interface for Searching (and Deleting) events.

upd. 16.12.2019

A small update on Zbrunk. First of all, I created a new API call that returns a list of object types in the database and number of this types for a certain period of time. Without it, debugging was rather inconvenient.

$ curl -k https://127.0.0.1:8088/services/searcher -d '{"get_types":"True", "search": {"time":{"from":"1471613579","to":"1471613580"}}, "output_mode": "json", "max_count":"10000000", "auth_token":"8DEE8A67-7700-4BA7-8CBF-4B917CE23512"}'

{"results": ["test_event"], "results_count": 1, "all_results_count": 0, "text": "Types found", "code": 0}

I also added some examples of working with Zbrunk http API from python3. Rewriting them from pure curl was not so trivial. πŸ˜… Flask is rather moody, so I had to abandon the idea of making requests exactly the same as in Splunk. πŸ˜“ But the differences are cosmetic. It is now assumed that events will be passed to collector in valid json (not as a file with json events separated by ‘\n’). I also send all params of requests as json, not data. But for the compatibility reasons previous curl examples will also work. πŸ˜‰

Barapass console Password Manager

I decided to publish my simple console Password Manager. I called it barapass (github). I’ve been using It for quite some time in Linux and in Windows (in WSL). Probably it will also work natively in Windows and MacOS with minimal fixes, but I haven’t tried it yet.

Barapass logo

Why do people use password managers?

Well, with password manager it’s possible to avoid remembering passwords and make them arbitrarily complex and long. And no one will be able to brute force them. Of course, you can simply store passwords in text files, but password managers are better than this because:

  • no one will see your password over your shoulder;
  • if an attacker gains access to the files on your host, it won’t possible to read your passwords from the encrypted file or storage (well, ideally);
  • it’s easier to search for objects in the password manager and copy values from it.

I wanted something as simple as editing a text file with the key-value content. And I wanted it to be stored in a secure manner, and security could be easily checked, “simple and stupid”.

Continue reading

Zbrunk universal data analysis system

Zbrunk logo

Zbrunk project (github) began almost like a joke. And in a way it is. 😜 In short, my friends and I decided to make an open-source (MIT license) tool, which will be a kind of alternative to Splunk for some specific tasks. So, it will be possible to:

  • Put structured JSON events in Zbrunk using http collector API
  • Get the events from Zbrunk using http search API
  • Make information panels based on these search requests and place them on dashboards

Why is it necessary? Well, I’ve worked a lot with Splunk in recent years. I like the main concepts, and I think working with the events is a very effective and natural way of processing and presenting data. But for my tasks (Asset Management, Compliance Management, Vulnerability Management) with several hundred megabytes of raw data per day to process and dashboards that need to be updated once or several times a day Splunk felt like an overkill. You really don’t need such performance for these tasks.

And, considering the price, it only makes sense if your organization already uses Splunk for other tasks. After Splunk decision to leave Russian market, this became even more obvious, so many people began to look for alternatives for possible and, as far as possible, painless migration.

We are realistic, the performance and search capabilities of Zbrunk will be MUCH worse. It’s impossible to make such universal and effective solution as a pet project without any resources. So, don’t expect something that will process terabytes of logs in near real time, the goal is completely different. But if you want same basic tool to make dashboards, it worth a try. πŸ™‚

Now, after first weekend of coding and planning it’s possible to send events to Zbrunk just like you do it using the Splunk HTTP Event Collector and they appear in MongoDB:

$ echo -e '{"time":"1471613579", "host":"test_host", "event":{"test_key":"test_line1"}}\n{"time":"1471613580", "host":"test_host", "event":{"test_key":"test_line2"}}' > temp_data
$ curl -k https://127.0.0.1:8088/services/collector -H 'Authorization: Zbrunk 8DEE8A67-7700-4BA7-8CBF-4B917CE2352B' -d @temp_data
{"text": "Success", "code": 0}

In Mongo:

> db.events.find()
{ "_id" : ObjectId("5d62d7061600085d80bb1ea8"), "time" : "1471613579", "host" : "test_host", "event" : { "test_key" : "test_line1" }, "event_type" : "test_event" }
{ "_id" : ObjectId("5d62d7061600085d80bb1ea9"), "time" : "1471613580", "host" : "test_host", "event" : { "test_key" : "test_line2" }, "event_type" : "test_event" }

Thus, it will be very easy to use your existing custom connectors if you already have some. The next step is to make basic http search API, prepare dashboard data using these search requests and somehow show these dashboards, for example, in Grafana. Stay tuned and welcome to participate. πŸ˜‰

zbrunk_madskillz.jpg

Open Positioner: my new project for tracking IT and security jobs

The idea of my new project is to retrieve the data from job-searching websites and provide better filtering, searching and visualization.

I think for the most people who read this, searching for a job in Internet is a pretty common activity. Even if you are not going to change job right now, it might be quite interesting to know what skills are currently the most valuable for your specialization and what is going on on the Global labor market.

Continue reading

Packabit project: building Nmap deb packages for Ubuntu

During the long New Year holidays (30 dec – 8 jan) I started a new project: Vagrant-based Linux package builder called Packabit. I thought it might be nice to have scripts that will automatically build a Linux packages from sources and will NOT litter main system with unnecessary packages. Something like a very simplified build server.

Packabit - Vagrant-based Linux package builder

Why might someone want to build Linux packages on their own?

Official repositories of Linux distros usually contain pretty old versions of packages. Let’s take, for example, Nmap. The only version of Nmap available in the repository for Ubuntu 18.04.1 LTS is 7.60 , and the latest stable version of Nmap available on official website is 7.70.

Release dates according Nmap changelog:

  • Nmap 7.70 [2018-03-20]
  • Nmap 7.60 [2017-07-31]

The latency is more than a year.

Is it really necessary to use the latest version? Actually, yes. Every new version of Nmap contain more banner detects and service detects and produce more adequate results. Nmap project offers official packages only for RPM-based distributions: CentOS, RHEL, Fedora, etc. And if you need packages for DEB-based distributions, Debian, Ubuntu, Kali, etc. you should build them yourself.

What is the Packabit?

Currently it’s just 2 bash scripts for building stable Nmap package for Ubuntu 18.04: one is for creation and launching Vagrant virtual machine, the other runs on guest virtual machine and build Nmap package from the sources.

How to try it

I want this project to be as opensource as possible. If anyone wants to give it a try, get it from Github. For the end-user it looks like this: run the script build_nmap.sh, wait for 10 minutes and get new package from the packages directory. There won’t be any litter in a process. Each time a new virtual machine for building will be automatically created and then destroyed.

Continue reading