Category Archives: Concept

The first Zbrunk dashboard and other news

The long New Year holiday season in Russia was not in vain. I had time to work on Zbrunk. πŸ˜‰ As you can see, I made my first dashboard and added other features.

The first Zbrunk dashboard

No more timestamps in code

I added functions to get Unix timestamps from lines in human-readable time format, e.g. “2019.12.10 13:00:00”.

Instead of a date, you can use words:

  • Today
  • Yesterday
  • N days ago
  • Beginning of Time
  • End of Time

API requests will continue to support only Unix timestamps.

Continue reading

Zbrunk search launcher and event types statistics

I also changed the priorities. Now I think it would be better not to integrate with Grafana, but to create own dashboards and GUI. And to begin with, I created a simple interface for Searching (and Deleting) events.

upd. 16.12.2019

A small update on Zbrunk. First of all, I created a new API call that returns a list of object types in the database and number of this types for a certain period of time. Without it, debugging was rather inconvenient.

$ curl -k https://127.0.0.1:8088/services/searcher -d '{"get_types":"True", "search": {"time":{"from":"1471613579","to":"1471613580"}}, "output_mode": "json", "max_count":"10000000", "auth_token":"8DEE8A67-7700-4BA7-8CBF-4B917CE23512"}'

{"results": ["test_event"], "results_count": 1, "all_results_count": 0, "text": "Types found", "code": 0}

I also added some examples of working with Zbrunk http API from python3. Rewriting them from pure curl was not so trivial. πŸ˜… Flask is rather moody, so I had to abandon the idea of making requests exactly the same as in Splunk. πŸ˜“ But the differences are cosmetic. It is now assumed that events will be passed to collector in valid json (not as a file with json events separated by ‘\n’). I also send all params of requests as json, not data. But for the compatibility reasons previous curl examples will also work. πŸ˜‰

CentOS 8 with IceWM Desktop Environment

Do you need CentOS 8 with IceWM as desktop Operating System? Most likely not. Especially if you want it to work smoothly without any worries and troubles. However, if you enjoy playing with new desktop environments, you might find it fun.

CentOS 8 with IceWM desktop environment

My reasons were as follows:

  1. I wanted to use the same Linux distribution for server and desktop. Just to minimize possible surprises during the deployment.
  2. I wanted to know what is going on in the RPM-based part of Linux world. The only way to achieve this is to use such distribution every day.
  3. I was tired of problems with the Virtual Box guest additions in CentOS 7 (yes , I run it all as a virtual machine), especially after the 3.10 kernel updates. It was time to move on.
  4. I didn’t want to use Gnome 3, because it’s slow and ugly (however it’s fully functional!). And there were no other DEs in CentOS 8 repositories at that time.

So, I tried CentOS 8 with IceWM (installed it from source) and it worked. IceWM is small, very fast, ascetic, and in some ways quite intuitive. There were some problems with the clipboard (in xTerm and with VBox shared clipboard) and with language switching, but I figured it out and I think that I would probably continue to use it. Below are some notes on how I installed it and resolved the issues.

Continue reading

Barapass console Password Manager

I decided to publish my simple console Password Manager. I called it barapass (github). I’ve been using It for quite some time in Linux and in Windows (in WSL). Probably it will also work natively in Windows and MacOS with minimal fixes, but I haven’t tried it yet.

Barapass logo

Why do people use password managers?

Well, with password manager it’s possible to avoid remembering passwords and make them arbitrarily complex and long. And no one will be able to brute force them. Of course, you can simply store passwords in text files, but password managers are better than this because:

  • no one will see your password over your shoulder;
  • if an attacker gains access to the files on your host, it won’t possible to read your passwords from the encrypted file or storage (well, ideally);
  • it’s easier to search for objects in the password manager and copy values from it.

I wanted something as simple as editing a text file with the key-value content. And I wanted it to be stored in a secure manner, and security could be easily checked, “simple and stupid”.

Continue reading

Zbrunk universal data analysis system

Zbrunk logo

Zbrunk project (github) began almost like a joke. And in a way it is. 😜 In short, my friends and I decided to make an open-source (MIT license) tool, which will be a kind of alternative to Splunk for some specific tasks. So, it will be possible to:

  • Put structured JSON events in Zbrunk using http collector API
  • Get the events from Zbrunk using http search API
  • Make information panels based on these search requests and place them on dashboards

Why is it necessary? Well, I’ve worked a lot with Splunk in recent years. I like the main concepts, and I think working with the events is a very effective and natural way of processing and presenting data. But for my tasks (Asset Management, Compliance Management, Vulnerability Management) with several hundred megabytes of raw data per day to process and dashboards that need to be updated once or several times a day Splunk felt like an overkill. You really don’t need such performance for these tasks.

And, considering the price, it only makes sense if your organization already uses Splunk for other tasks. After Splunk decision to leave Russian market, this became even more obvious, so many people began to look for alternatives for possible and, as far as possible, painless migration.

We are realistic, the performance and search capabilities of Zbrunk will be MUCH worse. It’s impossible to make such universal and effective solution as a pet project without any resources. So, don’t expect something that will process terabytes of logs in near real time, the goal is completely different. But if you want same basic tool to make dashboards, it worth a try. πŸ™‚

Now, after first weekend of coding and planning it’s possible to send events to Zbrunk just like you do it using the Splunk HTTP Event Collector and they appear in MongoDB:

$ echo -e '{"time":"1471613579", "host":"test_host", "event":{"test_key":"test_line1"}}\n{"time":"1471613580", "host":"test_host", "event":{"test_key":"test_line2"}}' > temp_data
$ curl -k https://127.0.0.1:8088/services/collector -H 'Authorization: Zbrunk 8DEE8A67-7700-4BA7-8CBF-4B917CE2352B' -d @temp_data
{"text": "Success", "code": 0}

In Mongo:

> db.events.find()
{ "_id" : ObjectId("5d62d7061600085d80bb1ea8"), "time" : "1471613579", "host" : "test_host", "event" : { "test_key" : "test_line1" }, "event_type" : "test_event" }
{ "_id" : ObjectId("5d62d7061600085d80bb1ea9"), "time" : "1471613580", "host" : "test_host", "event" : { "test_key" : "test_line2" }, "event_type" : "test_event" }

Thus, it will be very easy to use your existing custom connectors if you already have some. The next step is to make basic http search API, prepare dashboard data using these search requests and somehow show these dashboards, for example, in Grafana. Stay tuned and welcome to participate. πŸ˜‰

zbrunk_madskillz.jpg

The most magnificent thing about Vulnerabilities and who is behind the magic

What I like the most about software vulnerabilities is how “vulnerability”, as a quality of a real object (and the computer program is real), literally appears from nothing.

The most magnificent thing about Vulnerabilities and who is behind the magic

Let’s say we have a fully updated server. We turn it off, lock it in a safe and forget about it for half a year. Six months later, we get it, turn it on. It is the same and works absolutely the same. But now it is also exposed to dozens of critical vulnerabilities that, with some (un)luck, can be exploited by any script kiddie. New important characteristic of the material object appeared from nowhere, isn’t this magnificent? 🀩

Continue reading

First steps with Docker: installation in CentOS 7, vulnerability assessment, interactive mode and saving changes

Docker and containerization are literally everywhere. IMHO, this changes the IT landscape much more than virtualization and clouds. Let’s say you have a host, you checked it and find out that there are no vulnerable packages. But what’s the point if this host runs Docker containers with their own packages that may be vulnerable? Add to this the issues with complex orchestration systems, such as Kubernetes, completely different DevOps subculture with their own terms, slang, beliefs, priorities, and the situation begins to look like complete IT Hell. πŸ™‚

First steps with Docker

But it seems that Docker will be here for a long time, so we will have to live with it. πŸ˜‰ Here I will not write what Docker is and how it works. There are many publications about this. I personally interested in what actually we can do with these weird “virtual machines”, how can we run and assess them.

Continue reading