Elasticsearch + Nginx Access Log using Kibana and FileBeat

Long time no see. Very short post today, very fast howto to implement access log logging to elasticsearch using simple utilities.

First of all, I expect you have already setup your elastic cluster with Kibana or Grafana or whatever.

Rsyslog + Elasticsearch/Redis backend template

Here is example of template when using redis or Elasticsearch backend for rsyslog. Very usefull along with logstash and kibana.

 

module(load="omhiredis")
template(name="ls_json" type="list" option.json="on")
   { constant(value="{")
     constant(value="\"timestamp\":\"")         property(name="timegenerated" dateFormat="rfc3339")
     constant(value="\",\"message\":\"")         property(name="msg")
     constant(value="\",\"host\":\"")            property(name="fromhost")
     constant(value="\",\"host_ip\":\"")         property(name="fromhost-ip")
     constant(value="\",\"logsource\":\"")       property(name="fromhost")
     constant(value="\",\"severity_label\":\"")  property(name="syslogseverity-text")
     constant(value="\",\"severity\":\"")        property(name="syslogseverity")
     constant(value="\",\"facility_label\":\"")  property(name="syslogfacility-text")
     constant(value="\",\"facility\":\"")        property(name="syslogfacility")
     constant(value="\",\"program\":\"")         property(name="programname")
     constant(value="\",\"pid\":\"")             property(name="procid")
     constant(value="\",\"syslogtag\":\"")       property(name="syslogtag")
     constant(value="\"}\n")
   } 
*.* action(
  name="push_redis"
  type="omhiredis"
  server="127.0.0.1"
  mode="queue"
  key="syslog"
  template="ls_json"
)

 

RIPE-Atlas Anchor installation

Ripe-Atlas project Anchor installation.

Anchor is high capacity RIPE-Atlas probe for internet measurements.

Anchor mainly consist of  Soekris Net6501-70 board (can find on kd85.com) and running Centos 6.x.

 

 

[gallery link="file" ids="802,801,800,799"]

Analyzing Spam - Visualization

Beautiful image of visualization spam traffic on primary mailserver.

Every blue path is spam detected and delivered to mailbox spam folder, every yellow path is rejected spam.

RIPE-Atlas Traceroute monitoring

Last day I did some measurements using RIPE-Atlas project.

I did traceroute to Livesport infrastructure using 1000 probes around the globe.

Here is some nice visualization of ASN paths.

 

How to handle thousands of reports from servers - Logstash, ElasticSearch, Kibana

Many services and server audit utilities like logcheck, logwatch, cron daemon, aide, ZRM, etc. are sending emails to local user or root. Some of them, can be easily configured but some of them not.

Best way in my case is to deal with emails directly, but how ?

Gitolite recover from lost authorized_keys

Sometimes bad things happed and you have to recover your git server from backup or you have corrupted authorized_keys for git user.

First of all, you should know what paths and user you using. For this example we have:

User: git

Home: /home/git

Repositories: /data/git (linked into /home/git as repositories)

Samsung SSD 840 PRO - performance degradation

About year ago I wrote blogpost about endurance and performance test of Samsung SSD 840 PRO. Some things has changed, especially firmware of disks.

I performed test on disk with firmware DXM04B0Q. When you want to buy this disk now you probably have firmware DXM05B0Q or DXM06B0Q.

Problem is, both new firmwares has the same performance degradation issue.

Since my first blogpost we bought about 100 disks and after few months we're started observing some problems on database servers. Servers and disks was very slow, we are talking about 10 MB/s continuous write speed.

First of all, we repeat our direct disk test, then we started to test disk with different filesystems.

Command Line Tool for iRedMail (MySQL backend only)

Hi,

if anyone is interested in  open source mail server solution iRedMail and use MySQL as backend should now use my small cli script. Script has limited functions but it's perfect for things like importing new domains or creating many email accounts.

Script is opensource and use some functions from original iredadmin web management. So you need iredadmin installed, which is default option.

iRedMail CLI Tool on Github

Samsung SSD 840: Endurance Destruct Test

When you operate a Datacenter with many servers you also probably have big number of installed disks. In most cases even you have cluster, especialy in small companies, there is still SPOF (Single point of failure) somewhere.

In this SPOF it's critical to know condition of the system. When system using HDD it's important to know HDD condition to prevent failures, plan maintenance etc.

We will talk about Samsung SSD 840 PRO series. We realize this SSD has very good performance and lifetime. Before we use it in production we must know how to monitor condition. There is many articles and technical specification but we had a lot of questions without answer.

(Update: 10.10.2013)

(Samsung performance issues, read new post, 03.12.2014)

Forgotten Skills: Build your own linux kernel

Firt we will talk about WHY. Many people have many opinions on that and I'd like to present you mine.

I work as System Specialist. During my work I often came to point that normal kernel from distribution doesn't work. Whole or part of server was unusable because one of these common reasons:

• Virtualization support with combination of new HW


• Missing driver for some Raid Controllers


• Missing driver for network cards


• Security hardening patches for specific systems

I don't remember which of the reason was the first when I must build my own kernel to be able use server or computer with all it's hardware. In few next years another big reason appear.

• New kernel have fixed some performance issues

Now we can talk about how to do it.