A Journey of a Thousand Miles Begins with a Single Step

Make docker ps readable

Does it bother you, that docker ps has too wide output?

Read more

Show docker container size

If you build docker containers, ensure that you don’t write any data within the containers. Therefore you can use mapped volumes or data containers. Basically your docker containers that hosts the application or service should be immutable. I won’t go into details why, but how to check that a docker container does not grow.

Read more

Correct type mapping in index template for Elasticsearch

If you use Dropwizard Metrics and the Metrics Reporter you might come into the situation, that the max value is not reported as long value. If it is reported as double Elasticsearch will complain you have an invalid mapping type, since a previous one has the type long. To avoid the situation, you can define in the index template, the type Elasticsearch for new indices from that template.

Read more

Use Ansible for cluster management

Having a Rasbperry Pi Cluster allows you to learn, test and experiment concepts about distributed computing. The first challenge is always how to keep the pi’s up to date. Having four nodes seems still manageable, but what about 64 nodes? Although it is a little bit repetitive, to enter the exact same commands over and over again. Another friend would see just write a bash script that executes the exact command on each node via SSH. Hmm, wait, there is Ansible that does exactly that for you. So a quick jump into Ansible under Ubuntu.

Read more

Groups of Groups in Ansible

Ansible is a free-software platform for configuring and managing computers which combines multi-node software deployment, ad hoc task execution, and configuration management by Red Hat. It is also possible to make groups of groups using the :children suffix.

Read more

Reindex data in Elasticsearch

Today we have reached more than 3000 shards in our elasticsearch clusters. Digging a little deeper, that is definitely too much. Since a shard (primary or replica) is a Lucene index, it consumes file handles, memory, and CPU resources. Each search request will touch a copy of every shard in the index, which isn’t a problem when the shards are spread across several nodes. Contention arises and performance decreases when the shards are competing for the same hardware resources. :- If you keep the logstash daily default, you will come in the situation very soon. I choose now a monthly basis. The latest 2-3 months are kept and old indices are deleted. The outcome is to merge all daily indices of a month to a big index. Therefore the Elasticsearch Reindex API is very useful.

Read more

Handling logstash input multiline codec

The multiline codec will collapse multiline messages and merge them into a single event. The default limit is 500 lines. If there are over 500 lines appended, the multiline codec split the message to the next 500 lines and so forth. This post demonstrates how to deal with this situation. Elasticsearch receives in tags the multiline_codec_max_lines_reached.

Read more

Localisation problem while deinstalling Oracle 11g on Windows

Well I stumbled over a funny fact, that Oracle could not handle localization properly. They ask in german for a choice with ja/nein means yes/no. See for yourself to tell the deinstaller my choice.

Read more

NonStop SQL MX with IntelliJ or DataGrip

NonStop SQL is a commercial relational database management system that is designed for fault tolerance and scalability for the HP NonStop. The latest version of the product is SQL MX 3.2.1 which was released in February 2013. This post describes how to setup IntelliJ or respective DataGrip to work with SQL MX.

Read more