This post is older than a year. Consider some information might not be accurate anymore.
Running Elasticsearch as docker container is straightforward. If you don’t have a cluster manager like Kubernetes, monit can help you to keep track of the container lifecycle.
An exemplary monit configuration:
CHECK PROCESS elasticsearch WITH MATCHING "org.elasticsearch.bootstrap.Elasticsearch"
CHECK PROGRAM elasticsearch_container WITH PATH "/usr/bin/docker top elasticsearch"
if status != 0 then alert
alert warning@cinhtau.net
group elkstack
CHECK HOST elasticsearch_healthcheck WITH ADDRESS cinhtau.net
if failed url http://cinhtau.net:9200 for 5 cycles
then alert
alert warning@cinhtau.net BUT not on { action, instance }
depends on elasticsearch_container
group elkstack
CHECK FILE elasticsearch_logfile with path /var/log/elasticsearch/test-cluster.log
if match "ERROR" for 2 times within 5 cycles then alert
alert elasticsearch@cinhtau.net BUT not on { action, instance, nonexist }
depends on elasticsearch_container
group elkstack
Pay attention to the nonexist option. Monit does an implicit check if the logifle exists. Elasticsearch writes a log file. Our housekeeping, logrotate or some kind of janitor script, rename, compress or delete this file. So if the file is missing, monit would complain without the option. If the file doesn’t exists, which is basically good for prod, you don’t want to be notified or warned. No logs, no errors, no worries.
