This post is older than a year. Consider some information might not be accurate anymore.
Used: elasticsearch 6.1.2 fluentd 0.12 docker 17.12
By default, Docker captures the standard output (and standard error) of all your containers, and writes them in files using the JSON format. It is advised to set a max size, otherwise you will run out of disk space. Having unified logging with Elasticsearch allows you to investigate logs in a single point of view. Sending the logs to Elasticsearch from the Docker containers is quite easy. Fluentd is a data collector, which a Docker container can use by omitting the option --log-driver=fluentd.
docker run --log-driver=fluentd ubuntu echo 'Hello Fluentd!'All we have to do, is to run Fluentd with the Elasticsearch output plugin. We can run Fluentd with Docker itself 
. Build fluentd Docker image with Elasticsearch plugin. This is a slight modified version of the official Docker image.
FROM fluent/fluentd:v0.12-onbuild
ENV http_proxy http://10.0.2.2:3128
ENV https_proxy https://10.0.2.2:3128
ENV no_proxy localhost,my-elasticsearch-cluster
USER root
RUN adduser -S fluent && apk add --update --virtual .build-deps \
sudo build-base ruby-dev \
&& gem install \
fluent-plugin-elasticsearch \
&& gem sources --clear-all \
&& apk del .build-deps \
&& rm -rf /var/cache/apk/* \
/home/fluent/.gem/ruby/2.3.0/cache/*.gemThe fluentd configuration:
<source>
@type forward
port 24224
bind 0.0.0.0
</source>
<match docker.consul>
@type elasticsearch
host my-es-cluster
user cinhtau
password "lemapper"
port 9200
tag_key @le_mapper
flush_interval 5s # for testing
logstash_format true
logstash_prefix consul
type_name doc
pipeline consul
</match>
<match docker.*>
@type elasticsearch
host my-es-cluster
user cinhtau
password "lemapper"
port 9200
tag_key @le_mapper
flush_interval 5s
logstash_format true
logstash_prefix docker
type_name doc
</match>Consul logs are ingested into the consul index name pattern which matches the docker.consul tag. Other output goes to the docker index.
The tag for fluentd is omitted by the run command:
docker run --log-driver=fluentd --log-opt fluentd-tag=docker.consul consul