Update Documents By Query

This post is older than a year. Consider some information might not be accurate anymore.

I got a use case, where I needed to grok some text. Therefore I created this exemplary pipeline.

curl -XPUT "http://localhost:9200/_ingest/pipeline/ems_flooding" -H 'Content-Type: application/json' -d'
{
  "description" : "grok the flood counters of an ems message",
   "processors" : [
    {
      "grok" : {
        "field": "event",
        "patterns": ["%{GREEDYDATA}\\(\\<\\<%{DATA:flood.data}\\>\\>\\)\\? %{GREEDYDATA}"],
        "ignore_missing": true,
        "ignore_failure" : true
      }
    }
  ],
   "on_failure" : [
          {
            "set" : {
              "field" : "error",
              "value" : ""
            }
          }
        ]
}'

This pipeline can be used in the Update By Query, that will apply the pipeline to each document.

curl -XPOST "http://localhost:9200/ems/_update_by_query?pipeline=ems_flooding&conflicts=proceed&pretty" -H 'Content-Type: application/json' -d'
{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "foapplication.keyword": "AOME2PPP"
          }
        },
        {
          "query_string": {
            "fields": [
              "event"
            ],
            "query": "MSGPA"
          }
        }
      ]
    }
  }
}'

To check the current task:

curl -XGET 'localhost:9200/_tasks?detailed=true&actions=*byquery&pretty'