Loading...

Remove field from Elasticsearch document

:heavy_exclamation_mark: This post is older than a year. Consider some information might not be accurate anymore. :heavy_exclamation_mark:

Lets assume you have some unwanted field in a document. In my case this is an error field from a pipeline.

GET ems/_search
{
  "query": {
    "simple_query_string": {
      "query": "_exists_:error"
    }
  }
}

To get rid of it in a single document:

POST /fo-ems-2017.04/logs/AVuzBCphEwjYNH5brv6M/_update
{
  "script": "ctx._source.remove(\"error\")"
}

We can also utilize update by query to remove it from all documents, that have the error field.

POST ems/logs/_update_by_query?wait_for_completion=false&conflicts=proceed
{
  "script": {
    "inline": """ctx._source.remove("error")""",
    "lang": "painless"
  },
  "query": {
    "bool": {
      "must": [
        {
          "exists": {
            "field": "error"
          }
        }
      ]
    }
  }
}

The field is logically removed. It still exists in the Apache Lucene segment. In order to delete it physically, a reindex operation and delete index operation is required.

Please remember the terms for blog comments.