By cinhtau / 25 July 2017 / data / est. 2 min read

Elasticsearch Range Query

This post is older than a year. Consider some information might not be accurate anymore.

An accident in the Elasticsearch universe. Instead writing to an daily index it was index to a yearly index. Now I had to check the date range of the documents. The Elasticsearch Date Math is a great help for the Range Query.

Detect Boundaries

First check lower and upper bound

Getting lower bound with sorting on date field

GET fo-log-2017/_search
{
  "_source": "datetime_host",
  "size": 1,
   "sort": [
    {
      "datetime_host": {
        "order": "asc"
      }
    }
  ]
}

Getting upper bound

GET fo-log-2017/_search
{
  "_source": "datetime_host",
  "size": 1,
   "sort": [
    {
      "datetime_host": {
        "order": "desc"
      }
    }
  ]
}

Get Docs Count

Check how many documents exist for a specific day

GET fo-log-2017/_search
{
  "size": 0,
  "query": {
    "range": {
      "datetime_host": {
        "gte": "2017-07-24 00:00",
        "lte": "2017-07-25 00:00",
        "format": "yyyy-MM-dd HH:mm"
      }
    }
  }
}

Example output

{
  "took": 59,
  "timed_out": false,
  "_shards": {
    "total": 2,
    "successful": 2,
    "failed": 0
  },
  "hits": {
    "total": 9576222,
    "max_score": 0,
    "hits": []
  }
}

Using Date Math

GET fo-log-2017/_search
{
  "size": 0,
  "query": {
    "range": {
      "datetime_host": {
        "gte": "now/d",
        "lte": "now+1d/d",
        "format": "yyyy-MM-dd"
      }
    }
  }
}

{
  "took": 9,
  "timed_out": false,
  "_shards": {
    "total": 2,
    "successful": 2,
    "failed": 0
  },
  "hits": {
    "total": 1627667,
    "max_score": 0,
    "hits": []
  }
}

Reindex with Range Query

Now use it to transfer the data to the daily index

POST _reindex
{
  "source": {
    "index": "fo-log-2017",
    "query": {
      "range": {
        "datetime_host": {
          "gte": "2017-07-25 00:00",
          "lte": "2017-07-26 00:00",
          "format": "yyyy-MM-dd HH:mm"
        }
      }
    }
  },
  "dest": {
    "index": "fo-log-2017.07.25"
  }
}

Delete with Range Query

The range query can also be utilized in the Delete By Query API. For example wrong indexed documents of the wrong month.

curl -XPOST "http://localhost:9200/fo-log-2017.05.24/_delete_by_query" -H 'Content-Type: application/json' -d'
{
  "query": {
    "range": {
      "datetime_host": {
        "gte": "2017-07-24 00:00",
        "lte": "2017-07-25 00:00",
        "format": "yyyy-MM-dd HH:mm"
      }
    }
  }
}'

elasticsearch