Package org.elasticsearch.xpack.esql.optimizer.rules.physical.local

Class ReplaceRoundToWithQueryAndTags

java.lang.Object
org.elasticsearch.xpack.esql.rule.Rule<EvalExec,PhysicalPlan>
org.elasticsearch.xpack.esql.rule.ParameterizedRule<EvalExec,PhysicalPlan,LocalPhysicalOptimizerContext>
org.elasticsearch.xpack.esql.optimizer.PhysicalOptimizerRules.ParameterizedOptimizerRule<EvalExec,LocalPhysicalOptimizerContext>
org.elasticsearch.xpack.esql.optimizer.rules.physical.local.ReplaceRoundToWithQueryAndTags
public class ReplaceRoundToWithQueryAndTags extends PhysicalOptimizerRules.ParameterizedOptimizerRule<EvalExec,LocalPhysicalOptimizerContext>
ReplaceRoundToWithQueryAndTags builds a list of ranges and associated tags base on the rounding points defined in a RoundTo function. It then rewrites the EsQueryExec.query() into a corresponding list of QueryBuilders and tags, each mapped to its respective range.

Here are some examples:

  1. Aggregation with date_histogram.

    The DATE_TRUNC function in the query below can be rewritten to RoundTo by ReplaceDateTruncBucketWithRoundTo. This rule pushes down the RoundTo function by creating a list of QueryBuilderAndTags, so that EsPhysicalOperationProviders can build LuceneSliceQueue with the corresponding list of QueryAndTags to process further. 

        | STATS COUNT(*) BY d = DATE_TRUNC(1 day, date)
    becomes, the rounding points are calculated according to SearchStats and predicates from the query. 
        | EVAL d = ROUND_TO(hire_date, 1697760000000, 1697846400000, 1697932800000)
    | STATS COUNT(*) BY d
    becomes 
        [QueryBuilderAndTags[query={
     "esql_single_value" : {
      "field" : "date",
      "next" : {
       "range" : {
         "date" : {
           "lt" : "2023-10-21T00:00:00.000Z",
           "time_zone" : "Z",
           "format" : "strict_date_optional_time",
           "boost" : 0.0
         }
       }
      },
      "source" : "date_trunc(1 day, date)@2:25"
     }
    }, tags=[1697760000000]], QueryBuilderAndTags[query={
    "esql_single_value" : {
     "field" : "date",
     "next" : {
       "range" : {
         "date" : {
           "gte" : "2023-10-21T00:00:00.000Z",
           "lt" : "2023-10-22T00:00:00.000Z",
           "time_zone" : "Z",
           "format" : "strict_date_optional_time",
           "boost" : 0.0
         }
       }
      },
      "source" : "date_trunc(1 day, date)@2:25"
     }
    }, tags=[1697846400000]], QueryBuilderAndTags[query={
    "esql_single_value" : {
     "field" : "date",
     "next" : {
       "range" : {
         "date" : {
           "gte" : "2023-10-22T00:00:00.000Z",
           "time_zone" : "Z",
           "format" : "strict_date_optional_time",
           "boost" : 0.0
         }
       }
      },
      "source" : "date_trunc(1 day, date)@2:25"
     }
    }, tags=[1697932800000]], QueryBuilderAndTags[query={
    "bool" : {
     "must_not" : [
       {
         "exists" : {
           "field" : "date",
           "boost" : 0.0
         }
       }
     ],
     "boost" : 1.0
    }
   }, tags=[null]]]
  2. Aggregation with date_histogram and the other pushdown functions

    When there are other functions that can also be pushed down to Lucene, this rule combines the main query with the RoundTo ranges to create a list of QueryBuilderAndTags. The main query is then applied to each query leg. 

        | WHERE keyword : "keyword"
    | STATS COUNT(*) BY d = DATE_TRUNC(1 day, date)
    becomes 
        | EVAL d = ROUND_TO(hire_date, 1697760000000, 1697846400000, 1697932800000)
    | STATS COUNT(*) BY d
    becomes 
        [QueryBuilderAndTags[query={
    "bool" : {
     "filter" : [
       {
         "match" : {
           "keyword" : {
             "query" : "keyword",
             "lenient" : true
           }
         }
       },
       {
         "esql_single_value" : {
           "field" : "date",
           "next" : {
             "range" : {
               "date" : {
                 "lt" : "2023-10-21T00:00:00.000Z",
                 "time_zone" : "Z",
                 "format" : "strict_date_optional_time",
                 "boost" : 0.0
               }
             }
           },
           "source" : "date_trunc(1 day, date)@3:25"
         }
       }
     ],
     "boost" : 1.0
     }
    }, tags=[1697760000000]], QueryBuilderAndTags[query={
    "bool" : {
     "filter" : [
       {
         "match" : {
           "keyword" : {
             "query" : "keyword",
             "lenient" : true
           }
         }
       },
       {
         "esql_single_value" : {
           "field" : "date",
           "next" : {
             "range" : {
               "date" : {
                 "gte" : "2023-10-21T00:00:00.000Z",
                 "lt" : "2023-10-22T00:00:00.000Z",
                 "time_zone" : "Z",
                 "format" : "strict_date_optional_time",
                 "boost" : 0.0
               }
             }
           },
           "source" : "date_trunc(1 day, date)@3:25"
         }
       }
     ],
     "boost" : 1.0
     }
    }, tags=[1697846400000]], QueryBuilderAndTags[query={
   "bool" : {
     "filter" : [
       {
         "match" : {
           "keyword" : {
             "query" : "keyword",
             "lenient" : true
           }
         }
       },
       {
         "esql_single_value" : {
           "field" : "date",
           "next" : {
             "range" : {
               "date" : {
                 "gte" : "2023-10-22T00:00:00.000Z",
                 "time_zone" : "Z",
                 "format" : "strict_date_optional_time",
                 "boost" : 0.0
               }
             }
           },
           "source" : "date_trunc(1 day, date)@3:25"
         }
       }
     ],
     "boost" : 1.0
     }
    }, tags=[1697932800000]], QueryBuilderAndTags[query={
   "bool" : {
     "filter" : [
       {
         "match" : {
           "keyword" : {
             "query" : "keyword",
             "lenient" : true
           }
         }
       },
       {
         "bool" : {
           "must_not" : [
             {
               "exists" : {
                 "field" : "date",
                 "boost" : 0.0
               }
             }
           ],
           "boost" : 0.0
         }
       }
     ],
     "boost" : 1.0
     }
    }, tags=[null]]]

There are some restrictions:

  1. Tags are not supported by LuceneTopNSourceOperator, if the sort is pushed down to Lucene, this rewrite does not apply.
  2. Tags are not supported by TimeSeriesSourceOperator, this rewrite does not apply to timeseries indices.
  3. Tags are not supported by LuceneCountOperator, this rewrite does not apply to EsStatsQueryExec, count with grouping is not supported by EsStatsQueryExec today.