Jason Gerlowski created SOLR-18201:
--------------------------------------
Summary: "Range Field" types should have docValues support and
associated optimizations
Key: SOLR-18201
URL: https://issues.apache.org/jira/browse/SOLR-18201
Project: Solr
Issue Type: Improvement
Components: query
Reporter: Jason Gerlowski
Assignee: Jason Gerlowski
Solr's recently-added "range field" types were added initially without support
for docValues. At the time this was done primarily for simplicity. But also
because the field types aren't particularly compelling candidates for sorting
or faceting - doing so isn't necessarily tricky to implement, but it was hard
to imagine use cases that would require users to even want to do this. However
- in hindsight there are other benefits of docValues that make them worth
supporting in these field types
Consider the example query: `myStrField:someSelectiveFilter AND
my_range_field:10`.
Without docValues, the `my_range_field` portion of this query will traverse its
entire BKD tree of field-values at creation time in order to build a bitset of
docs matching this clause. This is an expensive operation, and the query
doesn't benefit at all from the low-cost selective query clause that it is
AND'd with. This can make the query quite slow if the Lucene "point" data
structure for this field is large.
DocValues allows this case to be optimized. The range-field clause can be
wrapped in Lucene's "IndexOrDocValuesQuery", which is smart enough to use
docValues if it detects that a lower-cost query is being used to drive
result-set iteration.
We should add docValues support to our "range field" types, and ensure that
they make use of this potential optimization. *Our "range field" types haven't
gone out the door yet, so we can make this change without worrying about
backcompat, so it'd be great if we can get this in prior to 10.1's release. *
Trying to add it post-10.1 would add a good bit of complexity.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
