[
https://issues.apache.org/jira/browse/SOLR-7631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-7631:
---------------------------
Description:
Working through SOLR-7605, I've confirmed that the underlying problem exists
for regular {{field.facet}} situations, regardless of distrib mode, for Trie
fields that have a non-zero precisionStep. *this has only been reproduced when
the RandomCodec was in use*
The problem, when it manifests, is that faceting on a TrieIntField, using
{{facet.mincount=0}}, causes the facet results to include three instances of
facet the value "0" listed with a count of "0" -- even though no document in
the index contains this value at all...
{noformat}
[junit4] > <lst name="facet_fields">
[junit4] > <lst name="foo_ti">
[junit4] > <int name="20">32</int>
...
[junit4] > <int name="50">21</int>
[junit4] > <int name="0">0</int>
[junit4] > <int name="0">0</int>
[junit4] > <int name="0">0</int>
{noformat}
This is concerning for a few reasons:
* In the case of PivotFaceting, getting duplicate values back from a single
shard like this triggers an assert in distributed queries and the request fails
-- even if asserts aren't enabled, the bogus "0" value can be propogated to
clients if they ask for facet.pivot.mincount=0
* Client code expecting a single (value,count) pair for each value may equally
be confused/broken by this response where the same "value" is returned multiple
times
* w/o knowing the root cause, It seems very possible that other nonsense values
may be getting returned -- ie: if the error only happens with fields utilizing
precisionStep, then it's likely related to the synthetic values used for faster
range queries, and other synthetic values may be getting included with bogus
counts
A Patch with a simple test that can demonstrate the bug fairly easily will be
attached shortly
was:
Working through SOLR-7605, I've confirmed that the underlying problem exists
for regular {{field.facet}} situations, regardless of distrib mode, for Trie
fields that have a non-zero precisionStep -- there's still ome other missing
piece of the puzzle i haven't figured out, but it relates in some way to some
of randomized factors we use in our tests (Codec? PostingFormat? ... no idea)
The problem, when it manifests, is that faceting on a TrieIntField, using
{{facet.mincount=0}}, causes the facet results to include three instances of
facet the value "0" listed with a count of "0" -- even though no document in
the index contains this value at all...
{noformat}
[junit4] > <lst name="facet_fields">
[junit4] > <lst name="foo_ti">
[junit4] > <int name="20">32</int>
...
[junit4] > <int name="50">21</int>
[junit4] > <int name="0">0</int>
[junit4] > <int name="0">0</int>
[junit4] > <int name="0">0</int>
{noformat}
This is concerning for a few reasons:
* In the case of PivotFaceting, getting duplicate values back from a single
shard like this triggers an assert in distributed queries and the request fails
-- even if asserts aren't enabled, the bogus "0" value can be propogated to
clients if they ask for facet.pivot.mincount=0
* Client code expecting a single (value,count) pair for each value may equally
be confused/broken by this response where the same "value" is returned multiple
times
* w/o knowing the root cause, It seems very possible that other nonsense values
may be getting returned -- ie: if the error only happens with fields utilizing
precisionStep, then it's likely related to the synthetic values used for faster
range queries, and other synthetic values may be getting included with bogus
counts
A Patch with a simple test that can demonstrate the bug fairly easily will be
attached shortly
Summary: RandomCodec can cause Faceting on multivalued Trie fields with
precisionStep != 0 can produce bogus value="0" in some test seeds (was:
Faceting on multivalued Trie fields with precisionStep != 0 can produce bogus
value="0" in some situations)
re-reading my long comment from last night, i realized i kind of buried the
lead, which is: I was not able to reproduce this bug using any explicitly
specified -Dtests.codec other then "random"
> RandomCodec can cause Faceting on multivalued Trie fields with precisionStep
> != 0 can produce bogus value="0" in some test seeds
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-7631
> URL: https://issues.apache.org/jira/browse/SOLR-7631
> Project: Solr
> Issue Type: Bug
> Reporter: Hoss Man
> Attachments: SOLR-7631_test.patch, SOLR-7631_test.patch, log.tgz
>
>
> Working through SOLR-7605, I've confirmed that the underlying problem exists
> for regular {{field.facet}} situations, regardless of distrib mode, for Trie
> fields that have a non-zero precisionStep. *this has only been reproduced
> when the RandomCodec was in use*
> The problem, when it manifests, is that faceting on a TrieIntField, using
> {{facet.mincount=0}}, causes the facet results to include three instances of
> facet the value "0" listed with a count of "0" -- even though no document in
> the index contains this value at all...
> {noformat}
> [junit4] > <lst name="facet_fields">
> [junit4] > <lst name="foo_ti">
> [junit4] > <int name="20">32</int>
> ...
> [junit4] > <int name="50">21</int>
> [junit4] > <int name="0">0</int>
> [junit4] > <int name="0">0</int>
> [junit4] > <int name="0">0</int>
> {noformat}
> This is concerning for a few reasons:
> * In the case of PivotFaceting, getting duplicate values back from a single
> shard like this triggers an assert in distributed queries and the request
> fails -- even if asserts aren't enabled, the bogus "0" value can be
> propogated to clients if they ask for facet.pivot.mincount=0
> * Client code expecting a single (value,count) pair for each value may
> equally be confused/broken by this response where the same "value" is
> returned multiple times
> * w/o knowing the root cause, It seems very possible that other nonsense
> values may be getting returned -- ie: if the error only happens with fields
> utilizing precisionStep, then it's likely related to the synthetic values
> used for faster range queries, and other synthetic values may be getting
> included with bogus counts
> A Patch with a simple test that can demonstrate the bug fairly easily will be
> attached shortly
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
