[
https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-2894:
---------------------------
Attachment: SOLR-2894.patch
I let my laptop hammer away on TestCloudPivotFacet while i was looking at some
other stuff, and got a new reproducible failure...
{noformat}
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestCloudPivotFacet
-Dtests.method=testDistribSearch -Dtests.seed=EE02505B2F4046AC
-Dtests.nightly=true -Dtests.slow=true -Dtests.locale=fi
-Dtests.timezone=Asia/Aqtobe -Dtests.file.encoding=UTF-8
[junit4] FAILURE 56.9s | TestCloudPivotFacet.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError:
{main(facet=true&facet.pivot=pivot_y_s%2Cpivot_b&facet.pivot=pivot_tdt1&facet.limit=4&facet.offset=5&facet.pivot.mincount=17&facet.missing=false&facet.sort=index),extra(rows=0&q=id%3A%5B*+TO+786%5D&_test_min=17&_test_miss=false&_test_sort=index)}
==> pivot_y_s,pivot_b:
{params(rows=0),defaults({main(rows=0&q=id%3A%5B*+TO+786%5D&_test_min=17&_test_miss=false&_test_sort=index),extra(fq=%7B%21term+f%3Dpivot_y_s%7Dg)})}
expected:<22> but was:<50>
[junit4] > at
__randomizedtesting.SeedInfo.seed([EE02505B2F4046AC:6FE4DE43581F2690]:0)
[junit4] > at
org.apache.solr.cloud.TestCloudPivotFacet.assertPivotCountsAreCorrect(TestCloudPivotFacet.java:239)
[junit4] > at
org.apache.solr.cloud.TestCloudPivotFacet.doTest(TestCloudPivotFacet.java:187)
[junit4] > at
org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
[junit4] > at java.lang.Thread.run(Thread.java:744)
[junit4] > Caused by: java.lang.AssertionError: pivot_y_s,pivot_b:
{params(rows=0),defaults({main(rows=0&q=id%3A%5B*+TO+786%5D&_test_min=17&_test_miss=false&_test_sort=index),extra(fq=%7B%21term+f%3Dpivot_y_s%7Dg)})}
expected:<22> but was:<50>
[junit4] > at
org.apache.solr.cloud.TestCloudPivotFacet.assertNumFound(TestCloudPivotFacet.java:507)
[junit4] > at
org.apache.solr.cloud.TestCloudPivotFacet.assertPivotCountsAreCorrect(TestCloudPivotFacet.java:257)
[junit4] > at
org.apache.solr.cloud.TestCloudPivotFacet.assertPivotCountsAreCorrect(TestCloudPivotFacet.java:229)
[junit4] > ... 42 more
{noformat}
At first i thought this was simply an issue in how "needRefinementAtThisLevel"
assumed we never need refinement for sort=index -- that's too general of an
assertion, we can only asume no refinement is needed if mincount=0. But fixing
that still didn't solve the problem.
Thinking about the PivotFacetField.queuePivotRefinementRequests logic however
made me realize that all of the logic in that method (and it's use of
"countThreshold") really only works with sort=count ... for sort=index we
shouldn't make any assumptions about the cutoff based on the count.
Before digging into a fix, I started working on more sort=index tests to try
and better excercise this code, and quickly encountered a new (unrelated?)
failure that seems to related to mincount==0 on sub pivots...
I distilled the new mincount failure out into a new isolated test query (that
doesn't use sort=index) in DistributedFacetPivotLargeTest:
{code}
rsp = query( "q", "*:*",
"rows", "0",
"facet","true",
"facet.pivot","place_s,company_t",
FacetParams.FACET_LIMIT, "50",
FacetParams.FACET_PIVOT_MINCOUNT,"0");
{code}
...which leads to...
{noformat}
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=DistributedFacetPivotLargeTest -Dtests.method=testDistribSearch
-Dtests.seed=63DFE6A839DD2C9F -Dtests.slow=true -Dtests.locale=es_NI
-Dtests.timezone=Asia/Bishkek -Dtests.file.encoding=UTF-8
[junit4] FAILURE 43.5s | DistributedFacetPivotLargeTest.testDistribSearch <<<
[junit4] > Throwable #1: junit.framework.AssertionFailedError:
.facet_counts.facet_pivot.place_s,company_t[1].pivot.length:3!=50
[junit4] > at
__randomizedtesting.SeedInfo.seed([63DFE6A839DD2C9F:E23968B04E824CA3]:0)
{noformat}
...i haven't dug into what exactly is going on here, i've been focusng on more
tests for the sort=index refinement bug first (since it's easy to reproduce
even w/o sub-pivots)
----
In addition to the above mentioned addition to DistributedFacetPivotLargeTest,
this new patch also adds some new queries/assertions to
DistributedFacetPivotSmallTest that seem to demo the problem with
facet.sort=index as the randomized failure (at least ... i *think* it's the
same problem).
i'm going to work on fixing queuePivotRefinementRequests to account for
sort=index tomorow.
----
Andrew, Brett: I don't suppose the mincount=0 bug jumps out at you guys as
something with an obvious fix?
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Assignee: Hoss Man
> Fix For: 4.9, 5.0
>
> Attachments: SOLR-2894-mincount-minification.patch,
> SOLR-2894-reworked.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894_cloud_test.patch, dateToObject.patch,
> pivot_mincount_problem.sh
>
>
> Following up on SOLR-792, pivot faceting currently only supports
> undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
