[
https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-2894:
---------------------------
Attachment: SOLR-2894.patch
I started reviewing again this afternoon and made a few more tweaks but then
quickly encountered a troubling situation:
There seems to be some set of circumstances that can cause pivot refinement to
go into an (infinite?) ridiculously long loop.
Here's an example log snippet from a test run that i eventually had to
explicitly kill after several minutes (normally it finishes in ~40 seconds on
my laptop)..
{noformat}
...
[junit4] 2> 365476 T48 C473 P35623 oasc.SolrCore.execute [collection1]
webapp= path=/select
params={facet.limit=14&facet.pivot={!fpt%3D3557}pivot_y_s,pivot_l1&isShard=true&distrib=false&facet=true&shard.url=https://127.0.0.1:35623/collection1/|https://127.0.0.1:35174/collection1/&version=2&q=*:*&NOW=1403905534861&facet.pivot.mincount=-1&rows=0&fpt3557=-8197981690463795098&fpt3557=-7333481702750443698&fpt3557=-5750361150833026124&fpt3557=-1254664925684537075&fpt3557=-790491513359287891&fpt3557=-259812169693239119&fpt3557=5005&fpt3557=5023&fpt3557=434325197357513755&fpt3557=1208379606676285112&fpt3557=2157244738088160377&fpt3557=4049867752092041147&wt=javabin}
hits=384 status=0 QTime=3
[junit4] 2> 365484 T53 C473 P35623 oasc.SolrCore.execute [collection1]
webapp= path=/select
params={facet.limit=14&facet.pivot={!fpt%3D3558}pivot_y_s,pivot_l1&isShard=true&distrib=false&facet=true&shard.url=https://127.0.0.1:35623/collection1/|https://127.0.0.1:35174/collection1/&version=2&q=*:*&NOW=1403905534861&facet.pivot.mincount=-1&rows=0&fpt3558=-8197981690463795098&fpt3558=-7333481702750443698&fpt3558=-5750361150833026124&fpt3558=-1254664925684537075&fpt3558=-790491513359287891&fpt3558=-259812169693239119&fpt3558=5005&fpt3558=5023&fpt3558=434325197357513755&fpt3558=1208379606676285112&fpt3558=2157244738088160377&fpt3558=4049867752092041147&wt=javabin}
hits=384 status=0 QTime=3
[junit4] 2> 365493 T50 C473 P35623 oasc.SolrCore.execute [collection1]
webapp= path=/select
params={facet.limit=14&facet.pivot={!fpt%3D3559}pivot_y_s,pivot_l1&isShard=true&distrib=false&facet=true&shard.url=https://127.0.0.1:35623/collection1/|https://127.0.0.1:35174/collection1/&version=2&q=*:*&NOW=1403905534861&facet.pivot.mincount=-1&rows=0&fpt3559=-8197981690463795098&fpt3559=-7333481702750443698&fpt3559=-5750361150833026124&fpt3559=-1254664925684537075&fpt3559=-790491513359287891&fpt3559=-259812169693239119&fpt3559=5005&fpt3559=5023&fpt3559=434325197357513755&fpt3559=1208379606676285112&fpt3559=2157244738088160377&fpt3559=4049867752092041147&wt=javabin}
hits=384 status=0 QTime=5
...
{noformat}
A few things to note about those above log lines:
* with the seed used in this run there was only 740 total docs in the index
* all three of those requests were made to the same shard/core (C473) on the
same port (P35623)
* the "pivot_l1" field being refined in these requests is a single valued long
field - which means even if every random value generated for it were unique, in
an index with 740 docs there can only be 740 possible long values here.
* these requests are already upto fpt=3559 -- _way_ more refinements then
should be neccessary for this field
* the shard is being asked to refine the same pivot values over and over again
(but with increasing "fpt#####" keys)
Unfortunately while trying to get to the bottom of this, i realized the way the
test was picking the random pivots it used wasn't reproducible with a
consistent test seed. I've fixed that, but now i need to hammer on this test
some more to try and reproduce again with a reliable seed.
----
Small changes to the patch ...
* TestCloudPivotFacet
** added explicit sort to String[] fieldNames so buildRandomPivot would
reproduce with consistent seed
* SimpleFacets tweaks i made before encountering the test bug:
** more javadocs on some subtly diff methods
** change the new getTermCounts(String,Integer,DocSet) to private since it's
only used as a helper for the other public methods
> Implement distributed pivot faceting
> ------------------------------------
>
> Key: SOLR-2894
> URL: https://issues.apache.org/jira/browse/SOLR-2894
> Project: Solr
> Issue Type: Improvement
> Reporter: Erik Hatcher
> Fix For: 4.9, 5.0
>
> Attachments: SOLR-2894-mincount-minification.patch,
> SOLR-2894-reworked.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
> SOLR-2894.patch, SOLR-2894.patch, SOLR-2894_cloud_test.patch,
> dateToObject.patch, pivot_mincount_problem.sh
>
>
> Following up on SOLR-792, pivot faceting currently only supports
> undistributed mode. Distributed pivot faceting needs to be implemented.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
