[
https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13558794#comment-13558794
]
Markus Jelsma commented on SOLR-4260:
-------------------------------------
Here's the debug output of the same query executed on both the leader and the
replica. This set up uses and overridden BM25Similarity that returns docCount()
for IDF instead of maxDoc. In this case both have the equal number of documents
in the index so something else doesn't seem right. Facet counts add up, both
leader and replicaa have the same number of documents per domain.
{code}
43.960983 = (MATCH) sum of:
43.960983 = (MATCH) max plus 0.35 times others of:
29.059849 = (MATCH) weight(title_nl:amsterdam^6.4 in 14437) [], result of:
29.059849 = score(doc=14437,freq=3.0 = termFreq=3.0
), product of:
6.4 = boost
2.889473 = idf(docFreq=18368, docCount=330335)
1.5714288 = tfNorm, computed from:
3.0 = termFreq=3.0
1.2 = parameter k1
0.0 = parameter b (norms omitted for field)
4.651832 = (MATCH) weight(content_nl:amsterdam^1.6 in 14437) [], result of:
4.651832 = score(doc=14437,freq=1.0 = termFreq=1.0
), product of:
1.6 = boost
2.9073951 = idf(docFreq=18039, docCount=330285)
1.0 = tfNorm, computed from:
1.0 = termFreq=1.0
1.2 = parameter k1
0.0 = parameter b (norms omitted for field)
32.161896 = (MATCH) weight(url:amsterdam^3.64 in 14437) [], result of:
32.161896 = score(doc=14437,freq=2.0 = termFreq=2.0
), product of:
3.64 = boost
6.328843 = idf(docFreq=608, docCount=341068)
1.396098 = tfNorm, computed from:
2.0 = termFreq=2.0
1.2 = parameter k1
0.75 = parameter b
4.227131 = avgFieldLength
4.0 = fieldLength
{code}
{code}
45.993042 = (MATCH) sum of:
45.993042 = (MATCH) max plus 0.35 times others of:
28.35725 = (MATCH) weight(title_nl:amsterdam^6.4 in 170479) [], result of:
28.35725 = score(doc=170479,freq=3.0 = termFreq=3.0
), product of:
6.4 = boost
2.8196125 = idf(docFreq=16736, docCount=280676)
1.5714288 = tfNorm, computed from:
3.0 = termFreq=3.0
1.2 = parameter k1
0.0 = parameter b (norms omitted for field)
4.577688 = (MATCH) weight(content_nl:amsterdam^1.6 in 170479) [], result of:
4.577688 = score(doc=170479,freq=1.0 = termFreq=1.0
), product of:
1.6 = boost
2.8610551 = idf(docFreq=16054, docCount=280631)
1.0 = tfNorm, computed from:
1.0 = termFreq=1.0
1.2 = parameter k1
0.0 = parameter b (norms omitted for field)
34.465813 = (MATCH) weight(url:amsterdam^3.64 in 170479) [], result of:
34.465813 = score(doc=170479,freq=2.0 = termFreq=2.0
), product of:
3.64 = boost
6.798851 = idf(docFreq=323, docCount=290119)
1.3926809 = tfNorm, computed from:
2.0 = termFreq=2.0
1.2 = parameter k1
0.75 = parameter b
4.189095 = avgFieldLength
4.0 = fieldLength
{code}
It's clear that not only docCount is different but also docFreq while both
should be equal on the leader and replica. This makes a mess of the final score!
Anyone else here that has seen this issue?
> Inconsistent numDocs between leader/replica
> -------------------------------------------
>
> Key: SOLR-4260
> URL: https://issues.apache.org/jira/browse/SOLR-4260
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
> Affects Versions: 5.0
> Environment: 5.0.0.2013.01.04.15.31.51
> Reporter: Markus Jelsma
> Priority: Critical
> Fix For: 5.0
>
>
> After wiping all cores and reindexing some 3.3 million docs from Nutch using
> CloudSolrServer we see inconsistencies between the leader and replica for
> some shards.
> Each core hold about 3.3k documents. For some reason 5 out of 10 shards have
> a small deviation in then number of documents. The leader and slave deviate
> for roughly 10-20 documents, not more.
> Results hopping ranks in the result set for identical queries got my
> attention, there were small IDF differences for exactly the same record
> causing a record to shift positions in the result set. During those tests no
> records were indexed. Consecutive catch all queries also return different
> number of numDocs.
> We're running a 10 node test cluster with 10 shards and a replication factor
> of two and frequently reindex using a fresh build from trunk. I've not seen
> this issue for quite some time until a few days ago.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
