[ https://issues.apache.org/jira/browse/SOLR-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13793213#comment-13793213 ]
Yonik Seeley edited comment on SOLR-5330 at 10/12/13 2:30 AM: -------------------------------------------------------------- So I instrumented the faceting code like so: {code} seg.tempBR = seg.tenum.next(); if (seg.tempBR.bytes == val.bytes) { System.err.println("##########SHARING DETECTED: val.offset="+val.offset + " val.length="+val.length + " new.offset="+seg.tempBR.offset + " new.length="+seg.tempBR.length); if (val.offset == seg.tempBR.offset) { System.err.println("!!!!!!SHARING USING SAME OFFSET"); } {code} And it detects tons of sharing (the returned bytesref still pointing to the same byte[]) of course... but the thing is, it never generates an invalid result. calling next() on the term enum never changes the bytes that were previously pointed to... it simply points to a different part of the same byte array. I can never detect a case where the original bytes are changed, thus invalidating the shallow copy. Example output: {code} ##########SHARING DETECTED: val.offset=1 val.length=4 new.offset=6 new.length=4 {code} was (Author: ysee...@gmail.com): So I instrumented the faceting code like so: {code} seg.tempBR = seg.tenum.next(); if (seg.tempBR.bytes == val.bytes) { System.err.println("##########SHARING DETECTED: val.offset="+val.offset + " val.length="+val.length + " new.offset="+seg.tempBR.offset + " new.length="+seg.tempBR.length); if (val.offset == seg.tempBR.offset) { System.err.println("!!!!!!SHARING USING SAME OFFSET"); } {code} And it detects tons of sharing (the returned bytesref still pointing to the same byte[]) of course... but the thing is, it never generates an invalid result. calling next() on the term enum never changes the bytes that were previously pointed to... it simply points to a different part of the same byte array. I can never detect a case where the original bytes are changed, thus invalidating the shallow copy. > PerSegmentSingleValuedFaceting overwrites facet values > ------------------------------------------------------ > > Key: SOLR-5330 > URL: https://issues.apache.org/jira/browse/SOLR-5330 > Project: Solr > Issue Type: Bug > Affects Versions: 4.2.1 > Reporter: Michael Froh > Assignee: Yonik Seeley > Attachments: solr-5330.patch > > > I recently tried enabling facet.method=fcs for one of my indexes and found a > significant performance improvement (with a large index, many facet values, > and near-realtime updates). Unfortunately, the results were also wrong. > Specifically, some facet values were being partially overwritten by other > facet values. (That is, if I expected facet values like "abcdef" and "123", I > would get a value like "123def".) > Debugging through the code, it looks like the problem was in > PerSegmentSingleValuedFaceting, specifically in the getFacetCounts method, > when BytesRef val is shallow-copied from the temporary per-segment BytesRef. > The byte array assigned to val is shared with the byte array for seg.tempBR, > and is overwritten a few lines down by the call to seg.tenum.next(). > I managed to fix it locally by replacing the shallow copy with a deep copy. > While I encountered this problem on Solr 4.2.1, I see that the code is > identical in 4.5. Unless the behavior of TermsEnum.next() has changed, I > believe this bug still exists. -- This message was sent by Atlassian JIRA (v6.1#6144) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org