Ishan Chattopadhyaya created SOLR-8687: ------------------------------------------
Summary: Race condition with RTGs during soft commit Key: SOLR-8687 URL: https://issues.apache.org/jira/browse/SOLR-8687 Project: Solr Issue Type: Bug Reporter: Ishan Chattopadhyaya I am facing a problem with stress testing SOLR-5944, even though I think this problem persists in Solr even without my changes. The symptom is that during a stress test (similar to TestStressReorder), RTG gets a document which is older version than that of the last acknowledged write. Possible reason: {code} (DUH2's commit()) ... 1: if (cmd.softCommit) { 2: // ulog.preSoftCommit(); 3: synchronized (solrCoreState.getUpdateLock()) { 4: if (ulog != null) ulog.preSoftCommit(cmd); 5: core.getSearcher(true, false, waitSearcher, true); 6: if (ulog != null) ulog.postSoftCommit(cmd); 7: } 8: callPostSoftCommitCallbacks(); 9: } ... {code} * Before line 1, there was an update (say id=2) which was in ulog's map. Maps are, say, `map={2=LogPtr(1234)}, prevMap={...}, prevMap2={...}` * Due to line 4 (ulog.preSoftCommit()), the maps were rotated. Now, the id=2 is in prevMap: `map={}, prevMap={2=LogPtr(1234)}, prevMap2={...}`. Till now RTG for id=2 will work. * Due to line 5, a new searcher is due to be opened. But this is asynchronous, and lets assume this doesn't complete before few more lines are executed. * Due to line 6 (ulog.postSoftCommit()), the previous maps are cleared out. Now the maps are: `map={}, prevMap=null, prevMap2=null` * If there's an RTG for id=2, it will not work from the ulog's maps, so it will fall through to be searched using the last searcher. But, the searcher due to be opened in line 5 hasn't yet been opened. In this case, the returned document will be whatever version of id=2 that was present in the previous searcher. Can someone please confirm if this is a potential problem? If so, any suggestions for a fix, please? I tried opening a ulog.openRealtimeSearcher() in the above synchronized block, but the problem still persists, but I haven't looked into why that could be. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org