Yes, you’ll get stale values. There’s no way I know of to change that, it’s a
fundamental result of Lucene’s design.
There’s a “segment_n” file that contains pointers to the current valid
segments. When a commit happens, segments are closed and the very last
operation is to update that file.
In the abnormal termination case, that file has not been updated with docs that
are in the tlog. So when Solr opens a searcher, it has no record at all of any
new segments created since the last hard commit. So there are only two choices:
1> refuse to seve any searches at all
2> allow searches on the last snapshot of the index while the tlog replays
The latter is the choice we’ve made and I agree with it. While theoretically
you could refuse to open a searcher while the tlog was replaying, I’d rather
get some results than none at all.
Especially when this only happens when Solr is abnormally terminated.
You can mitigate the time frame here by setting your hard commit interval to,
say, 15 seconds, which should be the upper bound of getting stale docs when the
tlog is replayed.
It’s also good practice to have the autocommit interval relatively short for a
variety of reasons, not the least of which is that it’ll grow infinitely until
a hard commit happens.
Best,
Erick
> On Mar 8, 2019, at 8:48 AM, Rahul Goswami <rahul196...@gmail.com> wrote:
>
> What I am observing is that Solr is fully started up even before it has
> finished playing the tlog. In the logs I see that a searcher is registered
> first and the "Log replay finished" appears later. During that time if I
> search, I do get stale values. Below are the log lines that I captured :
>
> WARN - 2019-03-08 16:33:42.126; [ x:techproducts]
> org.apache.solr.update.UpdateLog$LogReplayer; Starting log replay
> tlog{file=C:\Work\Solr\solr-7.2.1\Installation\solr-7.2.1\example\techproducts\solr\techproducts\data\tlog\tlog.0000000000000000009
> refcount=2} active=false starting pos=0 inSortedOrder=false
> INFO - 2019-03-08 16:33:42.141; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] webapp=null path=null
> params={q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&event=firstSearcher}
> hits=3 status=0 QTime=37
> INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
> org.apache.solr.core.QuerySenderListener; QuerySenderListener done.
> INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
> org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener;
> Loading spell index for spellchecker: default
> INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
> org.apache.solr.handler.component.SpellCheckComponent$SpellCheckerListener;
> Loading spell index for spellchecker: wordbreak
> INFO - 2019-03-08 16:33:42.157; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] Registered new searcher
> Searcher@63c5631[techproducts]
> main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.1):C32/1:delGen=1)
> Uninverting(_5(7.2.1):C1)))}
> INFO - 2019-03-08 16:34:07.373; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
> params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=2
> INFO - 2019-03-08 16:34:08.818; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
> params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=1
> INFO - 2019-03-08 16:34:14.422; [ x:techproducts]
> org.apache.solr.update.DirectUpdateHandler2; start
> commit{flags=2,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> INFO - 2019-03-08 16:34:16.353; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
> params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=1
> INFO - 2019-03-08 16:34:18.948; [ x:techproducts]
> org.apache.solr.update.SolrIndexWriter; Calling setCommitData with
> IW:org.apache.solr.update.SolrIndexWriter@266e1192 commitCommandVersion:0
> INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> org.apache.solr.search.SolrIndexSearcher; Opening
> [Searcher@5c6044f1[techproducts]
> main]
> INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
> INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> org.apache.solr.core.QuerySenderListener; QuerySenderListener sending
> requests to Searcher@5c6044f1[techproducts]
> main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.1):C32/1:delGen=1)
> Uninverting(_6(7.2.1):C1)))}
> INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> org.apache.solr.core.QuerySenderListener; QuerySenderListener done.
> INFO - 2019-03-08 16:34:19.040; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] Registered new searcher
> Searcher@5c6044f1[techproducts]
> main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.1):C32/1:delGen=1)
> Uninverting(_6(7.2.1):C1)))}
> INFO - 2019-03-08 16:34:19.056; [ x:techproducts]
> org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor;
> [techproducts] {add=[SP2514N (1627455755076501504)]} 0 36923
> WARN - 2019-03-08 16:34:19.056; [ x:techproducts]
> org.apache.solr.update.UpdateLog$LogReplayer; Log replay finished.
> recoveryInfo=RecoveryInfo{adds=1 deletes=0 deleteByQuery=0 errors=0
> positionOfStart=0}
> INFO - 2019-03-08 16:34:23.523; [ x:techproducts]
> org.apache.solr.core.SolrCore; [techproducts] webapp=/solr path=/select
> params={q=id:SP2514N&wt=json&_=1552062352063} hits=1 status=0 QTime=1
>
> On Thu, Mar 7, 2019 at 11:36 PM Zheng Lin Edwin Yeo <edwinye...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Do you mean that when you startup Solr, it will automatically do the search
>> request even before the Solr is fully started up?
>>
>> Regards,
>> Edwin
>>
>>
>> On Fri, 8 Mar 2019 at 10:13, Rahul Goswami <rahul196...@gmail.com> wrote:
>>
>>> Hello Solr gurus,
>>>
>>> I am using Solr 7.2.1 (non-SolrCloud). I have a situation where Solr got
>>> killed before it could commit updates to the disk resulting in log replay
>>> on startup. During this interval, I observe that a searcher is opened
>> even
>>> before log replay has finished, resulting in some stale results, which in
>>> turn has a cascading effect on other parts of the application. Is there a
>>> setting in Solr which would prevent Solr from serving search requests
>>> before log replay has finished?
>>>
>>> Thanks,
>>> Rahul
>>>
>>
