Re: Too many Soft commits and opening searchers realtime
Cool ! So actually you were not using the default you defined in th
Solrconfig, but it was loaded from a java environment property set to be
"3" ms ?
Cheers
2015-07-09 4:21 GMT+01:00 Summer Shire :
> Yonik, Mikhail, Alessandro
>
> After a lot of digging around and isolation, All u guys were right. I was
> using property based value and there was one place where it was 30 secs and
> that was overriding my main props.
>
> Also Yonik thanks for the explanation on the real time searcher. I wasn't
> sure if the maxwarmingSearcher error I was getting also had something to do
> with it.
>
> Thanks a lot
>
> > On Jul 8, 2015, at 5:28 AM, Yonik Seeley wrote:
> >
> > A realtime searcher is necessary for internal bookkeeping / uses if a
> > normal searcher isn't opened on a commit.
> > This searcher doesn't have caches and hence doesn't carry the weight
> > that a normal searcher would. It's also invisible to clients (it
> > doesn't change the view of the index for normal searches).
> >
> > Your hard autocommit at 8 minutes with openSearcher=false will trigger
> > a realtime searcher to open on every 8 minutes along with the hard
> > commit.
> >
> > -Yonik
> >
> >
> >> On Tue, Jul 7, 2015 at 5:29 PM, Summer Shire
> wrote:
> >> HI All,
> >>
> >> Can someone help me understand the following behavior.
> >> I have the following maxTimes on hard and soft commits
> >>
> >> yet I see a lot of Opening Searchers in the log
> >> org.apache.solr.search.SolrIndexSearcher - Opening Searcher@1656a258[main]
> realtime
> >> also I see a soft commit happening almost every 30 secs
> >> org.apache.solr.update.UpdateHandler - start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> >>
> >> 48
> >> false
> >>
> >>
> >>
> >> 18
> >>
> >> I tried disabling softCommit by setting maxTime to -1.
> >> On startup solrCore recognized it and logged "Soft AutoCommit: disabled"
> >> but I could still see softCommit=true
> >> org.apache.solr.update.UpdateHandler - start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> >>
> >> -1
> >>
> >>
> >> Thanks,
> >> Summer
>
--
--
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Too many Soft commits and opening searchers realtime
Yonik, Mikhail, Alessandro
After a lot of digging around and isolation, All u guys were right. I was using
property based value and there was one place where it was 30 secs and that was
overriding my main props.
Also Yonik thanks for the explanation on the real time searcher. I wasn't sure
if the maxwarmingSearcher error I was getting also had something to do with it.
Thanks a lot
> On Jul 8, 2015, at 5:28 AM, Yonik Seeley wrote:
>
> A realtime searcher is necessary for internal bookkeeping / uses if a
> normal searcher isn't opened on a commit.
> This searcher doesn't have caches and hence doesn't carry the weight
> that a normal searcher would. It's also invisible to clients (it
> doesn't change the view of the index for normal searches).
>
> Your hard autocommit at 8 minutes with openSearcher=false will trigger
> a realtime searcher to open on every 8 minutes along with the hard
> commit.
>
> -Yonik
>
>
>> On Tue, Jul 7, 2015 at 5:29 PM, Summer Shire wrote:
>> HI All,
>>
>> Can someone help me understand the following behavior.
>> I have the following maxTimes on hard and soft commits
>>
>> yet I see a lot of Opening Searchers in the log
>> org.apache.solr.search.SolrIndexSearcher - Opening Searcher@1656a258[main]
>> realtime
>> also I see a soft commit happening almost every 30 secs
>> org.apache.solr.update.UpdateHandler - start
>> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
>>
>> 48
>> false
>>
>>
>>
>> 18
>>
>> I tried disabling softCommit by setting maxTime to -1.
>> On startup solrCore recognized it and logged "Soft AutoCommit: disabled"
>> but I could still see softCommit=true
>> org.apache.solr.update.UpdateHandler - start
>> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
>>
>> -1
>>
>>
>> Thanks,
>> Summer
Re: Too many Soft commits and opening searchers realtime
A realtime searcher is necessary for internal bookkeeping / uses if a
normal searcher isn't opened on a commit.
This searcher doesn't have caches and hence doesn't carry the weight
that a normal searcher would. It's also invisible to clients (it
doesn't change the view of the index for normal searches).
Your hard autocommit at 8 minutes with openSearcher=false will trigger
a realtime searcher to open on every 8 minutes along with the hard
commit.
-Yonik
On Tue, Jul 7, 2015 at 5:29 PM, Summer Shire wrote:
> HI All,
>
> Can someone help me understand the following behavior.
> I have the following maxTimes on hard and soft commits
>
> yet I see a lot of Opening Searchers in the log
> org.apache.solr.search.SolrIndexSearcher- Opening Searcher@1656a258[main]
> realtime
> also I see a soft commit happening almost every 30 secs
> org.apache.solr.update.UpdateHandler - start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
>
> 48
> false
>
>
>
> 18
>
> I tried disabling softCommit by setting maxTime to -1.
> On startup solrCore recognized it and logged "Soft AutoCommit: disabled"
> but I could still see softCommit=true
> org.apache.solr.update.UpdateHandler - start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
>
> -1
>
>
> Thanks,
> Summer
Re: Too many Soft commits and opening searchers realtime
Summer,
A log excerpt usually helps to troubleshoot any magic. Would you mind to
provide one?
On Wed, Jul 8, 2015 at 2:30 PM, Alessandro Benedetti <
benedetti.ale...@gmail.com> wrote:
> So you are saying that no-one is triggering any commit, and that the auto
> soft commit solution is not actually waiting the proper time ?
> I suspect something is not like described, because if the Auto Soft commit
> was not working I would expect thousands of bugs raised.
>
> let's dig a little bit into details…
> What are you using exactly to index content ?
> Maybe some commit is actually hidden there :)
>
> Cheers
>
> 2015-07-08 2:21 GMT+01:00 Summer Shire :
>
> > No the client lets solr handle it.
> >
> >
> > > On Jul 7, 2015, at 2:38 PM, Mike Drob wrote:
> > >
> > > Are the clients that are posting updates requesting commits?
> > >
> > >> On Tue, Jul 7, 2015 at 4:29 PM, Summer Shire
> > wrote:
> > >>
> > >> HI All,
> > >>
> > >> Can someone help me understand the following behavior.
> > >> I have the following maxTimes on hard and soft commits
> > >>
> > >> yet I see a lot of Opening Searchers in the log
> > >> org.apache.solr.search.SolrIndexSearcher - Opening Searcher@1656a258
> > [main]
> > >> realtime
> > >> also I see a soft commit happening almost every 30 secs
> > >> org.apache.solr.update.UpdateHandler - start
> > >>
> >
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> > >>
> > >> 48
> > >> false
> > >>
> > >>
> > >>
> > >> 18
> > >>
> > >> I tried disabling softCommit by setting maxTime to -1.
> > >> On startup solrCore recognized it and logged "Soft AutoCommit:
> disabled"
> > >> but I could still see softCommit=true
> > >> org.apache.solr.update.UpdateHandler - start
> > >>
> >
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> > >>
> > >> -1
> > >>
> > >>
> > >> Thanks,
> > >> Summer
> >
>
>
>
> --
> --
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
Re: Too many Soft commits and opening searchers realtime
So you are saying that no-one is triggering any commit, and that the auto
soft commit solution is not actually waiting the proper time ?
I suspect something is not like described, because if the Auto Soft commit
was not working I would expect thousands of bugs raised.
let's dig a little bit into details…
What are you using exactly to index content ?
Maybe some commit is actually hidden there :)
Cheers
2015-07-08 2:21 GMT+01:00 Summer Shire :
> No the client lets solr handle it.
>
>
> > On Jul 7, 2015, at 2:38 PM, Mike Drob wrote:
> >
> > Are the clients that are posting updates requesting commits?
> >
> >> On Tue, Jul 7, 2015 at 4:29 PM, Summer Shire
> wrote:
> >>
> >> HI All,
> >>
> >> Can someone help me understand the following behavior.
> >> I have the following maxTimes on hard and soft commits
> >>
> >> yet I see a lot of Opening Searchers in the log
> >> org.apache.solr.search.SolrIndexSearcher - Opening Searcher@1656a258
> [main]
> >> realtime
> >> also I see a soft commit happening almost every 30 secs
> >> org.apache.solr.update.UpdateHandler - start
> >>
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> >>
> >> 48
> >> false
> >>
> >>
> >>
> >> 18
> >>
> >> I tried disabling softCommit by setting maxTime to -1.
> >> On startup solrCore recognized it and logged "Soft AutoCommit: disabled"
> >> but I could still see softCommit=true
> >> org.apache.solr.update.UpdateHandler - start
> >>
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> >>
> >> -1
> >>
> >>
> >> Thanks,
> >> Summer
>
--
--
Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti
"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"
William Blake - Songs of Experience -1794 England
Re: Too many Soft commits and opening searchers realtime
No the client lets solr handle it.
> On Jul 7, 2015, at 2:38 PM, Mike Drob wrote:
>
> Are the clients that are posting updates requesting commits?
>
>> On Tue, Jul 7, 2015 at 4:29 PM, Summer Shire wrote:
>>
>> HI All,
>>
>> Can someone help me understand the following behavior.
>> I have the following maxTimes on hard and soft commits
>>
>> yet I see a lot of Opening Searchers in the log
>> org.apache.solr.search.SolrIndexSearcher - Opening Searcher@1656a258[main]
>> realtime
>> also I see a soft commit happening almost every 30 secs
>> org.apache.solr.update.UpdateHandler - start
>> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
>>
>> 48
>> false
>>
>>
>>
>> 18
>>
>> I tried disabling softCommit by setting maxTime to -1.
>> On startup solrCore recognized it and logged "Soft AutoCommit: disabled"
>> but I could still see softCommit=true
>> org.apache.solr.update.UpdateHandler - start
>> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
>>
>> -1
>>
>>
>> Thanks,
>> Summer
Re: Too many Soft commits and opening searchers realtime
Are the clients that are posting updates requesting commits?
On Tue, Jul 7, 2015 at 4:29 PM, Summer Shire wrote:
> HI All,
>
> Can someone help me understand the following behavior.
> I have the following maxTimes on hard and soft commits
>
> yet I see a lot of Opening Searchers in the log
> org.apache.solr.search.SolrIndexSearcher- Opening Searcher@1656a258[main]
> realtime
> also I see a soft commit happening almost every 30 secs
> org.apache.solr.update.UpdateHandler - start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
>
> 48
> false
>
>
>
> 18
>
> I tried disabling softCommit by setting maxTime to -1.
> On startup solrCore recognized it and logged "Soft AutoCommit: disabled"
> but I could still see softCommit=true
> org.apache.solr.update.UpdateHandler - start
> commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
>
> -1
>
>
> Thanks,
> Summer
Too many Soft commits and opening searchers realtime
HI All,
Can someone help me understand the following behavior.
I have the following maxTimes on hard and soft commits
yet I see a lot of Opening Searchers in the log
org.apache.solr.search.SolrIndexSearcher- Opening Searcher@1656a258[main]
realtime
also I see a soft commit happening almost every 30 secs
org.apache.solr.update.UpdateHandler - start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
48
false
18
I tried disabling softCommit by setting maxTime to -1.
On startup solrCore recognized it and logged "Soft AutoCommit: disabled"
but I could still see softCommit=true
org.apache.solr.update.UpdateHandler - start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
-1
Thanks,
Summer
