Re: [Solr4 beta] error 503 on commit

[Radim Kolar]

 After investigating more, here is the tomcat log herebelow. It is 
indeed the same problem: exceeded limit of maxWarmingSearchers=2,.


could not be solr able to close oldest warming searcher and replace it 
by new one?

Re: [Solr4 beta] error 503 on commit

[Yonik Seeley]

On Tue, Sep 11, 2012 at 10:52 AM, Radim Kolar h...@filez.com wrote:
 After investigating more, here is the tomcat log herebelow. It is indeed
 the same problem: exceeded limit of maxWarmingSearchers=2,.

 could not be solr able to close oldest warming searcher and replace it by
 new one?

That approach can easily lead to starvation (i.e. you never get a new
searcher usable for queries).

-Yonik
http://lucidworks.com

Re: [Solr4 beta] error 503 on commit

[Radim Kolar]

could not be solr able to close oldest warming searcher and replace it by
new one?

That approach can easily lead to starvation (i.e. you never get a new
searcher usable for queries).

It will not. If there is more then 1 warming searcher. Look at this schema:

1. current in use searcher
2. 1st warming searcher
3. 2nd warming searcher

if new warming searcher is needed, close (3) and create a new one (3).
(2) will finish work uninterrupted and it will replace (1)

Re: [Solr4 beta] error 503 on commit

[Antoine LE FLOC'H]

Hoss,

After investigating more, here is the tomcat log herebelow. It is indeed
the same problem: exceeded limit of maxWarmingSearchers=2,.
It is an indexing box and the comment says that we could rise this number
to 4 or something. I can do that but I have four questions though:
  - is it something that can happen anyway ?
  - what are the actions in case it does (since re-committing the same docs
with a try-again mechanism is not re-committing) ?
  - I was in the impression that no new searchers are created if I don't do
search. And I was not searching at that time. Where is this searcher coming
from ?
  - is there a way to disable searchers during indexing since I precisely
don't want warming during indexing ?
Thanks a lot.

11 sept. 2012 15:25:08 org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
11 sept. 2012 15:25:08 org.apache.solr.core.SolrCore registerSearcher
INFO: [lg_fr_alpha6_full] Registered new searcher
Searcher@61f5e795main{StandardDirectoryReader(segments_qo:13989:nrt
_13d(4.0.0.1):C4636752/2696195 _26s(4.0.0.1):C4422269
_329(4.0.0.1):C2409200 _3me(4.0.0.1):C4534687/1 _4df(4.0.0.1):C3745599/731
_4np(4.0.0.1):C3660400/5317 _4qr(4.0.0.1):C2422935/138
_4w4(4.0.0.1):C261418 _4qw(4.0.0.1):C293154 _56y(4.0.0.1):C833572
_4vc(4.0.0.1):C138593 _53b(4.0.0.1):C410764 _4yg(4.0.0.1):C168744
_51j(4.0.0.1):C134313 _56s(4.0.0.1):C151121 _55m(4.0.0.1):C40342
_59i(4.0.0.1):C167117 _58i(4.0.0.1):C82564 _57r(4.0.0.1):C79488
_57k(4.0.0.1):C2589 _5a7(4.0.0.1):C64667 _59x(4.0.0.1):C14142
_58v(4.0.0.1):C1618 _58x(4.0.0.1):C2219 _595(4.0.0.1):C15193
_590(4.0.0.1):C17855 _598(4.0.0.1):C3528 _59a(4.0.0.1):C5998
_59l(4.0.0.1):C10234 _59k(4.0.0.1):C6524 _59q(4.0.0.1):C3115
_5al(4.0.0.1):C3502 _5a4(4.0.0.1):C1602 _5a6(4.0.0.1):C1797
_5a5(4.0.0.1):C1530 _5ad(4.0.0.1):C7351 _5ac(4.0.0.1):C5797
_5ab(4.0.0.1):C8330 _5aa(4.0.0.1):C6436 _5a9(4.0.0.1):C5944
_5ag(4.0.0.1):C36424 _5aj(4.0.0.1):C7 _5ai(4.0.0.1):C26770
_5ak(4.0.0.1):C29729 _5af(4.0.0.1):C35554 _5ah(4.0.0.1):C23804)}
11 sept. 2012 15:25:08 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
11 sept. 2012 15:25:08 org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: [lg_fr_alpha6_full] webapp=/solr path=/update
params={waitSearcher=falsecommit=truewt=xmlsoftCommit=falseversion=2.2}
{commit=} 0 107492
11 sept. 2012 15:25:08 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=false,expungeDeletes=false,softCommit=false}
11 sept. 2012 15:25:08 org.apache.solr.search.SolrIndexSearcher init
INFO: Opening Searcher@62677672 main
11 sept. 2012 15:25:08 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
11 sept. 2012 15:25:08 org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: [lg_fr_alpha6_full] webapp=/solr path=/update
params={waitSearcher=falsecommit=truewt=xmlsoftCommit=falseversion=2.2}
{commit=} 0 93563
11 sept. 2012 15:25:08 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=false,expungeDeletes=false,softCommit=false}
11 sept. 2012 15:25:08 org.apache.solr.core.SolrCore getSearcher
ATTENTION: [lg_fr_alpha6_full] PERFORMANCE WARNING: Overlapping
onDeckSearchers=2
11 sept. 2012 15:25:08 org.apache.solr.search.SolrIndexSearcher init
INFO: Opening Searcher@4479f66a main
11 sept. 2012 15:25:08 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
11 sept. 2012 15:25:08 org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: [lg_fr_alpha6_full] webapp=/solr path=/update
params={waitSearcher=falsecommit=truewt=xmlsoftCommit=falseversion=2.2}
{commit=} 0 93178
11 sept. 2012 15:25:08 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=false,expungeDeletes=false,softCommit=false}
11 sept. 2012 15:25:08 org.apache.solr.core.SolrCore getSearcher
ATTENTION: [lg_fr_alpha6_full] Error opening new searcher. exceeded limit
of maxWarmingSearchers=2, try again later.
11 sept. 2012 15:25:08 org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: [lg_fr_alpha6_full] webapp=/solr path=/update
params={waitSearcher=falsecommit=truewt=xmlsoftCommit=falseversion=2.2}
{} 0 93137
11 sept. 2012 15:25:08 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit{flags=0,_version_=0,optimize=false,openSearcher=true,waitSearcher=false,expungeDeletes=false,softCommit=false}
11 sept. 2012 15:25:08 org.apache.solr.core.SolrCore getSearcher
ATTENTION: [lg_fr_alpha6_full] Error opening new searcher. exceeded limit
of maxWarmingSearchers=2, try again later.
11 sept. 2012 15:25:08 org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: [lg_fr_alpha6_full] webapp=/solr path=/update
params={waitSearcher=falsecommit=truewt=xmlsoftCommit=falseversion=2.2}
{} 0 90799
11 sept. 2012 15:25:08 

[Solr4 beta] error 503 on commit

[Antoine LE FLOC'H]

Hello,

Using package org.apache.solr.client.solrj;

when I do:

UpdateResponse ur = solrServer.commit(false, false);

I get sometimes (not often):
SolrException e  where   e.code() ==
SolrException.ErrorCode.SERVICE_UNAVAILABLE.code

When I catch this exception, I try to commit again, the call doesn't throw,
but the docs are not committed. Am I supposed to add docs again before
committing
solrServer.add(docs);
What am I supposed to do basically ?

Please note that I was not getting this on the Solr4 trunk from december,
but I am getting it sometimes (not systematic) since the Solr4 alpha.

Thanks for your help.

Re: [Solr4 beta] error 503 on commit

[Chris Hostetter]

: I get sometimes (not often):
: SolrException e  where   e.code() ==
: SolrException.ErrorCode.SERVICE_UNAVAILABLE.code

Are there any errors in your solr server logs?
Are you using the DistributedUpdateProcessor (ie: SolrCloud) ?

There aren't many places in Solr that will throw a 503 status code, if i 
had to guess i would suspect that your problem is you are committing too 
often relative to the amount of warming you are doing, and exceeding the 
max number of open searchers...

https://wiki.apache.org/solr/FAQ#What_does_.22exceeded_limit_of_maxWarmingSearchers.3DX.22_mean.3F

: When I catch this exception, I try to commit again, the call doesn't throw,
: but the docs are not committed. Am I supposed to add docs again before

Hmmm... that is odd, i think we'd definitely need to see the logs from one 
of these errors (and the lines leading up to it) to understand what might 
be happening.

-Hoss

RE: [Solr4 beta] error 503 on commit

[Markus Jelsma]

Hi,

We've seen this too on one of the test nodes yesterday, it ran on a build of a 
few days old. The node receiving documents complained it could not forward them 
to the fifth node and returned a 503. The fifth node itself only logged a NPE 
and the 503, nothing more, no stack traces.

There was no heavy committing going on (we only use auto commit of a few 
seconds) and it were only about 500 documents. The first two batches were 
accepted but then it died. It failed very consistently. Perhaps it accepted the 
first batches because no document was sent to the shard leader on that node yet.

Restarting the node fixed the trouble.


-Original message-
 From:Chris Hostetter hossman_luc...@fucit.org
 Sent: Fri 07-Sep-2012 23:02
 To: solr-user@lucene.apache.org
 Subject: Re: [Solr4 beta] error 503 on commit
 
 
 : I get sometimes (not often):
 : SolrException e  where   e.code() ==
 : SolrException.ErrorCode.SERVICE_UNAVAILABLE.code
 
 Are there any errors in your solr server logs?
 Are you using the DistributedUpdateProcessor (ie: SolrCloud) ?
 
 There aren't many places in Solr that will throw a 503 status code, if i 
 had to guess i would suspect that your problem is you are committing too 
 often relative to the amount of warming you are doing, and exceeding the 
 max number of open searchers...
 
 https://wiki.apache.org/solr/FAQ#What_does_.22exceeded_limit_of_maxWarmingSearchers.3DX.22_mean.3F
 
 : When I catch this exception, I try to commit again, the call doesn't throw,
 : but the docs are not committed. Am I supposed to add docs again before
 
 Hmmm... that is odd, i think we'd definitely need to see the logs from one 
 of these errors (and the lines leading up to it) to understand what might 
 be happening.
 
 -Hoss