OK, but the consider what you can do to keep from having to create such things. 1> for infix notation (leading and trailing wildcards) you can use ngrams to turn them into simple queries. These are performance-killers. 2> Use reverseWildcardFactory to deal with leading wildcards 3> restrict the wildcard search to at least two "real" characters. I can (and have) argued that a* is not useful to a user at all.
The point is that your responsiveness will suffer if you come anywhere near 100K OR clauses. Maybe that's OK, but it's lurking out there. And no I can't see the image, the Apache server is pretty aggressive about stripping those too. Best, Erick On Tue, Jun 28, 2016 at 7:19 AM, Pablo Anzorena <anzorena.f...@gmail.com> wrote: > Hi Erick, thanks for answering. > > I attached the image to the body so you can see it. > > > > Why do I need so many clauses? > It is because I have two text fields that contains in average 25 words > with a lot of typos (which I'm not cleaning it) and on top of that the > index consists of 25 million records. And we let the users make queries > with phrases, wildcards prefix and suffix. So for example the following > query is valid *q=text_field:"*ban*" AND text_field2:"foo* bar*"* (I > ommited the query parser syntax that allows this). > > > 2016-06-28 2:08 GMT-03:00 Erick Erickson <erickerick...@gmail.com>: > >> That error sometimes gets reported inappropriately, as long as the >> servers are live >> you can pretty much ignore it. >> >> Attachments pretty much all get stripped by the mail server so we can't >> see your >> screen shot. >> >> Setting your max clause count to over 100K is pretty much an >> anti-pattern, what in the world >> are you doing that would require it to be that high? You haven't >> really shown us the query >> you're sending, but I bet it's amazing. Frankly, anything over the >> default of 1K is suspect. >> >> If this is some clause like id:(1 OR 2 OR 3 OR 4.......) you really >> want to try using the >> TermsQueryParser (note the 's' as opposed to TermQueryParser (no 's'). >> See: >> >> https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-TermQueryParser >> >> And if you use the TermsQueryParser, it's probably a good idea to sort >> the list of terms, it's >> more efficient. >> >> Or do a join or... because anything with that many terms will be >> rather slow to say the least. >> >> Best, >> Erick >> >> >> On Mon, Jun 27, 2016 at 8:38 AM, Pablo Anzorena <anzorena.f...@gmail.com> >> wrote: >> > Hi, >> > >> > I have an ensemble zookeeper consisting of 3 machines and 2 machines >> with >> > solrcloud. >> > >> > With a high frequency I see in the logging: >> > No live SolrServers available to handle this >> > request:[ >> http://solr2:8983/solr/usa_bills_imp_2016_20160623000000_shard1_replica1, >> > >> http://solr3:8983/solr/usa_bills_imp_2016_20160623000000_shard1_replica2] >> > >> > and the state.json is: >> > >> > {"usa_bills_imp_2016_20160623000000":{ >> > "replicationFactor":"2", >> > "shards":{"shard1":{ >> > "range":"80000000-7fffffff", >> > "state":"active", >> > "replicas":{ >> > "core_node1":{ >> > "core":"usa_bills_imp_2016_20160623000000_shard1_replica2", >> > "base_url":"http://solr3:8983/solr", >> > "node_name":"solr3:8983_solr", >> > "state":"active"}, >> > "core_node2":{ >> > "core":"usa_bills_imp_2016_20160623000000_shard1_replica1", >> > "base_url":"http://solr2:8983/solr", >> > "node_name":"solr2:8983_solr", >> > "state":"active", >> > "leader":"true"}}}}, >> > "router":{"name":"compositeId"}, >> > "maxShardsPerNode":"1", >> > "autoAddReplicas":"false"}} >> > >> > And the full stacktrace of the error is: >> > >> > null:org.apache.solr.common.SolrException: >> > org.apache.solr.client.solrj.SolrServerException: No live SolrServers >> > available to handle this >> > request:[ >> http://solr3:8983/solr/usa_bills_imp_2014_20160603115528_shard1_replica2, >> > >> http://solr3:8983/solr/usa_bills_imp_2016_20160623000000_shard1_replica2, >> > >> http://solr2:8983/solr/usa_bills_imp_2014_20160603115528_shard1_replica1, >> > >> http://solr2:8983/solr/usa_bills_imp_2015_20160610125230_shard1_replica1] >> > at >> > >> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:375) >> > at >> > >> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143) >> > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064) >> > at >> org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654) >> > at >> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450) >> > at >> > >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227) >> > at >> > >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196) >> > at >> > >> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) >> > at >> > >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) >> > at >> > >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) >> > at >> > >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) >> > at >> > >> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) >> > at >> > >> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) >> > at >> > >> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) >> > at >> > >> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) >> > at >> > >> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) >> > at >> > >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) >> > at >> > >> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) >> > at >> > >> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) >> > at >> > >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >> > at org.eclipse.jetty.server.Server.handle(Server.java:497) >> > at >> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) >> > at >> > >> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) >> > at >> > >> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) >> > at >> > >> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) >> > at >> > >> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) >> > at java.lang.Thread.run(Thread.java:745) >> > Caused by: org.apache.solr.client.solrj.SolrServerException: No live >> > SolrServers available to handle this >> > request:[ >> http://solr3:8983/solr/usa_bills_imp_2014_20160603115528_shard1_replica2, >> > >> http://solr3:8983/solr/usa_bills_imp_2016_20160623000000_shard1_replica2, >> > >> http://solr2:8983/solr/usa_bills_imp_2014_20160603115528_shard1_replica1, >> > >> http://solr2:8983/solr/usa_bills_imp_2015_20160610125230_shard1_replica1] >> > at >> > >> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:355) >> > at >> > >> org.apache.solr.handler.component.HttpShardHandlerFactory.makeLoadBalancedRequest(HttpShardHandlerFactory.java:246) >> > at >> > >> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:221) >> > at >> > >> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:183) >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> > at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> > at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> > at >> > >> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148) >> > at >> > >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> > at >> > >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> > ... 1 more >> > Caused by: >> > org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: >> Error >> > from server at >> > >> http://solr3:8983/solr/usa_bills_imp_2014_20160603115528_shard1_replica2: >> > maxClauseCount is set to 102400 >> > at >> > >> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560) >> > at >> > >> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235) >> > at >> > >> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227) >> > at >> > >> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376) >> > at >> > >> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:344) >> > ... 10 more >> > >> > So my question is if it is the maxClauseCount triggering the "No live >> > SolrServers" or if it is something else. >> > I attach the Admin UI of one of the SolrServers. >> > The zookeeper log does not throw any errors regarding to this. >> > >> > >> > Thanks. >> > >> > >