Re: TooManyClauses: maxClauseCount is set to 1024

[Erick Erickson]

Another alternative is to use the TermsQueryParser which is intended to
deal with very large lists of values that should be ORed together. It may
be useful if your query pattern matches its intent. see:
https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-TermsQueryParser

Best,
Erick


On Fri, Aug 5, 2016 at 8:23 AM, Shawn Heisey  wrote:
> On 8/3/2016 12:49 AM, liubiaoxin1 wrote:
>> set exery core solrconfig.xml: 4096
>
> Are you absolutely certain that you have set maxBooleanClauses on
> *EVERY* core in that Solr instance?
>
> This value is global, across the entire JVM, and the last core that
> starts will set the value for all cores.
>
> It is not possible to explicitly control the exact starting order of
> your cores.  If the config option is missing from the last core that
> gets started, then the global setting will be reset back to 1024.
>
> I tried to address this once, but the change was vetoed, and by Apache's
> rules, I wasn't allowed to do it.
>
> https://issues.apache.org/jira/browse/SOLR-4586
>
> I hope to try again and make this situation better for Solr.
>
> Thanks,
> Shawn
>

Re: TooManyClauses: maxClauseCount is set to 1024

[Shawn Heisey]

On 8/3/2016 12:49 AM, liubiaoxin1 wrote:
> set exery core solrconfig.xml: 4096

Are you absolutely certain that you have set maxBooleanClauses on
*EVERY* core in that Solr instance?

This value is global, across the entire JVM, and the last core that
starts will set the value for all cores.

It is not possible to explicitly control the exact starting order of
your cores.  If the config option is missing from the last core that
gets started, then the global setting will be reset back to 1024.

I tried to address this once, but the change was vetoed, and by Apache's
rules, I wasn't allowed to do it.

https://issues.apache.org/jira/browse/SOLR-4586

I hope to try again and make this situation better for Solr.

Thanks,
Shawn

Re: TooManyClauses: maxClauseCount is set to 1024

[liubiaoxin1]

<http://lucene.472066.n3.nabble.com/file/n4290158/QQ%E6%88%AA%E5%9B%BE20160803145022.jpg>
 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/TooManyClauses-maxClauseCount-is-set-to-1024-tp4056965p4290158.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: TooManyClauses: maxClauseCount is set to 1024

[liubiaoxin1]

set exery core  solrconfig.xml: 4096



--
View this message in context: 
http://lucene.472066.n3.nabble.com/TooManyClauses-maxClauseCount-is-set-to-1024-tp4056965p4290157.html
Sent from the Solr - User mailing list archive at Nabble.com.

maxClauseCount is set to 1024

[Andreas Owen]

 Daniel 
(DWU)2/int   int name=Zimmermann Daniel (DZI)1/int   int 
name=Zimmermann Peter (PZN)1/int   int name=Zingg Roland 
(RZI)1/int /lst lst name=sachverstaendiger_s   int 
name=91/int   int
name=Füllemann Stefan (SFU)1/int   int name=Reinhard Martin 
(MRE)11/int   int name=Ruchti Patrick (PRU)3/int   int 
name=Wäschle Jeannine (JWA)3/int /lst lst 
name=veranstaltung_s   int name=Ausstellungen / Messen20/int   
int name=CEO / Bankrat15/int   int name=GB Firmenkunden 
(intern)59/int   int name=GB Private Banking (intern)11/int   
int name=GB Privatkunden (intern)36/int   int 
name=Kundenevents204/int   int name=Marketing / Sponsoring18/int 
  int name=Medien / PR2/int   int 
name=Mitarbeiterevents25/int /lst   /lst   lst 
name=facet_dates lst name=last_modified   int 
name=2011-03-01T00:00:00Z7/int   int 
name=2011-04-01T00:00:00Z5/int   int 
name=2011-05-01T00:00:00Z2/int   int
name=2011-06-01T00:00:00Z7/int   int 
name=2011-07-01T00:00:00Z3/int   int 
name=2011-08-01T00:00:00Z7/int   int 
name=2011-09-01T00:00:00Z10/int   int 
name=2011-10-01T00:00:00Z10/int   int 
name=2011-11-01T00:00:00Z4/int   int 
name=2013-10-01T00:00:00Z1/int   int 
name=2013-11-01T00:00:00Z34/int   int 
name=2013-12-01T00:00:00Z4/int   int 
name=2014-01-01T00:00:00Z35/int   int 
name=2014-02-01T00:00:00Z25/int   int 
name=2014-03-01T00:00:00Z9/int   str name=gap+1MONTH/str   
date name=start2011-03-01T00:00:00Z/date   date 
name=end2014-04-01T00:00:00Z/date   int name=after0/int 
/lst   /lst   lst name=facet_ranges/ /lst lst name=error   str 
name=msgmaxClauseCount is set to 1024/str   str
name=traceorg.apache.lucene.search.BooleanQuery$TooManyClauses: 
maxClauseCount is set to 1024at 
org.apache.lucene.search.ScoringRewrite$1.checkMaxClauseCount(ScoringRewrite.java:72)
at 
org.apache.lucene.search.ScoringRewrite$ParallelArraysTermCollector.collect(ScoringRewrite.java:152)
 at 
org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:79)
   at org.apache.lucene.search.ScoringRewrite.rewrite(ScoringRewrite.java:108)  
   at org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:288)  
   at 
org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:217)
 at 
org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:99)
  at 
org.apache.lucene.search.highlight.WeightedSpanTermExtractor.getWeightedSpanTerms(WeightedSpanTermExtractor.java:469)
at
org.apache.lucene.search.highlight.QueryScorer.initExtractor(QueryScorer.java:217)
  at 
org.apache.lucene.search.highlight.QueryScorer.init(QueryScorer.java:186)at 
org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:199)
at 
org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:528)
at 
org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:415)
 at 
org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:139)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1820)at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:656) 
 at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:359)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
 at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
  at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)   at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)   
 at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:560)  at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)
   at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382) 
   at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
 at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
   at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)   
 at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
  at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116

Re: maxClauseCount is set to 1024

[Jack Krupansky]

The clause limit covers all clauses (terms) in one Lucene BooleanQuery - one 
level of a Solr query, where a parenthesized sub-query is a separate level 
and counts as a single clause in the parent query.


In this case, it appears that the wildcard is being expanded/rewritten to a 
long list of terms.


What version of Solr are you using? Newer releases of Solr should expand 
wildcards as constant score queries rather than enumerating all the terms. 
Although, it looks like the difficulty may be in highlighting - do you 
really need to highlight the roles field?


-- Jack Krupansky

-Original Message- 
From: Andreas Owen

Sent: Monday, March 10, 2014 4:07 AM
To: solr-user@lucene.apache.org
Subject: maxClauseCount is set to 1024


does this maxClauseCount go over each field individually or all put 
together? is it the date fields?



when i execute a query i get this error:


lst name=responseHeader   int name=status500/int   int 
name=QTime93/int   lst name=params str 
name=indenttrue/str str name=qEin PDFchen als Dokument 
roles:*/str str name=_1394436617394/str str 
name=wtxml/str   /lst /lst result name=response numFound=499 
start=0 maxScore=0.40899447

doc
   .
   float name=score0.10604319/float
/doc
/result lst name=facet_counts   lst name=facet_queries/   lst 
name=facet_fields lst name=inhaltstyp_s   int 
name=Agenda390/int   int name=Formular2/int   int 
name=Formulare27/int   int name=Für Dokumente only1/int 
int name=Für Websiten only1/int   int name=Hilfsmittel3/int 
int name=Information3/int   int name=Präsentation1/int 
int name=Regelung8/int   int name=Schulung10/int   int 
name=Schulung_ONL1/int   int name=Test14/int   int 
name=Weisung37/int   int name=test1/int /lst lst 
name=doctype   int name=doc1/int   int name=docx4/int 
int name=htm8/int   int name=pdf44/int   int 
name=pptx4/int   int name=vsd1/int   int 
name=xlsx6/int
/lst lst name=thema_f   int name=1_57/int   int 
name=1_Anleitungen11/int   int name=1_Anleitungen_Ausbildung 
[Anleitungen]11/int   int name=1_Ausbildung3/int   int 
name=1_Ausbildung_Weiterbildung3/int   int 
name=1_Beratung4/int   int name=1_Beratung_Beratungsportal 
FK1/int   int name=1_Beratung_Beratungsportal PK2/int 
int name=1_Beratung_Beratungsprozess1/int   int 
name=1_Handlungsempfehlung2/int   int 
name=1_Handlungsempfehlung_a2/int   int 
name=1_Marktbearbeitung2/int   int 
name=1_Marktbearbeitung_Events2/int   int 
name=1_Produkte29/int   int name=1_Weisungen1/int   int 
name=1_Weisungen_Workplace [Weisungen]1/int /lst lst 
name=author_s   int name=17/int   int name=Aeschlimann 
Monika
(MAE)1/int   int name=Ancora Carlo (CAA)1/int   int 
name=Bannwart Markus (MBA)4/int   int name=Basse Detlev 
(DBS)1/int   int name=Beerli Dominik (DBI)3/int   int 
name=Bollinger Beat (BBO)5/int   int name=Brunner Elisabeth 
(EBN)1/int   int name=Brüschweiler Otto (OBR)5/int   int 
name=Buric Aleksandra (ABC)1/int   int name=Bächtold Eliane 
(EBA)2/int   int name=Chieco Daniela (DCH)1/int   int 
name=D'Adamo-Gähler Karin (KDA)1/int   int name=Dannecker Dietmar 
(DDA)1/int   int name=De Biasio Claudio (CDB)35/int   int 
name=Donatsch Roman (RDO)1/int   int name=Eberhart Livia 
(LET)2/int   int name=Etter Alice (AET)26/int   int 
name=Fankhauser Hausi (HFA)2/int   int name=Frei Beat 
(BFI)1/int   int name=Frick
Patrick (PFR)2/int   int name=Grasset André (AGT)3/int 
int name=Grava Reto (RGV)1/int   int name=Gunterswiler Walter 
(WGU)1/int   int name=Gürkan Simon (SGN)1/int   int 
name=Heimbeck Markus (MHI)27/int   int name=Helbling Andreas 
(AHG)3/int   int name=Held Hans-Jörg (HHE)1/int   int 
name=Helg Christoph (CHL)1/int   int name=Hofer Astrid 
(AHO)3/int   int name=Huber Kalevi (KHU)1/int   int 
name=Huber Paul (PHU)1/int   int name=Häberli Peter 
(PHI)3/int   int name=Häfliger Gabriela (GHA)6/int   int 
name=Hümbeli Isabelle (IHE)3/int   int name=Isler Myriam 
(MIS)1/int   int name=Jäger Andreas (AJA)2/int   int 
name=Kasper Markus (MKP)2/int   int name=Keller Reto 
(RKE)2/int   int name=Knecht Urs
(UKN)2/int   int name=Kutter Benedikt (BKU)2/int   int 
name=Kälin-Klay Sonja (SKY)28/int   int name=Lutz René 
(RLU)4/int   int name=Matanovic Jacques (JMT)2/int   int 
name=Monti Mirko (MMO)1/int   int name=Märki Susanne 
(SMA)16/int   int name=Olimpio Marco (MOL)46/int   int 
name=Pfister Nicole (NPF)1/int   int name=Pozzi Anthony 
(ANP)5/int   int name=Reinhard Martin (MRE)11/int   int 
name=Reutlinger Graf Caroline (CRE)58/int   int name=Roth Rolf 
(ROR)1/int   int name=Rutz Mirco (MRT)2/int   int 
name=Salvisberg Adrian (ASA)29/int   int name=Sassano Marianna 
(MSN)2/int

TooManyClauses: maxClauseCount is set to 1024

[sawanverma]

Its quite confusing about this error.

I had a situation where i have to turn on the highlighting. In some cases
though the number of docs found for a particular query was for example say
2, the highlighting was coming only for 1. I did some checks and found that
that particular text searched was in the bigger document and was towards the
end of the document.

So i increased the hl.maxAnalyzedChar from default value of 51200 to a
bigger value say 500. And then it started working, i mean now the
highlighting was working properly.

Now, i have encountered one more problem with the same error.

There is a document which is returning the maxClauseCount error when i do
search on content:*. The document is quite big big in size and the
hl.maxAnalyzedChars was default i.e. 51200.

I tried decreasing that i found that the error is coming exactly at 31375
char (this is did my setting the hl.maxAnalyzedChars to 31375. it worked
fine till 31374).

Solutions are most welcome as i am in great need of this.

Sample query is as below
http://localhost:8983/solr/test/select/?q=content:* AND
obs_date:[2010-01-01T00:00:00Z%20TO%202011-12-31T23:59:59Z]fl=contenthl=truehl.fl=contenthl.snippets=1hl.fragsize=1500hl.requireFieldMatch=truehl.alternateField=contenthl.maxAlternateFieldLength=1500hl.maxAnalyzedChars=31375facet.limit=200facet.mincount=1start=64rows=1sort=obs_date%20desc

Regards,
Sawan



--
View this message in context: 
http://lucene.472066.n3.nabble.com/TooManyClauses-maxClauseCount-is-set-to-1024-tp4056965.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: TooManyClauses: maxClauseCount is set to 1024

[pravesh]

Just increase the value of /maxClauseCount/ in your solrconfig.xml. Keep it
large enough.

Best
Pravesh



--
View this message in context: 
http://lucene.472066.n3.nabble.com/TooManyClauses-maxClauseCount-is-set-to-1024-tp4056965p4056966.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: TooManyClauses: maxClauseCount is set to 1024

[Yonik Seeley]

Can you provide a full stack trace of the exception?

There's a maxClauseCount in solrconfig.xml that you can increase to
work around the issue.

-Yonik
http://lucidworks.com


On Thu, Apr 18, 2013 at 7:31 AM, sawanverma sawan.ve...@glassbeam.com wrote:
 Its quite confusing about this error.

 I had a situation where i have to turn on the highlighting. In some cases
 though the number of docs found for a particular query was for example say
 2, the highlighting was coming only for 1. I did some checks and found that
 that particular text searched was in the bigger document and was towards the
 end of the document.

 So i increased the hl.maxAnalyzedChar from default value of 51200 to a
 bigger value say 500. And then it started working, i mean now the
 highlighting was working properly.

 Now, i have encountered one more problem with the same error.

 There is a document which is returning the maxClauseCount error when i do
 search on content:*. The document is quite big big in size and the
 hl.maxAnalyzedChars was default i.e. 51200.

 I tried decreasing that i found that the error is coming exactly at 31375
 char (this is did my setting the hl.maxAnalyzedChars to 31375. it worked
 fine till 31374).

 Solutions are most welcome as i am in great need of this.

 Sample query is as below
 http://localhost:8983/solr/test/select/?q=content:* AND
 obs_date:[2010-01-01T00:00:00Z%20TO%202011-12-31T23:59:59Z]fl=contenthl=truehl.fl=contenthl.snippets=1hl.fragsize=1500hl.requireFieldMatch=truehl.alternateField=contenthl.maxAlternateFieldLength=1500hl.maxAnalyzedChars=31375facet.limit=200facet.mincount=1start=64rows=1sort=obs_date%20desc

 Regards,
 Sawan



 --
 View this message in context: 
 http://lucene.472066.n3.nabble.com/TooManyClauses-maxClauseCount-is-set-to-1024-tp4056965.html
 Sent from the Solr - User mailing list archive at Nabble.com.

RE: TooManyClauses: maxClauseCount is set to 1024

[sawanverma]

Thanks Pravesh.

But won't that hit the query performance? Still what would be the ideal value 
to increase? Say this error may come even if we increase the value from 1024 to 
say 5120?
Have tried increasing the value and it had hit the performance.

Regards,
Sawan

From: pravesh [via Lucene] [mailto:ml-node+s472066n4056966...@n3.nabble.com]
Sent: Thursday, April 18, 2013 5:06 PM
To: Sawan Verma
Subject: Re: TooManyClauses: maxClauseCount is set to 1024

Just increase the value of maxClauseCount in your solrconfig.xml. Keep it large 
enough.

Best
Pravesh

If you reply to this email, your message will be added to the discussion below:
http://lucene.472066.n3.nabble.com/TooManyClauses-maxClauseCount-is-set-to-1024-tp4056965p4056966.html
To unsubscribe from TooManyClauses: maxClauseCount is set to 1024, click 
herehttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4056965code=c2F3YW4udmVybWFAZ2xhc3NiZWFtLmNvbXw0MDU2OTY1fC0xMTI5MDQ2NDY1.
NAMLhttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml




--
View this message in context: 
http://lucene.472066.n3.nabble.com/TooManyClauses-maxClauseCount-is-set-to-1024-tp4056965p4056968.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: TooManyClauses: maxClauseCount is set to 1024

[pravesh]

Update:

Also remove your range queries from the main query and specify it as a
filter query.


Best
Pravesh



--
View this message in context: 
http://lucene.472066.n3.nabble.com/TooManyClauses-maxClauseCount-is-set-to-1024-tp4056965p4056969.html
Sent from the Solr - User mailing list archive at Nabble.com.

RE: TooManyClauses: maxClauseCount is set to 1024

[sawanverma]

Hi Yonik,

Thanks for your reply.

I tried increasing the maxClauseCount to a bigger value. But what could be the 
ideal value and will not that hit the performance? What are the chances that if 
we increase the value we will not face this issue again?

As you asked pasting below the full trace of the error

Problem accessing /solr/ar/select/. Reason:
maxClauseCount is set to 1024

org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 
1024
at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:136)
at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:127)
at 
org.apache.lucene.search.ScoringRewrite$1.addClause(ScoringRewrite.java:51)
at 
org.apache.lucene.search.ScoringRewrite$1.addClause(ScoringRewrite.java:55)
at 
org.apache.lucene.search.ScoringRewrite$3.collect(ScoringRewrite.java:95)
at 
org.apache.lucene.search.TermCollectingRewrite.collectTerms(TermCollectingRewrite.java:38)
at 
org.apache.lucene.search.ScoringRewrite.rewrite(ScoringRewrite.java:93)
at 
org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:312)
at 
org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:158)
at 
org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:98)
at 
org.apache.lucene.search.highlight.WeightedSpanTermExtractor.getWeightedSpanTerms(WeightedSpanTermExtractor.java:391)
at 
org.apache.lucene.search.highlight.QueryScorer.initExtractor(QueryScorer.java:216)
at 
org.apache.lucene.search.highlight.QueryScorer.init(QueryScorer.java:185)
at 
org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:205)
at 
org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:490)
at 
org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:401)
at 
org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:131)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:186)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at 
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at 
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at 
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at 
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at 
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at 
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at 
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at 
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at 
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at 
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
at 
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

Regards,
Sawan

From: Yonik Seeley-4 [via Lucene] 
[mailto:ml-node+s472066n4056967...@n3.nabble.com]
Sent: Thursday, April 18, 2013 5:09 PM
To: Sawan Verma
Subject: Re: TooManyClauses: maxClauseCount is set to 1024

Can you provide a full stack trace of the exception?

There's a maxClauseCount in solrconfig.xml that you can increase to
work around the issue.

-Yonik
http://lucidworks.com


On Thu, Apr 18, 2013 at 7:31 AM, sawanverma [hidden 
email]/user/SendEmail.jtp?type=nodenode=4056967i=0 wrote:

 Its quite confusing about this error.

 I had a situation where i have to turn on the highlighting. In some cases
 though the number of docs found for a particular query was for example say
 2, the highlighting was coming only for 1. I did some checks and found that
 that particular text searched was in the bigger document and was towards the
 end of the document.

 So i increased the hl.maxAnalyzedChar from default value of 51200

RE: TooManyClauses: maxClauseCount is set to 1024

[sawanverma]

Yonik,

When i remove the sort part from the query below it works fine. But with
sort it throws the exception

http://localhost:8983/solr/test/select/?q=content:*fl=contenthl=truehl.fl=contenthl.maxAnalyzedChars=31375start=64rows=1sort=obs_date%20desc
--  Throws Exception



http://localhost:8983/solr/test/select/?q=content:*fl=contenthl=truehl.fl=contenthl.maxAnalyzedChars=31375start=64rows=1
-- Works fine.

From the above its clear that sort is causing the problem. Any idea why is
this happening and how to fix this?

Regards,
sawan





--
View this message in context: 
http://lucene.472066.n3.nabble.com/TooManyClauses-maxClauseCount-is-set-to-1024-tp4056965p4056974.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: TooManyClauses: maxClauseCount is set to 1024

[Shawn Heisey]

On 4/18/2013 6:02 AM, sawanverma wrote:

Hi Yonik,

Thanks for your reply.

I tried increasing the maxClauseCount to a bigger value. But what could be the 
ideal value and will not that hit the performance? What are the chances that if 
we increase the value we will not face this issue again?


Changing the maxBooleanClauses value does not affect performance.  It's 
just an arbitrary limit on query complexity.  You can make it as big as 
you want and Solr's performance will not change.  For most people, 1024 
is plenty.  For others, we have no idea how many clauses are needed.


The queries themselves with large numbers of clauses are what affects 
performance, and the only way to improve it is to decrease the query 
complexity.  Chances are good that you are already experiencing the 
performance hit associated with large queries.  Adding more clauses to a 
query will reduce performance.  If you find yourself in a situation 
where you continually need more boolean clauses, you may need to start 
over and create a better design.


The maxBooleanClauses value is just a safety net, created long ago when 
Lucene worked differently than it does now.  There is a discussion 
currently happening among committers about whether that limit even needs 
to exist.  Very likely the limit in Solr will be removed in the near future.


Thanks,
Shawn

RE: TooManyClauses: maxClauseCount is set to 1024

[sawanverma]

Shawn,

Thanks a lot for your reply. But I am confused again if the following query is 
complex.
http://localhost:8983/solr/test/select/?q=content:*fl=contenthl=truehl.fl=contenthl.maxAnalyzedChars=31375start=64rows=1sort=obs_date%20desc

Is that because of content : *? The only unusual thing is the field size of the 
content. In this particular case content field has enormously big data. Since 
this problem comes only when we do a search on * for content field. Is there a 
way that we can split the doc size?

Regards,
Sawan

From: Shawn Heisey-4 [via Lucene] 
[mailto:ml-node+s472066n4057027...@n3.nabble.com]
Sent: 18 April 2013 PM 09:38
To: Sawan Verma
Subject: Re: TooManyClauses: maxClauseCount is set to 1024

On 4/18/2013 6:02 AM, sawanverma wrote:
 Hi Yonik,

 Thanks for your reply.

 I tried increasing the maxClauseCount to a bigger value. But what could be 
 the ideal value and will not that hit the performance? What are the chances 
 that if we increase the value we will not face this issue again?

Changing the maxBooleanClauses value does not affect performance.  It's
just an arbitrary limit on query complexity.  You can make it as big as
you want and Solr's performance will not change.  For most people, 1024
is plenty.  For others, we have no idea how many clauses are needed.

The queries themselves with large numbers of clauses are what affects
performance, and the only way to improve it is to decrease the query
complexity.  Chances are good that you are already experiencing the
performance hit associated with large queries.  Adding more clauses to a
query will reduce performance.  If you find yourself in a situation
where you continually need more boolean clauses, you may need to start
over and create a better design.

The maxBooleanClauses value is just a safety net, created long ago when
Lucene worked differently than it does now.  There is a discussion
currently happening among committers about whether that limit even needs
to exist.  Very likely the limit in Solr will be removed in the near future.

Thanks,
Shawn



If you reply to this email, your message will be added to the discussion below:
http://lucene.472066.n3.nabble.com/TooManyClauses-maxClauseCount-is-set-to-1024-tp4056965p4057027.html
To unsubscribe from TooManyClauses: maxClauseCount is set to 1024, click 
herehttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4056965code=c2F3YW4udmVybWFAZ2xhc3NiZWFtLmNvbXw0MDU2OTY1fC0xMTI5MDQ2NDY1.
NAMLhttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml




--
View this message in context: 
http://lucene.472066.n3.nabble.com/TooManyClauses-maxClauseCount-is-set-to-1024-tp4056965p4057060.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: TooManyClauses: maxClauseCount is set to 1024

[Shawn Heisey]

On 4/18/2013 11:53 AM, sawanverma wrote:

Shawn,

Thanks a lot for your reply. But I am confused again if the following query is 
complex.
http://localhost:8983/solr/test/select/?q=content:*fl=contenthl=truehl.fl=contenthl.maxAnalyzedChars=31375start=64rows=1sort=obs_date%20desc


I hardly know anything about highlighting, so nothing that I say here 
may have any relevance to your situtation at all.


A query of content:* strikes me as an invalid query.  If you are 
shooting for all documents where content exists and excluding those 
where it doesn't exist, I would think that 'q=content:[* TO *]' (the TO 
must be uppercase) would be a better option.


Exactly how your query gets expanded into a something that exceeds 
maxBooleanClauses is a complete mystery to me, and probably does have 
something to do with the highlighting.


Thanks,
Shawn

RE: TooManyClauses: maxClauseCount is set to 1024

[sawanverma]

Shawn,

Giving content:[* TO *] gives the same error but when I give content:[a TO z] 
it works fine. Can you please explain what does it mean when I give content:[a 
TO z]? Can I use this as workaround? The datatype of content field is text_en.

Thanks again for you replies and suggestions.

Regards,
Sawan

From: Shawn Heisey-4 [via Lucene] 
[mailto:ml-node+s472066n4057074...@n3.nabble.com]
Sent: Friday, April 19, 2013 12:33 AM
To: Sawan Verma
Subject: Re: TooManyClauses: maxClauseCount is set to 1024

On 4/18/2013 11:53 AM, sawanverma wrote:
 Shawn,

 Thanks a lot for your reply. But I am confused again if the following query 
 is complex.
 http://localhost:8983/solr/test/select/?q=content:*fl=contenthl=truehl.fl=contenthl.maxAnalyzedChars=31375start=64rows=1sort=obs_date%20desc

I hardly know anything about highlighting, so nothing that I say here
may have any relevance to your situtation at all.

A query of content:* strikes me as an invalid query.  If you are
shooting for all documents where content exists and excluding those
where it doesn't exist, I would think that 'q=content:[* TO *]' (the TO
must be uppercase) would be a better option.

Exactly how your query gets expanded into a something that exceeds
maxBooleanClauses is a complete mystery to me, and probably does have
something to do with the highlighting.

Thanks,
Shawn



If you reply to this email, your message will be added to the discussion below:
http://lucene.472066.n3.nabble.com/TooManyClauses-maxClauseCount-is-set-to-1024-tp4056965p4057074.html
To unsubscribe from TooManyClauses: maxClauseCount is set to 1024, click 
herehttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=4056965code=c2F3YW4udmVybWFAZ2xhc3NiZWFtLmNvbXw0MDU2OTY1fC0xMTI5MDQ2NDY1.
NAMLhttp://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewerid=instant_html%21nabble%3Aemail.namlbase=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespacebreadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml




--
View this message in context: 
http://lucene.472066.n3.nabble.com/TooManyClauses-maxClauseCount-is-set-to-1024-tp4056965p4057181.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: TooManyClauses: maxClauseCount is set to 1024

[Shawn Heisey]

On 4/18/2013 11:02 PM, sawanverma wrote:
 Giving content:[* TO *] gives the same error but when I give content:[a TO z] 
 it works fine. Can you please explain what does it mean when I give 
 content:[a TO z]? Can I use this as workaround? The datatype of content field 
 is text_en.

That syntax is a range query.  The [* TO *] basically means that you are
requesting all documents where the content field exists (has a value).
It's not very likely that [a TO z] will include all possible documents -
it would not include a value like zap for instance, because
alphabetically, that is after z.

I am a little bit confused - why would you want to do highlighting on a
query that matches all documents that contain the content field, or even
all documents?  The point of highlighting is to show the parts of the
text that matched your query text, but you don't have any query text.

I think it may be time to back up and tell us what you want to actually
accomplish, rather than trying to deal directly with the error message.
 Because it has to do with highlighting, I may not be able to help, but
there are plenty of very smart people here who do understand highlighting.

Thanks,
Shawn