subject:"\[jira\] \[Commented\] \(SOLR\-4586\) Increase default maxBooleanClauses"

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-05 Thread Shawn Heisey (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197896#comment-14197896
 ] 

Shawn Heisey commented on SOLR-4586:


If this looks acceptable, my plan is to commit to 5x, then merge to trunk and 
remove the config parsing and deprecation log message.

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-05 Thread Yonik Seeley (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14198367#comment-14198367
 ] 

Yonik Seeley commented on SOLR-4586:


Unfortunately this issue never got consensus as Robert reiterated his -1


 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-05 Thread David Smiley (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14198371#comment-14198371
 ] 

David Smiley commented on SOLR-4586:


[~rcmuir] now has a conflict-of-interest that he didn't have at the time he 
voted.

+1 to commit to 5x.

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-04 Thread Robert Parker (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196180#comment-14196180
 ] 

Robert Parker commented on SOLR-4586:
-

I've opened a bug in response to this behavior:  
https://issues.apache.org/jira/browse/SOLR-6695

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-04 Thread Yonik Seeley (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196210#comment-14196210
 ] 

Yonik Seeley commented on SOLR-4586:


IMO, maxBooleanClauses should just go away for 5.0.
It's a no-brainer... one of the first things I fixed in Heliosearch: 
http://heliosearch.org/download/#0.04

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-04 Thread David Smiley (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196212#comment-14196212
 ] 

David Smiley commented on SOLR-4586:


I'd like to point out that if you are submitting queries with a lot of terms 
then you really should be using the new [Terms 
QParser|https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-TermsQueryParser]
 in 4.10.  That said, I continue to think maxBooleanClauses ought to be 
unlimited by default.

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-04 Thread Jack Krupansky (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196260#comment-14196260
 ] 

Jack Krupansky commented on SOLR-4586:
--

[~yo...@apache.org], I think you just stumbled upon the single most compelling 
reason for releasing and attracting people to Solr 5.0 - No more Max Boolean 
Clauses!

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-04 Thread Jack Krupansky (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196266#comment-14196266
 ] 

Jack Krupansky commented on SOLR-4586:
--

[~reparker], yeah, this is the known behavior - the first core loaded sets this 
setting and any subsequent core loads ignore any new setting. So, yes, you need 
the bounce to change it.

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-04 Thread Ramkumar Aiyengar (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196963#comment-14196963
 ] 

Ramkumar Aiyengar commented on SOLR-4586:
-

Since it's been mentioned somewhere above in this issue that maxBooleanClauses 
mainly serves the purpose of not tripping users from creating bad queries, 
wanted to point out that we recently enountered a case where it was impossible 
to control this setting.

ComplexPhraseQuery rewrites queries like foo* bar* to BQs (because it needs 
visibility into the structure of the query), and there's no way to properly set 
maxClauses without actually knowing term distribution (to make things worse, 
per shard in a distributed setup, such a query could fail in some shards and 
succeed in others).

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-04 Thread Shawn Heisey (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197467#comment-14197467
]

Shawn Heisey commented on SOLR-4586:

bq. yeah, this is the known behavior - the first core loaded sets this setting
and any subsequent core loads ignore any new setting. So, yes, you need the
bounce to change it.

The way I read the code, it shouldn't behave in exactly that way. For every
core that gets loaded, it will read the maxBooleanClauses value out of the
config and set the global limitation, overwriting any value set by any
previously parsed configs. If the setting is missing from the config during
SolrCore initialization, Solr asks BooleanQuery what the current setting is and
uses that. This effectively means that the last initialized config (not the
first) will set the value for all cores. From what I've seen in the logs
during Solr startup, the exact core load order may not be completely
predictable.

Getting rid of the limit entirely as Yonik did for Heliosearch seems like the
best option to me. There are a lot of people that legitimately need to create
queries with a very large number of boolean clauses. The users who ask for
help with this are probably the tip of the iceberg. I think that many users
who exceed maxBooleanClauses are able to figure out on their own how to fix the
problem.

The advice to use the terms qparser is reasonable, but I see that as a
performance option ... Solr should honor any valid query syntax, even if the
performance sucks.

Increase default maxBooleanClauses
--

Key: SOLR-4586
URL: https://issues.apache.org/jira/browse/SOLR-4586
Project: Solr
Issue Type: Improvement
Affects Versions: 4.2
Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch,
SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch

In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to
someone asking a question about queries. Mark Miller told me that
maxBooleanClauses no longer applies, that the limitation was removed from
Lucene sometime in the 3.x series. The config still shows up in the example
even in the just-released 4.2.
Checking through the source code, I found that the config option is parsed
and the value stored in objects, but does not actually seem to be used by
anything. I removed every trace of it that I could find, and all tests still
pass.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-04 Thread Shawn Heisey (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197627#comment-14197627
]

Shawn Heisey commented on SOLR-4586:

[~jkrupan], you were right and I was wrong. I found the code that does exactly
what you described ... and the same code also causes SOLR-6695.

Increase default maxBooleanClauses
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-04 Thread Shawn Heisey (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14197692#comment-14197692
 ] 

Shawn Heisey commented on SOLR-4586:


Currently this limit is set in the SolrConfig constructor.  This will get 
executed multiple times on a multicore installation.  That's probably not a 
performance issue, but it aggravates my unclean code sense, so I was hoping 
to put it someplace that only gets executed once.  A likely candidate is the 
CoreContainer constructor, but I'm not sure whether it would be a good idea to 
introduce a dependency on a lucene class into solr code at that level.


 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-03 Thread Robert Parker (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195112#comment-14195112
 ] 

Robert Parker commented on SOLR-4586:
-

Under Solr 4.10.2 in solrcloud configuration, if I upload a change to 
solrconfig.xml to zookeeper that raises maxBooleanClauses from 1024 to 2048 and 
then reload the collection, the cores do not recongnize a new value for 
maxBooleanClauses unlike other changes to schema.xml and solrconfig.xml.  I 
have to bounce Solr on each node before queries will honor the new value for 
maxBooleanClauses.  This seems like unintentional behavior.  I should be able 
to make any change to schema.xml and solrconfig.xml, then upload those to 
zookeeper and have each node in the cluster instantly honor all new values 
after a core/collection reload.

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-03 Thread Shawn Heisey (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195223#comment-14195223
 ] 

Shawn Heisey commented on SOLR-4586:


[~reparker], maxBooleanClauses is a global Lucene setting across the entire 
application, and the last thing to set that value will win every time.

If you have any configs with the default of 1024 and you reload any of those 
cores after reloading the one that sets it to 2048, then it will be changed 
back -- for the entire application.  The best option is to set the higher limit 
in *every* solrconfig.xml file, or remove the setting from all of them except 
one.

The javadocs for the Lucene setter method do not indicate this global nature, 
but I assure you that I have looked at the code, and it is indeed global.

http://lucene.apache.org/core/4_10_0/core/org/apache/lucene/search/BooleanQuery.html#setMaxClauseCount%28int%29

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-03 Thread Robert Parker (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195233#comment-14195233
 ] 

Robert Parker commented on SOLR-4586:
-

Ive only got one collection and one config in zookeeper, and thats the one that 
is being changed.  Each core had its solrconfig.xml updated on disk, but since 
its a SolrCloud config, only the zookeeper version should matter, correct?  

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-11-03 Thread Shawn Heisey (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14195253#comment-14195253
]

Shawn Heisey commented on SOLR-4586:

That's correct.

It is entirely possible that this is one config setting that does not get
re-handled on a core reload. I'd have to trace the code to know for sure, and
I'm not very familiar with it, so it could take quite a while to trace. I
don't have the time for that at the moment.

Increase default maxBooleanClauses
--

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-08-20 Thread Bragadeesh (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14103900#comment-14103900
 ] 

Bragadeesh commented on SOLR-4586:
--

I bumped through this issue recently. Is this something planned for a release 
sooner ? 

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2014-08-20 Thread Erick Erickson (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14104249#comment-14104249
 ] 

Erick Erickson commented on SOLR-4586:
--

Bragadeesh:

You can set it in Solr via a setting in solrconfig.xml, and in Lucene by the 
appropriate setter method.

This is just a default, not a hard limit.



 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-07-21 Thread Jack Krupansky (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13714848#comment-13714848
]

Jack Krupansky commented on SOLR-4586:
--

Huh... I though we had settled this for 4.4. Oh well.

My latest thoughts.

1. For BooleanQuery, remove it. People writing or generating Solr queries are
very aware when they are doing something big.

2. For rewrite of queries, have a warning and cutoff of the generated terms,
rather than a hard fail. With FSTs, do we still have a rewriting problem?

3. Any query that can generate lots of terms should have an additional
constructor parameter to specify the limit, or -1 if unlimited. No more global
limit. Each query parser should have a term expansion limit parameter.

4. The default for the term expansion limit parameter should be expanded as
hardware capabilities expand. Say 10,000 for a proposed Lucene limit and Solr
config default limit as well.

Increase default maxBooleanClauses
--

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-11 Thread Shawn Heisey (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629036#comment-13629036
 ] 

Shawn Heisey commented on SOLR-4586:


In situations where true consensus cannot be found, but most people agree work 
is needed, how do we decide what to do?

Back to the issue - is there a way in Lucene to specify something like the 
following query (using all of the different kinds of boolean logic) without 
BooleanQuery?

(+author:tolkien -title:hobbit) isbn:( ... 20 different ISBNs ...)

If so, how's the performance compared to BooleanQuery, and could Solr reliably 
construct such a query?  That would be the ideal long-term fix, if it's 
possible, and that would make this issue a band-aid until the real fix can 
happen.


 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-11 Thread Yonik Seeley (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629048#comment-13629048
 ] 

Yonik Seeley commented on SOLR-4586:


FYI, I'm currently working on testing that setting Integer.MAX_VALUE has no 
unintended side-effects (i.e. no huge priority queues should be created, etc.)

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-11 Thread Jack Krupansky (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629052#comment-13629052
 ] 

Jack Krupansky commented on SOLR-4586:
--

{bq}
 is there a way in Lucene to specify something like the following query (using 
all of the different kinds of boolean logic) without BooleanQuery?

{code}
(+author:tolkien -title:hobbit) isbn:( ... 20 different ISBNs ...)
{code}
{bq}

Sure, just break it up into 1024-term chunks:

{code}
(+author:tolkien -title:hobbit) isbn:( ( ... 1024 ISBNs ... ) ( ... 1024 ISBNs 
...) ... repeat 200 times ... )

{code}

The parentheses for the sub-queries should cause the query parser to generate a 
nested BooleanQuery for each sub-query.

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-11 Thread Shawn Heisey (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629068#comment-13629068
 ] 

Shawn Heisey commented on SOLR-4586:


My specific query example might have been bad, as it would exceed a 2MB POST 
buffer, but ignore that and focus on the idea.

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-11 Thread Shawn Heisey (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629080#comment-13629080
]

Shawn Heisey commented on SOLR-4586:

bq. Sure, just break it up into 1024-term chunks:

I didn't know that was possible. I thought I had remembered (using debugQuery)
that Solr parses nested queries down to remove nesting, but I could be wrong
about that.

I was actually asking if it was possible to do it without BooleanQuery at all.
I'm glad to have the knowledge you've imparted, though.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-11 Thread Yonik Seeley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629096#comment-13629096
]

Yonik Seeley commented on SOLR-4586:

bq. Sure, just break it up into 1024-term chunks:

Right. This will be slightly less efficient than one bigger boolean query, but
it does work around the arbitrary limit. The other workaround is to remove the
arbitrary limit ;-)

For those that might not have the historical context: in the past, some
auto-expanding term queries (like fuzzy queries) would generate more and more
terms to search until max clauses was hit - so the limit had a purpose. If
you go back far enough, other queries (like prefix queries) generated unbounded
boolean queries, and an exception would be thrown when too many terms were
generated. The latter was very bad (things worked in production until your
index grew and then would blow up). Solr luckily never had this issue as it
used constant scoring techniques with no limits for both prefix and range
queries from the beginning.

The historical reasons for this limit no longer apply. No multi-term queries
that solr uses explode into a huge boolean query if you remove the limit (this
is what I'm busy verifying with tests right now). This only affects explicitly
created queries, where it's very bad to have an artificial limit like this set
since it may work in testing and initially in production, and then blow up in
the future because the arbitrary limit was crossed.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-11 Thread Jack Krupansky (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629098#comment-13629098
 ] 

Jack Krupansky commented on SOLR-4586:
--

And, technically, the query parser's query generator could generate 
daisy-chained BooleanQuery's, where every 1024th clause is a reference to a 
nested BQ that has than same form. Or, the query parser could generate every 
1024 clauses down at a second level and reference them from the main BQ, so 
that you could have a single BQ that could have 1024 clauses, each of each is 
either a term or BQ, with the second level of BQ being terms only, which would 
let you have 1024x1024 or 1M terms.

Not that I am proposing any of this, and I suspect that Lucene will perform 
better if it sees all the clauses in one, single group rather than scattered 
into multiple BQ's.


 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-11 Thread Jack Krupansky (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629301#comment-13629301
 ] 

Jack Krupansky commented on SOLR-4586:
--

I was thinking that maybe there should be a tombstone deprecated comment in 
solrconfig for maxBooleanClauses so that somebody doing a file compare could 
know that they can/should clean up their pre-4.3 solrconfig as opposed to some 
feature that may have simply disappeared or moved.


 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch, SOLR-4586.patch, SOLR-4586_verify_maxClauses.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-11 Thread Robert Muir (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629302#comment-13629302
]

Robert Muir commented on SOLR-4586:
---

I'm still -1 to removing the limit. Its even more bogus to just silently ignore
a configuration parameter that used to exist completely: i guess this is
typical Solr though.

{noformat}
Back to the issue - is there a way in Lucene to specify something like the
following query (using all of the different kinds of boolean logic) without
BooleanQuery?

(+author:tolkien -title:hobbit) isbn:( ... 20 different ISBNs ...)
{noformat}

This is a fantastic example of why the limit can still be useful. it forces you
to think about what is going on here... why do your queries have 200,000 terms?

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-11 Thread Yonik Seeley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629336#comment-13629336
]

Yonik Seeley commented on SOLR-4586:

bq. Its even more bogus to just silently ignore a configuration parameter

That's part of the problem - most people will not have explicitly configured it
(except those trying to avoid exceptions), but just got it by virtue of copying
the example solrconfig.xml as a starting point.

bq. (+author:tolkien -title:hobbit) isbn:( ... 20 different ISBNs ...)
bq. This is a fantastic example of why the limit can still be useful. it forces
you to think about what is going on here..

Unfortunately, it doesn't force anyone to think about those issues early on.
Maybe setting a very low limit like 10 or 20 terms would... but 1024 definitely
does not.
What happens is that you can have a working, tested application that constructs
queries like this, and it only breaks at some undetermined future point in time
when the magic limit is exceeded (caused by index growth or whatever). There
is no warning to the user that the query structure that they are using will
break hard after going over this magic limit (that no longer serves a purpose).
It's simply a trap, and we should remove it.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-11 Thread Shawn Heisey (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629399#comment-13629399
]

Shawn Heisey commented on SOLR-4586:

bq. Here's a simple patch that removes the max clauses limitation from Solr.

I would also remove maxBooleanClauses from the many test-related solrconfig.xml
files that have included it. If you don't have the time right now, I am
willing to make a new patch. I like Jack's idea for a tombstone comment in the
example solrconfig.xml file.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-11 Thread Yonik Seeley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13629427#comment-13629427
]

Yonik Seeley commented on SOLR-4586:

bq. I would also remove maxBooleanClauses from the many test-related
solrconfig.xml files that have included it. If you don't have the time right
now, I am willing to make a new patch.

Thanks, I had missed that.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-05 Thread Shawn Heisey (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13623631#comment-13623631
]

Shawn Heisey commented on SOLR-4586:

Even before someone was crazy enough to make me a committer, I developed a lot
of respect for Robert, watching the dev list in general and Jira in particular.
He knows most of this code backwards and forwards, and often keeps the rest of
us from doing things we'll regret later. Is this one of those things? I can't
tell.

My gut tells me that we need to increase the value and take the choice away
from non-savvy users. MAX_INT seems too high. Even the default I've proposed
is probably too high, but what value will work? I haven't done any
experimenting to know when performance becomes unacceptable. Would 16K be
better? 32K? 256K? IMHO, if we choose a value low enough that pathological
user code will easily result in a too many clauses exception, there will be a
significant subset of Solr users that will legitimately need more. We've got
potential support issues either way. Perhaps my 4x solution needs to be left
in place long-term, and we need a little more discussion about the default
value.

bq. if the BooleanQuery instead logged a one-time warning if it gets used with
1024 clauses? And if assertions are enabled then fail. The idea here is to
try and alert the developer that they are most likely using BooleanQuery
incorrectly, and suggest TermsFilter might be suitable instead.

For Lucene, choosing TermsFilter is probably a viable solution much of the
time, but this issue is for Solr. It's linked to a similar issue for Lucene.
My Lucene knowledge is very pitiful, I know enough to be dangerous but not
useful. In my head, I imagine that reworking Solr to eliminate/reduce the use
of BooleanQuery is a monumental task. Am I right about that?

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-05 Thread David Smiley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13623654#comment-13623654
]

David Smiley commented on SOLR-4586:

bq. For Lucene, choosing TermsFilter is probably a viable solution much of the
time, but this issue is for Solr. It's linked to a similar issue for Lucene. My
Lucene knowledge is very pitiful, I know enough to be dangerous but not useful.
In my head, I imagine that reworking Solr to eliminate/reduce the use of
BooleanQuery is a monumental task. Am I right about that?

The solr user would be un-impacted because they very well may choose to ignore
the warning. If they have some basic Lucene knowledge, they could write a
QParser that builds a ConstantScoreQuery from a TermsFilter. It's a piece of
cake and if I wasn't so darned busy I'd do it myself because it's something
Solr should have.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-05 Thread Mark Miller (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13623682#comment-13623682
]

Mark Miller commented on SOLR-4586:
---

bq. Even the default I've proposed is probably too high, but what value will
work?

I don't think it's a very good argument because of this silly guessing. We
could try and protect a user from these silly situations everywhere. Should we
also limit the number of filters allowed in an analyzer? Should we watch how
many docs you are putting into the index? Should we make sure you don't add a
query filter thats too long and crazy? It's a silly direction to go down. And
that's why you won't find it anywhere else. This is an aberration thats been
around before pretty much any of us showed up. The idea that this is helpful to
users is absurd - I've been around lucene and solr and users for too long to
fall for that.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-05 Thread Jack Krupansky (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13623691#comment-13623691
]

Jack Krupansky commented on SOLR-4586:
--

My revised proposal:

1. Unless the Lucene guys cave, leave it in Lucene. Same Lucene default.
2. Leave it in Solr as well, on the theory that if Lucene has a setting, Solr
should provide access to it.
3. Raise the Solr default to 100K - both in the code and example solrconfig. 1
million is too scary.
4. Change the Solr semantics so that the Lucene limit is raised whenever a
collection is loaded that has a higher limit.
5. Change in both trunk and 4x since it won't impact existing apps.

Today, if you want to change the limit, you must shutdown and reload the Solr
server since only the initial core load will set it.

My proposal is trying to satisfy two goals:

1. A higher default limit.
2. Can change the value, at least to raise it, without a full Solr server
shutdown and restart.
3. Somewhat simpler semantics - you don't need to read the fine print to use
it, and rarely would you need to use it.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-05 Thread Yonik Seeley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13623852#comment-13623852
]

Yonik Seeley commented on SOLR-4586:

bq. Leave it in Solr as well, on the theory that if Lucene has a setting, Solr
should provide access to it.

Solr isn't a Lucene configuration engine - we don't allow for all of the
flexibility that lucene provides, and never will.
A setting should make sense on it's own. This particular setting no longer
makes sense. I don't think it makes sense for Lucene or Solr, but the
arguments need not be the same for both since the use cases are different.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-04 Thread Shawn Heisey (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13622910#comment-13622910
]

Shawn Heisey commented on SOLR-4586:

I have been working on a new patch. Here's my current plan:

4x:
Support maxBooleanClauses in solrconfig.xml. The highest value found wins. If
the first core loaded is missing the config option, the new
CoreContainer.DEFAULT_MAX_BOOLEAN_CLAUSES constant (1048576) will be used, and
is therefore likely to be that winner. Robert's theoretical value of 5 is
doable by putting it in the first core config, or in all of them. The example
solrconfig.xml for collection1 has some comments. The maxBooleanClauses
section is entirely removed from all other solrconfig.xml files in the source
tree, which mostly means configs for tests.

{noformat}
!-- Max Boolean Clauses

Maximum number of clauses in each BooleanQuery, an exception
is thrown if a query exceeds this.

** WARNING **

This option actually modifies a global Lucene property that
will affect all SolrCores. If multiple solrconfig.xml files
disagree on this property, the highest value found will be
the one that applies. As of Solr 4.3, if the first solrconfig.xml
file loaded does not have this specified, a default of 1048576
will be used. The default in older versions was 1024.

--
!--
maxBooleanClauses1024/maxBooleanClauses
--
{noformat}

trunk:
Set maxBooleanClauses to DEFAULT_MAX_BOOLEAN_CLAUSES constant in CoreContainer.
All other code for maxBooleanClauses is gone. If someone needs more than
1048576 clauses, they'll have to modify the source.

I hope the 4x direction is correct. The trunk change seems right to me, but I
can make them the same if that would be better. If any existing tests are
affected, I will fix them. I would like to build some tests specifically for
this issue, but as I said before, I'll need help with that.

If trunk and 4x do end up being different, should they be entirely separate
commits? I have learned that standard practice is to commit to trunk, then
merge the trunk commit into the release branch and do any necessary cleanup.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-04 Thread Robert Muir (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13622924#comment-13622924
]

Robert Muir commented on SOLR-4586:
---

I already stated i dont think it should be completely removed. doing it in
trunk only doesnt make it any better.

I think its fine to *carefully* increase the default, but its also useful to
have limits rather than performance traps (I fundamentally disagree with the
entire idea of just letting things be slow instead of catching errors).

I don't think its going to be any easier to commit this as a solr issue vs. as
lucene one just because I'm being a pain in the ass. I can be a pain in the ass
on this issue, too.

Just increasing the default isnt enough anyway, its a must to ensure that
priority queues of size 1m arent being created by default where they were 1k
before, too.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-04 Thread Yonik Seeley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13622994#comment-13622994
]

Yonik Seeley commented on SOLR-4586:

bq. (I fundamentally disagree with the entire idea of just letting things be
slow instead of catching errors).

It's extremely trappy to fail hard at a low arbitrary limit (and it's silly
calling it an error if it goes above that limit).
maxBooleanClauses should be removed from Solr (or effectively removed via
setting it to MAX_INT) provided there are no bugs in Lucene that create any
priority queues of this size.

bq. All other code for maxBooleanClauses is gone.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-04-04 Thread David Smiley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13623383#comment-13623383
]

David Smiley commented on SOLR-4586:

Rob, how would you feel if the BooleanQuery instead logged a one-time warning
if it gets used with 1024 clauses? And if assertions are enabled then fail.
The idea here is to try and alert the developer that they are most likely using
BooleanQuery incorrectly, and suggest TermsFilter might be suitable instead.
At least then, the application won't experience an avoidable failure in
production -- the primary concern of everyone else on this thread.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-27 Thread Robert Muir (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13615068#comment-13615068
]

Robert Muir commented on SOLR-4586:
---

{quote}
1) Remove the solr.xml additions.
{quote}

-1. thats where it belongs. its not a per-core thing.

{quote}
2) Log a deprecation warning when maxBooleanClauses is found in solrconfig.xml,
but honor it.
{quote}

=1. As i stated, I want to keep this parameter. its useful.

{quote}
2a) Should we make it possible to go lower than Lucene's default? The current
patch won't.
{quote}

Dude. it should be nothing more than doing what the config file says.
Seriously, anything else is a bug.
If i want to have a max of 5, thats my choice.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-27 Thread Yonik Seeley (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13615267#comment-13615267
 ] 

Yonik Seeley commented on SOLR-4586:


-1 to put maxBooleanClauses in solr.xml, it does not belong there.
This is arguably a Lucene bug (for being a system-wide static) and this problem 
should not be propagated into Solr.

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-27 Thread Mark Miller (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13615312#comment-13615312
]

Mark Miller commented on SOLR-4586:
---

bq. if the static max clauses had never existed, would it be added today? I
don't think so.

Agreed. An ugly relic of the past that wouldn't get through the door today.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-26 Thread Shawn Heisey (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13614407#comment-13614407
]

Shawn Heisey commented on SOLR-4586:

Here's my proposal:

The current 4x patch goes in largely as it is now. Changes:
1) Remove the solr.xml additions.
2) Log a deprecation warning when maxBooleanClauses is found in solrconfig.xml,
but honor it.
2a) Should we make it possible to go lower than Lucene's default? The current
patch won't.
3) Make some tests to verify behavior. I'm willing to do this, but I will need
a little guidance.

With the current POST buffer default size of 2MiB, you could include just under
2^20 boolean clauses, if each clause were only 1 byte, a highly contrived and
illogical query. For that reason, I think that 2^20 is a reasonable default
value. Also, I think that performance would become intolerable long before you
reached that many clauses, and I think that will continue to be the case for
the foreseeable future.

For 5.0, we remove the maxBooleanClauses config entirely. If someone really
did have a viable use case for more than 2^20 clauses, they would very likely
have the expertise required to modify Solr code.

Would it be a good idea to file another issue to have Solr use a better
solution than BooleanQuery when possible?

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-23 Thread Robert Muir (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611735#comment-13611735
]

Robert Muir commented on SOLR-4586:
---

My recommendation at solr.xml was to fix the bugs and not have complicated
logic.

I look at the issue differently: to me the important thing is that its bogus
and confusing today to have this configuration parameter in solrconfig.xml,
when it really should be in solr.xml, since it cannot be applied per-core. This
is just a straight up bug! And I dont think the limit should be adjusted until
this bug is fixed: otherwise it will just create more bugs (or at least
confusing logic, like the last one or highest one or whatever is picked).

So it should be in solr.xml, and no one gets confused, and there is no need for
confusing logic (picking the last one, or the highest one, or whatever). If
someone has it specified in solrconfig.xml, some action should be taken: a nice
person might argue for just a warning or similar in 4.x and just a straight up
error (refuse to start) in 5.x.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-23 Thread Yonik Seeley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611742#comment-13611742
]

Yonik Seeley commented on SOLR-4586:

bq. when it really should be in solr.xml, since it cannot be applied per-core.

If it were truly a limit that one would want applied to all cores, then it
should be in solr.xml
But I don't think that's the case. The real bug here is this global static
(max clauses) that has always had far reaching effects.

I'd like to just set the limit to Integer.MAX_VALUE (essentially removing the
limit), but it's not clear if that has any negative effects.
I'm not concerned with any user queries that explicitly have many terms, but am
concerned about MultiTerm queries or other things that Lucene may do internally
based on this number.

For example, in TopTermsRewrite:
{code}
/** return the maximum size of the priority queue (for boolean rewrites this
is BooleanQuery#getMaxClauseCount). */
protected abstract int getMaxSize();
{code}

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-23 Thread Robert Muir (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611744#comment-13611744
 ] 

Robert Muir commented on SOLR-4586:
---

I disagree with Integer.MAX_VALUE (as i said on LUCENE-4835), and I disagree 
with removing the limit completely too.

Today the way its implemented though, its a bug.

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-23 Thread Yonik Seeley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611757#comment-13611757
]

Yonik Seeley commented on SOLR-4586:

bq. I disagree with Integer.MAX_VALUE (as i said on LUCENE-4835)

The argument there seemed to be more about Lucene (a programmer bug that kept
adding clauses), although this is an extremely weird way to help catch such an
error.
Put another way... if the static max clauses had never existed, would it be
added today? I don't think so. It was added in the past for historical
reasons (auto-expanding queries) that hopefully no longer apply today.

Anyway, for Solr-land, if we want to enforce on a per-core basis then we can do
that at the Solr query parser level. I see no reason to do that though. I
think we should just remove the limit altogether at the solr level.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-23 Thread Robert Muir (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611760#comment-13611760
]

Robert Muir commented on SOLR-4586:
---

Like i said: I disagree. if someone has more than 1024 terms in their query,
more often than not: something is wrong and not being done efficiently.

They hit the limit and realize that, and its useful for that reason. If they
want to increase/decrease the limit with some configuration parameter, then
that configuration parameter should actually work and not be buggy.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-23 Thread Yonik Seeley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611764#comment-13611764
]

Yonik Seeley commented on SOLR-4586:

We've seen tons of instances of people wanting to send in queries of more than
1024 terms. Making them edit a config file first is very unfriendly. It's
*really* unfriendly if the system is already in production.

We don't limit the size of a query in bytes, the number of prefix queries, or
fuzzy queries, or range queries, or any other query clauses that can take a
large amount of resources. We shouldn't limit the number of clauses in general
either - it made sense in the past, but it no longer does.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-23 Thread Jack Krupansky (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611842#comment-13611842
]

Jack Krupansky commented on SOLR-4586:
--

I would note that neither SpanOrQuery or SpanMultiTermQueryWrapper implements a
limit as BooleanQuery does. Shouldn't there be a consistent limit policy
between BooleanQuery and SpanOrQuery, both of which take an arbitrary number of
clauses?

So, I'm back to the proposal that:

1. The boolean query caluse limit be deprecated in 4.x and removed in 5.0, in
both Solr and Lucene.
2. Solr define a default boolean clause limit in 4.x that is much higher than
the Lucene limit.
3. We otherwise preserve the semantics of clause limits for 4.x.
4. Separately consider a query parser option to limit the maximum query size,
in both source length and number of terms.

I would classify the current Solr core-related semantics for the clause limit
as messy rather than buggy. If there is a clear and indisputable bug in
the current semantics, fine, let's fix it. The current semantics are a holdover
from the old single-core days when it made no difference that the solrconfig
limit was for the core since that was all there was. Sure, that semantics
should have been refined when multiple collections became the norm, but this is
essentially a by design by default rather than an outright logic flaw.
Besides, if you know the current semantics and have an app that depends on
them, changing the rules would be a compatibility issue. Easier just to leave
well enough alone, and properly fix the semantics in 5.0 by completely
removing the unnecessary feature - that SpanOrQuery proves is not necessary.

I would agree that there is some (limited) value in specifying a limit to catch
buggy code, but I just don't find it truly persuasive. Or at least not as
persuasive as the annoyance of having the artificially low default and the need
for a user-level config setting that is global for all Lucene indexes in the
same JVM. (Why not make the limit be a Lucene IndexConfig setting? Please
Don't!!).

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-23 Thread David Smiley (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611972#comment-13611972
]

David Smiley commented on SOLR-4586:

FWIW My opinion is consistent with Yonik Jack's.

BTW, when I see code that creates a massive BooleanQuery, it's always been of
TermsQueries in a situation where the BooleanQuery could have been replaced
with a ConstantScoreQuery wrapping a TermsFilter -- which runs faster too. Of
course no scores but, again, the use-cases where I've seen BQ's limit hit don't
actually care about the score. It'd be nice if Solr had a QParser for that.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-22 Thread Shawn Heisey (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611368#comment-13611368
 ] 

Shawn Heisey commented on SOLR-4586:


Side note: I notice that the default value for persistent in solr.xml when it's 
not present is false.  Shouldn't that be true, particularly with increased 
reliance on API changes from SolrCloud?  Is this no longer applicable with the 
new properties file replacement for solr.xml?


 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-22 Thread Mark Miller (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611383#comment-13611383
 ] 

Mark Miller commented on SOLR-4586:
---

Yeah, it probably would be a good change except it will be irrelevant with the 
new stuff.

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-22 Thread Jack Krupansky (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611411#comment-13611411
 ] 

Jack Krupansky commented on SOLR-4586:
--

This issue has morphed more than a bit since the original proposal to simply 
remove the maxBooleanClauses limit from Solr.

At this stage, I'd be more in favor of simply increasing the default 
maxBooleanClauses in Solr to 1 million (or whatever), but keeping all of the 
other semantics the exact same so that nobody sees any change other than the 
larger default limit, and all the rules (crazy as they are) for when and how 
the limit gets set remain unchanged. In other words, nobody needs to dig and 
try to understand What does this really mean?? because the new default limit 
means quite simply Don't worry about it.

I'm tempted to push for deprecation and removal of the limit, but I can see 
some merit in an artificial limit for testing and to limit resource consumption.

In short, the main goals here are: 1) Make the old limit a non-issue for 
typical applications, 2) don't mess with the semantics or create new rules for 
where or when or how to set the limit, and 3) preserve the limit capability for 
special situations where it is desirable to REUDCE the limit, as opposed to 
being forced to increase the limit as some applications need today.


 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch, 
 SOLR-4586.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-22 Thread Shawn Heisey (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13611537#comment-13611537
]

Shawn Heisey commented on SOLR-4586:

Yes, it has morphed quite a lot. I had some fundamental misunderstandings
early on. I suppose a true long-term goal would be to have Solr stop creating
BooleanQuery objects whenever possible and use newer query objects. I don't
know enough about the low-level details to know if that IS possible.

In the 4.x short term, we probably need to support setting a low limit as
mentioned in existing configs, and set a high limit if it's not configured.
That's what I've attempted to do in the newest patches.

Exactly what to do for 5.x and beyond is something I'm not sure about. If
we're going to support a user setting a low limit, Robert's idea of putting it
into solr.xml or its replacement strikes me as the right thing to do. We could
also set a very high limit and not let the user configure it.

Is there any consensus to be found here?

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-18 Thread Robert Muir (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13605943#comment-13605943
 ] 

Robert Muir commented on SOLR-4586:
---

Instead of the hair involving reloading and stuff, why not move it to solr.xml? 
Isn't that where it belongs (not per-core)?

 Increase default maxBooleanClauses
 --

 Key: SOLR-4586
 URL: https://issues.apache.org/jira/browse/SOLR-4586
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.2
 Environment: 4.3-SNAPSHOT 1456767M - ncindex - 2013-03-15 13:11:50
Reporter: Shawn Heisey
 Attachments: SOLR-4586.patch, SOLR-4586.patch, SOLR-4586.patch


 In the #solr IRC channel, I mentioned the maxBooleanClauses limitation to 
 someone asking a question about queries.  Mark Miller told me that 
 maxBooleanClauses no longer applies, that the limitation was removed from 
 Lucene sometime in the 3.x series.  The config still shows up in the example 
 even in the just-released 4.2.
 Checking through the source code, I found that the config option is parsed 
 and the value stored in objects, but does not actually seem to be used by 
 anything.  I removed every trace of it that I could find, and all tests still 
 pass.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4586) Increase default maxBooleanClauses

2013-03-18 Thread Shawn Heisey (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-4586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13606058#comment-13606058
]

Shawn Heisey commented on SOLR-4586:

bq. Instead of the hair involving reloading and stuff, why not move it to
solr.xml? Isn't that where it belongs (not per-core)?

That seems like a good idea to me. It'll take me a while to figure out how.

Increase default maxBooleanClauses
--

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

58 matches

Mail list logo