true. but how can you find documents containing that field without expanding
1000 clauses?
<br><br><br>------- Original Message -------
On 3/19/2012 07:24 AM Erick Erickson wrote:<br>bq: So all I want to do is a simple
"all docs with something in this field,
<br>and to highlight the field"
<br>
<br>But that doesn't really make sense to do at the Solr/Lucene level. All
<br>you're saying is that you want that field highlighted. Wouldn't it be much
<br>easier to just do this at the app level whenever your field had anything
<br>returned in it?
<br>
<br>Best
<br>Erick
<br>
<br>On Sat, Mar 17, 2012 at 8:07 PM, Darren Govoni <dar...@ontrenet.com> wrote:
<br>> Thanks for the tip Hoss.
<br>>
<br>> I notice that it appears sometimes and was varying because my index runs
<br>> would sometimes have different amount of docs, etc.
<br>>
<br>> So all I want to do is a simple "all docs with something in this field,
<br>> and to highlight the field".
<br>>
<br>> Is the query expansion to "all possible terms in the index" really
<br>> necessary? I could have 100's of thousands of possible terms. Why should
<br>> they all become explicit query elements? Seems overkill and
<br>> underperformant.
<br>>
<br>> Is there a another way with Lucene or not really?
<br>>
<br>> On Thu, 2012-03-08 at 16:18 -0800, Chris Hostetter wrote:
<br>>> : I am suddenly getting a maxClauseCount exception for no reason. I am
<br>>> : using Solr 3.5. I have only 206 documents in my index.
<br>>>
<br>>> Unless things have changed the reason you are seeing this is because
<br>>> _highlighting_ a query (clause) like "type_s:[*+TO+*]" requires rewriting
<br>>> it into a giant boolean query of all the terms in that field -- so even
if
<br>>> you only have 206 docs, if you have more then 206 values in that field in
<br>>> your index, you're going to go over 1024 terms.
<br>>>
<br>>> (you don't get this problem in a basic query, because it doens't need to
<br>>> enumerate all the terms, it rewrites it to a ConstantScoreQuery)
<br>>>
<br>>> what you most likeley want to do, is move some of those clauses like
<br>>> "type_s:[*+TO+*]: and "usergroup_sm:admin") out of your main "q" query
and
<br>>> into "fq" filters ... so they can be cached independently, won't
<br>>> contribute to scoring (just matching) and won't be used in highlighting.
<br>>>
<br>>> :
params={hl=true&hl.snippets=4&hl.simple.pre=<b></b>&fl=*,score&hl.mergeContiguous=true&hl.usePhraseHighlighter=true&hl.requireFieldMatch=true&echoParams=all&hl.fl=text_t&q={!lucene+q.op%3DOR+df%3Dtext_t}+(+kind_s:doc+OR+kind_s:xml)+AND+(type_s:[*+TO+*])+AND+(usergroup_sm:admin)&rows=20&start=0&wt=javabin&version=2}
hits=204 status=500 QTime=166 |#]
<br>>>
<br>>> : [#|2012-02-22T13:40:13.131-0500|SEVERE|glassfish3.1.1|
<br>>> : org.apache.solr.servlet.SolrDispatchFilter|
<br>>> :
_ThreadID=22;_ThreadName=Thread-2;|org.apache.lucene.search.BooleanQuery
<br>>> : $TooManyClauses: maxClauseCount is set to 1024
<br>>> : at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:136)
<br>>> ...
<br>>> : at
<br>>> :
org.apache.lucene.search.MultiTermQuery.rewrite(MultiTermQuery.java:304)
<br>>> : at
<br>>> :
org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:158)
<br>>>
<br>>> -Hoss
<br>>>
<br>>
<br>>
<br>
<br>
