Thanks Erick & Paul, I also found a great example of a custom filter in LIA (6.4 Using a custom filter)
Here's my updated testcase if anybody is interested... ===== QueryParserTest.java ================================================ ... public class QueryParserTest extends LuceneTestCase { ... private static int MAX_HITS = 10; public void testCatchTooManyClauses() throws Exception { System.out.println("===>testCatchTooManyClauses"); Vector docList = null; try { causeTooManyClauses(); } catch(BooleanQuery.TooManyClauses ex) { Term term = new Term(field, queryStr); final BitSet bs = new BitSet(reader.maxDoc()); TermDocs termDocs = reader.termDocs(); WildcardTermEnum wte = new WildcardTermEnum(reader, term); int cnt = 0; docList = new Vector(MAX_HITS); /* the methods termDocs.next() and reader.document() go to different places in the Lucene index so this will send the disk head up and down. see http://lucene.apache.org/java/docs/fileformats.html */ for (term = null; (term = wte.term()) != null && cnt < MAX_HITS; wte.next()) { // get doc ids from .frq file termDocs.seek(term); while (termDocs.next() && cnt++ < MAX_HITS) { bs.set(termDocs.doc()); } } termDocs.close(); // retrieve the Document's in numerical order for(int i=bs.nextSetBit(0); i>=0; i=bs.nextSetBit(i+1)) { docList.add(reader.document(i)); } } System.out.println("found:" + docList.size()); assertTrue(docList.size() == MAX_HITS); } ... ===== QueryParserTest.java ================================================ > -----Original Message----- > From: Paul Elschot [mailto:[EMAIL PROTECTED] > Sent: Sunday, 16 April 2006 5:13 AM > To: java-user@lucene.apache.org > Subject: Re: Catching BooleanQuery.TooManyClauses > > > On Saturday 15 April 2006 13:44, Erick Erickson wrote: > > With the warning that I'm not the most experienced Lucene > user in the > > world... > > > > I *think*, that rather than search for each term, it's more > efficient to > > just use IndexReader.termDocs..... i.e. > > > > Indexreader ir = <whatever>; > > TermDocs termDocs = ir.TermDocs(); > > WildcardTermEnum wildEnum = <whatever>; > > > > for (Term term = null; (term = wildEnum.term()) != null; > wildEnum.next()) { > > termDocs.seek(term); > > This avoids the buffer space needed for each TermDocs by > using each term > separately. A BooleanQuery over all the terms will use the > termDocs.next() and > termDocs.doc() for all terms at the same time. It has to, > because more terms > might match each document and it has to compute the query > score for each > document. > > > while (termDocs.next()) { > > Document doc = reader.document(termDocs.doc()) > > The methods termDocs.next() and reader.document() > go to different places in the Lucene index (see the index format), > so this will send the disk head up and down. > It's better to collect the termDocs.doc() values first, for > example in a > BitSet, and then retrieve the Document's in numerical order. > Btw., this is what the ConstantScoreRangeQuery does to avoid > using all terms > at the same time. > > > } > > } > > > > I know that for loop looks odd, but I just peeked at the > source code for the > > TermEnum classes and see why it works. > > > > One warning, as the folks on the board have pointed out to > me is that the > > Hits object is not entirely efficient when you fetch lots > of docs (more than > > 100 has been mentioned) and you should think about TopDocs > or some such. > > > > Also, if you can avoid fetching the document (i.e. get > everything you want > > from the index) you'll add efficiency. I have no clue how > much you're > > returning to the user, so I don't know whether that would > work for you..... > > In other words, one can use the above BitSet in a Filter lateron > during an IndexSearcher.search() (or in a ConstantScoreQuery), > and use Hits or TopDocs for document retrieval. > > Regards, > Paul Elschot. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- > No virus found in this incoming message. > Checked by AVG Free Edition. > Version: 7.1.385 / Virus Database: 268.4.1/313 - Release > Date: 15/04/2006 > > -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.1.385 / Virus Database: 268.4.2/314 - Release Date: 16/04/2006 --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]