Hey, Sorry, I will explain a bit more about my collect method. Currently my collect method is executing IndexSearcher.doc(id) and storing some stuff in a Map which I can then retrieve from the HitCollector (much like the example in the Lucene In Action book). Of course that's somewhat expensive, so I'd like to do some statistical sampling based on the result set size to try and speed things up.
The way I was thinking about doing this was, during the collect method only executing IndexSearcher.doc(id) on every Nth document, where N is calculated dynamically based on a minimum number X. The rule would be: N = Max(1,(numResults / X)) In order to do this in the collect method, I need to know the total number of results before ever invoking the collect method right? That seemed to make a case for the BitSet/QueryFilter in the constructor. In addition, someone else on the list mentioned that one of the reasons calling IndexSearcher.doc(id) in the collect method was that it caused the disk to do a lot of seeking. Maybe that's a moot point if one is using a RAMDirectory or an FSDirectory small enough that it gets cached by the OS anyway, but if it's not, then I thought it might be more performant to have the hitcollector set the Bits in the collect method and then do another pass to do the statistical sampling. Either way it seems that to do the statistical sampling that I envision I either need to calculate the total result count/document id set in the constructor, before calling the collect method, or calculate the total result count/document id set in the collect method and then execute some sort of post-collect method, right? So I was just wondering which method was better/faster. Thanx. JAMES --- Chris Hostetter <[EMAIL PROTECTED]> wrote: > > : I'm using a HitCollector and would like to know > the > : total number of results that matched a given > query. > : Based on the JavaDoc, I this will do the trick: > > you don't need a BitSet in that case, you could find > that out just using > an int... > > public CountingCollector extends HitCollector { > public int count = 0; > public void collect(int doc, float score) { > count++ }; > } > CountingCollector c = new CountingCollector(); > searcher.search(query, c) > int numResults = c.count; > > : If I want to know the total number of results > inside > : of the HitCollector, i.e. before the collect > method > : has ever been called, I think I could pass the > Query > : and Searcher objects into the HitCollector and do > this > : in its constructor: > : > : BitSet bits = (new > : > QueryFilter(query)).bits(searcher.getIndexReader()); > : int numResults = bits.cardinality(); > > This question doesn't make a lot of sense to me, why > do you need to know > the total number ofresults before the collect method > is called? .. what > you are suggesting here (using QueryFilter in this > way) is perfectly > legal, but it's going to do just as much work as > using a HitCollector will > (possibly more, i can't remember). > > : Is Lucene executing another pass over the index in > : order to populate the BitSet and then doing > another > : pass while calling the collect method? Thanx. > > in your last example, you never us your > HitCollector, so i'm not sure what > you mean, but assuming you aresking about combining > those examples into > something like this.... > > Searcher searcher = new > IndexSearcher(indexReader); > BitSet bits = (new > QueryFilter(query)).bits(searcher.getIndexReader()); > final int numResults = bits.cardinality(); > searcher.search(query, new HitCollector() { > public void collect(int doc, float score) { > /* do something with numResults and doc > and score */ > } > }); > > ...then yes, you are most definitely making two > passes to do do that. > > > > -Hoss > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]