UnInvertedField performance on faceted fields containing many unique terms

2009-06-15 Thread Kent Fitch
Hi, This may be of interest to other users of SOLR's UnInvertedField who have a very large number of unique terms in faceted fields. Our setup is : - about 34M lucene documents of bibliographic and full text content - index currently 115GB, will at least double over next 6 months - moving to

Re: UnInvertedField performance on faceted fields containing many unique terms

2009-06-15 Thread Yonik Seeley
Great writeup Ken, All the constants you see in UnInvertedField were a best guess - I wasn't working with any real data. It's surprising that a big array allocation every 4096 terms is so significant - I had figured that the work involved in processing that many terms would far outweigh

Re: UnInvertedField performance on faceted fields containing many unique terms

2009-06-15 Thread Kent Fitch
Hi Yonik, On Tue, Jun 16, 2009 at 10:52 AM, Yonik Seeleyyo...@lucidimagination.com wrote: All the constants you see in UnInvertedField were a best guess - I wasn't working with any real data.  It's surprising that a big array allocation every 4096 terms is so significant - I had figured that