Scaling Lucene to 500 million+ documents - preferred architecture

2007-07-07 Thread muraalee
Hi Everybody, We are building a search infrastructure using lucene to scale upto 500 million document with search < 500 ms. Here is my rough math on the size of content & index : Total Documents = 500 million documents Size / Document = 10k / document Index Size / Million = 2 GB / million documen

Re: Scaling Lucene to 500 million documents - preferred architecture

2007-07-16 Thread Otis Gospodnetic
as your 8 CPU/64GB ones) and well over 1B docs. Otis -- Lucene Consulting -- http://lucene-consulting.com/ - Original Message From: muraalee <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Sent: Saturday, July 7, 2007 2:25:06 AM Subject: Scaling Lucene to 500 million documents - prefer