have you considered hadoop "light" mesagging RPC, should have significantly smaller latencies than RMI
----- Original Message ---- From: Simon Wistow <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Wednesday, 4 October, 2006 3:26:38 PM Subject: Re: Searching documents on big index by using ParallelMultiSearcher is slow... On Wed, Oct 04, 2006 at 08:14:38AM -0400, Haines, Ronald C. (LNG-DAY) said: > I too am interested in learning more about a large scale distributed > Lucene model. I'm also building a large scale (billions of documents) Lucene index. Prelimary experimentation with a RemoteSearch/ParallelMultiSearcher combination found that there were issues with the RMI causing significant blocking. I'm currently playing around with trying alternative messaging approaches so that I can also load balance requests. One of the things I haven't decided on is the optimum size for the sub indexes. Does anybody have any data on the overhead of file handle opens to index searching size or have anecodatal evidence of similar experiences? Simon --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
