On Thu, May 19, 2011 at 10:09 AM, Jason Rutherglen <jason.rutherg...@gmail.com> wrote: >> When >> you mmap them you let the OS decide when to swap stuff out which mean >> you pick up potentially high query latency waiting for these pages to >> swap back in > > Right, however if one is using lets say SSDs, and the query time is > less important, then MMap'ing would be fine. Also it prevents deadly > OOMs in favor of basic 'slowness' of the query. If there is no > performance degradation I think MMap'ing is a great option. A common > use case is an index that's far too large for a given server will > simply not work today, whereas with MMap'ed field caches the query > would complete, just extremely slowly. If the user wishes to improve > performance it's easy enough to add more hardware.
Well, be careful: if you just don't have enough memory to accomodate all the RAM data structures Lucene needs... you're gonna be in trouble with mmap too. True, you won't hit OOMEs anymore, but instead you'll be in a swap fest and your app is nearly unusable. SSDs, while orders of magnitude faster than spinning magnets, are still orders of magnitude slower than RAM. But, yes, they obviously help substantially. It's a one-way door... you'll never go back once you've switched to SSDs. And I do agree there are times when mmap is appropriate, eg if query latency is unimportant to you, but it's not a panacea and it comes with serious downsides. I wish I could have the opposite of mmap from Java -- the ability to pin the pages that hold important data structures. Mike http://blog.mikemccandless.com --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
