Another option is to use tmpfs (memory driven file system) as a backing storage for index. This allows to minimize disc access latency. In average case this doesn't make sense, because OS will cache file system access quite effective if you have enough memory. But in worst case (if you measure response time 99 percentile or higher, for example) this method can give you more stable performance.
On Jul 20, 2012, at 10:32 PM, Dragon Fly <[email protected]> wrote: > > Thank you. > >> From: [email protected] >> Date: Thu, 19 Jul 2012 13:34:26 +0200 >> Subject: Re: RAM or SSD... >> To: [email protected] >> >> Read this: >> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html >> >> Dawid >> >> On Thu, Jul 19, 2012 at 1:32 PM, Dragon Fly <[email protected]> >> wrote: >>> >>> The slowest part of my application is to read the search hits from disk. I >>> was hoping that using an SSD or RAMDirectory/MMapDirectory would speed that >>> up. I read the JavaDoc for MMapDirectory but didn't really understand how >>> that differs from RAMDirectory. Could someone please explain? >>> >>>> Date: Wed, 18 Jul 2012 23:46:51 -0700 >>>> Subject: Re: RAM or SSD... >>>> From: [email protected] >>>> To: [email protected] >>>> >>>> I was referring to *RAMDirectory*. >>>> >>>> On Wed, Jul 18, 2012 at 11:04 PM, Lance Norskog <[email protected]> wrote: >>>>>> You do not want to store 30 G of data in the JVM heap, no matter what >>>> library does this. >>>>> MMapDirectory does not store data in the JVM heap. It lets the >>>>> operating system manage the disk buffer space. Even if the JVM says "I >>>>> have 30G of memory space", it really does not. It only has address >>>>> space allocated by the OS but no memory. >>>>> >>>>> On Wed, Jul 18, 2012 at 10:39 PM, Toke Eskildsen >>>>> <[email protected]> >>>> wrote: >>>>>> On Wed, 2012-07-18 at 17:50 +0200, Dragon Fly wrote: >>>>>>> If I want to improve performance, which of the following is better and >>>> why? >>>>>>> >>>>>>> 1. Buy a machine with a lot of RAM and use a RAMDirectory for the index. >>>>>> >>>>>> As others has pointed out, MMapDirectory should work better than >>>>>> RAMDirectory. I am sure it will work fine with a relative small index >>>>>> such as yours. However, it does not scale that well with index size. >>>>>> >>>>>>> 2. Put the index on a solid state drive. >>>>>> >>>>>> Why anyone buys computers without SSD's is a mystery to me. Use SSDs for >>>>>> the small low-latency stuff and a secondary spinning drive for the large >>>>>> slow stuff. Nowadays, a 30GB index (or 100GB for that matter) falls into >>>>>> the small low-latency bucket. SSDs speeds up almost everything, saves >>>>>> RAM and spares a lot of work hours optimizing I/O-speed. >>>>>> >>>>>> Regards, >>>>>> Toke Eskildsen >>>>>> >>>>>> >>>>>> --------------------------------------------------------------------- >>>>>> To unsubscribe, e-mail: [email protected] >>>>>> For additional commands, e-mail: [email protected] >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Lance Norskog >>>>> [email protected] >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: [email protected] >>>>> For additional commands, e-mail: [email protected] >>>>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > --- Denis Bazhenov <[email protected]> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
