Another very simple alternative to using RAMDirectory is the use of RAM FS:
http://search.yahoo.com/search?p=ramfs http://search.yahoo.com/search?p=tmpfs Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- > From: Chris Hostetter <[EMAIL PROTECTED]> > To: java-user@lucene.apache.org > Sent: Tuesday, August 12, 2008 10:55:28 PM > Subject: Re: possible to read index into memory? > > > : On one index, I am seeing no speed change when flipping between > : RAMDirectory IndexSearcher and file system version. > > that is probably because even if you just use an FSDirectory, your OS will > cache the disk "pages" in RAM for you -- all using a RAMDirectory does for > you is garuntee that the entire index is copied into the heap you allocate > for your JVM. If you've got 16GB or RAM, and a 5GB index, and you > allocated 12GB of RAM to the JVM and read your index into a RAMDirectory, > your index will always be in RAM, no matter what other processes do on > your machine. > > If instead you only allocate 6GB of RAM to the JVM, and nothing else is > using up the rest of your RAM, the OS has plenty to load the whole index > into RAM as part of the filesystem cache once you use it -- but if another > process comes along and really needs that RAM (or if something reads a lot > of other pages of disk) your index might get bumped from the filesystem > cache, and the next few reads could be slow. > > : Creating the RAMDirectory from the on-disk index only takes 0.09 > : seconds. It appears it is not loading the data into memory, but maybe > : just the file names of the index? > > passing an FSDIrectory to the constructor of a RAMDIrectory uses the > Directory.copy() method whose source is fairly straight forward and easy > to read -- unless your index is ginormous it's not suprising that it's > "fast" particularly if it's already in the filesystem cache. > > > > > -Hoss > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
