Hmm... posix_fadvise lets you do this with a file descriptor; this would be better for Lucene (per descriptor not per mapped region of RAM) since we could "advise" independent of which FSDir impl is in use...
Mike On Tue, Jun 16, 2009 at 10:32 AM, Uwe Schindler<[email protected]> wrote: > But to use it, we should change MMapDirectory to also use the mapping when > writing to files. I thought about it, it is very simple to implement (just > copy the IndexInput and change all gets() to sets()) > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: [email protected] > >> -----Original Message----- >> From: Michael McCandless [mailto:[email protected]] >> Sent: Tuesday, June 16, 2009 4:22 PM >> To: [email protected] >> Cc: Alan Bateman; [email protected] >> Subject: Re: madvise(ptr, len, MADV_SEQUENTIAL) >> >> Lucene could really make use of this method. When a segment merge >> takes place, we can read & write many GB of data, which without >> madvise on many OSs would effectively flush the IO cache (thus hurting >> our search performance). >> >> Mike >> >> On Mon, Jun 15, 2009 at 6:01 PM, Jason >> Rutherglen<[email protected]> wrote: >> > Thanks Alan. >> > >> > I cross posted this to the Lucene dev list where we are discussing using >> > madvise for minimizing unnecessary IO cache usage when merging segments >> > (where we really want the newly merged segments in the IO cache rather >> than >> > the old segment files). >> > >> > How would the advise method work? Would there need to be a hint in the >> > FileChannel.map method? >> > >> > -J >> > >> > On Mon, Jun 15, 2009 at 12:36 AM, Alan Bateman <[email protected]> >> wrote: >> >> >> >> Jason Rutherglen wrote: >> >>> >> >>> Is there going to be a way to do this in the new Java IO APIs? >> >> >> >> Good question, as it has come up a few times and is needed for some >> >> important use-cases. A while back I looked into adding a >> >> MappedByteBuffer#advise method to allow the application provide hints >> on the >> >> expected usage but didn't complete it. We should probably look at this >> again >> >> for jdk7. >> >> >> >> -Alan. >> >> >> > >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
