Except, you don't know the size of the file to be written upfront. One probable solution is to map output file in pages. As a complementary solution you can map a huge area of the file, and hope few real memory is allocated by OS unless you actually write all over that area. Dunno. The idea of using mmapped write has stopped looking interesting to me.
On Tue, Jun 16, 2009 at 18:32, Uwe Schindler<[email protected]> wrote: > But to use it, we should change MMapDirectory to also use the mapping when > writing to files. I thought about it, it is very simple to implement (just > copy the IndexInput and change all gets() to sets()) > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: [email protected] > >> -----Original Message----- >> From: Michael McCandless [mailto:[email protected]] >> Sent: Tuesday, June 16, 2009 4:22 PM >> To: [email protected] >> Cc: Alan Bateman; [email protected] >> Subject: Re: madvise(ptr, len, MADV_SEQUENTIAL) >> >> Lucene could really make use of this method. When a segment merge >> takes place, we can read & write many GB of data, which without >> madvise on many OSs would effectively flush the IO cache (thus hurting >> our search performance). >> >> Mike >> >> On Mon, Jun 15, 2009 at 6:01 PM, Jason >> Rutherglen<[email protected]> wrote: >> > Thanks Alan. >> > >> > I cross posted this to the Lucene dev list where we are discussing using >> > madvise for minimizing unnecessary IO cache usage when merging segments >> > (where we really want the newly merged segments in the IO cache rather >> than >> > the old segment files). >> > >> > How would the advise method work? Would there need to be a hint in the >> > FileChannel.map method? >> > >> > -J >> > >> > On Mon, Jun 15, 2009 at 12:36 AM, Alan Bateman <[email protected]> >> wrote: >> >> >> >> Jason Rutherglen wrote: >> >>> >> >>> Is there going to be a way to do this in the new Java IO APIs? >> >> >> >> Good question, as it has come up a few times and is needed for some >> >> important use-cases. A while back I looked into adding a >> >> MappedByteBuffer#advise method to allow the application provide hints >> on the >> >> expected usage but didn't complete it. We should probably look at this >> again >> >> for jdk7. >> >> >> >> -Alan. >> >> >> > >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- Kirill Zakharenko/Кирилл Захаренко ([email protected]) Home / Mobile: +7 (495) 683-567-4 / +7 (903) 5-888-423 ICQ: 104465785 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
