Aha! Yes, Uwe, now I remember you explained this to me, that we can now do direct IO purely in java. I think we should fix up NativeUnixDirectory, and then run some more benchmarks to see if it helps? I'll open an issue.
And definitely big +1 to give us fadvise/madvise in Java so we can test that too. It's better long term to give hints to the kernel and then let it manage its buffer cache appropriately. Mike McCandless http://blog.mikemccandless.com On Tue, Sep 17, 2019 at 1:16 PM Uwe Schindler <u...@thetaphi.de> wrote: > We discussed this already on Berlinbuzzwords (Mike and Michael). Yes it's > possible and may work for merges where block io is possible. But most of us > said: it's fine to not use io cache for merging, but it won't make pages > hot. So merges are invisible to OS, so you have to warm merged segments if > you write directly. If you read directly on merging, you won't pollute > cache with one time reads, but it also won't use cache if already cached. > We should better make a proposal for f/madvise. The jdk people are open > for that, and I am jdk committer now, so I can make a prototype. > > Uwe > > Am September 17, 2019 4:48:26 PM UTC schrieb Dawid Weiss < > dawid.we...@gmail.com>: >> >> Isn't that restricted to aligned block-only access though? I can >> imagine this would complicate the implementation if somebody wanted to >> use it directly. >> >> Dawid >> >> On Tue, Sep 17, 2019 at 5:37 PM Michael McCandless >> <luc...@mikemccandless.com> wrote: >> >>> >>> Whoa! That would be awesome -- no more JNI to use Direct I/O? >>> Looks like you use it like this: >>> >>> FileChannel fc = FileChannel.open(p, StandardOpenOption.WRITE, >>> ExtendedOpenOption.DIRECT >>> >>> But it looks like you need to enable the jdk.unsupported module, added >>> with http://openjdk.java.net/jeps/260 >>> >>> Mike McCandless >>> >>> http://blog.mikemccandless.com >>> >>> >>> On Mon, Sep 16, 2019 at 11:55 AM Michael Sokolov <msoko...@gmail.com> >>> wrote: >>> >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8189192 makes it appear that >>>> Direct I/O is (or may be?) available now in JDK's since JDK10. Should >>>> we try using that API in NativeUnixDirectory in order to avoid JNI >>>> calls? >>>> ------------------------------ >>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>>> For additional commands, e-mail: dev-h...@lucene.apache.org >>>> >>>> ------------------------------ >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >> > -- > Uwe Schindler > Achterdiek 19, 28357 Bremen > https://www.thetaphi.de >
