Thanks Remi. Of course that would work, but the goal was to try VarHandles as generic way to access on-heap and off-heap structures. Maybe Paul Sandoz has an answer to this API inconsistency.
Uwe Am 23. Februar 2017 22:31:35 MEZ schrieb [email protected]: >Hi Uwe, >I've not the definitive answer, but you can create a MethodHandle on >ByteBuffer::get and see a VarHandle as a MethodHandle (with >VarHandler.toMethodHandle) so both access can be represented as method >handles. > >Rémi > >[1] >http://download.java.net/java/jdk9/docs/api/java/lang/invoke/VarHandle.html#toMethodHandle-java.lang.invoke.VarHandle.AccessMode- > >----- Mail original ----- >> De: "Uwe Schindler" <[email protected]> >> À: "Remi Forax" <[email protected]>, "Vitaly Davidovich" ><[email protected]> >> Cc: "jigsaw-dev" <[email protected]> >> Envoyé: Jeudi 23 Février 2017 19:16:44 >> Objet: RE: sun.nio.ch.DirectBuffer and jdk9/jigsaw > >> Hi, >> >>> I believe that what you are looking for is >>> MethodHandles.byteBufferViewVarHandle [1] that creates a VarHandle >that >>> allows to do aligned/unaligned access on a ByteBuffer. >> >> Related question: I wonder why there is now way to get a view on a >ByteBuffer as >> a byte[]-typed Varhandle (byte[] and boolean[] views are explicitly >excluded - >> why????)? It throws UnsupportedOperationExeception! To me this is >inconsistent. >> You can get a Varhandle for byte[] arrays, but not ByteBuffers. I >know it >> brings not much, but you would have the possibility to have "generic >code" for >> all types. >> >> And by looking at the source code, there is no reason to not generate >the byte[] >> view, in fact it is already available in the generated class for >array-views. >> >> Should I open a feature request? This was something we were stuck >when trying to >> use VarHandles inside Lucene to access memory mapped files as >VarHandles (would >> be really cool). But we need a byte[] view, too!!! >> >> Uwe >> >>> It should be a little slower that using Unsafe because of the bound >check >>> (and the nullcheck but it is usually removed by the VM). >>> That's said, unaligned access are faster with a VarHandle because >>> sun.misc.Unsafe (which is stuck in the Java 8 world unlinke >>> jdk.internal.misc.Unsafe) does not provide a way to do an unaligned >access >>> so you have do multiple read which is slower. >>> >>> cheers, >>> Rémi >>> >>> >http://download.java.net/java/jdk9/docs/api/java/lang/invoke/MethodHan >>> >dles.html#byteBufferViewVarHandle-java.lang.Class-java.nio.ByteOrder- >>> >>> ----- Mail original ----- >>> > De: "Vitaly Davidovich" <[email protected]> >>> > À: "Uwe Schindler" <[email protected]> >>> > Cc: "jigsaw-dev" <[email protected]> >>> > Envoyé: Jeudi 23 Février 2017 18:05:36 >>> > Objet: Re: sun.nio.ch.DirectBuffer and jdk9/jigsaw >>> >>> > On Thu, Feb 23, 2017 at 11:54 AM, Uwe Schindler >>> <[email protected]> >>> > wrote: >>> > >>> >> Hi, >>> >> >>> >> Why do you need the address at all in the Java code? Java code >can use >>> the >>> >> official ByteBuffer methods to access the memory you are >wrapping. In >>> Java >>> >> 9 that’s optimized very good by Hotspot and should be almost as >fast as >>> >> array accesses (we proved that in Apache Lucene - congrats to the >Hotspot >>> >> committers). If you need special access modes like volatile >access, then >>> >> you can use Java 9's VarHandles. You can get a VarHandle to the >backing >>> >> direct buffer using the MethodHandles API. >>> >> >>> > I mentioned this upthread - the base address is used for index >calculations >>> > to read/write data using Unsafe directly. I don't know about Java >9 as >>> > I've not tried it yet, but the generated assembly for using BB >methods vs >>> > Unsafe did not favor BB. There are also various safety checks in >>> > DBB/Buffer internals that won't get optimized away. >>> > >>> > Also, my general experience with looking at C2 optimizations has >led me to >>> > the conclusion that the optimizations are "unstable" - innocent >code >>> > changes, differences in order of how classes are loaded, >differences in >>> > what callsites trigger compilation first, and a bunch of other >otherwise >>> > benign things can interfere with inlining decisions, which is >typically the >>> > reason things go sideways in terms of optimization. >>> > >>> > As for MethodHandles and VarHandles, that's one possibility I'm >>> considering >>> > as a way to migrate off using DirectBuffer (although I'd still >like to know >>> > if there's any plan to standardize/formalize some notion of a >direct >>> > buffer). However, my understanding is that using MH will still >require me >>> > to crack into jdk code (to get access to the DBB and friends) and >thus >>> > require addExports. DirectBuffer is still accessible if using >addExports, >>> > but it's a wrinkle I was hoping to iron out, hence what started >this email >>> > chain. >>> > >>> >> >>> >> Uwe >>> >> >>> >> ----- >>> >> Uwe Schindler >>> >> [email protected] >>> >> ASF Member, Apache Lucene PMC / Committer >>> >> Bremen, Germany >>> >> http://lucene.apache.org/ >>> >> >>> >> > -----Original Message----- >>> >> > From: jigsaw-dev [mailto:[email protected]] >On >>> Behalf >>> >> > Of Vitaly Davidovich >>> >> > Sent: Thursday, February 23, 2017 5:30 PM >>> >> > To: Chris Hegarty <[email protected]> >>> >> > Cc: jigsaw-dev <[email protected]> >>> >> > Subject: Re: sun.nio.ch.DirectBuffer and jdk9/jigsaw >>> >> > >>> >> > On Thu, Feb 23, 2017 at 11:10 AM, Chris Hegarty >>> >> > <[email protected]> >>> >> > wrote: >>> >> > >>> >> > > >>> >> > > > On 23 Feb 2017, at 11:30, Vitaly Davidovich ><[email protected]> >>> >> wrote: >>> >> > > >> ... >>> >> > > > The buffers are reused by having them point to different >native >>> >> memory >>> >> > > > block addresses; those blocks are managed by native code. >As >>> >> > mentioned, >>> >> > > > the ByteBuffer (DirectByteBuffer concretely) is used as the >Java >>> >> level >>> >> > > > interface/view of native memory, allowing Java and native >code to >>> >> > > > communicate. >>> >> > > >>> >> > > So a DBB, under your code, may report a different address at >some >>> time >>> >> > > in the future, to that of what it currently reports? >>> >> > >>> >> > Correct. >>> >> > >>> >> > > I was not aware of this >>> >> > > usecase. Is any similar code available on the web, or >elsewhere, so we >>> >> > > could try to determine why this is being done? >>> >> > > >>> >> > Unfortunately it's not open source code, and I don't >immediately know >>> of >>> >> > anything similar on the web (or otherwise). However, the gist >is the >>> >> > following: >>> >> > 1) Allocate a 0-size DBB (i.e. ByteBuffer.allocateDirect(0)). >This gives >>> >> > you a Java "handle", if you will, to some native memory. But, >since this >>> >> > DBB will be attached/reattached to different memory >dynamically, >>> there's >>> >> no >>> >> > need for an actual allocation. >>> >> > 2) Native code wants to expose a segment of memory to Java. In >JNI, it >>> >> > sets the address and capacity of this DBB to the pointer where >the native >>> >> > memory segment starts, and to the capacity (it knows how big >the native >>> >> > segment is). Java code asks for this DBB to be "attached" to, >say, some >>> >> > sort of message, and the JNI/native code perform these >functions. >>> >> > 3) Java gets the attached DBB back, and can then use its API >>> >> > (getXXX/setXXX) to read/write that native block. Once the >operation >>> >> > completes, the DBB is recycled for reuse (i.e. can be attached >to a >>> >> > different native segment again). >>> >> > >>> >> > Obviously, we can use >>> >> > >>> >http://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions >>> >> . >>> >> > html#GetDirectBufferAddress >>> >> > to get the address and then expose that via a JNI helper - in >fact, >>> >> that's >>> >> > what was done before. But, there's a JNI call penalty here for >what is >>> >> > otherwise a memory read. DirectBuffer::address() solves that >nicely, >>> and >>> >> > also plays well with the C2 JIT (as mentioned) because the >callsites >>> >> where >>> >> > this is used only see DBB, and then the whole invokeinterface >call is >>> >> > devirtualized and inlined into a quick type check and Java >field read - >>> >> the >>> >> > performance of this is, as you can imagine, significantly >better than the >>> >> > JNI approach. >>> >> > >>> >> > If you think of what a DBB really is, it's pretty much what >it's name >>> >> > suggests - it's an API to read/write to native memory, rather >than Java >>> >> > heap memory (i.e. HeapByteBuffer). But, there's no reason the >native >>> >> > memory backing the DBB has to also be allocated via Unsafe >itself, >>> >> although >>> >> > that's the more common scenario. >>> >> > >>> >> > On the Java side, consumers of this have a common and >conventional >>> API >>> >> > over >>> >> > a byte buffer, i.e. ByteBuffer, which can optionally be used in >the >>> >> manner >>> >> > above (obviously callers will need to know what mode they're >using). >>> >> > >>> >> > >>> >> > > -Chris. >>> >> > > >>> >> > >
