Hello Steve, Thank you for addressing all concerns raised in the discussion.
For my points: Using BufferSupplier or similar as the interface name can be mixed with class/constructor names which describe the actual type of memory. However yes I agree that “Memory” has the additional benefit that it can be used directly as a postfix in the class name, so thinking more about it, I agree with the nice and concrete “Memory” name. I shortened the method to “allocate()”, but you are right it might be better to make it a bit more specific if extensions are to be expected. My main motivation behind a long key was, that in most scenarios which come to my mind (NVRAM, Flash, mmaped, shmem) the position of the allocated segment is relevant for cross VM (concurrent or sequential, as in the persisted case) sharing. As a ByteBuffer is not only a window into those memory segments but also a active datastructure (pos) it would be ugly to request only a single buffer to cover a large region. Especially with a 32bit size limit. But yes it might also be possible to setup the position/sharing details with the Memory constructor request a large ByteBuffer and then split the actual buffers mapped on top of it (and do the allocation management based on start positions by hand) The meaning of size in combination with the allocation key would be provider dependent. One option would be to refuse overlapping allocations but a more common implementation would simply allocate the segments (and if two VMs use overlapping regions they will exactly receive them). In all cases it should describe the capacity of the buffer. Greetings bernd -- http://bernd.eckenfels.net Von: Dohrmann, Steve Gesendet: Mittwoch, 6. April 2016 03:59 An: core-libs-dev@openjdk.java.net Betreff: JDK 9 proposal: allocating ByteBuffers on heterogeneous memory Re: JDK-8153111<https://bugs.openjdk.java.net/browse/JDK-8153111> Hi, Below are responses to some of the points brought up in the discussion as well as is a little expansion of the reasoning that went into the proposed API. One motivation we saw for doing anything beyond a concrete ByteBuffer class was application code (e.g. Cassandra) that allocates many off-heap ByteBuffers using ByteBuffer#allocateDirect. We reasoned that if the allocation of ByteBuffers could be done using a common memory interface then only the code that provisioned instances of the the memory spaces would have to change in order to switch or mix memory types. We did think of overloading the ByteBuffer#allocateDirect method with memory space info and avoid an allocation interface. We ended up with a separate user called interface scheme because we imagined that extensions of the memory interface would enable new memory functionality that required new methods (e.g. memory transactions for persistence). Without a separate callable interface, the static method space in ByteBuffer might have to change again. For any API In general we saw the need for two things 1) something to represent a memory space from which objects are allocated (a Memory instance) and 2) a broadly usable familiar "anchor" type as the common data object (java.nio.ByteBuffer). The two points for extension with the current proposal are: 1) Constructors on Memory implementation classes -- allow implementors the ability to offer features on the memory space itself (e.g. partitioning, size limits) and 2) future extensions on the Memory interface -- for example PersistentMemory. Regarding a more elaborate scheme for memory space creation -- we did consider a factory scheme for memory object allocation but could not get comfortable with either a) standardized method signatures suitable for various kinds of memory or b) the complexity that something like a general-purpose "spec" format would add, even if we could come up with it. Direct construction does expose more to the user but it seems sufficient and might be the best given what we can say about the future of heterogeneous memory at this point. Regarding the suggested addition of keyed access to ByteBuffers, we are assuming this is only proposed to enable sharing? We thought it might take a while to properly explore the details (i.e. semantics / thread safety / predicable performance) of sharing such that it would work well and maybe even extend well to things like process-shared and cluster-shared ByteBuffers. We elected to propose nothing for JDK 9 beyond what developers can already do with schemes based on e.g. ByteBuffer#duplicate. We were thinking shared buffers could appear later, possibly as an extension of the Memory interface. The keyed access scheme is simple and appealing, however. One question: how is the request method's size parameter to be interpreted? The suggestion of parameterizing the Memory interface bounded by ByteBuffers seems useful as it gives a clean way to support extended ByteBuffers. Not sure if the change of the allocation method name from #allocateByteBuffer to #allocate was incidental or not? Alternates to the "Memory" interface name might be preferred, BufferSupplier is certainly reasonable. We imagined instances that name different memory spaces (e.g. OffHeapRAM) for allocation rather that the role played -- thinking the role is explicit in the allocation method name (allocateByteBuffer). Regarding changing the allocation size parameter to a long, this would be very nice to have. We avoided it in order to match the existing ByteBuffer API. If 64-bit ByteBuffers are planned, sticking with int sizes might have been the wrong call. We are still coming up to speed on VarHandles (JEP 193), atomics for ByteBuffers (JDK-8008240), and ByteBuffer consistency (JDK-6476827). We hope there is continued interest in this proposal and happy to provide a modified patch. Best regards, Steve Dohrmann
