Hi Stuart,
On 26/09/18 03:19, Stuart Marks wrote:
> I've been starting to look at some of the buffer-related issues and I've
> been discussing this issue with Alan.
I'd be interested to hear more details if the discussion has gone far
enough for any of it to be aired online.
> On 9/25/18 2:01 AM, Andrew Haley wrote:
>> . . .
>> I'm baffled by this assertion. It's true that the 2Gb limit is turning
>> into a real pain, but all of the rest are nothing to do with
>> ByteBuffers, which are just raw memory. Adding structure is something
>> that can be done by third-party libraries or by some future OpenJDK
>> API.
>
> If you look around Java SE for a public API to represent raw memory,
> then MappedByteBuffer is the obvious choice; there isn't any realistic
> alternative right now. By asking whether MBB is "the right way to
> expose" raw memory, I think Alan is really saying, is MBB the best API
> to use to expose raw memory in the long run? I think the answer is
> clearly No.
Sorry, it may well be my fault but it's not really clear to me. You
mention two issues below, buffer size limit and API verbosity.
I acknowledge the former is a problem but i) there is a proposal
(JDK-8180628, referred to in the JEP) to deal with this limitation by
adding extra methods that support the creation of larger buffers and use
of long indices and ii) there are existing Java libraries built over
ByteBuffer that overcome this issue (as Sandhya pointed out in a note
somewhere near this one). Sure, both of these remedies have limitations
which /might/ lead to problems but I don't see (yet, at least) that they
are manifestly unworkable.
As regards the latter issue, I am not really sure what you are
suggesting would be a better alternative to using ByteBuffer get and put
methods? Are you perhaps thinking of some way of overlaying a record (or
object?) structure over the mapped memory that might allow a compiler to
provide an equivalent to these ByteBuffer method calls as direct memory
loads and stores? Of course, a Java library built on top of this
proposal could provide a similar abstraction, although perhaps not with
as firm guarantees for compiler optimization and certainly not with the
possibility of direct language integration.
Copying might indeed be an issue but surely that depends on the type of
data being written, the library design and the way the client needs to
operate in order to use it (essentially on whether it can size in
advance a data area in which to write the contents direct vs build a
separate copy as distinct pieces and then serialize them).
Anyway, I hope the above explains why I'm not sure about your use of the
the words 'clearly' or (in a in a later comment) 'insurmountable'.
Perhaps more details of your conversation with Alan would help.
> There are, however, certain things that can be done with buffers in the
> short term to make things work better. For example, JDK-5029431 absolute
> bulk put/get methods. I suspect this will be quite helpful for the NVM
> case, and indeed it's been something that's been asked for repeatedly
> for quite some time.
Would it be enough to add a comment to the Risks and Assumptions of the
JEP to point out this limitation and the potential need to address it,
mentioning this specific JDK issue -- much as was done with JDK-8180628.
> If you (collectively) are aware of this and other limitations, then
> sure, let's proceed with this JEP.
Well, I'm very happy to proceed if Alan is in agreement. One thing he
suggested in an earlier post was splitting off the functionality to
create a persistent ByteBuffer into a separate method so as to avoid any
issues if we have to deprecate this model at a alter date. I think
that's a quite reasonable precaution and I'd be happy to propose an
alternative API or let Alan suggest one. Perhaps Alan can comment?
> Agree, "waiting for Panama" is certainly not something that anyone wants
> to do.
Yes, indeed, there are already several important use cases waiting in
the wings.
regards,
Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in England and Wales under Company Registration No. 03798903
Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander
