Re: notes from Valhalla meeting 5/24/17

[Paul Sandoz]

> On 21 Jun 2017, at 01:46, John Rose  wrote:
> 
> On Jun 19, 2017, at 6:28 PM, John Rose  wrote:
>> 
>> I have updated the javadoc portion of the spec for constant-dynamic.
>> There are small but strategic changes to MethodType and MethodHandle,
>> as well as new classes to represent the "pull mode" of constant resolution.
>> The key changes relevant to the JVM are in package-info.html,
>> and (of course) in the forthcoming JVM spec. changes.
>> 
>> http://cr.openjdk.java.net/~jrose/jvm/specdiff-condy-2017-0619.zip
>> 
> 
> The forthcoming has come forth.  Here is a semi-formatted and
> color-annotated diff of the JVMS outlining proposed support for
> dynamic constants, as well as some enhancements to the processing
> of bootstrap methods:
> 
> http://cr.openjdk.java.net/~jrose/jvm/condy-jvms-2017-0620.html
> 

Nice, a thorougher set of changes.

I like the trigger on arity. There is no dependency on the signature of the BSM 
(although as an implementation detail we can optimize).

Trivially (perhaps as note?) a BSM is not limited to being solely referenced by 
a CONSTANT_ConstantDynamic_info or a CONSTANT_InvokeDynamic_info. Just like 
previously a BSM was not limited to being solely referenced by one kind of 
CONSTANT_InvokeDynamic_info.

It’s now much easier to share BSMs if that is ones desire.

Paul.

Re: notes from Valhalla meeting 5/24/17

[Paul Sandoz]

> On 21 Jun 2017, at 09:03, John Rose  wrote:
> 
> On Jun 21, 2017, at 8:46 AM, Remi Forax  > wrote:
>> 
>> but i do not like with this proposal as is, i will explain why and how to 
>> fix it:
>> - condy is linked to a static final field but unlike invokedynamic which is 
>> a link from an invokedynamic instruction to a CONSTANT_InvokeDynamic_info,
>>  there is no link from the static final field to the 
>> CONSTANT_ConstantDynamic_info.
>>  Why not reuse the ConstantValue attribute [1] to reference the 
>> CONSTANT_ConstantDynamic_info instead (the constantvalue_index can be 
>> extended to allow a CONSTANT_ConstantDynamic).
>> 
>> - condy if a 'dy' like indy, so it should do late late binding, i.e. being 
>> initialized (run the bootstrap method) only the first time someone access to 
>> the static field exactly like with indy the bsm is called the first time you 
>> try to access the instruction.
>> 
>> 
>> In term of semantics, my proposal does not introduce an item in the constant 
>> pool which is resolved only by the virtue of being in the constant pool 
>> unlike any other items. If condy is linked to the ConstantValue of a field, 
>> the condy item is resolved when necessary as usual. With my ASM hat, i see 
>> how to implement it easily without having to surface the constant pool 
>> itself (at least until the items are pointed by the j.l.i.BootstrapCallInfo).
> 
> Indeed, repurposing ConstantValue in the way you describe is an add-on to 
> this proposal.

Can we get away with changing all static final fields to be lazily initialized 
without some explicit opt-in? It would be nice but it might induce subtle 
changes in behaviour and expectations (especially for where exceptions may 
occur).

Paul.

> I almost threw it in, but didn't want to muddy the basic proposal.
> In the basic proposal, condy is *not* linked to static finals.
> It only repurposes the concept of field names and field types
> (as if from Fieldref but not using Fieldref) but does not actually link to 
> fields.
> 
> — John

Re: notes from Valhalla meeting 5/24/17

[Paul Sandoz]

> On 19 Jun 2017, at 18:28, John Rose  wrote:
> 
> On Jun 9, 2017, at 1:51 PM, John Rose  wrote:
>> 
>> We have talked about "condy" (a constant-pool friend for indy).
>> Enclosed is the javadoc portion of a draft spec.
>> The JVM spec. is not ready and will be sent separately.
>> This javadoc reflects code checked into the condy branch of Amber.
> 
> I have updated the javadoc portion of the spec for constant-dynamic.
> There are small but strategic changes to MethodType and MethodHandle,
> as well as new classes to represent the "pull mode" of constant resolution.
> The key changes relevant to the JVM are in package-info.html,
> and (of course) in the forthcoming JVM spec. changes.
> 
> http://cr.openjdk.java.net/~jrose/jvm/specdiff-condy-2017-0619.zip
> 


package-summary
—

"Bytecode may contain dynamic call sites equipped with equipped with bootstrap 
methods"

s/equipped with equipped with/equipped with/


"This allows the bootstrap logic the ability to order the resolution of 
constants and catch linkage exceptions.”

And possibly still link a constant rather than re-throwing a linkage exception?


"The second-to-last example assumes that all extra arguments are of type 
CONSTANT_String”

s/CONSTANT_String/String


BootstrapCallInfo
—

"This information include the method”

s/include/includes

Paul.

> (I've given up sending attachments for now since the server
> scrubs them.)
> 
> — John

Re: notes from Valhalla meeting 5/24/17

[Paul Sandoz]

> On 21 Jun 2017, at 12:37, fo...@univ-mlv.fr wrote:
> 
> 
> 
> De: "John Rose" 
> À: "Rémi Forax" 
> Cc: "Valhalla Expert Group Observers" 
> , 
> valhalla-spec-experts@openjdk.java.net
> Envoyé: Mercredi 21 Juin 2017 18:03:49
> Objet: Re: notes from Valhalla meeting 5/24/17
> 
> [...]
> It only repurposes the concept of field names and field types
> (as if from Fieldref but not using Fieldref) but does not actually link to 
> fields.
> 
> — John
> 
> Ok, i do not understand exactly what you mean here,
> i will do a complete reading of the spec tomorrow so i hope things will be 
> more clear for me.
> 

Relevant bit:

  CONSTANT_ConstantDynamic_info {
  u1 tag;
  u2 bootstrap_method_attr_index;
  u2 name_and_type_index;
  }

  The value of the name_and_type_index item must be a valid index into the 
constant_pool table. The constant_pool entry at
  that index must be a CONSTANT_NameAndType_info structure (§4.4.6) 
representing a field name and field descriptor (§4.3.2).

i.e. name and descriptor indexed by CONSTANT_NameAndType_info must conform to a 
field name and field descriptor respectively, but there is no such actual field.

Hth,
Paul.

Re: notes from Valhalla meeting 5/24/17

[Paul Sandoz]

> On 21 Jun 2017, at 16:40, John Rose  wrote:
> 
> Thanks; all fixed.
> 
> On Jun 21, 2017, at 1:45 PM, Paul Sandoz  wrote:
>> 
>> 
>> "This allows the bootstrap logic the ability to order the resolution of 
>> constants and catch linkage exceptions.”
>> 
>> And possibly still link a constant rather than re-throwing a linkage 
>> exception?
>> 
> 
> No, there's no way to force the JVM to change a linkage decision once made.
> 

A BSM called to resolve a constant can choose when pulling it’s static args to 
catch and swallow linkage exceptions, keep on trucking, and return a value. 
Should this be allowed?

Paul.

Re: notes from Valhalla meeting 5/24/17

[Paul Sandoz]

> On 21 Jun 2017, at 16:49, John Rose  wrote:
> 
> On Jun 21, 2017, at 4:44 PM, Paul Sandoz  <mailto:paul.san...@oracle.com>> wrote:
>> 
>> A BSM called to resolve a constant can choose when pulling it’s static args 
>> to catch and swallow linkage exceptions, keep on trucking, and return a 
>> value. Should this be allowed?
> 
> Absolutely.  The BSM might have an API-specific fallback.
> 

Thanks, i thought so.

The returned value could capture the ConstantGroup and do more resolution 
lazily on demand. At some point if the value is tickled in the right way a 
linkage error might result.

Paul.

> I'm also thinking that we could add a query for the original symbolic 
> reference,
> which the BSM could look and and perhaps resolve "by hand".
> This is similar in spirit to the original "invokedynamic" proposals
> as a form of "messageNotUnderstood".
> 
> — John

Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

> On 23 Jun 2017, at 13:33, Karen Kinnear  wrote:

> VWithfield - propose for MVT - allow package private access - since there are 
> no methods on the derived value class
>and the value capable class can’t have any methods with vbytecodes since 
> generated by javac
>- plan to make private when we add factory methods to value classes with a 
> compiler (and we have nest support)
> 

I am unsure if it’s necessary for MVT purposes to dial back the accessibility 
then dial it up again later on.

ValueType.findWither can be used in conjunction with 
MethodHandle.privateLookupIn. It’s a little odd but works. What am i missing?

Paul.

Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

> On 5 Jul 2017, at 07:26, Karen Kinnear  wrote:
> 
> Paul,
> 
> What we were discussing was the ability to use the byte code itself - not the 
> ValueType.findWither API.
> John’s longer term plan is that ultimately the byte code can only be executed 
> in the value class itself, and
> since the derived value class has no methods, we need a temporary approach.
> 
> Did I misunderstand what you were saying?
> 

No, i was missing aspect that you were referring to byte code generation.

What if we added a safe replacement for:

  Unsafe.defineAnonymousClass
  https://bugs.openjdk.java.net/browse/JDK-8171335

(Which we anyway have to do.)

?

Then the VCC or DVT could be used as the host class. However, i dunno if that 
would be sufficient to cover the use-cases of byte code generation.

Paul.

> thanks,
> Karen
> 
>> On Jun 26, 2017, at 2:52 PM, Paul Sandoz  wrote:
>> 
>> 
>>> On 23 Jun 2017, at 13:33, Karen Kinnear  wrote:
>> 
>>> VWithfield - propose for MVT - allow package private access - since there 
>>> are no methods on the derived value class
>>>  and the value capable class can’t have any methods with vbytecodes since 
>>> generated by javac
>>>  - plan to make private when we add factory methods to value classes with a 
>>> compiler (and we have nest support)
>>> 
>> 
>> I am unsure if it’s necessary for MVT purposes to dial back the 
>> accessibility then dial it up again later on.
>> 
>> ValueType.findWither can be used in conjunction with 
>> MethodHandle.privateLookupIn. It’s a little odd but works. What am i missing?
>> 
>> Paul.
>> 
>> 
>> 
>> 
> 

Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

I strongly suspect we can specify a safe version of Unsafe.defineAnonymousClass 
(minus constant pool patching) independent of nest mates. I believe that is 
desirable on it’s own as part of the replace unsafe functionality and if that 
can help MVT then even better!

Paul.

> On 5 Jul 2017, at 10:48, Karen Kinnear  wrote:
> 
> Agree with John’s clarification - yes we are planning longterm for nest mate 
> access.
> And your proposal of using a safe replacement for Unsafe.defineAnonymousClass 
> with
> appropriate access to add into the nest makes sense.
> 
> At this time we are building an Early Access that needs to go out sooner than 
> nest mates.
> We should re-evaluate adding a nest mate dependency when we get closer to 
> deadlines and
> see if that works for our partners and our own timing.
> 
> thanks,
> Karen
> 
>> On Jul 5, 2017, at 12:28 PM, Paul Sandoz  wrote:
>> 
>> 
>>> On 5 Jul 2017, at 07:26, Karen Kinnear  wrote:
>>> 
>>> Paul,
>>> 
>>> What we were discussing was the ability to use the byte code itself - not 
>>> the ValueType.findWither API.
>>> John’s longer term plan is that ultimately the byte code can only be 
>>> executed in the value class itself, and
>>> since the derived value class has no methods, we need a temporary approach.
>>> 
>>> Did I misunderstand what you were saying?
>>> 
>> 
>> No, i was missing aspect that you were referring to byte code generation.
>> 
>> What if we added a safe replacement for:
>> 
>> Unsafe.defineAnonymousClass
>> https://bugs.openjdk.java.net/browse/JDK-8171335
>> 
>> (Which we anyway have to do.)
>> 
>> ?
> 
>> 
>> Then the VCC or DVT could be used as the host class. However, i dunno if 
>> that would be sufficient to cover the use-cases of byte code generation.
> 
>> 
>> Paul.
>> 
>>> thanks,
>>> Karen
>>> 
>>>> On Jun 26, 2017, at 2:52 PM, Paul Sandoz  wrote:
>>>> 
>>>> 
>>>>> On 23 Jun 2017, at 13:33, Karen Kinnear  wrote:
>>>> 
>>>>> VWithfield - propose for MVT - allow package private access - since there 
>>>>> are no methods on the derived value class
>>>>> and the value capable class can’t have any methods with vbytecodes since 
>>>>> generated by javac
>>>>> - plan to make private when we add factory methods to value classes with 
>>>>> a compiler (and we have nest support)
>>>>> 
>>>> 
>>>> I am unsure if it’s necessary for MVT purposes to dial back the 
>>>> accessibility then dial it up again later on.
>>>> 
>>>> ValueType.findWither can be used in conjunction with 
>>>> MethodHandle.privateLookupIn. It’s a little odd but works. What am i 
>>>> missing?
>>>> 
>>>> Paul.
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>> 
> 

Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

> On 5 Jul 2017, at 11:06, John Rose  wrote:
> 
> On Jul 5, 2017, at 10:59 AM, Paul Sandoz  <mailto:paul.san...@oracle.com>> wrote:
>> 
>> I strongly suspect we can specify a safe version of 
>> Unsafe.defineAnonymousClass (minus constant pool patching) independent of 
>> nest mates. I believe that is desirable on it’s own as part of the replace 
>> unsafe functionality and if that can help MVT then even better!
> 
> We have an API point already for this, with a pre-planned "hole" for the 
> functionality you want.
> 
> If the VCC says "L=MethodHandles.lookup().in(DVT.class)" the Lookup should 
> retain the PRIVATE access mode.
> 

Yes.


> Then, L.defineClass(...) is specified to throw an exception when it sees the 
> PRIVATE access mode,

Right, i was just looking at that, currently scoped for package access.


> but what is supposed to happen in the future is that the newly defined class 
> is inserted into the nest of the Lookup.lookupClass.
> 

I was unsure if we require a new method L.defineAnonClass or could leverage the 
existing L.defineClass. IIUC for expediency the current hooks in the VM lean 
towards anon classes, since there is already code to defer to the host class, 
whereas the general defineClass case will likely require more work, although i 
can potentially see some short cuts if we focus on VCC/DVT.

Paul.

> In the very specific case of a DVT, it would be reasonable to allow the 
> injection you want, Paul, without throwing the required exception.  After 
> all, the DVT's are going away, so there's no compatibility risk for future 
> changes to the API.
> 
> The challenge to provide this for general use is to specify what happens when 
> the Lookup has PRIVATE access mode.  The best way to spec. this (IMO) is as a 
> forcible injection into the nest of the Lookup.lookupClass.  That requires 
> nestmates to be specified *first*, and *then* we can specify what it means to 
> inject a new (dynamically defined) nestmate.
> 
> — John

Re: constant-dynamic specification, updated

[Paul Sandoz]

> On 5 Jul 2017, at 11:57, Brian Goetz  wrote:
> 
> This is very nice.
> 
> One thing that readers will wonder (doesn't belong in JVMS, but belongs 
> somewhere) is what suggested practice are for using invocation name / 
> constant name.  I don't see the name widely used in indy, but I could imagine 
> it more widely used in condy, because constants often have names (like Pi.)  
> For a BSM that effectively has a single String parameter, I think we'll get 
> more compact classfiles if we use the name and a shared no-arg 
> BootstrapMethods entry, but is that a good reason?  For example, for a 
> primitive-class bootstrap, would this be a good use of the name?
> 

That’s how it’s applied by javac and the BSM 
(j.l.i.DynamicConstants.primitiveClassBootstrap) in the amber branch. For a 
VarHandle to a field, the name corresponds to the field name (for an array it 
is redundant).

(FWIW the StringConcatFactory BSM could of used the name for the recipe 
encoding. Arguably with condy a richer recipe description could be used as a 
BSM argument rather than encoding it all into a string.)


> I don't understand this sentence: "A bootstrap specifier gives a method or 
> field descriptor, TD."
> 
> On the API:
> 
>  - The type parameter T to BCI is "dangling", in that there is nothing to 
> constrain it.  However, IIUC, it can only be MethodType or Class.  In an 
> ideal world, T would be bounded by the common supertype of these two things 
> (NominalConstantRepresentingMemberDescriptor).  We generally advise against 
> use of unconstrained tvars like this in APIs, since it provides only the 
> illusion of type safety, though I understand why you went this way -- you can 
> have a BSM
> 
> Foo bsm(Lookup lookup, BootstrapCallInfo bci) { ... }
> 
> and just proceed without casting, and it's both convenient and 
> self-documenting.
> 
> If we had a common supertype in the future, we'd not be able to add this as a 
> bound in a binary-compatible manner, because invocationType()Object would be 
> burned into client classfiles.  So it's kind of a dead-end.  In any case, I'd 
> add a stronger note about the range of T here.  THere's room to put javadoc 
> on tvars with "@param "; you could say "Must be Class or MethodType."
> 
>  - In ConstantGroup, do you want to clarify that constants described by 
> multiple CGs might be the same constant, and therefore might share the 
> one-time transition to resolved?  Readers could think that the one-transition 
> lifecycle is a property of "entry #3 in CG@234098", not the constant that 
> CG@234098 happens to refer to in slot 3.
> 
> 
> I'd like to see some guidance about what a constant is.  Condy will happily 
> serve up mutable objects in the guise of dynamic constants, and the VM will 
> have no regrets.  However, this is likely to lead to pain (or security 
> issues) for the user.  Some sort of guidance here would be good, perhaps in 
> the package javadoc.
> 
> Also, related: what sort of facilities might we want to provide for 
> caching/interning of equivalent constants across classes?  I could easily 
> imagine a BSM wanting to preserve the invariant that any LDC with a given set 
> of args from any class results in the same object.
> 

And also that the constant returned from a condy might store a ConstantGroup 
for later resolution of further values, namely the scope of resolution can 
exceed that of the BSM invocation (which is what i suspect might happen for 
pattern matching).

Paul.

Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

In terms of what we have today we could easily do:

  // lookup must have private access to the lookup class, which becomes the 
“host” class
  Class defineAnonymousClass(byte[] data)

is that ending gaining too much?

That still leaves the possibility of another method in the future say:

  Class defineClass(boolean isAnon, byte[] data, Object constant)

That’s a little fuzzy since it’s not clear to me how the generated class 
locates the constant (synthetic static final field of known name? substitute 
the last entry in the CP if appropriately defined in the class bytes as 
substitutable?).

Paul.

> On 5 Jul 2017, at 11:43, John Rose  wrote:
> 
> On Jul 5, 2017, at 11:39 AM, Paul Sandoz  wrote:
>> 
>> 
>> I was unsure if we require a new method L.defineAnonClass or could leverage 
>> the existing L.defineClass. IIUC for expediency the current hooks in the VM 
>> lean towards anon classes, since there is already code to defer to the host 
>> class, whereas the general defineClass case will likely require more work, 
>> although i can potentially see some short cuts if we focus on VCC/DVT.
>> 
> 
> Good point.  Yes, that part isn't designed yet.  It may manifest as an extra 
> argument or two to the L.dC.
> 
> At least two degrees of freedom apply there:  1. suppressing the name (no 
> system dictionary update),
> 2. providing some sort of "user data" to bind to the class (can be a single 
> ref.,  replaces CP patching).
> 
> For MVT we can just generate a throwaway name, like DVT.getName()+":29348".
> 
> — John

Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

> On 6 Jul 2017, at 15:58, John Rose  wrote:
> 
> On Jul 6, 2017, at 1:02 PM, Paul Sandoz  <mailto:paul.san...@oracle.com>> wrote:
>> 
>> In terms of what we have today we could easily do:
>> 
>>  // lookup must have private access to the lookup class, which becomes the 
>> “host” class
>>  Class defineAnonymousClass(byte[] data)
>> 
>> is that ending gaining too much?
> 
> Sure, that's OK, or else an 'isAnonymous' optional argument.
> The difficult part here is specifying exactly what is a "host class".
> 

Agreed, that’s the tricky bit.


>> That still leaves the possibility of another method in the future say:
>> 
>>  Class defineClass(boolean isAnon, byte[] data, Object constant)
>> 
>> That’s a little fuzzy since it’s not clear to me how the generated class 
>> locates the constant (synthetic static final field of known name? substitute 
>> the last entry in the CP if appropriately defined in the class bytes as 
>> substitutable?).
> 
> On Jul 6, 2017, at 1:12 PM, Remi Forax  <mailto:fo...@univ-mlv.fr>> wrote:
>> 
>> Lookup.getConstant() with a private Lookup ?
> 
> Exactly, an ad hoc API point like that.  Would fit nicely with a BSM and
> CONSTANT_ConstantDynamic and ldc.  Or a private static final and
> code in .

I like that.

Perhaps those associated constants could be squirrelled away in a ClassValue, 
and any class could participate and receive an ad-hoc constant.


> 
> If you need several of them, a Map would be your
> friend.  This doesn't need to be baked into the Lookup API, just a
> design pattern, supported nicely by a slightly different BSM.
> 

And there could be a condy for extracting a value for a given key from the Map 
constant.

Paul.

Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

Hi Karen,

Thanks, i understand the need for being conservative here in case we get things 
wrong. The pragmatic relaxation is entirely reasonable, although i would still 
prefer to go through a lookup mechanism if we could pull it off :-)

Paul.

> On 11 Jul 2017, at 14:42, Karen Kinnear  wrote:
> 
> Paul,
> 
> We are working hard on getting the nest mates requirements clarified.
> I would like to use that to support the Lookup.defineClass and not do a 
> Quick&Dirty
> in advance for MVT . I think we should stick with the reduced restrictions
> for withfield for early access. I think we should put our energy into getting
> nest mates and Lookup.defineClass ready.
> 
> We have three parts to Lookup.defineClass.
> 1) PRIVATE mode handling - which we believe we can support with nest mates
> - we do not want to do any versions of this based on unsafe.DAC current 
> behavior,
> - we are looking at cleaner behavior for nestmates
> 2) “temporary” name - i.e. find doesn’t work
> 3) CP patching/user data
> 
> My mental model is that we don’t have to add all of these at the same time,
> although we will want user feedback on when to remove 
> unsafe.DefineAnonymousClass.
> In fact, we would love more examples of current use cases, to help guide our
> design.
> 
> thanks,
> Karen
> 
>> On Jul 5, 2017, at 2:43 PM, John Rose > <mailto:john.r.r...@oracle.com>> wrote:
>> 
>> On Jul 5, 2017, at 11:39 AM, Paul Sandoz > <mailto:paul.san...@oracle.com>> wrote:
>>> 
>>> 
>>> I was unsure if we require a new method L.defineAnonClass or could leverage 
>>> the existing L.defineClass. IIUC for expediency the current hooks in the VM 
>>> lean towards anon classes, since there is already code to defer to the host 
>>> class, whereas the general defineClass case will likely require more work, 
>>> although i can potentially see some short cuts if we focus on VCC/DVT.
>>> 
>> 
>> Good point.  Yes, that part isn't designed yet.  It may manifest as an extra 
>> argument or two to the L.dC.
>> 
>> At least two degrees of freedom apply there:  1. suppressing the name (no 
>> system dictionary update),
>> 2. providing some sort of "user data" to bind to the class (can be a single 
>> ref.,  replaces CP patching).
>> 
>> For MVT we can just generate a throwaway name, like DVT.getName()+":29348".
>> 
>> — John
> 

off-list Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

Hi,

Off-list as i am not sure i wanna commit to this publicly :-)

In the interest of moving this forward independent of MVT i can prototype some 
of this if you like and use the three usages of U.DAC in the JDK as use-cases 
(see also [1]) Perhaps surprisingly grepcode.com reports no external usages of 
U.DAC.

Some use-cases for U.DAC could be replaced by an isolated method feature, which 
makes me wonder if that is really the actual public feature we really want 
rather than anon/nameless classes?

My gut feeling is 2) can be implemented straightforwardly on top of Unsafe.DAC 
with appropriate checks, if we are ok with the current HS implementation at 
least as a starting point. To me the tricky bit is the spec gymnastics, it 
would have to be subset of the eventual behaviour and that can only expand in 
scope with refinement of terms. Is that a reasonably accurate assessment?

I believe we can also prototype the constant pool patching in an indirect 
manner as indicated by John and Remi.

That leaves the more general nested named class support which requires more 
fundamental hotspot work.

So in terms of progress order it could be: 2), 3), 1). But if we throw isolated 
methods into the mix then the plot thickens.

Paul.

[1]
https://bugs.openjdk.java.net/browse/JDK-8078602
Support j.l.i.BoundMethodHandle$Species_* classes unloading
https://bugs.openjdk.java.net/browse/JDK-8168848
Too many anonymous classes that aren't unloaded


> On 11 Jul 2017, at 14:42, Karen Kinnear  wrote:
> 
> Paul,
> 
> We are working hard on getting the nest mates requirements clarified.
> I would like to use that to support the Lookup.defineClass and not do a 
> Quick&Dirty
> in advance for MVT . I think we should stick with the reduced restrictions
> for withfield for early access. I think we should put our energy into getting
> nest mates and Lookup.defineClass ready.
> 
> We have three parts to Lookup.defineClass.
> 1) PRIVATE mode handling - which we believe we can support with nest mates
> - we do not want to do any versions of this based on unsafe.DAC current 
> behavior,
> - we are looking at cleaner behavior for nestmates
> 2) “temporary” name - i.e. find doesn’t work
> 3) CP patching/user data
> 
> My mental model is that we don’t have to add all of these at the same time,
> although we will want user feedback on when to remove 
> unsafe.DefineAnonymousClass.
> In fact, we would love more examples of current use cases, to help guide our
> design.
> 
> thanks,
> Karen
> 
>> On Jul 5, 2017, at 2:43 PM, John Rose  wrote:
>> 
>> On Jul 5, 2017, at 11:39 AM, Paul Sandoz  wrote:
>>> 
>>> 
>>> I was unsure if we require a new method L.defineAnonClass or could leverage 
>>> the existing L.defineClass. IIUC for expediency the current hooks in the VM 
>>> lean towards anon classes, since there is already code to defer to the host 
>>> class, whereas the general defineClass case will likely require more work, 
>>> although i can potentially see some short cuts if we focus on VCC/DVT.
>>> 
>> 
>> Good point.  Yes, that part isn't designed yet.  It may manifest as an extra 
>> argument or two to the L.dC.
>> 
>> At least two degrees of freedom apply there:  1. suppressing the name (no 
>> system dictionary update),
>> 2. providing some sort of "user data" to bind to the class (can be a single 
>> ref.,  replaces CP patching).
>> 
>> For MVT we can just generate a throwaway name, like DVT.getName()+":29348".
>> 
>> — John
> 

Re: off-list Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

> On 11 Jul 2017, at 16:34, Paul Sandoz  wrote:
> 
> Hi,
> 
> Off-list as i am not sure i wanna commit to this publicly :-)
> 

Sigh, email fail, looks like i just have :-)

Paul.


> In the interest of moving this forward independent of MVT i can prototype 
> some of this if you like and use the three usages of U.DAC in the JDK as 
> use-cases (see also [1]) Perhaps surprisingly grepcode.com reports no 
> external usages of U.DAC.
> 
> Some use-cases for U.DAC could be replaced by an isolated method feature, 
> which makes me wonder if that is really the actual public feature we really 
> want rather than anon/nameless classes?
> 
> My gut feeling is 2) can be implemented straightforwardly on top of 
> Unsafe.DAC with appropriate checks, if we are ok with the current HS 
> implementation at least as a starting point. To me the tricky bit is the spec 
> gymnastics, it would have to be subset of the eventual behaviour and that can 
> only expand in scope with refinement of terms. Is that a reasonably accurate 
> assessment?
> 
> I believe we can also prototype the constant pool patching in an indirect 
> manner as indicated by John and Remi.
> 
> That leaves the more general nested named class support which requires more 
> fundamental hotspot work.
> 
> So in terms of progress order it could be: 2), 3), 1). But if we throw 
> isolated methods into the mix then the plot thickens.
> 
> Paul.
> 
> [1]
> https://bugs.openjdk.java.net/browse/JDK-8078602
> Support j.l.i.BoundMethodHandle$Species_* classes unloading
> https://bugs.openjdk.java.net/browse/JDK-8168848
> Too many anonymous classes that aren't unloaded
> 
> 
>> On 11 Jul 2017, at 14:42, Karen Kinnear  wrote:
>> 
>> Paul,
>> 
>> We are working hard on getting the nest mates requirements clarified.
>> I would like to use that to support the Lookup.defineClass and not do a 
>> Quick&Dirty
>> in advance for MVT . I think we should stick with the reduced restrictions
>> for withfield for early access. I think we should put our energy into getting
>> nest mates and Lookup.defineClass ready.
>> 
>> We have three parts to Lookup.defineClass.
>> 1) PRIVATE mode handling - which we believe we can support with nest mates
>> - we do not want to do any versions of this based on unsafe.DAC current 
>> behavior,
>> - we are looking at cleaner behavior for nestmates
>> 2) “temporary” name - i.e. find doesn’t work
>> 3) CP patching/user data
>> 
>> My mental model is that we don’t have to add all of these at the same time,
>> although we will want user feedback on when to remove 
>> unsafe.DefineAnonymousClass.
>> In fact, we would love more examples of current use cases, to help guide our
>> design.
>> 
>> thanks,
>> Karen
>> 
>>> On Jul 5, 2017, at 2:43 PM, John Rose  wrote:
>>> 
>>> On Jul 5, 2017, at 11:39 AM, Paul Sandoz  wrote:
>>>> 
>>>> 
>>>> I was unsure if we require a new method L.defineAnonClass or could 
>>>> leverage the existing L.defineClass. IIUC for expediency the current hooks 
>>>> in the VM lean towards anon classes, since there is already code to defer 
>>>> to the host class, whereas the general defineClass case will likely 
>>>> require more work, although i can potentially see some short cuts if we 
>>>> focus on VCC/DVT.
>>>> 
>>> 
>>> Good point.  Yes, that part isn't designed yet.  It may manifest as an 
>>> extra argument or two to the L.dC.
>>> 
>>> At least two degrees of freedom apply there:  1. suppressing the name (no 
>>> system dictionary update),
>>> 2. providing some sort of "user data" to bind to the class (can be a single 
>>> ref.,  replaces CP patching).
>>> 
>>> For MVT we can just generate a throwaway name, like DVT.getName()+":29348".
>>> 
>>> — John
>> 
> 

Re: Valhalla EG minutes 6/21/17

[Paul Sandoz]

> On 12 Jul 2017, at 11:10, Karen Kinnear  wrote:
> 
> I think we have been discussing two different time frames.
> For the MVT EA we won’t have nest mates, so there I was pushing back on the 
> dependency
> and we will stick with the package private access.
> 
> Post-EA - I totally agree with going through a lookup mechanism, and I would 
> like to design
> this around nest mates - and have the Lookup.defineClass(…) PRIVATE
> mode be an early use case of nest mates. Totally agree with trying to pull 
> this one off.
> 
> It would make sense for the VCC/DVT to be nest mates, the VCC would have to 
> be the
> nest-top so it could be loaded with or without the DVT. ( Non-MVT I would 
> expect the valhalla
> value class to be the nest-top.)
> 

Ok.


> For byte code generation uses - I assume you are generating additional 
> methods that
> you would want to add to the nest. Are those temporary classes?
> 

By temporary do you mean sort of like our existing VM anonymous classes? i.e. 
stuff that cannot be found and is not in the system dictionary. IIUC they are 
the same in that regard.

Code generation for lambda forms uses U.DAC (with LambdaForm as the host 
class), but could theoretically if supported could add new methods to a 
LambdaForm nest. (Which is sort of like what we are doing explicitly with 
pre-generation in the jlink stage for some lambda forms and also more 
explicitly for VarHandle linkage, see VarHandleGuards).


> And if I understand the proposal correctly, we are replacing constant pool 
> patching
> with Lookup.getConstant() with a private Lookup,

Yes, that looks like a promising direction to explore.


> which uses an ldc of condy underneath,
> so essentially the BSM is filling in new types in the condy constant pool 
> entries.
> 

Here is my understanding: there could be a condy BSM that calls 
Lookup.getConstant() and returns the result of invoking the returned method 
handle, or a class could do that explicitly in clinit and assigning the result 
in a static final field. If the constant is a Map there could be a condy with a 
key value constant, or likewise explicitly in clinit.

Off the top of my head:

  byte[] classBytes = ...
  Map constant = …
  Class c = l.defineClass(classBytes, Map.class, constant);
  ...
  Map _constant = (Map) MethodHandles.privateLookupIn(c, 
MethodHandles.lookup()).getConstant().invokeExact();
  assert(_constant.equals(constant));

Implementation wise underneath the covers there could be a map say of Class -> 
MethodHandle e.g.: somewhere in the defineClass impl:

  MethodHandle cmh = MethodHandles.constant(constantType, constantValue);
  constantMap.put(definedClass, cmh);

Then the Lookup.getConstant could be implemented as follows:

  return constantMap.get(lookupClass);

A ClassValue kind of map might be more efficient than say a ConcurrentHM.

Paul.



Paul.

Re: Valhalla EG meeting Sept 27 - and Sept 13 minutes

[Paul Sandoz]

> On 27 Sep 2017, at 06:50, Karen Kinnear  wrote:
> 
> Reminder - meeting TODAY, Sept 27 9am PT/noon ET/
> dial-in:  https://oracle.zoom.us/j/5249803466 
> 
> 
> AI: Karen send Dan H example in which a BSM can be used for both indy and 
> condy
> AI: Dan Smith version of Condy JVMS changes (not an appendix
> AI: nestmates JVMS comments: 
> http://cr.openjdk.java.net/~dlsmith/private-access.html 
> 
>  Remi proposal: rename attribute to “NestHost"
> AI: Karen: write-up preparation relative to selection
> AI: Karen - ask Paul progress on VarHandles and atomicity relative to Value 
> Types
> 

Alas very little :-( Most effort has been focused on bashing condy (minus BSCI) 
into shape for integration.

Paul.

Re: Final CONSTANT_Dynamic spec

[Paul Sandoz]

> On 18 Jan 2018, at 17:43, John Rose  wrote:
> 
> On Jan 18, 2018, at 4:14 PM, Dan Smith  wrote:
>> 
>> A proposed final spec for CONSTANT_Dynamic is here:
>> 
>> http://cr.openjdk.java.net/~dlsmith/constant-dynamic.html
> 
> 
> I gave it one more read.  It is excellent.  Let's do it.  — John


+1 thanks Dan,
Paul.

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[Paul Sandoz]

> On Jan 30, 2018, at 1:55 AM, David Holmes  wrote:
> MethodHandle API Changes:
> 
> - java/lang/invoke/MethodHandle.java
>   * A non-virtual method handle to a specific virtual method implementation
>   * can also be created.  These do not perform virtual lookup based on
>   * receiver type.  Such a method handle simulates the effect of
> - * an {@code invokespecial} instruction to the same method.
> + * an {@code invokespecial} instruction to the same non-private method;
> + * or an {@code invokevirtual} or {@code invokeinterface} instruction to the
> + * same private method (as applicable).
> I tried to clarify that non-virtual invocations are not limited to 
> invokespecial - as private invocations via invokevirtual or invokeinterface 
> are also non-virtual.
> 
> 

Why s/same method/same non-private method/ for the invokespecial?

It’s possible to look up a private method within the same class using 
Lookup.invokespecial and invoke it (and also look up a private constructor and 
invoke it).

Paul.

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[Paul Sandoz]

> On Jan 30, 2018, at 7:22 PM, David Holmes  wrote:
> 
> Hi Paul,
> 
> On 31/01/2018 12:24 PM, Paul Sandoz wrote:
>>> On Jan 30, 2018, at 1:55 AM, David Holmes >> <mailto:david.hol...@oracle.com>> wrote:
>>> 
>>> 
>>>  MethodHandle API Changes:
>>> 
>>> - java/lang/invoke/MethodHandle.java
>>>   * A non-virtual method handle to a specific virtual method implementation
>>>   * can also be created.  These do not perform virtual lookup based on
>>>   * receiver type.  Such a method handle simulates the effect of
>>> - * an {@code invokespecial} instruction to the same method.
>>> + * an {@code invokespecial} instruction to the same non-private method;
>>> + * or an {@code invokevirtual} or {@code invokeinterface} instruction to 
>>> the
>>> + * same private method (as applicable).
>>> 
>>> I tried to clarify that non-virtual invocations are not limited to 
>>> invokespecial - as private invocations via invokevirtual or invokeinterface 
>>> are also non-virtual.
>>> 
>>> 
>> Why s/same method/same non-private method/ for the invokespecial?
>> It’s possible to look up a private method within the same class using 
>> Lookup.invokespecial and invoke it (and also look up a private constructor 
>> and invoke it).
> 
> Yes you are right, but this text is not trying to describe how an 
> invokespecial might be used, but rather how "A non-virtual method handle to a 
> specific virtual method implementation can also be created.”
> 

Ok, i see now.


> But the notion of "specific virtual method implementation" is perhaps not 
> applicable to private methods in the first place.
> 
> Perhaps I just need to remove this change altogether and leave it as-is.
> 

I would be inclined just to leave it as is.

Paul.

Re: JVMS draft for L-world value types with support for nullability

[Paul Sandoz]

Hi Fred,

Some basic questions, which might be because the specification is transitioning 
from value-based to value classes.


> The ACC_ENUM flag indicates that this class or its superclass is declared as 
> an enumerated type. A class file must not have both ACC_ENUM and 
> ACC_VALUE_TYPE flags set.


Why can’t enum classes be values classes, where enum values are actual values? 
I think the answer may be below.


> The ACC_NON_NULLABLE flag indicates that this field must never store the null 
> reference. The field signature must be the signature of a class. The class 
> specified in the field’s signature is loaded during the loading phase of the 
> class declaring this field. The class of the field must be a value class. 
> This field must be initialized with the default value of this value class.


I suppose in theory this attribute could be applied to a non-value class?


> A field must not have both ACC_STATIC and ACC_NON_NULLABLE flags set. 

This would rule out the static fields of a enum class, although in this case i 
would presume an enum class is such that after static initializer block all 
static fields would be assigned since they are also marked final. 

So, if static fields can be final then why not non-nullable?

Thanks,
Paul.


> On Jan 31, 2018, at 11:38 AM, Frederic Parain  
> wrote:
> 
> Here’s a draft of the JVMS proposing a way to implement the L-world value 
> types,
> with support for value-based classes migration to value classes (essentially, 
> support
> for nullability):
> 
> http://cr.openjdk.java.net/~fparain/L-world/L-World-JVMS-3.pdf
> 
> The assumptions and key properties are listed in Karen’s document:
> 
> http://cr.openjdk.java.net/~acorn/LWorldValueTypesjan31.pdf
> 
> 
> Feedback and comments are welcome.
> 
> Fred
> 

Re: JVMS draft for L-world value types with support for nullability

[Paul Sandoz]

Hi Fred, John,

Thanks for the explanations, very informative.

Regarding enums i was not thinking of the case of implicitly retrofitting all 
enum classes to be value classes (that would be tricky as you point out). I was 
wondering if it might be possible for an enum class to also explicitly be a 
value class, e.g.:

__Value enum MyEnumValue { 
V1, V2, V3; 
}

(placing aside compatibility issues of an explicit transition, and even though 
it inherits from abstract class Enum), since enum is anyway special cased in 
some ways, but perhaps not sufficiently to make it easy to do.

Thanks,
Paul.
 

> On Feb 1, 2018, at 6:34 AM, John Rose  wrote:
> 
> On Jan 31, 2018, at 9:38 PM, Frederic Parain  
> wrote:
>> 
>> Hi Paul,
>> 
>>> On Jan 31, 2018, at 15:50, Paul Sandoz  wrote:
>>> 
>>> Hi Fred,
>>> 
>>> Some basic questions, which might be because the specification is 
>>> transitioning from value-based to value classes.
>>> 
>>> 
>>>> The ACC_ENUM flag indicates that this class or its superclass is declared 
>>>> as an enumerated type. A class file must not have both ACC_ENUM and 
>>>> ACC_VALUE_TYPE flags set.
>>> 
>>> 
>>> Why can’t enum classes be values classes, where enum values are actual 
>>> values? I think the answer may be below.
>> 
>> First of all, there’s a backward compatibility issue with old enums. They 
>> have been defined with full identity,
>> which is incompatible with being a value type.
>> There’s also an issue with the super-type. The super-type of all enums is 
>> the abstract class java.lang.Enum,
>> but the super-type of a value class must be java.lang.Object.
>> One advantage of values types is that the JVM can flattened them, it is 
>> possible because of two properties:
>> being identity-less and having a default value. Without a default value, the 
>> JVM cannot initialized a non-nullable
>> field, so flattened cannot be performed,
> 
> Good summary.  Although it is desirable to evolve enums to value types,
> we would have to figure out a migration strategy that would allow us to
> recompile them into value types, but also allow old code to operate
> correctly on them.  I don't think this is an easy problem, in L-world or
> any other world.
> 
> A JVM could do a heroic optimization on a field of enum type, if the
> enum type were loaded before the class containing the field.  That
> would be an instructive project, I think, but it is not an important one.
> 
>> 
>>> 
>>> 
>>>> The ACC_NON_NULLABLE flag indicates that this field must never store the 
>>>> null reference. The field signature must be the signature of a class. The 
>>>> class specified in the field’s signature is loaded during the loading 
>>>> phase of the class declaring this field. The class of the field must be a 
>>>> value class. This field must be initialized with the default value of this 
>>>> value class.
>>> 
>>> 
>>> I suppose in theory this attribute could be applied to a non-value class?
>> 
>> No as it is define today, because of the lack of non-null default value for 
>> non-value class, which makes the initialization
>> of such field impossible for the JVM.
>> 
>> A future project might be to enable ACC_NON_NULLABLE for non-value class, but
>> it would require a much complex initialization scheme, probably involving 
>> indy or condy.
>> This is way beyond the scope of this draft.
> 
> I agree.  The JVM would have to somehow track the initialization state
> of a non-nullable field, forcing each constructor to initialize it, and 
> preventing
> access before initialization.  (Idea:  Use null internally, and have getfield
> throw NPE if there is accidental access before construction is complete.)
> 
>> 
>>> 
>>>> A field must not have both ACC_STATIC and ACC_NON_NULLABLE flags set. 
>>> 
>>> This would rule out the static fields of a enum class, although in this 
>>> case i would presume an enum class is such that after static initializer 
>>> block all static fields would be assigned since they are also marked final. 
>>> 
>>> So, if static fields can be final then why not non-nullable?
>> 
>> This is an open question.
>> 
>> The rational of the current choice is to avoid some circularity errors with 
>> some
>> common construct for static fields.
> 
> Don't we have similar circularity problems with non-static fields?  Why are
> static fields worse?  One answer:  You have to create the Class mir

Re: JVMS draft for L-world value types with support for nullability

[Paul Sandoz]

> On Feb 7, 2018, at 6:35 AM, Karen Kinnear  wrote:
> 
> Paul,
> 
> Just to make sure we are in sync for now:
> 

We are.


> I think we are all in agreement that current Enums can not migrate to be
> value types:
> 1. enums have identity
> 2. enums have java.lang.Enum abstract class as super-class, not 
> java.lang.Object
> 3. there is no clear default value
> 4. enums have mutable fields.
> 
> What I think you are wondering about is if there is a role for a new kind of 
> type,
> value-enums, that have value type characteristics.

Yes, it was a thought exercise which also conveniently pushed on some other 
areas like static fields.

Thanks,
Paul.

> I totally agree with John
> that the is a future exercise. And if you can figure out a migration story in
> future, more power to you, but we are not designing value types around that
> requirement.
> 
> A note - at least from the hotspot perspective, enums are not special-cased
> and as you can imagine we are trying to minimize special cases since they
> tend to be sources of bugs, so we would like to keep it that way.
> 
> thanks,
> Karen
> 
>> On Feb 1, 2018, at 2:05 PM, John Rose > <mailto:john.r.r...@oracle.com>> wrote:
>> 
>> On Feb 1, 2018, at 5:24 PM, Paul Sandoz > <mailto:paul.san...@oracle.com>> wrote:
>>> 
>>> an enum class to also explicitly be a value class
>> 
>> Yes, it's probably doable, but we would have to work out the
>> migration story, and also figure out how to manage the Enum
>> supertype.  Remember that one of the ongoing challenges
>> of VT's is the role of Object.  For value-enums, I think we
>> would reprise the role for Enum.  I'd rather have template
>> classes under my belt before tackling that, so I could make
>> Enum behave differently as an object vs. a value supertype.
>> 
> 

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[Paul Sandoz]

> On Feb 11, 2018, at 5:19 PM, David Holmes  wrote:
> 
> Hi Paul,
> 
> On 1/02/2018 2:48 AM, Paul Sandoz wrote:
>>> On Jan 30, 2018, at 7:22 PM, David Holmes  wrote:
>>> On 31/01/2018 12:24 PM, Paul Sandoz wrote:
>>>>> On Jan 30, 2018, at 1:55 AM, David Holmes >>>> <mailto:david.hol...@oracle.com>> wrote:
>>>>>  MethodHandle API Changes:
>>>>> 
>>>>> - java/lang/invoke/MethodHandle.java
>>>>>   * A non-virtual method handle to a specific virtual method 
>>>>> implementation
>>>>>   * can also be created.  These do not perform virtual lookup based on
>>>>>   * receiver type.  Such a method handle simulates the effect of
>>>>> - * an {@code invokespecial} instruction to the same method.
>>>>> + * an {@code invokespecial} instruction to the same non-private method;
>>>>> + * or an {@code invokevirtual} or {@code invokeinterface} instruction to 
>>>>> the
>>>>> + * same private method (as applicable).
>>>>> 
>>>>> I tried to clarify that non-virtual invocations are not limited to 
>>>>> invokespecial - as private invocations via invokevirtual or 
>>>>> invokeinterface are also non-virtual.
>>>>> 
>>>>> 
>>>> Why s/same method/same non-private method/ for the invokespecial?
>>>> It’s possible to look up a private method within the same class using 
>>>> Lookup.invokespecial and invoke it (and also look up a private constructor 
>>>> and invoke it).
>>> 
>>> Yes you are right, but this text is not trying to describe how an 
>>> invokespecial might be used, but rather how "A non-virtual method handle to 
>>> a specific virtual method implementation can also be created.”
>>> 
>> Ok, i see now.
>>> But the notion of "specific virtual method implementation" is perhaps not 
>>> applicable to private methods in the first place.
>>> 
>>> Perhaps I just need to remove this change altogether and leave it as-is.
>>> 
>> I would be inclined just to leave it as is.
> 
> I decided to restore the original text and then add the following - as I do 
> want to make it clear that non-virtual is not restricted to invokespecial:
> 
> * A non-virtual method handle can also be created to simulate the effect
> * of an {@code invokevirtual} or {@code invokeinterface} instruction on
> * a private method (as applicable).
> 
> I have updated specdiffs for all of the changes here:
> 
> http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang/overview-summary.html

If a class has no nestmates is it implicitly a nest of itself? (is that 
considered something valid?)

  Class c = … // some class with no nest mates
  assert c.getNestHost() == c
  assert c.isNestmateOf(c)
  assert Stream.of(c.getNestMembers()).anyMatch(_c -> _c == c);

?

If class nh is a nest host with nest mates (in addition to itself):

  Class nh = … // some class that is a nest host

  assert nh.getNestHost() == c
  assert nh.isNestmateOf(nh)
  assert Stream.of(nh.getNestMembers()).anyMatch(_c -> _c == nh);
  assert Stream.of(nh.getNestMembers()).allMatch(_c -> _c.isNestmakeOf(nh));
  assert Stream.of(nh.getNestMembers()).allMatch(
_c -> Stream.of(nh.getNestMembers()).allMatch(__c -> __c.isNestmakeOf(_c)));

?

getNestMembers
—

"The list of nest members in the classfile is permitted to contain duplicates, 
or to explicitly include the nest host. It is not required that an 
implementation of this method removes these duplicates."

The "or to explicitly include the nest host” suggests it might not include the 
nest host, but a prior statement says it will be present in the zeroth element.

Why the ambiguity over duplicates? is this motivated by performance? this may 
just push the cost to clients that have to always remove duplicates to function 
reliably and may be cause bugs if duplicates are rare and induced by certain 
relationships or loading patterns. My inclination would be for the returned 
array to not contain duplicates.


> http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.invoke/overview-summary.html
> http://cr.openjdk.java.net/~dholmes/8010319/specs/java.lang.reflect/overview-summary.html
> 

Looks good to me.

Paul.

> Please ignore the spurious "version" changes.
> 
> Thanks,
> David
> -
> 
>> Paul.

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[Paul Sandoz]

> On Feb 12, 2018, at 1:55 PM, David Holmes  wrote:
>> getNestMembers
>> —
>> "The list of nest members in the classfile is permitted to contain 
>> duplicates, or to explicitly include the nest host. It is not required that 
>> an implementation of this method removes these duplicates."
>> The "or to explicitly include the nest host” suggests it might not include 
>> the nest host, but a prior statement says it will be present in the zeroth 
>> element.
> 
> The "or" pertains to the list of nest members in the classfile - ie the 
> contents of the NestMembers attribute. The returned array of nestmembers will 
> always contain the nesthost as the zeroeth element, but may also contain it 
> somewhere else if the classfile explicitly listed it in nestmembers.
> 

I see, it’s easy to misread, well i did :-) Perhaps call out the attribute and 
provide a link to the JVMS? 


>> Why the ambiguity over duplicates? is this motivated by performance? this 
>> may just push the cost to clients that have to always remove duplicates to 
>> function reliably and may be cause bugs if duplicates are rare and induced 
>> by certain relationships or loading patterns. My inclination would be for 
>> the returned array to not contain duplicates.
> 
> Yes performance. Having to check for duplicates adds a cost to every single 
> well formed call to this API to account for something that the specification 
> allows to happen but which we don't expect to happen and which javac will 
> never produce. This has all been discussed previously.
> 

Ok, it’s unfortunate that the cost will be placed on the developer who has to 
code defensively in case there might be duplicates i.e. the performance cost is 
pushed to an unbounded set of places (or places where bugs may lurk).

Paul.

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[Paul Sandoz]

> On Feb 12, 2018, at 6:24 PM, David Holmes  wrote:
> 
> On 13/02/2018 11:45 AM, Paul Sandoz wrote:
>>> On Feb 12, 2018, at 1:55 PM, David Holmes  wrote:
>>>> getNestMembers
>>>> —
>>>> "The list of nest members in the classfile is permitted to contain 
>>>> duplicates, or to explicitly include the nest host. It is not required 
>>>> that an implementation of this method removes these duplicates."
>>>> The "or to explicitly include the nest host” suggests it might not include 
>>>> the nest host, but a prior statement says it will be present in the zeroth 
>>>> element.
>>> 
>>> The "or" pertains to the list of nest members in the classfile - ie the 
>>> contents of the NestMembers attribute. The returned array of nestmembers 
>>> will always contain the nesthost as the zeroeth element, but may also 
>>> contain it somewhere else if the classfile explicitly listed it in 
>>> nestmembers.
>>> 
>> I see, it’s easy to misread, well i did :-) Perhaps call out the attribute 
>> and provide a link to the JVMS?
> 
> I can add a link to JVMS if that is what we normally do. As for misreading 
> ... the subject of the sentence is "The list of nest members in the 
> classfile". ;-)
> 

I know :-) most developers will not be thinking at the classfile level so some 
link for those that are interested or care is helpful i think.


>>>> Why the ambiguity over duplicates? is this motivated by performance? this 
>>>> may just push the cost to clients that have to always remove duplicates to 
>>>> function reliably and may be cause bugs if duplicates are rare and induced 
>>>> by certain relationships or loading patterns. My inclination would be for 
>>>> the returned array to not contain duplicates.
>>> 
>>> Yes performance. Having to check for duplicates adds a cost to every single 
>>> well formed call to this API to account for something that the 
>>> specification allows to happen but which we don't expect to happen and 
>>> which javac will never produce. This has all been discussed previously.
>>> 
>> Ok, it’s unfortunate that the cost will be placed on the developer who has 
>> to code defensively in case there might be duplicates i.e. the performance 
>> cost is pushed to an unbounded set of places (or places where bugs may lurk).
> 
> There's really no expectation that the developer will need to program 
> defensively here as we don't expect compilers to produce such classfiles.

But the developer will not know that since they will be reading the JavaDoc.


> But I, for one, prefer a "user pays" scheme over an "everyone pays" scheme 
> (which is what disallowing duplicates would also be).

It’s an awkward situation, experience suggests this type of thing bites back 
later on, so my inclination is to take the hit in the JDK and not return dups. 
The implementation might be able to cache information on the internal 
ReflectionData class.
 
Paul.

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[Paul Sandoz]

> On Feb 12, 2018, at 8:04 PM, David Holmes  wrote:
> 
> On 13/02/2018 12:51 PM, David Holmes wrote:
>> On 13/02/2018 12:39 PM, Paul Sandoz wrote:
>>>> On Feb 12, 2018, at 6:24 PM, David Holmes  wrote:
>>>> 
>>>> On 13/02/2018 11:45 AM, Paul Sandoz wrote:
>>>>>> On Feb 12, 2018, at 1:55 PM, David Holmes  
>>>>>> wrote:
>>>>>>> getNestMembers
>>>>>>> —
>>>>>>> "The list of nest members in the classfile is permitted to contain 
>>>>>>> duplicates, or to explicitly include the nest host. It is not required 
>>>>>>> that an implementation of this method removes these duplicates."
>>>>>>> The "or to explicitly include the nest host” suggests it might not 
>>>>>>> include the nest host, but a prior statement says it will be present in 
>>>>>>> the zeroth element.
>>>>>> 
>>>>>> The "or" pertains to the list of nest members in the classfile - ie the 
>>>>>> contents of the NestMembers attribute. The returned array of nestmembers 
>>>>>> will always contain the nesthost as the zeroeth element, but may also 
>>>>>> contain it somewhere else if the classfile explicitly listed it in 
>>>>>> nestmembers.
>>>>>> 
>>>>> I see, it’s easy to misread, well i did :-) Perhaps call out the 
>>>>> attribute and provide a link to the JVMS?
>>>> 
>>>> I can add a link to JVMS if that is what we normally do. As for misreading 
>>>> ... the subject of the sentence is "The list of nest members in the 
>>>> classfile". ;-)
>>>> 
>>> 
>>> I know :-) most developers will not be thinking at the classfile level so 
>>> some link for those that are interested or care is helpful i think.
>> Okay will see what I can reasonably add.
> 
> Added JVMS ref:
> 
> * The list of nest members in the classfile (JVMS 4.1) is permitted to
> * contain duplicates, or to explicitly include the nest host.
> 

Ok, you can also add @jvms, see examples in StackWalker (alas JavaDoc does not 
support generating links to the specification).

Paul.

Re: API Updates: 8191116: [Nestmates] Update core reflection, MethodHandle and varhandle APIs to allow for nestmate access

[Paul Sandoz]

> On Feb 12, 2018, at 6:51 PM, David Holmes  wrote:
>>> But I, for one, prefer a "user pays" scheme over an "everyone pays" scheme 
>>> (which is what disallowing duplicates would also be).
>> It’s an awkward situation, experience suggests this type of thing bites back 
>> later on, so my inclination is to take the hit in the JDK and not return 
>> dups. The implementation might be able to cache information on the internal 
>> ReflectionData class.
> 
> Please see John's response on this:
> 
> http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2017-December/000465.html
> 

Thanks for the context. You may be surprised to learn that i still don’t agree 
:-) so i will just state my objection more concisely and move on:

I don’t think as specified the method serves developers very well for the 
common use-case of iterating over the nest mates and performing actions with 
side-effects. A developer will have to write defensive code or, more likely, 
write buggy code that will blow up in the rare case a classfile contain 
duplicates e.g. generated by something other than javac. That rare case may be 
hard to track down and expensive to fix.

Paul.

Re: Valhalla EG minutes Feb 14, 2018

[Paul Sandoz]

> On Feb 20, 2018, at 7:52 AM, Karen Kinnear  wrote:
> 
> attendees: Tobi, Mr Simms, Dan H, Dan S, Frederic, Remi, Karen
> 
> I. Condy
> 
> 1. Condy reference implementation was pushed last week into JDK 11.
> 

A next round of development will focus on the BootstrapCallInfo API and BSM 
invocation accepting such an argument. It’s currently in the condy-folding 
branch of amber. Observation: this can make recursive resolution errors more 
explicit, resulting in a more “obvious” stack overflow. Plus constants can be 
resolved outside of the context of the BSM invocation (if say the BSM returns 
something that itself operates on the BCI instance passed to it).


> 
> 3. Planned uses for condy in jdk?
>   - Nothing in imminent plans
>   - expect longer term constant Lambdas to use condy - lightweight

It would be great if javac could at some point also “intern” duplicate lambda 
expressions.


>   - future: still exploring APIs for constants, switch, pattern match, …

> 
>  Remi: Python, JRuby - all lambdas are constant
>  Remi: wants support in javac behind a flag
>  Dan S: it is in Amber

Vicente recently pushed a patch to the condy-folding branch of amber [1] to 
remove the flag, this feature is now always on when building JDK source (and 
was always on by default for stuff outside the JDK).

Paul.

[1] http://hg.openjdk.java.net/amber/amber/rev/75e0076b3ef0 


>  Remi: wants a binary :-) - Dan S will pass on that message
> 

Re: value type hygiene

[Paul Sandoz]

Thanks for sharing this!

I like the null containment approach. It recognizes that nulls (for better or 
worse) are a thing in the ref world but stops the blighters from infecting the 
value world at the borders.

We will need to extend this hygiene to javac and the libraries. 

Javac could fail to compile when it knows enough, and in other cases place in 
explicit null checks if not otherwise performed by existing instructions so as 
to fail fast.

Certain APIs that rely on null as a signal will need careful reviewing and 
possible adaption if the prevention has some side effects, and maybe 
errors/warnings from javac. The poster child being Map.get, but others like 
Map.compute are problematic too (if a value is not present for a key, then a 
null value is passed to the remapping function). How we proceed might depend on 
whether specialized generics progresses at a slower rate rate than value types.

Paul.

> On May 6, 2018, at 2:17 AM, John Rose  wrote:
> 
> Like many of us, I have been thinking about the problems of keeping values, 
> nulls,
> and objects separate in L-world.  I wrote up some long-ish notes on the 
> subject.
> I hope it will help us wrap our arms around the problem, and get it solved.
> 
> TL;DR:  Remi was right in January.  We need a ValueTypes attribute.
> 
> http://cr.openjdk.java.net/~jrose/values/value-type-hygiene.html
> 
> Cheers!
> — John
> 
> P.S. Raw markdown source follows for the record.
> http://cr.openjdk.java.net/~jrose/values/value-type-hygiene.md
> 
> # Value Type Hygiene
> 
>  May 2018 _(v. 0.1)_
> 
>  John Rose and the Valhalla Expert Group
> 
> Briefly put, types in L-world are ambiguous, leading to unhygienic
> mixtures of value operations with reference operations, and
> uncontrolled pollution from `null`s infecting value code.
> 
> This note explores a promising proposal for resolving the key
> ambiguity.  It is a cleaner design than the ad hoc mechanisms tried so
> far.  The resulting system would seem to allow more predictable and
> debuggable behavior, a stronger backward compatibility story, and
> better optimization.
> 
> ## Problem statement
> 
> In the _L-world_ design for value types, the classfile type descriptor
> syntax is left unchanged, and the pre-existing descriptor form
> `"LFoo;"` is overloaded to denote value types as well as object types.
> A previous design introoduced new descriptors for value types of the
> form `"QFoo;"`, and possibly a union type `"UFoo;"`.  This design
> might be called _Q-world_.  In comparison with Q-world, the L-world
> design approach has two advantages--compatibility and migration--but
> also one serious disadvantage: ambiguity.
> 
> L-world is _backward compatible_ with tools that must parse classfile
> descriptors, since it leaves descriptor syntax unchanged.  There have
> been no changes to this syntax in almost thirty years, and there is a
> huge volume of code that depends on its stability.  The HotSpot JVM
> itself makes hundreds of distinct decisions based on descriptor syntax
> which would need careful review and testing if they were to be adapted
> to take account of a new descriptor type (`"QFoo;"`, etc.).
> 
> Because of its backward compatibility, L-world also has a distinctly
> simpler _migration story_ than previous designs.  Some _value-based
> classes_, such as `Optional` and `LocalTime`, have been engineered to
> be candidates for migration to proper value types.  We wish to allow
> such a migration without recompiling the world or forcing programmers
> to recode uses of the migrated types.  It is very difficult to sustain
> the illusion in Q-world that a value type `Qjava/util/Optional;` can
> be operated on in old code under the original object type
> `Ljava/util/Optional;`, since the descriptors do not match and a
> myriad of adapters must be spun (one for every mention of the wrong
> descriptor).  With L-world, we have the simpler problem (addressed in
> this document) of keeping straight the meaning of L-descriptors
> in each relevant context, whether freshly recompiled or legacy
> code; this is a simpler problem than spinning adapters.
> 
> But not all is well in L-world.  The compatibility of descriptors
> implies that, when a classfile must express a semantic distinction
> between a reference type and an object type, it must be allowed to do
> so unambiguously, in a side channel outside of the descriptor.
> 
> Our first thought was, "well, just load all the value types and then
> you will know the list of them".  If we have a global registry of
> classes (such as the HotSpot system dictionary), nobody needs to
> express any additional distinctions, since everybody can just ask the
> register which are the value types.
> 
> This simple idea has a useful insight, but it goes wrong in three
> ways.  First, for some use cases such as classfile transformation, it
> might be difficult to find such a global registry; in some cases we
> might prefer to rely on local information in the class

Re: value type hygiene

[Paul Sandoz]

> On May 10, 2018, at 8:52 AM, Brian Goetz  wrote:
> 
> Thanks for this great writeup.  I find much to agree with here, and a few 
> things to be concerned about (I’ll express the latter in a separate mail; Dan 
> touched on some of them.)  
> 
> Now that we see it, elevating from ACC_FLATTENABLE to the ValueTypes 
> attribute makes obvious sense.  The key thing to reify is whether V was a 
> value type at the time C was compiled.  This flows into many decisions within 
> C, and at the boundary of C and other V-users, so capturing it in one place 
> makes sense.  
> 
> I’ll add that this reminds me very much of loader constraints.  When class C 
> calls method D.m(P)R, we first textually match the call with m(P)R in D via 
> descriptor match, and then we additionally make sure that C and D agree on 
> any loader constraints, throwing an error if they do not.  In L-world, 
> whether C and D think V is a value or object class is another kind of 
> constraint.  At linkage time, if these constraints agree, they can use an 
> optimized protocol; if they disagree, rather than failing, the VM can 
> introduce hidden adaptation to iron out the disagreement.  

Also bridges generated by generics are naturally a place for such 
checks/adaptions from the ref world to the value world, the cast could be 
coopted to perform the null check and throw e.g. forEach'ing with a 
Consumer over a List.


> This is a big win over the use of bridges in Q-world, since the adaptors are 
> only generated at runtime when they are strictly needed, and as the ecosystem 
> gets recompiled over time to a more uniform view of V’s value-ness, will 
> steadily go away.  We saw shades of this in Albert’s first prototype of 
> heisenboxes, where the JIT compiled multiple versions of each method (if 
> needed) according to different views of value-ness, and then fit them 
> together, lego-style.  
> 
> A note on the responses:
> 
> - I think the Map.get() discussion is a red herring.  This is a signature 
> that simply makes no sense when V is a value.  We’ve looked at several 
> alternatives — optional-bearing, a pattern match (case withMapping(var v)), a 
> get-with-default, etc.  In Q-world, we observed that sometimes a method 
> doesn’t make it to the any-fied version; it becomes a restricted method that 
> only makes sense on reference types.  In L-world, we don’t necessarily have 
> “ref V” to fall back on (though we might), but there will need to be some way 
> to give Map.get() a gold watch and thank it for its service (and lament that 
> the best name has been retired from the namespace.)  
> 

Yes Map.get has to somehow retire (although i still think it represents a good 
use case of what to do at the boundary of the value and ref worlds, perhaps 
List.get is a better case to discuss in this regard) IMO that’s part of the 
hygiene we need to do to the libraries. I just don’t have a strong sense on how 
to retire this if value types and specialized generics proceed at different 
rates. We can start by deprecating it (with forRemoval? that might be tricky), 
but perhaps it requires some compiler and linkage error when used with values?

Paul.

> I’ll start a separate thread on my concerns.  
> 
> 

Re: value type hygiene

[Paul Sandoz]

> On May 10, 2018, at 11:53 AM, Brian Goetz  wrote:
> 
>> Well, yes, but
>> remember also that if `Q` started its career as an object type, it was
>> a value-based class, and such classes are documented as being
>> null-hostile.  The null-passers were in a fool's paradise.
> 
> Objection, Your Honor, assumes facts not in evidence!  The letters "n-u-l-l" 
> do not appear in the definition of value-based linked above.  Users have no 
> reasons to believe any of he following are bad for a VBC, among others:
> 
> V v = null;
> 
> if (v == null) { ... }
> 
> List list = new ArrayList<>();
> list.add(null);
> 
> In Q world, these uses of V were compiled to LV rather than QV, so these 
> idioms mapped to a natural and sensible translation.  Our story was that when 
> you went to recompile, then the stricter requirements of value-ness would be 
> enforced, and you'd have to fix your code.  (That is: making V a value is 
> binary compatible but not necessarily source compatible.  This was a pretty 
> valuable outcome.)
> 
> One of the possible coping strategies in Q world for such code is to allow 
> the box type LV to be denoted at source level in a possibly-ugly way, so that 
> code like the above could continue to work by saying "V.BOX" instead of "V".  
> IOW, you can opt into the old behavior, and even mix and match between int 
> and Integer.  So users who wanted to fight progress had some escape hatches.
> 
> While I don't have a specific answer here, I do think we have to back up and 
> reconsider the assumption that all uses of V were well informed about V's 
> null-hostility, and have a more nuanced notion of the boundaries between 
> V-as-value-world and V-as-ref-world.
> 

If source files tend to be migrated as whole units rather than bit by bit then 
perhaps this problem can considered similar to compiling with the --release 
flag? Thus there is no explicit intermixing of the two worlds within one 
compilation unit.

Paul.

Re: value type hygiene

[Paul Sandoz]

Hi John,

The answers below might depend on experimentation but what might you propose 
the behavior should be for the following code, assuming we have no specialized 
generics, ArrayList is not yet modified to cope better:

value class Point { … }

class VW {
  public static void main(String[] s) {
List l = new ArrayList<>();
l.add(P.default);
l.add(P.default); // assuming this works :-)

Point[] p = new Point[10]; // Flattened array is created
l.toArray(p); // What should happen here?  
  }
}

(I know toArray is value hostile and maybe should be deprecated or changed but 
I find it a useful example to think about as it may be indicative of legacy 
code in general.)
 
Should the call to l.toArray link? If so then i presume some form of array 
store exception will be thrown when ArrayList attempts to store null into the 
flattened array at index 2?

Or:

  Point[] p = l.toArray(new Point[2]);

a flattened array is returned (the argument)? assuming System.arraycopy works.

Or:

  Point[] p = l.toArray(new Point[1]);

a non-flattened array is returned? since Arrays.copyOf operates reflectively on 
the argument’s class and not additional runtime properties. 


What about:

  Object[] o = l.toArray();

A non-flattened array is returned containing elements that are instances of 
boxed Point?

Paul.


> On May 14, 2018, at 4:02 PM, John Rose  wrote:
> 
> On May 11, 2018, at 7:39 AM, Frederic Parain  
> wrote:
>> 
>> John,
>> 
>> I have a question about the semantic within legacy class files (class
>> files lacking a ValueTypes attribute). Your document clarifies the semantic
>> for fields as follow:
>> 
>> "Meanwhile, C is allowed to putfield and getfield null all day long into its 
>> own fields (and fields of other benighted legacy classes that it may be 
>> friends with). Thus, the getfield and putfield instructions link to slightly 
>> different behavior, not only based on the format of the field, but also 
>> based on “who’s asking”. Code in C is allowed to witness nulls in its Q 
>> fields, but code in A (upgraded) is not allowed to see them, even though 
>> it’s the same getfield to the same symbolic reference. Happily, fields are 
>> not shared widely across uncoordinated classfiles, so this is a corner case 
>> mainly for testers to worry about.”
>> 
>> But what’s about arrays? If I follow the same logic that “old code needs to
>> be left undisturbed if possible”, if a legacy class C creates an array of Q,
>> without knowing that Q is now a value type, C would expect to be allowed
>> to write and read null from this array, as it does from its own fields. Is 
>> it a
>> correct assumption?
> 
> Yes, I avoided this question in the write-up.  To apply the same move
> as fields, we could try to say that arrays of type V[] created by a legacy
> class C do not reject nulls, while arrays of type V[] created by normal
> classes (that recognize V as value types) are created as flattened.
> 
> But the analogy between fields and array elements doesn't work in this
> case.  While a class C can only define fields in itself, by creating arrays
> it is working with a common global type.  Think of V[] as a global type,
> and you'll see that it needs a global definition of what is flattened and
> what is nullable.  I think we will get away with migrating types and
> declaring that legacy classes that use their arrays will fail.  The mode
> of failure needs engineering via experiment.  We could go so far as
> to reject legacy classes that use anewarray to build arrays of value
> type, without putting those types on the ValueTypes list.
> 
> This means that if there is a current class C out there that is creating
> arrays of type Optional[] or type LocalDate[], then if one of those types
> is migrated to a value type, then C becomes a legacy class, and it will
> probably fail to operate correctly.  OTOH, since those classes use
> factories to create non-null values of type Optional or LocalDate, such
> a legacy class is likely to refrain from using nulls.  I think it's possible
> but not likely that the author of a legacy class will make some clever
> use of nulls, storing them into an array of upgraded type V.
> 
> In the end, some legacy code will not port forward without recompilation
> and even recoding.  Let's do what we can to make it easier to diagnose
> and upgrade such code, as long as it doesn't hurt the basic requirement
> of making values flattenable.  The idea of making fields nullable seems
> a reasonable low-cost compromise, but making elements nullable a
> much higher cost.
> 
> Any need for a boxy or nullable array is more easily served by an explicit
> reference array, of type Object[] or ValueRef[].  Overloading that 
> behavior
> into V[] is asking for long-term trouble with performance surprises.  Erased
> Object or interface arrays will fill this gap just as well as a first-class 
> nullable
> VT.BOX[], with few exceptions.  I think those exceptions are manageable by
> other means

Re: value type hygiene

[Paul Sandoz]

> On May 14, 2018, at 11:36 PM, John Rose  wrote:
> 
> On May 14, 2018, at 6:53 PM, Paul Sandoz  wrote:
>> 
>> Hi John,
>> 
>> The answers below might depend on experimentation but what might you propose 
>> the behavior should be for the following code, assuming we have no 
>> specialized generics, ArrayList is not yet modified to cope better:
> 
> This is the right sort of question to ask and answer about value types,
> and the sooner we get L-world up and running, the quicker we can
> validate our answers.
> 
> We'll probably end up with a bunch of rules of thumb about how to
> handle nulls.  In this case, null is a sentinel value used by the external
> API.  There are two main choices with cases like this:  (a) disallow
> the null when used with null-hostile types (V[]) and (b) adapt null
> to another value when a null-hostile type is detected.
> 
>> value class Point { … }
>> 
>> class VW {
>> public static void main(String[] s) {
>>   List l = new ArrayList<>();
>>   l.add(P.default);
>>   l.add(P.default); // assuming this works :-)
> 
> (There's no reason why it shouldn't work.)
> 

Agreed.


>>   Point[] p = new Point[10]; // Flattened array is created
>>   l.toArray(p); // What should happen here?  
>> }
>> }
>> 
>> (I know toArray is value hostile and maybe should be deprecated or changed 
>> but I find it a useful example to think about as it may be indicative of 
>> legacy code in general.)
> 
> In the case of List.toArray(.) the simple answer IMO is (b), and the
> sentinel value is clearly T.default, to be computed something
> like this:
> 
> diff --git a/src/java.base/share/classes/java/util/AbstractCollection.java 
> b/src/java.base/share/classes/java/util/AbstractCollection.java
> --- a/src/java.base/share/classes/java/util/AbstractCollection.java
> +++ b/src/java.base/share/classes/java/util/AbstractCollection.java
> @@ -186,13 +186,13 @@
> for (int i = 0; i < r.length; i++) {
> if (! it.hasNext()) { // fewer elements than expected
> if (a == r) {
> -r[i] = null; // null-terminate
> +r[i] = a.getClass().getDefaultValue()
> } else if (a.length < i) {
> return Arrays.copyOf(r, i);
> } else {
> System.arraycopy(r, 0, a, 0, i);
> if (a.length > i) {
> -a[i] = null;
> +a[i] = a.getClass().getDefaultValue()
> }
> }
> return a;
> 
> Eventually when int[] <: Object[], then 
> int[].class.getClass().getDefaultValue()
> will return an appropriate zero value, at which point the above behavior will
> "work like an int".
> 
> Another way to make this API point "work like an int" would be to throw an
> exception (ASE or the like), on the grounds that you can't store a null into
> an int[] so you shouldn't be able to store a null into a Point[].
> 

A third approach could be to check if the array is non-nullable and not store a 
default value, which may be surprising, but storing a default is arguably less 
useful in general for arrays of value types but it is suppose mostly harmless 
(i am thinking of cases where a value type has a default that is hostile to be 
operated on, like perhaps LocalDate).


>> Should the call to l.toArray link?
> 
> Yes, because Point[] <: Object[].  There's a separate question on whether
> the source language should allow the instance List; I think it should
> do so because that's more useful than disallowing it.
> 
>> If so then i presume some form of array store exception will be thrown when 
>> ArrayList attempts to store null into the flattened array at index 2?
> 
> In the case of the List API it's more useful for the container, which is 
> attempting
> to contain all kinds of data, to bend a little and store T.default as the 
> proper
> generalization of null.  Under this theory, Object.default == null, and 
> X.default
> is also null for any non-value, non-primitive X.  (Including Integer but not 
> int.)

Agreed, i just wanted to do the thought experiment given the current behavior 
of List/ArrayList as if it's unmodified legacy code.


> 
>> Or:
>> 
>> Point[] p = l.toArray(new Point[2]);
>> 
>> a flattened array is returned (the argument)? assuming System.arraycopy 
>> works.
> 
> It does.  (Or will.)  Reason:  Point[] <: Object[].
> 

Ok.


>> Or:
>> 
>> Point[] p = l.toArray(new Point[1]);
>> 
>>

Re: Towards Minimal L World

[Paul Sandoz]

> On May 16, 2018, at 7:53 PM, Dan Smith  wrote:
> 
>> On May 16, 2018, at 4:45 PM, Brian Goetz > > wrote:
>> 
>> No support for any interaction between values and generics _whatsoever_ (I 
>> said minimal!)
> 
> You clarified that this means the compiler actively rejects types like 
> List. Not clear to me what would prompt that—it's more work for the 
> compiler, the JVM doesn't care either way, and it's easy for users to work 
> around (use a raw type).
> 

Right, one could use a raw type, which would induce, by default, a warning from 
javac, so maybe a warning is sufficient by default for the List case.


> But the quality of language support will be "crappy ad-hoc" anyway, so, you 
> know, whatever works. I'm just happy to have a compiler at all!

Indeed, it's liberating :-)

Paul.