Summary notes from meeting 3/15/17: attendees: Brian, Doug Lea, Stas, Frederic, Bjorn, Tobias, Mr Simms, Vladimir I, Maurizio, Karen, John
1. Identity Major decision: Doug: It is ok if a value type box does NOT retain identity. Do not be overly concerned with sync, ===, hashcode. Let’s see if the benefits outweigh the problems. We need to set clear expectations for customers. ed. note: Maurizio clarified later that it is critical that a value type not lose identity within a given method, but it is ok to lose identity across methods. It is ok if the vm throws an exception - e.g. IllegalMonitorStateException when attempting to lock a Value Type, ok if we do that based on a command-line flag to not slow down normal execution. 2. Evolution approach can evolve experimental annex to JVMS - optional to implement, with minor change to classfile, byte codes, etc. no story for experimental JLS Doug suggested: could have an experimental javac documentation 3. vdefault/vwithfield - constructor as static factory method - vdefault - create 0-filled default value type - only valid within constructor - note a value type array - initial array allows default value to escape - vwithfield - copy-on-write a new value type with one modified field - allows final fields to be final (value type is immutable) - JIT can optimized via escape analysis - only within value class (which means both the value capable class and the DVT) 4. Requested Extensions: - exploring adding references - both IBM and Oracle believe customers will need this for use cases - support for VCC implementing interfaces - do users need this? - propose: go to Early Access without this and see if it is required - IBM and Oracle investigating - note: VM would do no magic boxing, and for MVT, would require boxing to invoke default methods 5. Packed Objects: AI: IBM - Doug requested a summary of differences between PackedObjects and MVT John note: Packed Objects have identity, others see changes through the reference 6. JVMS changes: AI: Oracle - need to propose a draft for the JVMS extensions 7. early access timing: Note: need to ensure that IBM and Oracle are in sync on Early Access timing Meeting March 29th - need to discuss any other Shady Values version 0.2 changes and timing. *** will need a Shady Values update *** thanks, Karen > On Mar 15, 2017, at 1:20 AM, Karen Kinnear <karen.kinn...@oracle.com> wrote: > > Summary notes from meeting 2/15/17 - embedded below, starting with #8 > > Meeting 3/16/17: > — there are a few more already identified topics to discuss > — welcome additional questions/suggestions > — a couple of topics to revisit (or perhaps put on an open issues list until > we have more information or feedback) > > thanks, > Karen > > >> On Feb 10, 2017, at 9:47 AM, Karen Kinnear <karen.kinn...@oracle.com >> <mailto:karen.kinn...@oracle.com>> wrote: >> >> John - please correct >> >> Bjorn, >> >> Good question. This was news to us as well, so John will have the final word. >> >> My assumption is that he means that: >> If you create an array of Value-Capable-Classes that you would not >> automagically also create an equivalent >> array of the Derived Value Types. >> That the MethodHandle APIs could explicitly, through new byte codes create >> arrays of the DVTs by passing a >> Derived Value Type as the “array ref” to anewarray or multianewarray. >> I presume that any explicit byte code generation by power users to go with >> this code could use the same mechanism. >> >> thanks, >> Karen >> >>> On Feb 9, 2017, at 10:56 PM, Bjorn B Vardal <bjorn...@ca.ibm.com >>> <mailto:bjorn...@ca.ibm.com>> wrote: >>> >>> Karen / John: Can you clarify this? Do you mean that they will only be >>> flattened when created using the reflection / MethodHandle API? >>> >>> > John: MVT 1.0 will only flatten arrays reflectively >>> >>> -- >>> Bjørn Vårdal >>> J9 Java Virtual Machine Developer >>> IBM Runtimes >>> >>> >>> ----- Original message ----- >>> From: Karen Kinnear <karen.kinn...@oracle.com >>> <mailto:karen.kinn...@oracle.com>> >>> Sent by: "valhalla-spec-experts" >>> <valhalla-spec-experts-boun...@openjdk.java.net >>> <mailto:valhalla-spec-experts-boun...@openjdk.java.net>> >>> To: John Rose <john.r.r...@oracle.com <mailto:john.r.r...@oracle.com>> >>> Cc: valhalla-spec-experts@openjdk.java.net >>> <mailto:valhalla-spec-experts@openjdk.java.net> >>> Subject: Re: Initial feedback on Minimal Value Types 0.2 for discussion >>> Date: Thu, Feb 9, 2017 6:31 PM >>> >>> Notes from discussion on Feb 01,2017. Feedback welcome. >>> >>> John - one question extracted at the top from the embedded notes. >>> >>>> 11. "interfaces (especially with default methods)" >>>> - please change p.6 to clarify that there are no value type >>>> interfaces period. >>> Ed note: There is a distinction here between >>> 11a) defining an interface as a VCC with a derived DVT and >>> 11b) whether the POJO which defines the VCC can implement interfaces. This >>> discussion was about whether >>> a POJO which defines the VCC can implement interfaces. >>> John: MVT 1.0 : No value capable interfaces for JVMT 1.0. >>> Ed note: was this the answer to 11a or 11b above please? >>>> On Feb 9, 2017, at 5:43 PM, Karen Kinnear <karen.kinn...@oracle.com >>>> <mailto:karen.kinn...@oracle.com>> wrote: >>>> >>>> >>>> (This is a resend of an email I sent to valhalla-...@openjdk.java.net >>>> <mailto:valhalla-...@openjdk.java.net> on January 23) >>>> >>>> Review of Minimal Value Types August 2016 Shady Edition (v 0.2) >>>> http://cr.openjdk.java.net/~jrose/values/shady-values.html >>>> <http://cr.openjdk.java.net/~jrose/values/shady-values.html> >>>> >>>> Questions/Comments: >>>> abbreviations used: VCC: value-capable class, DVT: derived value type >>>> >>>> 1. Goals >>>> -- might be worth adding to the bullets: >>>> Allow use of existing toolchain where possible including IDEs and >>>> debuggers >>> John: this is just a review comment, no discussion required. >>>> >>>> >>>> 2. Features: >>>> "Three" bytecode instructions ->"A small set of” >>> John: just a text edit to encompass extensions in #12 below, not yet >>> discussed as a group. >>>> >>>> >>>> 3. Typically, value-capable classes will not be exported. >>>> Is the reason for this to limit exposure since the expectation is that >>>> the >>>> initial APIs and mechanisms will change? >>> John: “yes”. This does not have to be enforced in the implementation. >>> ed. note: perhaps we could remove this from the MVT specification. >>>> >>>> >>>> 4. Value-capable classes: supporting methods >>>> >>>> p.3 "This design endows both boxes and pure values with a common set of >>>> methods; it "lifts" box methods >>>> onto the derived values." >>>> p.5 "The synthetic class has the given fields (unchanged) and has no >>>> methods" >>>> p.5 "Meanwhile, all methods (and other class features) stay on the >>>> value-capable class. The value type >>>> proper is just a "dumb struct" containing the extracted fields" >>>> >>>> - given that in the MVT model we are starting with a POJO box, and >>>> instance methods that clearly >>>> take an Object as a receiver, one proposal for the initial MVT >>>> approach would be to have >>>> all methods only supported by the box, and require boxing to invoke >>>> any methods >>>> >>>> - so I think the first quote would need either removal or >>>> modification >>> John: agreed. >>> Karen: See #9 below: Evolved proposal would keep the POJO, which we call >>> the Value Capable Class (VCC) unchanged, >>> and derive a Derived Value Type (DVT) which would only contain a copy of >>> the immutable instance fields, i.e. be >>> a “dumb struct”. >>> Bjorn: In this model, the source class would be the same as the box class, >>> if we leave the instance fields in it. And we >>> would box to invoke methods for the MVT 1.0 timeframe. >>> John: Agreed. >>> John: Longer-term - will want to invoke methods on values as soon as we >>> can. We will need source support for that. >>> >>> John: Minimal Value Type (MVT) programming models: >>> 1) source — only works for boxes >>> 2) Method Handle reflection - for early adopters >>> 3) bytecodes >>> The MethodHandle/ValueFactory approach is clearly described. >>> The language and byte code we will use longterm are still uncertain. >>> Bjorn: What would be enabled by having all members in the value vs. just >>> the [instance] fields in the value? >>> Karen: Challenge is instance methods, where the type of the receiver is >>> expected to be a VCC, not a DVT. This same >>> expectation applies to any method called from the instance method, or any >>> field in which the receiver is stored. >>> John: we are using existing javac support, therefore we need to define >>> value types indirectly, box first. This is not the longterm plan. >>> MethodHandles will provide a direct way of speaking of the values. The >>> MethodHandle runtime will spin byte codes. >>> >>> Maurizio: it is easy to just map the fields >>> Karen: Methods are coded on the box. Static fields we left on the box. So >>> we just lift the instance fields. >>> >>> John: We box the value to run methods. We want to preserve “vagueness”. >>> Legacy code could misuse identity e.g. equals, hashcode, sync >>> This only works for early adopters who are aware of value-capable-class >>> identitylessness and implications >>> >>> Maurizio: If you pass the VCC to another method you are passing the box and >>> no mechanical transformation is needed. >>> Karen: You could have a problem if you were to pass the DVT as an argument >>> when an object is expected >>> John: clarify distinction between QType and LType >>> QType: no identity, not nullable, not shared visible state, no sync, no >>> reference equality >>> LType: identity, nullable, shared visible state for sync and reference >>> equality >>> semantic mismatch: nulls, if_acmp_eq/ne >>> MVT provides a short-term hack - which is ok for early adopters >>> in future expect explicit boxes for QTypes which are LTypes which >>> actually are identityful unlike the temporary VCCs >>> Bjorn: differences between QTypes and LTypes like int and Integer, where >>> only Integer provides methods? >>> John: short-term there will be no methods on a DVT. >>> longer-term value types will have methods >>> verifier will not accept LType for a QType byte code >>> Bjorn: if QType is a subset of an LType is it ok to convert? >>> John: want ability farther future to have different behaviors and different >>> stack representation. >>> In progress, exploring possibly interpreter implementations, things >>> like allowing the vm to buffer QTypes off the java heap >>> presumes a universal type must declare whether it has identity or not >>> Bjorn: with boxing and unboxing, if you say you have “no identity” do you >>> still have something? >>> John: must box to call a method, no one can rely on the identity so you can >>> elide it. >>> Ed note: this is only possibly temporarily because the value-capable-class >>> is defined as not being able to rely on identity. >>> This will not carry over to a more general value type approach if we wish >>> to have value types box to identityful LTypes which >>> can be used by existing code that is expecting a subtype of Object or an >>> Interface as defined today. >>> John: after MVT 1.0, exploring a union type, “UType” which is a common type >>> that could contain either a QType or its corresponding >>> LType >>>> >>>> 5. Value bytecodes >>>> p.11 "Method handles and invokedynamic will always allow bytecode to >>>> invoke methods on Q-types". >>>> - is this still accurate in the context above? I know the comment says >>>> that internally >>>> the MH might box the Qtype, but do we still want to support MH and indy >>>> to appear to invoke methods on Q-types? >>> Ed note: I don’t if we answered this question. >>>> >>>> >>>> 6. Restrictions on the POJO: >>>> A. It would help to have a bulleted list of restrictions >>>> B. Clarify error/exception to throw - perhaps ClassFormatError for all of >>>> these? >>>> C. Request to not support VCC on interfaces at all for MVT 1.0 >>>> >>>> I know the restrictions are intermixed in the text today. This is what >>>> I extracted: >>>> (p.3-4, 6) >>>> >>>> VCC (and probably going forward) >>>> 1. VCC superclass must be Object (and should be omitted) >>>> 2. the class must be final >>>> 3. all fields must be final >>>> - please clarify - all instance fields must be final >>> John: yes >>>> >>>> 4. all constructors private >>> Maurizio: why? >>> Note that there is NO constructor for the DVT. It can be created via a >>> vunbox or vdefault + vwithfield >>> John: ok to change the spec so the VCC constructor is not limited to being >>> private. >>>> >>>> 5. must replace equals, hashcode, toString (with current Object syntax) >>>> 6. may not use any methods provided on Object >>>> specifically: may not use clone, finalize, wait, notify, notifyAll >>>> (directly) >>>> 7. may use getClass >>>> MVT 1.0 additional limitations >>>> 9. may contain primitive instance fields, but no reference instance >>>> fields >>>> - please update document to clarify this restriction is for >>>> instance fields only >>> Karen: both IBM and Oracle JVM engineers are interested in an optional >>> extension to support >>> references instance fields. >>> Maurizio: If only primitives allowed, then no support for generics is >>> needed. >>> Clarification for Karen: statics can have generics with erasure today, but >>> can not mention type variables. (thanks :-) >>> Bjorn: With today’s erased generics, this is not a problem >>> John: Ok to explore having references in instance fields, generics are ok. >>> No type variables in instance fields >>> and no “any” generics. > 2/15/17: rediscussed: > Maurizio - higher perceived benefit to users to have reference fields in > VCC/DVT - e.g. Strings > John concern: hard to do embedded references in values > Mr Simms: almost have it now > — agreed: let’s give it a short >>> >>>> >>>> 10. may not contain generic instance fields >>>> - please update document to clarify this restriction is for >>>> instance fields only >>>> - it is my understanding that you can’t have generic static fields >>>> at all >>>> 11. "interfaces (especially with default methods)" >>>> - please change p.6 to clarify that there are no value type >>>> interfaces period. >>> Ed note: There is a distinction here between >>> 11a) defining an interface as a VCC with a derived DVT and >>> 11b) whether the POJO which defines the VCC can implement interfaces. This >>> discussion was about whether >>> a POJO which defines the VCC can implement interfaces. >>> >>> Karen: concern about setting expectations. Current interfaces assume >>> identity. >>> Maurizio: could always box to call interface methods. >>> >>> John: Question: do early adopters need interfaces? >>> Vladimir Ivanov: Yes >>> >>> note: Vector API has no benefits using MVT 1.0. >>> Ed note: later email clarification from Vlad: >>> Interface-based Vector API version [1] does not benefit from MVT 1.0. All >>> operations are expressed as interface >>> calls and require vector boxes. >>> That is out of scope for MVT 1.0. >>> Vectors exploring an alternative API, exposing operations as MethodHandles. >>> This is less convenient to use, but >>> allows experimenting to find performance benefits. >>> >>> John: MVT: box to get to methods. Longterm get to call I.defaultmethod >>> without boxing >>> Ed. note: Interface default method will need restrictions. >>> >>> John: MVT 1.0 : No value capable interfaces for JVMT 1.0. >>> Ed note: was this the answer to 11a or 11b above please? > I believe John agreed that we can not use a VCC/DVT to define an interface > > Open Question: > Do we need VCC/DVT to support interfaces and require boxing to invoke default > methods? > Perhaps try early access without support for interfaces and get feedback? > > Concerns: existing interfaces with default methods - assume identity > Longer-term interfaces that can be implemented by valhalla value types will > need to be > special interfaces which support UTypes - union of LType and QType - so that > they > can be implemented by references or by value types. > UType - must not assume identity, must retain identity if it has it > QType - not assume identity > LType - must preserve identity > John/Brian - outline of road ahead - post MVT 1.0 - looking at a potential > carrier or wrapper for > an “any” type > Dan: If a local is a UType - will enforce single type through program? > John: simple answer - not fixed type slots > Dan: verification issues > John: like a cast - must be explicit in byte codes and it might fail > LType-> UType always valid, UType-> LType might fail >>> >>> John: longer-term: >>> L-Type: always identity >>> Q- Type: never identity >>> U-Type: do not assume identity, must preserve identity >>> >>>> >>>> 12. 0.2 version states: may not contain a value class as an instance >>>> field >>>> - see below for further discussion >>>> >>>> 7. potential extensions: >>>> 12. 0.2 version states: value class may not contain a value class as an >>>> instance field >>>> - we would like to propose supporting this - perhaps as an optional >>>> extension? >>>> - we would need to add an exception for handling circularity >>>> - note: no way to express this in java, but you could express in a >>>> classfile >>> John: NO for MVT 1.0. Potential ambiguity whether the field contains a >>> value capable class or a derived value type >>> javac just deals with boxes, so no flattening here. Wants same layouts >>> whether boxed or not. >>> >>> John: MVT 1.0 will only flatten arrays reflectively >>> >>> End of discussion >>> >>> thanks, >>> Karen >>> ======= >>> >>>> >>>> 8. Splitting the value type from the object type >>>> Propose not using the nested class approach, to not tie us into this >>>> relationship longer term >>>> - so remove example and the "looks like an inner class" >>>> >>>> - note: a key point here is how the user generating bytecodes would know >>>> the generated name of the DVT >>>> This will need further discussion. >>>> In the constant pool, references can use Qpackage.Class; rather than >>>> Lpackage.Class; >>>> >>>> Are there requirements for java sources to be able to refer to the >>>> derived value type by name? >>>> e.g. Class.forName()? If not, then perhaps the temporary naming >>>> convention could be kept internal? > #8: not use nested class relationship between VCC and DVT: > Bjorn: not an issue either way > *** remove from spec: Does not belong in the spec - this is an implementation > detail > We don’t want to expose this as a nested class > - reflection might expose differently > - private static member access needs special handling, but not via javac > trampolines > - Minimal Value Types will to have nest mate support > Note: box must be in the same package & module > Need a way to mark the DVT - e.g. ACC_VALUE (other options ok) > > Note : DVT name should not be exposed. Ok to use unsafe.defineAnonymousClass > to generate the DVT. The VM generates the DVT. > API ValueType.valueclass() returns, but doesn’t need to know how generated > > Goal: VM’s problem to resolve QPoint descriptor reference to DVT for Point > *** add to spec: requirement: not want byte code to resolve LPoint$12345 (i.e. > a temporary name) - want ClassNotFound here >>>> >>>> 9. Splitting the value type from the object type >>>> p.5 "The original class is given a new synthetic field of the new value >>>> type, to hold the state for the original class". >>>> - to simplify implementation, and allow experiments which go beyond the >>>> initial MVT plans, we propose >>>> * that the VCC is left untouched >>>> * the DVT has a copy of the immutable instance fields >>>> >>>> - We think this qualifies as "any equivalent technique" on p.5 >>>> - the quote above would need modifying or removing >>>> > Agreed. >>>> 10. Scoping of these features >>>> - exploring adding class file capability bits for experimental >>>> features, as a versioning approach >>>> - we will want to pin this down later in the cycle > Brainstormed possibility of adding capability bits in minor version. Would > need to go through JCP. > editor’s note: may need to revisit this - turns out minor version 45.3 was > already used and set a precedent > that changes in a minor version would all be present in follow-on major > versions. So probably not in scope for MVT. >>>> >>>> >>>> 11. JVM changes to support Q-types >>>> - "So when the class loader loads an object whose fields are Q_types, >>>> it must resolve (and perhaps load) >>>> the classes of those Q_types, ..." >>>> - for instance fields that are Q-types, I believe we need to explicitly >>>> specify temporary JVMS load/link/init rules >>>> (I will propose an early draft in a later email). >>>> - e.g. Specifically, for instance fields that are Q-types, we would >>>> propose requiring eager loading of the Q-types, >>>> modifying JVMS 5.3.5 Deriving a Class from a class File >>>> Representation. Bullets 3 and 4 described eager resolution >>>> of the direct superclass and direct superinterfaces. The expectation >>>> is that an additional bullet would be added >>>> for direct instance fields that are Q-types. >>>> - note that this change would make it the JVM's responsibility, not >>>> the class loaders' responsibility, >>>> to resolve the classes of those Q-types. >>>> >>>> Note: in the JVMS load/link/init rules I will also propose VCC/DVT >>>> load/link/init requirements. > Note: there are two discussions here relative to load/link/init rules. > One is for the DVT/VCC relationship. > Karen: always load VCC first > Bjorn: if MH API: always create VCC first > AI: Karen - send load/link/init proposal for MVT > > Another is for flattening. Note VCC and DVT have the same field reference > types, which means no > QType fields, so no flattened fields in a DVT. > > Note: VCC can not mention QTypes in source, since javac can not translate. > > Bjorn: Can we use QTypes in descriptors of methods and fields if we spin byte > codes? > John: MethodHandles work with QTypes, therefore need an internal translation, > therefore need to > handle QTypes in descriptors, therefore ok to spin byte codes like > MethodHandles can. > > This is not a gating goal for MVT, but is not forbidden. > > Is it ok to flatten a QType field? > > Maurizio: > VCC->DVT can NOT change descriptor, therefore can not contain a QType. > > John: For MethodHandles and classes spun from byte codes with QType fields, > it is ok to flatten the QType. > - implementation choice. > - not required, but ok if not hard > > (ed. note: I have other notes on flattening which I can not translate - if > others have notes > on non-array flattening please let me know) > > We will however support flattening for arrays. > Ed note: thanks to Mr. Simms - he realized we need to initialize (and > therefore link) any value type that > is the element type for anewarray/multianewarray. > > Array flattening: > this will not be done automagically > flatten array only if explicitly an array of value types, i.e. > anewarray/multianewarray with a DVT as the > objectref > > TODO: open questions - need to investigate: > - Need to check how core reflection would deal with seeing an object that > is a DVTarray > - What does core reflection do if handed the j.l.Class for a DVT? > - Class.forName should not be able to find a j.l.Class for a DVT > > >>>> >>>> 12. value bytecodes >>>> - the following are useful in the MethodHandle implementation, and >>>> likely to be useful for direct bytecode access >>>> - we would like to propose the following as the minimal bytecode set: >>>> in addition to vload, vstore, vreturn (and slot-specific variants) >>>> - vdefault/vwithfield > open issue > Maurizio: nice on paper > In practice: challenges - vwithfield write to final field > note: all fields in DVT are final > > If we do not provide vdefault/vwithfield, but only provide vbox/vunbox > MethodHandles will use unsafe to set fields and then unbox > ed. note - need to revisit >>>> - vbox/vunbox > YES > Dan: what about a2b vs. vbox/vunbox? > ed note: we initially investigated a general a2b and we don’t yet have a good > way to define a CONSTANT_Type > or equivalent which we would need, so we found it easier to explicitly do > vbox/vunbox. (need to bring up in next meeting) >>>> - vaload/vastore > YES >>>> - vgetfield (fetch a field from a value type) > YES >>>> - NOT vcmp_eq/ne (equality can be implemented as component-wise >>>> comparison) >>>> >>>> clarify that for MVT 1.0, statics are only available through the box. >>>> (TODO: where does this go in shady?) >>>> >>>> 13. value bytecodes >>>> - open issue >>>> - typed prefix vs. vbytecodes for initial prototype >>>> >>>> 14. Value bytecodes >>>> use of Qtype as class component: >>>> "Initially the only valid use of a Q-type [is] as the class component >>>> of a CONSTANT_Methodref or CONSTANT_Fieldref >>>> is as a CONSTANT_MethodHandle constant." >>>> - if we extend the bytecodes as above, and we disallow anyone >>>> (MethodHandles, bytecodes) from invoking methods on Qtypes, we >>>> could modify this to disallow Q-types for CONSTANT_Methodref or >>>> CONSTANT_InterfaceMethodRef completely. >>>> - but perhaps you want the MethodHandles to be able to invoke methods >>>> on DVTs by dynamically boxing them. This works as long >>>> as the methods don't assume identity. >>>> >>>> 15. Q-types and bytecodes >>>> We propose modifying anewarray and multianewarray to allow operands >>>> that are Q-types. >>>> >>>> 16. Value Type Reflection >>>> With the proposed modifications in #8 above: i.e. leaving the VCC >>>> untouched and copying the >>>> instance fields to the DVT, the VCC now matches the source file. >>>> So Class.forName() would return the VCC which is the original POJO >>>> which fits the backward compatibility model. >>>> So we don't need a separate SourceClass, but leaving it in the proposal >>>> provides implementation flexibility. >>>> >>>> 17. Q-type method handles & behaviors "possible bytecode" >>>> might want to change vnew to vdefault >>>> these are samples and evolving, so maybe not worth changing >>>> >>>> I did not do this level of detailed review for the Future Work yet. >>>> >>>> thanks, >>>> Karen >>>> >>>>> On Sep 1, 2016, at 8:08 PM, John Rose <john.r.r...@oracle.com >>>>> <mailto:john.r.r...@oracle.com>> wrote: >>>>> >>>>> On Aug 31, 2016, at 11:59 PM, John Rose <john.r.r...@oracle.com >>>>> <mailto:john.r.r...@oracle.com>> wrote: >>>>>> >>>>>> >>>>>> I have updated of this document to reflect comments so far. >>>>>> It is stored to CR (in place) and enclosed here. >>>>>> — John >>>>>> >>>>>> Link: http://cr.openjdk.java.net/~jrose/values/shady-values.html >>>>>> <http://cr.openjdk.java.net/~jrose/values/shady-values.html> >>>>>> >>>>>> <shady-values.html> >>>>> >>>>> I have updated the document again with small corrections and >>>>> clarifications.