Re: Updated VM-bridges document
I need to do many more additional examples offline.
I appreciate your trying to make overriding of forwarders simpler for the jvm.
I would like to continue to explore the option having the jvm do the
calculation of overriding
both direct and indirect forwarders until we’ve worked more examples.
If we can find a way to do it, it helps with backward compatibility
- old clients with old receivers don’t go through adaptors - so they miss
adaptations
that could either throw an exception or potentially lose data through narrowing.
- same issue for reflection - Class.getDeclaredMethods() which just returns
local methods
- would be nice if we could not lose existing method names here
I am also exploring invoke local with explicit local name of method - so we can
try
to reduce loops - I believe there will be steps at which we will need to
identify loops and throw
exceptions or not create a reverser.
That said, it is getting more complex, so glad you are exploring alternatives.
Link below spells out a bit more the rule I am exploring for creating reversers,
with both the example below and another example (Example II) which has three
migration steps,
F <: E <: D
all start with m(Date, Time)
step 1: D m(Date, Time) -> D.m(LDT, Time)
step 2: E.m(Date, Time) -> E.m( Date, LDT)
step 3: D.m(LDT, Time) -> D.m(LDT, LDT)
http://cr.openjdk.java.net/~acorn/Forwarders.pdf
thanks,
Karen
> On Apr 12, 2019, at 11:44 AM, Brian Goetz wrote:
>
>
>
>> A VM perspective:
>>
>> invocation
>> dynamic receiver
>> resolution
>> NOT invoked
>> selection:
>> actual execution
>> invokevirtual D::m(LDT)
>> D
>> D.m(LDT)
>> D.m(LDT)
>> invokevirtual D::m(LDT)
>> E
>> D.m(LDT)
>> E.m(LDT)
>> reverser: adapt LDT->Date
>>invoke local E.m(Date)
>>if return had changed, adapt return back
>> invokevirtual D::m(Date)
>> D
>> D.m(Date)
>> D.m(Date)
>> forwarder: adapt Date->LDT
>> invoke local m(LDT)
>> if return had changed, adapt
>> invokevirtual D.m(Date)
>> E
>> D.m(Date)
>> E.m(Date)
>> invokevirtual E.m(LDT)
>> E
>> E.m(LDT)
>> reverser)
>> E.m(LDT):
>> reverser: adapt LDT->Date
>>invoke local E.m(Date)
>>if return had changed, adapt return back
>> invokevirtual E.m(Date)
>> E
>> E.m(Date)
>> E.m(Date) // original - unchanged behavior
>>
>
> Let me try from the other direction, using the JVMS terminology rather than
> appealing to as-if. Where I think we're saying slightly different things is
> in the interpretation of the lines I colored in blue above (hope the
> formatting came through.) You are talking about the E.m(Date) that appears
> in E.class (good so far). But I'm talking about the _members_ of E. And the
> E.m(Date) that appears in E.class should _not_ be considered a (new-to-E)
> member of E. Instead, that E.m(Date) gives rise to a synthetic member
> E.m(LDT). I have colored two cases in red because I think this is where our
> assumptions really parted ways; will come back to this at the bottom.
>
> Here's why I'm harping on this distinction; we mark methods as "forwarders"
> and do something special when we see something override a forwarder. Taking
> the same hierarchy:
>
> // before
> class D {
> void m(Date) { }
> }
>
> class E extends D {
> void m(Date) { }
> }
>
> // middle -- D migrates, but E not yet
> class D {
> void m(LDT) { }
> @Forwarding( m(LDT) } void m(Date);
> }
>
> class E extends D {
> void m(Date) { }
> }
>
> // after -- E finally gets the memo
> class D {
> void m(LDT) { }
> @Forwarding( m(LDT) } void m(Date);
> }
>
> class E extends D {
> void m(LDT) { }
> }
>
> Now, let's draw inheritance diagrams (these are not vtables, they are member
> tables). I'll use your notation, where I think D.m(X) means "the Code
> attribute declared in D for m(X)".
>
> Before
> m(Date)
> D
> D.m(Date)
> E
> E.m(Date)
>
> This part is easy; D has m(Date), and E overrides it.
>
> Middle
> m(LDT)
> m(Date)
> D
> D.m(LDT)
> forwarder -> m(LDT)
> E
> reverser adapted from E.m(Date)
> inherits forwarder
>
> Now, both D and E have both m(Date) and m(LDT). D has a real method for
> m(LDT), and a forwarder for m(Date). E has an m(Date), which we see
> overrides a forwarder. So we adapt it to be an m(LDT), but we consider E to
> have inherited the forwarder from D. I'll come back to this in a minute.
>
> After m(LDT)
> m(Date)
> D
> D.m(LDT)
> forwarder -> m(LDT)
> E
> E.m(LDT)
> inherits forwarder
>
> In this nirvana, there is a forwarder still, but it doesn't affect E, because
> E has already gotten the memo. It sits around purely in the case that
> someone calls m(Date).
>
> OK, so why am I saying that membership has to be tilted this way? Let's go
> back to the middle case, and add
>
> class F extends E {
> void
Re: RefObject and ValObject
My original thought had been that this would benefit new code that knew it had to do something identity-full (lock, make weak references, etc) and wanted to enforce it would never see a value.
The migration path adds a cherry on top.
--Dan
- Original message -From: Brian Goetz To: Daniel Heidinga Cc: valhalla-spec-experts Subject: Re: RefObject and ValObjectDate: Fri, Apr 12, 2019 11:51 AM
High-order tradeoffs: - Having R/VObject be classes helps from the pedagogical perspective(it paints an accurate map of the object model.) - There were some anomalies raised that were the result of rewritingan Object supertype to RefObject, and some concerns about "all ourtables got one level deeper." I don't really have a strong opinion onthese. - Using interfaces is less intrusive, but less powerful. - None of the approaches give an obvious solution for the "make me alock Object" problem.I think the useful new observation in this line of discussion is this: - The premise of L-World is that legacy Object-consuming code can keepworking with values. - We think that's a good thing. - But we also think there will be some cases where that's not agood thing, and that code will wish it had said `m(RefObject)` insteadof `m(Object)`. [ this is the new thing ]Combining this with the migration stuff going on in a separate thread, Ithink what you're saying is you want to be able to take a method: m(Object o) { }and _migrate_ it to be m(RefObject o) { }with a forwarder @ForwardTo( m(RefObject) ) m(Object o);So that code could, eventually, be migrated to RefObject-consuming code,and all is good again. And the JIT can see that o is a RefObject andcredibly fall back to a legacy interpretation of ACMP and locking.On 4/12/2019 11:16 AM, Daniel Heidinga wrote:> During the last EG call, I suggested there are benefits to having both RefObject and ValObject be classes rather than interfaces.>> Old code should be able work with both values and references (that's the promise of L-World after all!). New code should be able to opt into whether it wants to handle only references or values as there are APIs that may only make sense for one or the other. A good example of this is java.lang.Reference-subtypes which can't reasonably deal with values. Having RefObject in their method signatures would ensure that they're never passed a ValObject. (ie: the ctor becomes WeakReference(RefObject o) {...})>> For good or ill, interfaces are not checked by the verifier. They're passed as though they are object and the interface check is delayed until invokeinterface, etc. Using interfaces for Ref/Val Object doesn't provide verifier guarantees that the methods will never be passed the wrong type. Javac may not generate the code but the VM can't count on that being the case due to bytecode instrumentation, other compilers, etc.>> Using classes does provide a strong guarantee to the VM which will help to alleviate any costs (acmp, array access) for methods that are declared in terms of RefObject and ensures that the user is getting exactly what they asked for when they declared their method to take RefObject.>> It does leave some oddities as you mention:> * new Object() -> returns a new RefObject> * getSuperclass() for old code may return a new superclass (though this may be the case already when using instrumentation in the classfile load hook)> * others?>> though adding interfaces does as well:> * getInterfaces() would return an interface not declared in the source> * Object would need to implement RefObject for the 'new Object()` case which would mean all values implemented RefObject (yuck!)>> Letting users say what they mean and have it strongly enforced by the verifier is preferable in my view, especially as getSuperclass() issue will only apply to old code as newly compiled code will have the correct superclass in its classfile.>> --Dan>>> -"valhalla-spec-experts" wrote: ->>> To: valhalla-spec-experts >> From: Brian Goetz>> Sent by: "valhalla-spec-experts">> Date: 04/08/2019 04:00PM>> Subject: RefObject and ValObject We never reached consensus on how to surface Ref/ValObject. Here are some places we might want to use these type names: - Parameter types / variables: we might want to restrict the domain>> of a parameter or variable to only hold a reference, or a value: void m(RefObject ro) { … } - Type bounds: we might want to restrict the instantiation of a>> generic class to only hold a reference (say, because we’re going to>> lock on it): class Foo { … } - Dynamic tests: if locking on a value is to throw, there must be a>> reasonable idiom that users can use to detect lockability without>> just trying to lock: if (x instanceof RefObject) {>> synchronized(x) { … }>> } - Ref- or Val-specific methods. This one is more vague, but its>> conceivable we may want methods on ValObject that are members of all>> values.>> There’s been three ways proposed (so far) that we
Re: RefObject and ValObject
High-order tradeoffs:
- Having R/VObject be classes helps from the pedagogical perspective
(it paints an accurate map of the object model.)
- There were some anomalies raised that were the result of rewriting
an Object supertype to RefObject, and some concerns about "all our
tables got one level deeper." I don't really have a strong opinion on
these.
- Using interfaces is less intrusive, but less powerful.
- None of the approaches give an obvious solution for the "make me a
lock Object" problem.
I think the useful new observation in this line of discussion is this:
- The premise of L-World is that legacy Object-consuming code can keep
working with values.
- We think that's a good thing.
- But we also think there will be some cases where that's not a
good thing, and that code will wish it had said `m(RefObject)` instead
of `m(Object)`. [ this is the new thing ]
Combining this with the migration stuff going on in a separate thread, I
think what you're saying is you want to be able to take a method:
m(Object o) { }
and _migrate_ it to be
m(RefObject o) { }
with a forwarder
@ForwardTo( m(RefObject) )
m(Object o);
So that code could, eventually, be migrated to RefObject-consuming code,
and all is good again. And the JIT can see that o is a RefObject and
credibly fall back to a legacy interpretation of ACMP and locking.
On 4/12/2019 11:16 AM, Daniel Heidinga wrote:
During the last EG call, I suggested there are benefits to having both
RefObject and ValObject be classes rather than interfaces.
Old code should be able work with both values and references (that's the
promise of L-World after all!). New code should be able to opt into whether it
wants to handle only references or values as there are APIs that may only make
sense for one or the other. A good example of this is
java.lang.Reference-subtypes which can't reasonably deal with values. Having
RefObject in their method signatures would ensure that they're never passed a
ValObject. (ie: the ctor becomes WeakReference(RefObject o) {...})
For good or ill, interfaces are not checked by the verifier. They're passed as
though they are object and the interface check is delayed until
invokeinterface, etc. Using interfaces for Ref/Val Object doesn't provide
verifier guarantees that the methods will never be passed the wrong type. Javac
may not generate the code but the VM can't count on that being the case due to
bytecode instrumentation, other compilers, etc.
Using classes does provide a strong guarantee to the VM which will help to
alleviate any costs (acmp, array access) for methods that are declared in terms
of RefObject and ensures that the user is getting exactly what they asked for
when they declared their method to take RefObject.
It does leave some oddities as you mention:
* new Object() -> returns a new RefObject
* getSuperclass() for old code may return a new superclass (though this may be
the case already when using instrumentation in the classfile load hook)
* others?
though adding interfaces does as well:
* getInterfaces() would return an interface not declared in the source
* Object would need to implement RefObject for the 'new Object()` case which
would mean all values implemented RefObject (yuck!)
Letting users say what they mean and have it strongly enforced by the verifier
is preferable in my view, especially as getSuperclass() issue will only apply
to old code as newly compiled code will have the correct superclass in its
classfile.
--Dan
-"valhalla-spec-experts"
wrote: -
To: valhalla-spec-experts
From: Brian Goetz
Sent by: "valhalla-spec-experts"
Date: 04/08/2019 04:00PM
Subject: RefObject and ValObject
We never reached consensus on how to surface Ref/ValObject.
Here are some places we might want to use these type names:
- Parameter types / variables: we might want to restrict the domain
of a parameter or variable to only hold a reference, or a value:
void m(RefObject ro) { … }
- Type bounds: we might want to restrict the instantiation of a
generic class to only hold a reference (say, because we’re going to
lock on it):
class Foo { … }
- Dynamic tests: if locking on a value is to throw, there must be a
reasonable idiom that users can use to detect lockability without
just trying to lock:
if (x instanceof RefObject) {
synchronized(x) { … }
}
- Ref- or Val-specific methods. This one is more vague, but its
conceivable we may want methods on ValObject that are members of all
values.
There’s been three ways proposed (so far) that we might reflect these
as top types:
- RefObject and ValObject are (somewhat special) classes. We spell
(at least in the class file) “value class” as “class X extends
ValObject”. We implicitly rewrite reference classes at runtime that
extend Object to extend RefObject instead. This has obvious
pedagogical value, but there are some (small) risks of anomalies.
- RefObject and ValObject are interfaces.
Re: Updated VM-bridges document
A VM perspective:
*invocation*
*dynamic receiver*
*resolution*
*NOT invoked*
*selection:*
*actual execution*
invokevirtual D::m(LDT)
D
D.m(LDT)
D.m(LDT)
invokevirtual D::m(LDT)
E
D.m(LDT)
E.m(LDT)
reverser: adapt LDT->Date
invoke local E.m(Date)
if return had changed, adapt return back
invokevirtual D::m(Date)
D
D.m(Date)
D.m(Date)
forwarder: adapt Date->LDT
invoke local m(LDT)
if return had changed, adapt
invokevirtual D.m(Date)
E
D.m(Date)
E.m(Date)
invokevirtual E.m(LDT)
E
E.m(LDT)
reverser)
E.m(LDT):
reverser: adapt LDT->Date
invoke local E.m(Date)
if return had changed, adapt return back
invokevirtual E.m(Date)
E
E.m(Date)
E.m(Date) // original - unchanged behavior
Let me try from the other direction, using the JVMS terminology rather
than appealing to as-if. Where I think we're saying slightly different
things is in the interpretation of the lines I colored in blue above
(hope the formatting came through.) You are talking about the E.m(Date)
that appears in E.class (good so far). But I'm talking about the
_members_ of E. And the E.m(Date) that appears in E.class should _not_
be considered a (new-to-E) member of E. Instead, that E.m(Date) gives
rise to a synthetic member E.m(LDT). I have colored two cases in red
because I think this is where our assumptions really parted ways; will
come back to this at the bottom.
Here's why I'm harping on this distinction; we mark methods as
"forwarders" and do something special when we see something override a
forwarder. Taking the same hierarchy:
// before
class D {
void m(Date) { }
}
class E extends D {
void m(Date) { }
}
// middle -- D migrates, but E not yet
class D {
void m(LDT) { }
@Forwarding( m(LDT) } void m(Date);
}
class E extends D {
void m(Date) { }
}
// after -- E finally gets the memo
class D {
void m(LDT) { }
@Forwarding( m(LDT) } void m(Date);
}
class E extends D {
void m(LDT) { }
}
Now, let's draw inheritance diagrams (these are not vtables, they are
member tables). I'll use your notation, where I think D.m(X) means "the
Code attribute declaredin D for m(X)".
Before
m(Date)
D
D.m(Date)
E
E.m(Date)
This part is easy; D has m(Date), and E overrides it.
Middle
m(LDT)
m(Date)
D
D.m(LDT)
forwarder -> m(LDT)
E
reverser adapted from E.m(Date)
inherits forwarder
Now, both D and E have both m(Date) and m(LDT). D has a real method for
m(LDT), and a forwarder for m(Date). E has an m(Date), which we see
overrides a forwarder. So we adapt it to be an m(LDT), but we consider
E to have inherited the forwarder from D. I'll come back to this in a
minute.
After m(LDT)
m(Date)
D
D.m(LDT)
forwarder -> m(LDT)
E
E.m(LDT)
inherits forwarder
In this nirvana, there is a forwarder still, but it doesn't affect E,
because E has already gotten the memo. It sits around purely in the
case that someone calls m(Date).
OK, so why am I saying that membership has to be tilted this way? Let's
go back to the middle case, and add
class F extends E {
void m(Date) { } // still didn't get the memo
}
Middle
m(LDT)
m(Date)
D
D.m(LDT)
forwarder -> m(LDT)
E
reverser adapted from E.m(Date)
inherits forwarder
F
reverser adapted from F.m(Date)
inherits forwarder
When we go to compute members, I want to see that _F.m(Date) overrides a
forwarder too_. If we merely put E.m(Date) in the (E, m(Date)) box,
then it looks like F is overriding an ordinary member, and no reverser
is generated. (Or, we have to keep walking up the chain to see if
E.m(Date) in turn overrides a forwarder -- yuck, plus, that makes
forwarder-overrides-forwarder even messier.
Now, back to your table. The above interpretation of what is going on
comes to the same answer for all of the rows of your table, except these:
*invocation*
*dynamic receiver*
*resolution*
*NOT invoked*
*selection:*
*actual execution*
invokevirtual D.m(Date)
E
D.m(Date)
E.m(Date)
invokevirtual E.m(Date)
E
E.m(Date)
E.m(Date) // original - unchanged behavior
You are thinking "E has a perfectly good m(Date), let's just select
that". Makes sense, but the cost of that is that it complicates
calculation of membership and overriding. I think I am content to let
invocations of m(Date) on receivers of type E go through both rounds of
adaptation: forward the call (with adaptation) to m(LDT), which, in the
case of E, does the reverse
Re: Updated VM-bridges document
This leads us to the next question, given that you can only override "locally" a forwarder, what if a forwarder overrides a forwarder ? You throw a LinkageError ? Yes, this could arise from inconsistent separate compilation (I thought I covered this in my doc?) Best choice is probably to let the override proceed, establishing a new forwarder in that slot. (A lot of the time when this happens, it will be forwarding to the same place anyway.) The is the same thing we do with bridges overriding bridges. A good mental model (for my brain) here is that forwarders act a little like final methods. When a method overrides a final method, we throw a hard error. But here, when we see a method overriding a "final-ish" method, if it is a regular method, we shunt it out of the way, and if it is a new final-ish method, we let it take over the slot.
Re: Updated VM-bridges document
- Mail original -
> De: "Brian Goetz"
> À: "Karen Kinnear"
> Cc: "valhalla-spec-experts"
> Envoyé: Vendredi 12 Avril 2019 01:04:15
> Objet: Re: Updated VM-bridges document
> On 4/11/2019 5:18 PM, Karen Kinnear wrote:
>>>
>>> OK, so at this point, the classfiles that have been loaded look like:
>>>
>>> class D {
>>> void m(LDT) { real method }
>>> @Forwarding(m(LDT)) abstract void m(Date);
>>> }
>>>
>>> class E extends D {
>>> @Override
>>> m(Date) { impl }
>>> }
>>>
>>> So D has members m(LTD) and m(Date), the latter is a forwarder.
>>> Therefore E has the same members (instance methods are inherited).
>> From a source perspective, E has the same names of members, although
>> it has overridden the contents of m(Date).
>>
>>>
>>> Here's how I would imagine this turns into in the VM:
>> not important, but this was m(LDT) not m(LTD)
>>>
>>> class D {
>>> void m(LTD) { real method }
>>> void m(Date d) { m(adapt(d)); } // generated forwarder
>>> }
>>>
>>> class E extends D {
>>> private void m$synthetic(Date d) { real method, body as
>>> present in classfile }
>> I would expect that the existing m(Date) with the real method would
>> stay unchanged - including
>> the name and the access controls - since there may be clients of
>> subclass E still trying to invoke it.
>
> I think this is our point of disconnect.
>
> The subclass has overridden a forwarder. What we want to do is "heal
> the rift" by rewriting the subclass as if it had _only_ overridden the
> real method. Hence, the "shunt it off to a synthetic" and create an
> overriding reverser that overrides the real method, adapting
> args/return, which delegates to the shuntee.
>
> If we left m(Date) in E, then this would be overriding the forwarder,
> effectively un-doing the effect of forwarding.
>
> Note that this is all "as if"; there are a hundred ways to _actually_ do
> it.
Another way to see this effect is to say that it actually override the
forwarder but locally, just for that class, in subclasses, the forwarder is
still present
This leads us to the next question, given that you can only override "locally"
a forwarder, what if a forwarder overrides a forwarder ? You throw a
LinkageError ?
Rémi
