Re: Updated VM-bridges document
I need to do many more additional examples offline.
I appreciate your trying to make overriding of forwarders simpler for the jvm.
I would like to continue to explore the option having the jvm do the
calculation of overriding
both direct and indirect forwarders until we’ve worked more examples.
If we can find a way to do it, it helps with backward compatibility
- old clients with old receivers don’t go through adaptors - so they miss
adaptations
that could either throw an exception or potentially lose data through narrowing.
- same issue for reflection - Class.getDeclaredMethods() which just returns
local methods
- would be nice if we could not lose existing method names here
I am also exploring invoke local with explicit local name of method - so we can
try
to reduce loops - I believe there will be steps at which we will need to
identify loops and throw
exceptions or not create a reverser.
That said, it is getting more complex, so glad you are exploring alternatives.
Link below spells out a bit more the rule I am exploring for creating reversers,
with both the example below and another example (Example II) which has three
migration steps,
F <: E <: D
all start with m(Date, Time)
step 1: D m(Date, Time) -> D.m(LDT, Time)
step 2: E.m(Date, Time) -> E.m( Date, LDT)
step 3: D.m(LDT, Time) -> D.m(LDT, LDT)
http://cr.openjdk.java.net/~acorn/Forwarders.pdf
thanks,
Karen
> On Apr 12, 2019, at 11:44 AM, Brian Goetz wrote:
>
>
>
>> A VM perspective:
>>
>> invocation
>> dynamic receiver
>> resolution
>> NOT invoked
>> selection:
>> actual execution
>> invokevirtual D::m(LDT)
>> D
>> D.m(LDT)
>> D.m(LDT)
>> invokevirtual D::m(LDT)
>> E
>> D.m(LDT)
>> E.m(LDT)
>> reverser: adapt LDT->Date
>>invoke local E.m(Date)
>>if return had changed, adapt return back
>> invokevirtual D::m(Date)
>> D
>> D.m(Date)
>> D.m(Date)
>> forwarder: adapt Date->LDT
>> invoke local m(LDT)
>> if return had changed, adapt
>> invokevirtual D.m(Date)
>> E
>> D.m(Date)
>> E.m(Date)
>> invokevirtual E.m(LDT)
>> E
>> E.m(LDT)
>> reverser)
>> E.m(LDT):
>> reverser: adapt LDT->Date
>>invoke local E.m(Date)
>>if return had changed, adapt return back
>> invokevirtual E.m(Date)
>> E
>> E.m(Date)
>> E.m(Date) // original - unchanged behavior
>>
>
> Let me try from the other direction, using the JVMS terminology rather than
> appealing to as-if. Where I think we're saying slightly different things is
> in the interpretation of the lines I colored in blue above (hope the
> formatting came through.) You are talking about the E.m(Date) that appears
> in E.class (good so far). But I'm talking about the _members_ of E. And the
> E.m(Date) that appears in E.class should _not_ be considered a (new-to-E)
> member of E. Instead, that E.m(Date) gives rise to a synthetic member
> E.m(LDT). I have colored two cases in red because I think this is where our
> assumptions really parted ways; will come back to this at the bottom.
>
> Here's why I'm harping on this distinction; we mark methods as "forwarders"
> and do something special when we see something override a forwarder. Taking
> the same hierarchy:
>
> // before
> class D {
> void m(Date) { }
> }
>
> class E extends D {
> void m(Date) { }
> }
>
> // middle -- D migrates, but E not yet
> class D {
> void m(LDT) { }
> @Forwarding( m(LDT) } void m(Date);
> }
>
> class E extends D {
> void m(Date) { }
> }
>
> // after -- E finally gets the memo
> class D {
> void m(LDT) { }
> @Forwarding( m(LDT) } void m(Date);
> }
>
> class E extends D {
> void m(LDT) { }
> }
>
> Now, let's draw inheritance diagrams (these are not vtables, they are member
> tables). I'll use your notation, where I think D.m(X) means "the Code
> attribute declared in D for m(X)".
>
> Before
> m(Date)
> D
> D.m(Date)
> E
> E.m(Date)
>
> This part is easy; D has m(Date), and E overrides it.
>
> Middle
> m(LDT)
> m(Date)
> D
> D.m(LDT)
> forwarder -> m(LDT)
> E
> reverser adapted from E.m(Date)
> inherits forwarder
>
> Now, both D and E have both m(Date) and m(LDT). D has a real method for
> m(LDT), and a forwarder for m(Date). E has an m(Date), which we see
> overrides a forwarder. So we adapt it to be an m(LDT), but we consider E to
> have inherited the forwarder from D. I'll come back to this in a minute.
>
> After m(LDT)
> m(Date)
> D
> D.m(LDT)
> forwarder -> m(LDT)
> E
> E.m(LDT)
> inherits forwarder
>
> In this nirvana, there is a forwarder still, but it doesn't affect E, because
> E has already gotten the memo. It sits around purely in the case that
> someone calls m(Date).
>
> OK, so why am I saying that membership has to be tilted this way? Let's go
> back to the middle case, and add
>
> class F extends E {
> void m(
Re: Updated VM-bridges document
A VM perspective:
*invocation*
*dynamic receiver*
*resolution*
*NOT invoked*
*selection:*
*actual execution*
invokevirtual D::m(LDT)
D
D.m(LDT)
D.m(LDT)
invokevirtual D::m(LDT)
E
D.m(LDT)
E.m(LDT)
reverser: adapt LDT->Date
invoke local E.m(Date)
if return had changed, adapt return back
invokevirtual D::m(Date)
D
D.m(Date)
D.m(Date)
forwarder: adapt Date->LDT
invoke local m(LDT)
if return had changed, adapt
invokevirtual D.m(Date)
E
D.m(Date)
E.m(Date)
invokevirtual E.m(LDT)
E
E.m(LDT)
reverser)
E.m(LDT):
reverser: adapt LDT->Date
invoke local E.m(Date)
if return had changed, adapt return back
invokevirtual E.m(Date)
E
E.m(Date)
E.m(Date) // original - unchanged behavior
Let me try from the other direction, using the JVMS terminology rather
than appealing to as-if. Where I think we're saying slightly different
things is in the interpretation of the lines I colored in blue above
(hope the formatting came through.) You are talking about the E.m(Date)
that appears in E.class (good so far). But I'm talking about the
_members_ of E. And the E.m(Date) that appears in E.class should _not_
be considered a (new-to-E) member of E. Instead, that E.m(Date) gives
rise to a synthetic member E.m(LDT). I have colored two cases in red
because I think this is where our assumptions really parted ways; will
come back to this at the bottom.
Here's why I'm harping on this distinction; we mark methods as
"forwarders" and do something special when we see something override a
forwarder. Taking the same hierarchy:
// before
class D {
void m(Date) { }
}
class E extends D {
void m(Date) { }
}
// middle -- D migrates, but E not yet
class D {
void m(LDT) { }
@Forwarding( m(LDT) } void m(Date);
}
class E extends D {
void m(Date) { }
}
// after -- E finally gets the memo
class D {
void m(LDT) { }
@Forwarding( m(LDT) } void m(Date);
}
class E extends D {
void m(LDT) { }
}
Now, let's draw inheritance diagrams (these are not vtables, they are
member tables). I'll use your notation, where I think D.m(X) means "the
Code attribute declaredin D for m(X)".
Before
m(Date)
D
D.m(Date)
E
E.m(Date)
This part is easy; D has m(Date), and E overrides it.
Middle
m(LDT)
m(Date)
D
D.m(LDT)
forwarder -> m(LDT)
E
reverser adapted from E.m(Date)
inherits forwarder
Now, both D and E have both m(Date) and m(LDT). D has a real method for
m(LDT), and a forwarder for m(Date). E has an m(Date), which we see
overrides a forwarder. So we adapt it to be an m(LDT), but we consider
E to have inherited the forwarder from D. I'll come back to this in a
minute.
After m(LDT)
m(Date)
D
D.m(LDT)
forwarder -> m(LDT)
E
E.m(LDT)
inherits forwarder
In this nirvana, there is a forwarder still, but it doesn't affect E,
because E has already gotten the memo. It sits around purely in the
case that someone calls m(Date).
OK, so why am I saying that membership has to be tilted this way? Let's
go back to the middle case, and add
class F extends E {
void m(Date) { } // still didn't get the memo
}
Middle
m(LDT)
m(Date)
D
D.m(LDT)
forwarder -> m(LDT)
E
reverser adapted from E.m(Date)
inherits forwarder
F
reverser adapted from F.m(Date)
inherits forwarder
When we go to compute members, I want to see that _F.m(Date) overrides a
forwarder too_. If we merely put E.m(Date) in the (E, m(Date)) box,
then it looks like F is overriding an ordinary member, and no reverser
is generated. (Or, we have to keep walking up the chain to see if
E.m(Date) in turn overrides a forwarder -- yuck, plus, that makes
forwarder-overrides-forwarder even messier.
Now, back to your table. The above interpretation of what is going on
comes to the same answer for all of the rows of your table, except these:
*invocation*
*dynamic receiver*
*resolution*
*NOT invoked*
*selection:*
*actual execution*
invokevirtual D.m(Date)
E
D.m(Date)
E.m(Date)
invokevirtual E.m(Date)
E
E.m(Date)
E.m(Date) // original - unchanged behavior
You are thinking "E has a perfectly good m(Date), let's just select
that". Makes sense, but the cost of that is that it complicates
calculation of membership and overriding. I think I am content to let
invocations of m(Date) on receivers of type E go through both rounds of
adaptation: forward the call (with adaptation) to m(LDT), which, in the
case of E, does the reverse
Re: Updated VM-bridges document
This leads us to the next question, given that you can only override "locally" a forwarder, what if a forwarder overrides a forwarder ? You throw a LinkageError ? Yes, this could arise from inconsistent separate compilation (I thought I covered this in my doc?) Best choice is probably to let the override proceed, establishing a new forwarder in that slot. (A lot of the time when this happens, it will be forwarding to the same place anyway.) The is the same thing we do with bridges overriding bridges. A good mental model (for my brain) here is that forwarders act a little like final methods. When a method overrides a final method, we throw a hard error. But here, when we see a method overriding a "final-ish" method, if it is a regular method, we shunt it out of the way, and if it is a new final-ish method, we let it take over the slot.
Re: Updated VM-bridges document
- Mail original -
> De: "Brian Goetz"
> À: "Karen Kinnear"
> Cc: "valhalla-spec-experts"
> Envoyé: Vendredi 12 Avril 2019 01:04:15
> Objet: Re: Updated VM-bridges document
> On 4/11/2019 5:18 PM, Karen Kinnear wrote:
>>>
>>> OK, so at this point, the classfiles that have been loaded look like:
>>>
>>> class D {
>>> void m(LDT) { real method }
>>> @Forwarding(m(LDT)) abstract void m(Date);
>>> }
>>>
>>> class E extends D {
>>> @Override
>>> m(Date) { impl }
>>> }
>>>
>>> So D has members m(LTD) and m(Date), the latter is a forwarder.
>>> Therefore E has the same members (instance methods are inherited).
>> From a source perspective, E has the same names of members, although
>> it has overridden the contents of m(Date).
>>
>>>
>>> Here's how I would imagine this turns into in the VM:
>> not important, but this was m(LDT) not m(LTD)
>>>
>>> class D {
>>> void m(LTD) { real method }
>>> void m(Date d) { m(adapt(d)); } // generated forwarder
>>> }
>>>
>>> class E extends D {
>>> private void m$synthetic(Date d) { real method, body as
>>> present in classfile }
>> I would expect that the existing m(Date) with the real method would
>> stay unchanged - including
>> the name and the access controls - since there may be clients of
>> subclass E still trying to invoke it.
>
> I think this is our point of disconnect.
>
> The subclass has overridden a forwarder. What we want to do is "heal
> the rift" by rewriting the subclass as if it had _only_ overridden the
> real method. Hence, the "shunt it off to a synthetic" and create an
> overriding reverser that overrides the real method, adapting
> args/return, which delegates to the shuntee.
>
> If we left m(Date) in E, then this would be overriding the forwarder,
> effectively un-doing the effect of forwarding.
>
> Note that this is all "as if"; there are a hundred ways to _actually_ do
> it.
Another way to see this effect is to say that it actually override the
forwarder but locally, just for that class, in subclasses, the forwarder is
still present
This leads us to the next question, given that you can only override "locally"
a forwarder, what if a forwarder overrides a forwarder ? You throw a
LinkageError ?
Rémi
Re: Updated VM-bridges document
On 4/11/2019 5:18 PM, Karen Kinnear wrote:
OK, so at this point, the classfiles that have been loaded look like:
class D {
void m(LDT) { real method }
@Forwarding(m(LDT)) abstract void m(Date);
}
class E extends D {
@Override
m(Date) { impl }
}
So D has members m(LTD) and m(Date), the latter is a forwarder.
Therefore E has the same members (instance methods are inherited).
From a source perspective, E has the same names of members, although
it has overridden the contents of m(Date).
Here's how I would imagine this turns into in the VM:
not important, but this was m(LDT) not m(LTD)
class D {
void m(LTD) { real method }
void m(Date d) { m(adapt(d)); } // generated forwarder
}
class E extends D {
private void m$synthetic(Date d) { real method, body as
present in classfile }
I would expect that the existing m(Date) with the real method would
stay unchanged - including
the name and the access controls - since there may be clients of
subclass E still trying to invoke it.
I think this is our point of disconnect.
The subclass has overridden a forwarder. What we want to do is "heal
the rift" by rewriting the subclass as if it had _only_ overridden the
real method. Hence, the "shunt it off to a synthetic" and create an
overriding reverser that overrides the real method, adapting
args/return, which delegates to the shuntee.
If we left m(Date) in E, then this would be overriding the forwarder,
effectively un-doing the effect of forwarding.
Note that this is all "as if"; there are a hundred ways to _actually_ do
it.
Re: Updated VM-bridges document
> On Apr 10, 2019, at 5:22 PM, Brian Goetz wrote:
>
> OK, so in the old world, D has m(Date).
>
>
> Now, D has m(LDT), with a forwarder from m(Date) -> m(LDT), with some sort of
> metadata stapled somewhere to effect the Date <--> LDT conversions.
>
>> class E extends D { m(Date); } which now overrides the forwarder.
>> We do not change class E. We do not recompile it
>
>> old class ClientD invokevirtual D.m(Date) receiver:E
>> Migration step 3: new class ClientD invokevirtual D.m(LDT) receiver:E
>>resolution: finds D.m(LDT)
>>selection: starts with E, there is no E.m(LDT) so call D.m(LDT)
(LDT is LocalDateTime)
>
> OK, so at this point, the classfiles that have been loaded look like:
>
> class D {
> void m(LDT) { real method }
> @Forwarding(m(LDT)) abstract void m(Date);
> }
>
> class E extends D {
> @Override
> m(Date) { impl }
> }
>
> So D has members m(LTD) and m(Date), the latter is a forwarder. Therefore E
> has the same members (instance methods are inherited).
From a source perspective, E has the same names of members, although it has
overridden the contents of m(Date).
>
> Here's how I would imagine this turns into in the VM:
not important, but this was m(LDT) not m(LTD)
>
> class D {
> void m(LTD) { real method }
> void m(Date d) { m(adapt(d)); } // generated forwarder
> }
>
> class E extends D {
> private void m$synthetic(Date d) { real method, body as present in
> classfile }
I would expect that the existing m(Date) with the real method would stay
unchanged - including
the name and the access controls - since there may be clients of subclass E
still trying to invoke it.
> void m(LTD ltd) { m$synthetic(adapt(ltd)); } // generated reverser
I think we are in agreement that there is a reverser:
void m(LDT) generated receiver:
1) adapt LDT -> Date
2) invoke local m(Date)
3) if return had changed, adapt back
// adaptations for reverser are the inverse as for the forwarder
> }
>
>
> resolves
> selects
> invokevirtual D::m(LTD)
> D::m(LTD)
> E::m(LTD)
> invokevirtual D::m(Date)
> D::m(Date)
> D::m(Date), forwards to invvir D::m(LTD)
> In turn, selects E::m(LTD)
> invokevirtual E::m(LTD)
> E::m(LTD)
> E::m(LTD)
> invokevirtual E::m(Date)
> D::m(Date)
> D::m(Date), forwards to invvir D::m(LTD)
> In turn, selects E::m(LTD)
> In other words, we arrange that once the vtable is laid out, it is as if no
> one ever overrides the forwarders -- they only override the real method.
> Hence the reverser is needed only where a class (like E) actually overrides a
> descriptor that corresponds to a forwarder.
A VM perspective:
invocation
dynamic receiver
resolution
NOT invoked
selection:
actual execution
invokevirtual D::m(LDT)
D
D.m(LDT)
D.m(LDT)
invokevirtual D::m(LDT)
E
D.m(LDT)
E.m(LDT)
reverser: adapt LDT->Date
invoke local E.m(Date)
if return had changed, adapt return back
invokevirtual D::m(Date)
D
D.m(Date)
D.m(Date)
forwarder: adapt Date->LDT
invoke local m(LDT)
if return had changed, adapt
invokevirtual D.m(Date)
E
D.m(Date)
E.m(Date)
invokevirtual E.m(LDT)
E
E.m(LDT)
reverser)
E.m(LDT):
reverser: adapt LDT->Date
invoke local E.m(Date)
if return had changed, adapt return back
invokevirtual E.m(Date)
E
E.m(Date)
E.m(Date) // original - unchanged behavior
Point 1: The resolved method is NOT invoked, it is only the selected method
that is invoked.
We do NOT follow forwarding for the resolved method.
If the resolved method happens to also be the selected method, we will now
execute it and will follow the forwarding.
Note, the same applies to fields - we will not get/set the resolved field. We
will get/set the
selected field, and follow the forwarding at that point.
Point 2: Hotspot’s vtable implementation is set up so that for
class E - a vtable (or itable) is a selection cache. It allows for fast virtual
dispatch.
For Hotspot, for class E, the vtable starts with the inherited vtable from
superclass D.
Any entries in the table are replaced when a method overrides an inherited
method.
Additional methods are appended.
So resolution gives you the offset in the vtable. Selection tells you which
vtable owner
to index based on that offset.
We KEEP the existing methods in the subclass so that they are executed exactly
the same
with no change in behavior (no exceptions due to narrowing etc.)
Agree that so far the only reverser need I have identified is when a class
overrides a forwarder.
>
>> It is my belief that the expected behavior is that we want to invoke
>> E.m(Date) with asType signature matching.
>> To do that, I propose that if the vm detects overriding of a forwarder, that
>> we need to generate a reverser:
>>
>>E.m(Date) overrides D.m(Date)// forwarder: Date->LDT/invoke
>> D.m(LDT)/retu
Re: Updated VM-bridges document
> De: "Brian Goetz"
> À: "valhalla-spec-experts"
> Envoyé: Jeudi 11 Avril 2019 21:52:23
> Objet: Re: Updated VM-bridges document
> This was received through a side channel:
>> From: [ mailto:sebastian.sickelm...@gmx.de | sebastian.sickelm...@gmx.de ]
>> Subject: Re: Updated VM-bridges document
>> Hi,
>> i have a question regarding the discussed forwarding schema for fields.
>> Should it be possible to forward field access to methods, so that
>> we can safely remove public fields from the jdk in a binary compatible
>> way? Some time ago I experimented[1] with such a feature but unfortunatly
>> i haven't the time to continue on this.
>> There are some other things to solve when we map field access to
>> method-calls.
>> One example is that we need a bootstraping-result for the put and the get I
>> think.
>> It would be nice if "forwarders for fields" could solve thie issue of public
>> fields
>> in jdk-api in future.
>> --
>> Sebastian
>> [0] [
>> http://mail.openjdk.java.net/pipermail/discuss/2015-December/003863.html |
>> http://mail.openjdk.java.net/pipermail/discuss/2015-December/003863.html ]
>> (last link to multple short threads)
> While the mechanism could surely be used for this (and more), I would prefer
> not
> to. The two main challenges for which we need field forwarders _at all_ are
> for
> access to fields through the wildcard type, and for migrating fields from
> value-based classes to value types. An example for wildcards:
> class Foo {
> T t;
> }
> The specialized type `Foo` will have a `t : int` field, but the wildcard
> type `Foo` must be seen to have a `t : Object` field. But in reality, there
> needs to be one field. This is the main challenge for which we need field
> forwarder support.
> Migrating from fields to methods is an attractive target, but with it comes a
> great risk for abuse: "lookie, here's properties." And the reason I don't want
> to support this is that field access `t.f` has a known cost model (field
> access
> is fast) and known error model (throws few exceptions.) The more that
> arbitrary
> code could be run at field access time, the more we undermine that. So I want
> to restrict field bridges to:
> - Bridges for an _actual field only_
> - Limited set of adaptations: cast/null-check only
> This is enough to support the wildcard case and the L->Q migration case. I
> would
> strongly prefer to stop there.
Also i don't see how you can support the VarHandle API if a field is actually
forwarded to methods, or forwarded to methods means that you have to
re-implement all the VarHandle API (at least, get_volatile, set_volatile and
CAS) when you want to forward a field ?
Rémi
>> Forwarders for fields
>> The forwarding strategy can be applied to fields as well. In this
>> case, the forwardee descriptor is that of a field descriptor, and the
>> behavior has the same semantics as adapting a target field accessor
>> method handle to the type of the bridge descriptor. (If the forwarder
>> field is static, then the field should be static too.)
Re: Updated VM-bridges document
This was received through a side channel:
From: sebastian.sickelm...@gmx.de
Subject: Re: Updated VM-bridges document
Hi,
i have a question regarding the discussed forwarding schema for fields.
Should it be possible to forward field access to methods, so that
we can safely remove public fields from the jdk in a binary compatible
way? Some time ago I experimented[1] with such a feature but unfortunatly
i haven't the time to continue on this.
There are some other things to solve when we map field access to method-calls.
One example is that we need a bootstraping-result for the put and the get I
think.
It would be nice if "forwarders for fields" could solve thie issue of public
fields
in jdk-api in future.
--
Sebastian
[0]http://mail.openjdk.java.net/pipermail/discuss/2015-December/003863.html
(last link to multple short threads)
While the mechanism could surely be used for this (and more), I would
prefer not to. The two main challenges for which we need field
forwarders _at all_ are for access to fields through the wildcard type,
and for migrating fields from value-based classes to value types. An
example for wildcards:
class Foo {
T t;
}
The specialized type `Foo` will have a `t : int` field, but the
wildcard type `Foo` must be seen to have a `t : Object` field. But
in reality, there needs to be one field. This is the main challenge for
which we need field forwarder support.
Migrating from fields to methods is an attractive target, but with it
comes a great risk for abuse: "lookie, here's properties." And the
reason I don't want to support this is that field access `t.f` has a
known cost model (field access is fast) and known error model (throws
few exceptions.) The more that arbitrary code could be run at field
access time, the more we undermine that. So I want to restrict field
bridges to:
- Bridges for an _actual field only_
- Limited set of adaptations: cast/null-check only
This is enough to support the wildcard case and the L->Q migration
case. I would strongly prefer to stop there.
Forwarders for fields
The forwarding strategy can be applied to fields as well. In this
case, the forwardee descriptor is that of a field descriptor, and the
behavior has the same semantics as adapting a target field accessor
method handle to the type of the bridge descriptor. (If the forwarder
field is static, then the field should be static too.)
Re: Updated VM-bridges document
OK, so in the old world, D has m(Date).
Migration step 1: author: Date -> LocalDateTime
old class D { m(Date); }
Migration step 2: change method declarer: new class D {m(LDT);}
and javac creates a forwarder m(Date); -> Date->LDT/m(LDT);
Now, D has m(LDT), with a forwarder from m(Date) -> m(LDT), with some
sort of metadata stapled somewhere to effect the Date <--> LDT conversions.
class E extends D { m(Date); } which now overrides the forwarder.
We do not change class E. We do not recompile it (I don’t know what
recompilation would do here?)
On recompilation, we could do one of three things:
1. Error: you're overriding a bridge, fix your program!
2. Warning: you're overriding a bridge, I'll fix it for you (compiler
adapts m(Date) to m(LDT).
3. Warning: you're overriding a bridge, I'll believe you, and the VM
will fix it for you (bringing us back to where you started: "we do not
change E."
Which we choose at compilation time doesn't really affect what the VM
has to do (you still have to deal with the unrecompiled E), so we can
make this decision later.
old class ClientD invokevirtual D.m(Date) receiver:E
Migration step 3: new class ClientD invokevirtual D.m(LDT) receiver:E
resolution: finds D.m(LDT)
selection: starts with E, there is no E.m(LDT) so call D.m(LDT)
OK, so at this point, the classfiles that have been loaded look like:
class D {
void m(LDT) { real method }
@Forwarding(m(LDT)) abstract void m(Date);
}
class E extends D {
@Override
m(Date) { impl }
}
So D has members m(LTD) and m(Date), the latter is a forwarder.
Therefore E has the same members (instance methods are inherited).
Here's how I would imagine this turns into in the VM:
class D {
void m(LTD) { real method }
void m(Date d) { m(adapt(d)); } // generated forwarder
}
class E extends D {
private void m$synthetic(Date d) { real method, body as present
in classfile }
void m(LTD ltd) { m$synthetic(adapt(ltd)); } // generated reverser
}
resolves
selects
invokevirtual D::m(LTD)
D::m(LTD)
E::m(LTD)
invokevirtual D::m(Date)
D::m(Date)
D::m(Date), forwards to invvir D::m(LTD)
In turn, selects E::m(LTD)
invokevirtual E::m(LTD)
E::m(LTD)
E::m(LTD)
invokevirtual E::m(Date)
D::m(Date)
D::m(Date), forwards to invvir D::m(LTD)
In turn, selects E::m(LTD)
In other words, we arrange that once the vtable is laid out, it is as if
no one ever overrides the forwarders -- they only override the real
method. Hence the reverser is needed only where a class (like E)
actually overrides a descriptor that corresponds to a forwarder.
It is my belief that the expected behavior is that we want to invoke E.m(Date)
with asType signature matching.
To do that, I propose that if the vm detects overriding of a forwarder, that we
need to generate a reverser:
E.m(Date) overrides D.m(Date)// forwarder: Date->LDT/invoke D.m(LDT)/return
conversion
The reverser that we want would be
E.m(LDT) overrides D.m(LDT) // reverser: LDT->Date/invoke E.m(Date)/return
reverse conversion
I think we want: a reverser for E::m(LTD), but not for E::m(Date). Are
we saying the same thing?
Re: Updated VM-bridges document
Brian,
Here is an example of when I believe we need to create a reverser - let me know
if my
assumptions don’t match yours.
Migration step 1: author: Date -> LocalDateTime
old class D { m(Date); }
Migration step 2: change method declarer: new class D {m(LDT);}
and javac creates a forwarder m(Date); -> Date->LDT/m(LDT);
class E extends D { m(Date); } which now overrides the forwarder.
We do not change class E. We do not recompile it (I don’t know what
recompilation would do here?)
old class ClientD invokevirtual D.m(Date) receiver:E
Migration step 3: new class ClientD invokevirtual D.m(LDT) receiver:E
resolution: finds D.m(LDT)
selection: starts with E, there is no E.m(LDT) so call D.m(LDT)
It is my belief that the expected behavior is that we want to invoke E.m(Date)
with asType signature matching.
To do that, I propose that if the vm detects overriding of a forwarder, that we
need to generate a reverser:
E.m(Date) overrides D.m(Date)// forwarder: Date->LDT/invoke D.m(LDT)/return
conversion
The reverser that we want would be
E.m(LDT) overrides D.m(LDT) // reverser: LDT->Date/invoke E.m(Date)/return
reverse conversion
Corrections welcome,
thanks,
Karen
> On Apr 4, 2019, at 8:33 AM, Brian Goetz wrote:
>
> At the BUR meeting, we discussed reshuffling the dependency graph to do
> forwarding+reversing bridges earlier, which has the effect of taking some
> pressure off of the descriptor language. Here’s an updated doc on
> forwarding-reversing bridges in the VM.
>
> I’ve dropped, for the time being, any discussion of replacing existing
> generic bridges with this mechanism; we can revisit that later if it makes
> sense. Instead, I’ve focused solely on the migration aspects. I’ve also
> dropped any mention of implementation strategy, and instead appealed to “as
> if” behavior.
>
>
> ## From Bridges to Forwarders
>
> In the Java 1.0 days, `javac` was little more than an "assembler" for
> the classfile format, translating source code to bytecode in a mostly
> 1:1 manner. And, we liked it that way; the more predictable the
> translation scheme, the more effective the runtime optimizations.
> Even the major upgrade of Java 5 didn't significantly affect the
> transparency of the resulting classfiles.
>
> Over time, we've seen small divergences between the language model and
> the classfile model, and each of these is a source of sharp edges. In
> Java 1.1 the addition of inner classes, and the mismatch between the
> accessibility model in the language and the JVM (the language treated
> a nest as a single entity; the JVM treat nest members as separate
> classes) required _access bridges_ (`access$000` methods), which have
> been the source of various issues over the years. Twenty years later,
> these methods were obviated by [_Nest-based Access Control_][jep181]
> -- which represents the choice to align the VM model to the language
> model, so these adaptation artifacts are no longer required.
>
> In Java 5, while we were able to keep the translation largely stable
> and transparent through the use of erasure, there was one point of
> misalignment; several situations (covariant overrides, instantiated
> generic supertypes) could give rise to the situation where two or more
> method descriptors -- which the JVM treats as distinct methods -- are
> treated by the language as if they correspond to the same method. To
> fool the VM, the compiler emits _bridge methods_ which forward
> invocations from one signature to another. And, as often happens when
> we try to fool the VM, it ultimately has its revenge.
>
> Example: covariant overrides
>
> Java 5 introduced the ability to override a method but to provide a
> more specific return type. (Java 8 later extended this to bridges in
> interfaces as well.) For example:
>
> ```{.java}
> class Parent {
>Object m() { ... }
> }
>
> class Child extends Parent {
>@Override
>String m() { ... }
> }
> ```
>
> `Parent` declares a method whose descriptor is `()Object`, and `Child`
> declares a method with the same name whose descriptor is `()String`.
> If we compiled this class in the obvious way, the method in `Child`
> would not override the method in `Parent`, and anyone calling
> `Parent.m()` would find themselves executing the wrong implementation.
>
> The compiler addresses this by providing an additional implementation
> of `m()`, whose descriptor is `()Object` (an actual override), marked
> with `ACC_SYNTHETIC` and `ACC_BRIDGE`, whose body invokes `m()String`
> (with `invokevirtual`), redirecting calls to the right implementation.
>
> Example: generic substitution
>
> A similar situation arises when we have a generic substitution with a
> superclass. For example:
>
> ```{.java}
> interface Parent {
>void m(T x);
> }
>
> class Child extends Parent {
>@Override
>void m(String x) { ... }
> }
> ```
>
> At the language level, it is clear that `Child::m` intends to overrid
Re: Updated VM-bridges document
> De: "Brian Goetz" > À: "Remi Forax" > Cc: "valhalla-spec-experts" > Envoyé: Mardi 9 Avril 2019 15:39:19 > Objet: Re: Updated VM-bridges document >>> OK, I see what you’re getting at now. Yes, this is one of the implementation >>> possibilities. I was mostly looking to validate the concepts before diving >>> into >>> the representational details. One key point is that the default case should >>> be >>> able to proceed with no bootstrap; a small set of adaptations handles the >>> most >>> important cases, and avoiding an upcall is probably pretty desirable if we >>> can >>> get away with it. >> It's an optimization, i prefer the VM to recognize a specific BSM and don't >> upcall it because its semantics is well known, it has the same effect but >> it's >> an implementation detail and not something that need to figure in the VM >> spec. > It’s an optimization, but not in the way you think. If the purity of the spec > were the only concern, then the approach you lay out would make perfect sense. > But, there are other engineering realities, and taking on the full cost of > bootstrap upcalls at this particular place in the VM — when we don’t have to > yet — may well be a significant (and not yet necessary) engineering effort. It doesn't seem wise to directly calls the BSM when you are filling the vtable but you can always generate an adapter (the descriptors from and to are known statically) that will call the BSM and replace itself. A strawman implementation is to generate during the parsing of the classfile, for each VM bridge a Java method with a bytecodes that does an invokedynamic call, the invokedynamic BSM taking as parameter the VM bridge BSM. Rémi
Re: Updated VM-bridges document
> OK, I see what you’re getting at now. Yes, this is one of the implementation > possibilities. I was mostly looking to validate the concepts before diving > into the representational details. One key point is that the default case > should be able to proceed with no bootstrap; a small set of adaptations > handles the most important cases, and avoiding an upcall is probably pretty > desirable if we can get away with it. > > It's an optimization, i prefer the VM to recognize a specific BSM and don't > upcall it because its semantics is well known, it has the same effect but > it's an implementation detail and not something that need to figure in the VM > spec. It’s an optimization, but not in the way you think. If the purity of the spec were the only concern, then the approach you lay out would make perfect sense. But, there are other engineering realities, and taking on the full cost of bootstrap upcalls at this particular place in the VM — when we don’t have to yet — may well be a significant (and not yet necessary) engineering effort.
Re: Updated VM-bridges document
> De: "Brian Goetz" > À: "Remi Forax" > Cc: "valhalla-spec-experts" > Envoyé: Lundi 8 Avril 2019 20:39:03 > Objet: Re: Updated VM-bridges document > OK, I see what you’re getting at now. Yes, this is one of the implementation > possibilities. I was mostly looking to validate the concepts before diving > into > the representational details. One key point is that the default case should be > able to proceed with no bootstrap; a small set of adaptations handles the most > important cases, and avoiding an upcall is probably pretty desirable if we can > get away with it. It's an optimization, i prefer the VM to recognize a specific BSM and don't upcall it because its semantics is well known, it has the same effect but it's an implementation detail and not something that need to figure in the VM spec. > But, given that, an index to a BSM entry is probably fine; it moves the > representation of “which arguments are adapted how” into a static argument > list, which is probably the best place for it. > One non-obvious point here is that the adaptation must work _in both > directions_. If we are migrating Collection::size from returning int to > returning long, not only do we want to widen the result implicitly when > invoked > by legacy callers, but we have to _narrow_ the result when overridden by > legacy > subclasses. i agree, a peculiar BSM has to be able to do the migration in both ways from old to new and new to old. Rémi >> On Apr 8, 2019, at 1:18 PM, [ mailto:fo...@univ-mlv.fr | fo...@univ-mlv.fr ] >> wrote: >>> De: "Brian Goetz" < [ mailto:brian.go...@oracle.com | >>> brian.go...@oracle.com ] > >>> À: "Remi Forax" < [ mailto:fo...@univ-mlv.fr | fo...@univ-mlv.fr ] > >>> Cc: "valhalla-spec-experts" < [ >>> mailto:valhalla-spec-experts@openjdk.java.net | >>> valhalla-spec-experts@openjdk.java.net ] > >>> Envoyé: Lundi 8 Avril 2019 16:24:21 >>> Objet: Re: Updated VM-bridges document >>>> The other thing is that Forwarding bridge should not use an adapter but a >>>> bootstrap method. >>> Can you explain exactly what you mean here? Because in my mind, the adapter >>> _is_ >>> a bootstrap method — it is code to which the VM upcalls at preparation / >>> link >>> time to help establish linkage. >>> Since you obviously have a more specific notion of “bootstrap method”, can >>> you >>> explain exactly what you mean? >> it's a kind of bootstrap method but with not with the same ceremony as the >> one >> used by indy or condy, i.e. no lookup and no constant bootstrap arguments >> (and >> no name and no descriptor), >> i propose to the same protocol as with indy or condy. >> Rémi
Re: Updated VM-bridges document
OK, I see what you’re getting at now. Yes, this is one of the implementation possibilities. I was mostly looking to validate the concepts before diving into the representational details. One key point is that the default case should be able to proceed with no bootstrap; a small set of adaptations handles the most important cases, and avoiding an upcall is probably pretty desirable if we can get away with it. But, given that, an index to a BSM entry is probably fine; it moves the representation of “which arguments are adapted how” into a static argument list, which is probably the best place for it. One non-obvious point here is that the adaptation must work _in both directions_. If we are migrating Collection::size from returning int to returning long, not only do we want to widen the result implicitly when invoked by legacy callers, but we have to _narrow_ the result when overridden by legacy subclasses. > On Apr 8, 2019, at 1:18 PM, fo...@univ-mlv.fr wrote: > > > > De: "Brian Goetz" > À: "Remi Forax" > Cc: "valhalla-spec-experts" > Envoyé: Lundi 8 Avril 2019 16:24:21 > Objet: Re: Updated VM-bridges document > > The other thing is that Forwarding bridge should not use an adapter but a > bootstrap method. > > Can you explain exactly what you mean here? Because in my mind, the adapter > _is_ a bootstrap method — it is code to which the VM upcalls at preparation / > link time to help establish linkage. > > Since you obviously have a more specific notion of “bootstrap method”, can > you explain exactly what you mean? > > it's a kind of bootstrap method but with not with the same ceremony as the > one used by indy or condy, i.e. no lookup and no constant bootstrap arguments > (and no name and no descriptor), > i propose to the same protocol as with indy or condy. > > Rémi >
Re: Updated VM-bridges document
> De: "Brian Goetz" > À: "Remi Forax" > Cc: "valhalla-spec-experts" > Envoyé: Lundi 8 Avril 2019 16:24:21 > Objet: Re: Updated VM-bridges document >> The other thing is that Forwarding bridge should not use an adapter but a >> bootstrap method. > Can you explain exactly what you mean here? Because in my mind, the adapter > _is_ > a bootstrap method — it is code to which the VM upcalls at preparation / link > time to help establish linkage. > Since you obviously have a more specific notion of “bootstrap method”, can you > explain exactly what you mean? it's a kind of bootstrap method but with not with the same ceremony as the one used by indy or condy, i.e. no lookup and no constant bootstrap arguments (and no name and no descriptor), i propose to the same protocol as with indy or condy. Rémi
Re: Updated VM-bridges document
> > The other thing is that Forwarding bridge should not use an adapter but a > bootstrap method. Can you explain exactly what you mean here? Because in my mind, the adapter _is_ a bootstrap method — it is code to which the VM upcalls at preparation / link time to help establish linkage. Since you obviously have a more specific notion of “bootstrap method”, can you explain exactly what you mean?
Re: Updated VM-bridges document
That’s just a typo. There’s just one attribute.
> On Apr 7, 2019, at 5:30 PM, Remi Forax wrote:
>
> Hi Brian,
> with an hat of JVM language implementor,
> the attribute ForwardingBridge is important because it allows to simulate
> things that are not co/contravariant for the VM but are from the language
> point of view, like between an int and a Long.
>
> I don't see the point of having two attributes, given that ForwardingBridge
> supports a superset of what Forwarding support, it seems to be a premature
> optimization to me.
>
> The other thing is that Forwarding bridge should not use an adapter but a
> bootstrap method.
> Historically, indy was using something similar to your adapter but this
> design doesn't support sharing code because here the adapter code has to be
> present in the same class as the forwarding method.
> If you want to share code, the shared code may want to access to the class
> containing the forwarding method, that's why you need a Lookup object and the
> share code may want to have some specific arguments, aka the bootstrap
> arguments.
>
> I think John has already proposed that we should support a bootstrap method
> that returns a MethodHandle instead of a CallSite, this is exactly what we
> need here.
>
> regards,
> Rémi
>
> - Mail original -
>> De: "Brian Goetz"
>> À: "valhalla-spec-experts"
>> Envoyé: Jeudi 4 Avril 2019 14:33:39
>> Objet: Updated VM-bridges document
>
>> At the BUR meeting, we discussed reshuffling the dependency graph to do
>> forwarding+reversing bridges earlier, which has the effect of taking some
>> pressure off of the descriptor language. Here’s an updated doc on
>> forwarding-reversing bridges in the VM.
>>
>> I’ve dropped, for the time being, any discussion of replacing existing
>> generic
>> bridges with this mechanism; we can revisit that later if it makes sense.
>> Instead, I’ve focused solely on the migration aspects. I’ve also dropped any
>> mention of implementation strategy, and instead appealed to “as if” behavior.
>>
>>
>> ## From Bridges to Forwarders
>>
>> In the Java 1.0 days, `javac` was little more than an "assembler" for
>> the classfile format, translating source code to bytecode in a mostly
>> 1:1 manner. And, we liked it that way; the more predictable the
>> translation scheme, the more effective the runtime optimizations.
>> Even the major upgrade of Java 5 didn't significantly affect the
>> transparency of the resulting classfiles.
>>
>> Over time, we've seen small divergences between the language model and
>> the classfile model, and each of these is a source of sharp edges. In
>> Java 1.1 the addition of inner classes, and the mismatch between the
>> accessibility model in the language and the JVM (the language treated
>> a nest as a single entity; the JVM treat nest members as separate
>> classes) required _access bridges_ (`access$000` methods), which have
>> been the source of various issues over the years. Twenty years later,
>> these methods were obviated by [_Nest-based Access Control_][jep181]
>> -- which represents the choice to align the VM model to the language
>> model, so these adaptation artifacts are no longer required.
>>
>> In Java 5, while we were able to keep the translation largely stable
>> and transparent through the use of erasure, there was one point of
>> misalignment; several situations (covariant overrides, instantiated
>> generic supertypes) could give rise to the situation where two or more
>> method descriptors -- which the JVM treats as distinct methods -- are
>> treated by the language as if they correspond to the same method. To
>> fool the VM, the compiler emits _bridge methods_ which forward
>> invocations from one signature to another. And, as often happens when
>> we try to fool the VM, it ultimately has its revenge.
>>
>> Example: covariant overrides
>>
>> Java 5 introduced the ability to override a method but to provide a
>> more specific return type. (Java 8 later extended this to bridges in
>> interfaces as well.) For example:
>>
>> ```{.java}
>> class Parent {
>> Object m() { ... }
>> }
>>
>> class Child extends Parent {
>> @Override
>> String m() { ... }
>> }
>> ```
>>
>> `Parent` declares a method whose descriptor is `()Object`, and `Child`
>> declares a method with the same name whose descriptor is `()String`.
>> If we compiled this class in the obvious way, the method in `Child`
>> would not override the method in `Parent`, and anyone calling
>> `Parent.m()` would find themselves executing the wrong implementation.
>>
>> The compiler addresses this by providing an additional implementation
>> of `m()`, whose descriptor is `()Object` (an actual override), marked
>> with `ACC_SYNTHETIC` and `ACC_BRIDGE`, whose body invokes `m()String`
>> (with `invokevirtual`), redirecting calls to the right implementation.
>>
>> Example: generic substitution
>>
>> A similar situation arises when we have a generic substitution with
Re: Updated VM-bridges document
Hi Brian,
with an hat of JVM language implementor,
the attribute ForwardingBridge is important because it allows to simulate
things that are not co/contravariant for the VM but are from the language point
of view, like between an int and a Long.
I don't see the point of having two attributes, given that ForwardingBridge
supports a superset of what Forwarding support, it seems to be a premature
optimization to me.
The other thing is that Forwarding bridge should not use an adapter but a
bootstrap method.
Historically, indy was using something similar to your adapter but this design
doesn't support sharing code because here the adapter code has to be present in
the same class as the forwarding method.
If you want to share code, the shared code may want to access to the class
containing the forwarding method, that's why you need a Lookup object and the
share code may want to have some specific arguments, aka the bootstrap
arguments.
I think John has already proposed that we should support a bootstrap method
that returns a MethodHandle instead of a CallSite, this is exactly what we need
here.
regards,
Rémi
- Mail original -
> De: "Brian Goetz"
> À: "valhalla-spec-experts"
> Envoyé: Jeudi 4 Avril 2019 14:33:39
> Objet: Updated VM-bridges document
> At the BUR meeting, we discussed reshuffling the dependency graph to do
> forwarding+reversing bridges earlier, which has the effect of taking some
> pressure off of the descriptor language. Here’s an updated doc on
> forwarding-reversing bridges in the VM.
>
> I’ve dropped, for the time being, any discussion of replacing existing generic
> bridges with this mechanism; we can revisit that later if it makes sense.
> Instead, I’ve focused solely on the migration aspects. I’ve also dropped any
> mention of implementation strategy, and instead appealed to “as if” behavior.
>
>
> ## From Bridges to Forwarders
>
> In the Java 1.0 days, `javac` was little more than an "assembler" for
> the classfile format, translating source code to bytecode in a mostly
> 1:1 manner. And, we liked it that way; the more predictable the
> translation scheme, the more effective the runtime optimizations.
> Even the major upgrade of Java 5 didn't significantly affect the
> transparency of the resulting classfiles.
>
> Over time, we've seen small divergences between the language model and
> the classfile model, and each of these is a source of sharp edges. In
> Java 1.1 the addition of inner classes, and the mismatch between the
> accessibility model in the language and the JVM (the language treated
> a nest as a single entity; the JVM treat nest members as separate
> classes) required _access bridges_ (`access$000` methods), which have
> been the source of various issues over the years. Twenty years later,
> these methods were obviated by [_Nest-based Access Control_][jep181]
> -- which represents the choice to align the VM model to the language
> model, so these adaptation artifacts are no longer required.
>
> In Java 5, while we were able to keep the translation largely stable
> and transparent through the use of erasure, there was one point of
> misalignment; several situations (covariant overrides, instantiated
> generic supertypes) could give rise to the situation where two or more
> method descriptors -- which the JVM treats as distinct methods -- are
> treated by the language as if they correspond to the same method. To
> fool the VM, the compiler emits _bridge methods_ which forward
> invocations from one signature to another. And, as often happens when
> we try to fool the VM, it ultimately has its revenge.
>
> Example: covariant overrides
>
> Java 5 introduced the ability to override a method but to provide a
> more specific return type. (Java 8 later extended this to bridges in
> interfaces as well.) For example:
>
> ```{.java}
> class Parent {
>Object m() { ... }
> }
>
> class Child extends Parent {
>@Override
>String m() { ... }
> }
> ```
>
> `Parent` declares a method whose descriptor is `()Object`, and `Child`
> declares a method with the same name whose descriptor is `()String`.
> If we compiled this class in the obvious way, the method in `Child`
> would not override the method in `Parent`, and anyone calling
> `Parent.m()` would find themselves executing the wrong implementation.
>
> The compiler addresses this by providing an additional implementation
> of `m()`, whose descriptor is `()Object` (an actual override), marked
> with `ACC_SYNTHETIC` and `ACC_BRIDGE`, whose body invokes `m()String`
> (with `invokevirtual`), redirecting calls to the right implementation.
>
> Example: generic substitution
>
> A similar situation arises when we have a generic substitution with a
> superclass. For example:
>
> ```{.java}
> interface Parent {
>void m(T x);
> }
>
> class Child extends Parent {
>@Override
>void m(String x) { ... }
> }
> ```
>
> At the language level, it is clear that `Child::m` inten
