Re: with and binary backward compatibility
- Original Message -
> From: "Dan Heidinga"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Tuesday, June 14, 2022 3:10:46 PM
> Subject: Re: with and binary backward compatibility
> Remi, to restate your concern slightly - it sounds like there may be
> binary (and source?) compatibility concerns with the implementation of
> with'ers. You've pitched a possible implementation using
> invokedynamic (every compiler writer's favourite swiss army knife) but
> prefer something that is more explicit in the bytecode.
Yes, BTW there is a simple way to avoid those binary compatibility issues,
restrict "with" to the nest or the package/module (like sealed).
> I'm still
> working my way through the full reconstruction document but I assume
> compatibility and implementation have been given some thought even if
> they don't show up in that document.
The problem is that Brian believes that all constructors and deconstructors are
dual, i feel this is the wrong model, it does not work that way, a constructor
if not canonical creates values before calling the canonical constructor while
all deconstructors all match the same way but some provide fewer bindings than
the others. The binary compatibility story of "with" shows that you can not do
a circle point -> bindings -> point using any constructors but the canonical
constructor.
>
> Let's aim for the high order bits first - figuring out if the feature
> is desirable and if the general direction works before deep diving
> into class file representation and binary compatibility.
agree
>
> So I'd suggest putting a pin in this topic and circling back to it
> after further discussion of the core concept.
>
> Just my two cents =)
>
> --Dan
Rémi
>
> On Tue, Jun 14, 2022 at 8:23 AM Remi Forax wrote:
>>
>> Hi all,
>> Let say we have a Point with 2 components
>> record Point(int x, int y) { }
>>
>> Then we change the record to add a 3rd components in a more or less backward
>> compatible way
>> record Point(int x, int y, int z) {
>> Point(int x, int y) {
>> this(x, y, 0); // creation of the new value 0
>> }
>> }
>>
>> Now, let say there is a 'with' somewhere in another code
>>
>> var newPoint = point with { x = 3; };
>>
>> If this code is compiled when the record Point had only two components, so
>> this
>> is equivalent to
>>
>> Point(int x, int y) = point; // i.e. int x = point.x(); int y = point.y();
>> x = 3;
>> var newPoint = new Point(x, y);
>>
>> The problem is that if we run that code with the new version of Point (the
>> one
>> with 3 components),
>> newPoint.z is not equals to point.z but to 0, so once there is a 'with'
>> somewhere, there is no backward compatibility anymore.
>>
>> We can try to restore the backward compatibility by compiling to a slightly
>> different code using invokedynamic and a desugared method corresponding to
>> the
>> body of the 'with'
>>
>> var newPoint = invokedynamic (point) [desugared$method]; // equivalent to
>> a
>> call using with
>>
>> static Point desugared$method(int x, int y, MethodHandle mh) { // content
>> of
>> the body
>> x = 3;
>> return mh.invokeExact(x, y);
>> }
>>
>> an at runtime, we generate a tree of method handles that does more or less
>> stub(Point point) {
>> return desugared$method(point.x(), point.y(), (a, b) -> new Point(a, b,
>> point.z())
>> }
>>
>> because this code is generated at runtime, it will be always compatible with
>> the
>> latest version of Point.
>>
>> If we want to support this encoding, it means that the local variables of the
>> enclosing method need to be effectively final so the body of with can be
>> lifted
>> to a private static method (exactly like a lambda).
>>
>>
>> If we generalize this a bit, we can also use the same trick for the record
>> pattern, in that case the pattern Point(int a, int b) is equivalent at
>> runtime
>> to Point(int a, int b, _) once the runtime found that the canonical
>> deconstructor emits the values of 3 components.
>> I'm not sure it's a path i want to follow because i would prefer the record
>> pattern to match the shape excatly, but i find it more attractive than the
>> idea
>> to have overloaded deconstructors.
>>
>> regards,
>> Rémi
with and binary backward compatibility
Hi all,
Let say we have a Point with 2 components
record Point(int x, int y) { }
Then we change the record to add a 3rd components in a more or less backward
compatible way
record Point(int x, int y, int z) {
Point(int x, int y) {
this(x, y, 0); // creation of the new value 0
}
}
Now, let say there is a 'with' somewhere in another code
var newPoint = point with { x = 3; };
If this code is compiled when the record Point had only two components, so this
is equivalent to
Point(int x, int y) = point; // i.e. int x = point.x(); int y = point.y();
x = 3;
var newPoint = new Point(x, y);
The problem is that if we run that code with the new version of Point (the one
with 3 components),
newPoint.z is not equals to point.z but to 0, so once there is a 'with'
somewhere, there is no backward compatibility anymore.
We can try to restore the backward compatibility by compiling to a slightly
different code using invokedynamic and a desugared method corresponding to the
body of the 'with'
var newPoint = invokedynamic (point) [desugared$method]; // equivalent to a
call using with
static Point desugared$method(int x, int y, MethodHandle mh) { // content of
the body
x = 3;
return mh.invokeExact(x, y);
}
an at runtime, we generate a tree of method handles that does more or less
stub(Point point) {
return desugared$method(point.x(), point.y(), (a, b) -> new Point(a, b,
point.z())
}
because this code is generated at runtime, it will be always compatible with
the latest version of Point.
If we want to support this encoding, it means that the local variables of the
enclosing method need to be effectively final so the body of with can be lifted
to a private static method (exactly like a lambda).
If we generalize this a bit, we can also use the same trick for the record
pattern, in that case the pattern Point(int a, int b) is equivalent at runtime
to Point(int a, int b, _) once the runtime found that the canonical
deconstructor emits the values of 3 components.
I'm not sure it's a path i want to follow because i would prefer the record
pattern to match the shape excatly, but i find it more attractive than the idea
to have overloaded deconstructors.
regards,
Rémi
Re: "With" for records
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Saturday, June 11, 2022 8:16:26 PM
> Subject: Re: "With" for records
> We also probably want a rule to _prevent_ assignment to any locals *other
> than*
> the synthetic component locals. Assigning to uplevel locals from within a
> `with` block seems like asking for trouble; the with block is like a transform
> on the component locals.
perhaps any locals other that the synthetic ones and the ones declared inside
the block. For example:
record Couple(int first, int second) {
Couple withMax() {
return this with {
var max = Math.max(first, second); // new local variable, it's ok
first = max;
second = max;
};
}
}
> However, we may need a story (hope not) for _accessing_ uplevel shadowed
> locals.
> For example:
> record R(A contents) { }
> record A(B contents) { }
> record B(int contents) { }
> R r = ...
> R rr = r with { contents = contents with { contents = 3 }}
> it is possible that the inner block might want additional information from one
> of the enclosing `contents` variables.
or inside the block we may want to have access to the parameters, like in:
record Complex(double re, double im) {
Complex withRe(double re) {
return this with { re = re_from_outer_scope; } // find a syntax here !
}
}
we can introduce an intermediary local variable but i wonder if there is a
better solution ?
record Complex(double re, double im) {
Complex withRe(double re) {
var re_from_outer_scope = re;
return this with { re = re_from_outer_scope; }
}
}
and two other related questions about the syntax
- do we allow to not use curly braces if there is only one assignment
complex with re = 3
- or do we allow to avoid the last semicolon if there is only one assignment
like in your example
complex with { re = 3 }
Rémi
> On 6/10/2022 11:25 AM, Brian Goetz wrote:
>>> About the declaration of local variables, in Java, there is no
>>> hiding/shadowing
>>> between local variables, so a code like this is rejected ? Or do we
>>> introduce a
>>> special rule for the hiding of implicit variables ?
>> Yes, we probably do need a special rule for this. The component names are
>> fixed,
>> and collisions are likely, so these synthetic variables will probably have to
>> be allowed to shadow other locals.
Re: "With" for records
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Friday, June 10, 2022 2:44:51 PM
> Subject: "With" for records
> In
> [
> https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md
> |
> https://github.com/openjdk/amber-docs/blob/master/eg-drafts/reconstruction-records-and-classes.md
> ]
> we explore a generalized mechanism for `with` expressions, such as:
> Point shadowPos = shape.position() with { x = 0 }
> The document evaluates a general mechanism involving matched pairs of
> constructors (or factories) and deconstruction patterns, which is still
> several
> steps out, but we can take this step now with records, because records have
> all
> the characteristics (significant names, canonical ctor and dtor) that are
> needed. The main reason we might wait is if there are uncertainties in the
> broader target.
> Our C# friends have already gone here, in a way that fits into C#, using
> properties (which makes sense, as their language is built on that):
> object with { property-assignments }
> The C# interpretation is that the RHS of this expression is a sort of "DSL",
> which permits property assignment but nothing else. This is cute, but I think
> we can do better.
> In our version, the RHS is an arbitrary block of Java code; you can use loops,
> assignments, exceptions, etc. The only thing that makes it "special" is that
> that the components of the operand are lifted into mutable locals on the RHS.
> So inside the RHS when the operand is a Point, there are fresh mutable locals
> `x` and `y` which are initialized with the X and Y values of the operand.
> Their
> values are committed at the end of the block using the canonical constructor.
GREAT !
I've several questions, that we will have to answer later.
The block is also special because there is an implicit return at the end ? so i
believe "return" should be disallowed inside the block, right ?
Does the lifting that appears at the beginning of the block call all accessors
? or the compiler try to be clever and not call an accessor when its value is
not needed.
For example, in
point with { y = 3; }
calling point.y() is useless, or is it something the JIT will take care of ?
Do we allow "with" followed by an empty block ? As a way to clone a record ?
About the declaration of local variables, in Java, there is no hiding/shadowing
between local variables, so a code like this is rejected ? Or do we introduce a
special rule for the hiding of implicit variables ?
int x = Integer.parseInt();
...
foo(point with { y = 3; }); // x is declared twice because there is an implicit
int x = point.x();
> This should remind people of the *compact constructor* in a record; the body
> is
> allowed to freely mutate the special variables (who also don't have obvious
> declarations), and their terminal values determine the state of the record.
> Just as we were able to do record patterns without having full-blown
> deconstructors, we can do with expressions on records as well, because (a) we
> still have a canonical ctor, (b) we have accessors, and (c) we know the names
> of the components.
> Obviously when we get value types, we'll want classes to be able to expose (or
> not) such a mechanism (both for internal or external use).
yes, Complex.default with { re = 3; im = 4; } seems a great fit for value
classes.
Rémi
Re: Simplifying switch labels
- Original Message - > From: "Brian Goetz" > To: "daniel smith" , "amber-spec-experts" > > Sent: Thursday, June 2, 2022 8:08:14 PM > Subject: Re: Simplifying switch labels >> In this framing, the restrictions about sets of elements in a single label >> don't >> apply, because we're talking about two different labels. But we have rules to >> prevent various abuses. Examples: >> >> case 23: case Pattern: // illegal before and now, due to fallthrough Pattern >> rule > > Ideally, the fallthrough rule should be about _bindings_, not > _patterns_. If P an Q are patterns with no binding variables, then it > should be OK to say: > > case P: > case Q: > > The rule about fallthrough is to prevent falling into code where the > bindings are not DA. > >> Note that the second kind of Pattern SwitchLabel is especially weird—it binds >> 'null' to a pattern variable, and requires the pattern to be a (possibly >> parenthesized) type pattern. So it's nice to break it out as its own >> syntactic >> case. I'd also suggest rethinking whether "case null," is the right way to >> express the two kinds of nullable SwitchLabels, but anyway now it's really >> clear that they are special. > > This is a painful compromise. While this may be a transitional > phenomena, the rule "switch matches null only when `null` appears in a > case" is a helpful way to transition people away from "switch always > throws on null." It also allows to express that a switch statement on an enum should be exhaustive by adding case null -> throw null; Rémi
Re: Named record pattern
> From: "Brian Goetz"
> To: "Tagir Valeev"
> Cc: "amber-spec-experts"
> Sent: Tuesday, May 31, 2022 6:12:06 PM
> Subject: Re: Named record pattern
> Gavin reminded me that we are not finalizing patterns in switch in 19 (hard to
> keep track, sometimes), so we have a little bit of time to figure out what we
> want here.
> One thing that is potentially confusing is that patterns work indirectly in a
> number of ways. For example, if we have a declared deconstruction pattern for
> Point, you can't *invoke* it as you can a method; the language runtime invokes
> it on your behalf under the right situations. (In this way, a deconstructor is
> a little like a static initializer; it is a body of code that you declare, but
> you can't invoke it directly, the runtime invokes it for you at the right
> time,
> and that's fine.)
> I had always imagined the relationship with locals being similar; a pattern
> causes a local to be injected into certain scopes, but the pattern itself is
> not a local variable declaration. Obviously there is more than one way to
> interpret this, so we should make a more deliberate decision.
> As a confounding example that suggests that pattern variables are not "just
> locals", in the past we talked about various forms of "merging":
> if (t instanceof Box(String s) || t instanceof Bag(String s)) { ... }
> or
> case Box(String s):
> case Bag(String s):
> common-code;
> If pattern variables could be annotated, then the language would be in the
> position of deciding what happens with
> case Box(@Foo(1) String s):
> case Bag(@Foo(2) String s):
> (This is the "annotation merging" problem, which is why annotations are not
> inherited in the first place.)
> I don't have an answer here, but I'm going to think about the various issues
> and
> try to capture them in more detail before proposing an answer.
For me, there is a difference between a binding and a local variable, is that
bindings are the one declared inside a pattern and a local variables is how a
binding is transformed to be usable inside the boby if the pattern match.
So we can merge bindings and as a result have one local variable to be used in
the body.
About the annotations, if we follow the data orientation principle (data is
more important than code), we can have a record
record Person(@NonNull String name) { }
and a record pattern
case Person(@NonNull String s) -> ...
We want to be able to declare @NonNull in the record pattern so if the data is
changed, if the component name of the record becomes nullable by example, the
pattern will fail to compile.
So i think we should allow annotations because it's a way to enforce that data
are more important that code.
As you said, annotation merging is an issue because the correct merging
requires to know the semantics of the annotations and the compiler has no way
to know that.
But there is a simple solution, annotation are trees, so they can be compared
structurally. Thus we can do bindings merging if the annotations are the same
structurally.
regards,
Rémi
Re: Named record pattern
> From: "Maurizio Cimadamore"
> To: "Brian Goetz" , "Tagir Valeev"
> Cc: "amber-spec-experts"
> Sent: Tuesday, May 31, 2022 7:41:21 PM
> Subject: Re: Named record pattern
> While merging is an issue (if supported), I think early in the design we
> settled
> on the principle that a binding variable should always only have one
> corresponding declaration (in the places where it is defined).
> So, from a language perspective, I don’t see an immediate problem, in the
> sense
> that the annotations and finality for “s” would come from the place where “s”
> has been declared.
> From a compiler perspective, the big question is whether, even in the absence
> of
> merging in the flow scoping rules, we could still perform merging under the
> hood, in order to speed up computation. Consider:
> sealed interface Node
> record Pair (Node fst, Node snd) { } record A (String s) implements Node {}
> record B ( int i) implements Node {}
> And then:
> case Pair (A(String s1) , A (String s2) ) -> ... case Pair (A(String s3) , B (
> int i1) ) -> ... case Pair (B( int i2) , A (String s4) ) -> ... case Pair (B(
> int i3) , B ( int i4) ) -> ...
> (for simplicity, all binding names are different, so that we do not depend on
> whether the block after -> … completes normally or not)
> Of course the first two patterns share the common subpattern A(String s1) (if
> you ignore naming differences). So it might be possible, in principle, for the
> compiler/runtime to generate an optimized decision tree in which the String
> value for the first A(…) sub-pattern is computed only once, and then shared in
> the two cases. (A similar reasoning applies to the last two patterns).
> But if the types in the first and second pattern could be annotated
> differently,
> then we are faced with a number of challenges, as the compiler would not be
> able to just “blindly” reuse the cached values (as those values would be
> shared, and, therefore, unannotated). Instead, the compiler would have to
> assign the cached value into a fresh , correctly annotated local variable that
> is used only inside the given case . This is not impossible of course, but
> adds
> complexity to the translation strategy, and/or might affect the number of
> moves
> we might be able to do.
The same issue arise if the patterns parts are generated at runtime using an
invokedynamic.
I don't think it's an issue if we consider that inside the pattern tree, we
have bindings and that once "->" is crossed, those bindings are materialized as
local variables with annotations.
I kind of easier to see it if there an invokedynamic and a carrier object. A
switch(value) is translated to
Object carrier = invokedynamic pattern_match(value) [send the patterns as a
constantpool constant];
int index = invokdynamic extract (carrier)[binding 0 /* index */];
switch(index) {
case 0 -> {
// here we rematerialize the local variables
String s1 = invokdynamic extract (carrier)[binding 1];
String s2 = invokdynamic extract (carrier)[binding 2];
...
}
case 1 -> {
// we materialize s3 and i
String s3 = invokdynamic extract (carrier)[binding 1];
int i1 = invokdynamic extract (carrier)[binding 2];
...
}
...
}
In a sense, this is similar to a lambda, it's not the parameter of the method
of the lambda proxy which is annotated but the parameter of the static method
desugared from the lambda body.
Here, it's not the patterns part (that one that matches) which store the
annotations but the part that extract the value from the carrier into local
variables that stores the annotation.
I think this idea also work if we do not use invokedynamic, the matching par
can use a Tree or a DAG that fuses several bindings to one, it's not an issue
if we generate local variables initialized from the binding values afterward.
regards
> Maurizio
Rémi
> On 31/05/2022 17:12, Brian Goetz wrote:
>> As a confounding example that suggests that pattern variables are not "just
>> locals", in the past we talked about various forms of "merging":
>> if (t instanceof Box(String s) || t instanceof Bag(String s)) { ... }
>> or
>> case Box(String s):
>> case Bag(String s):
>> common-code;
>> If pattern variables could be annotated, then the language would be in the
>> position of deciding what happens with
>> case Box(@Foo(1) String s):
>> case Bag(@Foo(2) String s):
>> (This is the "annotation merging" problem, which is why annotations are not
>> inherited in the first place.)
>
Re: It's the data, stupid !
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Monday, May 30, 2022 8:40:33 PM
> Subject: Re: It's the data, stupid !
>> The problem is that what you propose is a leaky abstraction, because pattern
>> matching works on classes and not on types, so it's not a reverse link.
> ("Leaky abstraction" is sort of an inflammatory term.)
> What I think you're getting at is that some objects will have state that you
> can
> "put in", but can't "take out". The mathematical relationship here is
> "embedding projection pair" (this is similar to an adjoint functor pair in
> some
> ways.)
> A good example of this relationship is int and Integer. Every int corresponds
> to
> an Integer, and *almost* every Integer (except null) corresponds to an int.
> Imagine there are two functions e : int -> Integer and p : Integer -> int,
> where p(null) = bottom. Composing e-then-p is an identity; composing p-then-e
> can lose some information, but we can characterize the information loss.
> Records form this same relationship with their cartesian product space
> (assuming you follow the refined contract outlined in Record::equals). When
> you
> have this relationship, you get some very nice properties, such as "withers"
> and serialization basically for free. The relationship between a ctor and the
> corresponding dtor also has this structure. So yes, going backwards is
> "lossy",
> but in a controlled way. This turns out to be good enough for a lot of things.
I don't disagree about everything you are saying, because the problem is
elsewhere.
>> Let say we have a class with two shapes/deconstruction
>> class A {
>> deconstructor (B) { ... }
>> deconstructor (C) { ... }
>> }
>> With the pattern A(D d), D is a runtime class not a type, you have no idea
>> if it
>> means
>> instanceof A a && B b = a.deconstructor() && b instanceof D
>> or
>> instanceof A a && C c = a.deconstructor() && c instanceof D
> You can have types in the dtor bindings, just as you can have types in the
> constructor arguments. Both may use the class type variables, as they are
> instance "members".
The problem is not at callee site, as you said you have deconstructor binding
like you have constructor parameter, the problem is at callsite, when you have
a Type Pattern, a type pattern does not declare a type that can be used at
compile time but a class that is used at runtime (to do the instanceof).
So the problem is not how to declare several deconstructors, it's how to select
the right one without type information.
This is a problem specific to the switch or instanceof, if you have an
assignment context, here because the matching is total, you can extract the
types from the expression you are matching.
By example with
A(D d) = a;
here D is a type, not a class, so finding the right descriptor is easy, you can
apply the same algorithm as the overloading selection.
But with instanceof,
a instanceof A(D d)
here D is a class, you can not find the corresponding type, so you can not run
the overloading selection.
>> As i said, it's a question where OOP and DOD (data oriented design ?)
>> disagree
>> one with the other.
> I don't think they disagree at all. They are both useful tools for modeling
> things; one is good for modeling entities and processes, the other for
> modeling
> data, using a common vocabulary. Our systems may have both!
yes, the question is when both disagree, who is winning ? for all cases where
they both disagree.
>> And this is a problem specific to the deconstructor, for named pattern
>> method,
>> there is no such problem, obviously a user can add as many pattern methods
>> he/she want.
> Because there's no name, we are limited to overloads that are distinct up to
> erasure; constructors have the same restriction.
nope, because if we have a type pattern / record pattern we have no type
information.
If named pattern method allows overloading, we will have the same issue, there
is not enough type information at callsite.
>>> But for each way of putting together the data, there should be a
>>> corresponding
>>> way to take it apart.
>> if the pattern matching was a real inverse link, yes, maybe.
> I think the word "real" is doing too much lifting in that sentence.
>> The problem is that varargs pattern can also recognizes a record with no
>> record
>> or class with a deconstructor with no varargs.
> As can a constructor.
no, there is no spread operator in Java unlike by example in JavaScript, so y
Re: It's the data, stupid !
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Monday, May 30, 2022 6:40:22 PM
> Subject: Re: It's the data, stupid !
>> First, i've overlook the importance of the record pattern as a check of the
>> shape of the data.
>> Then if we say that data are more important than code and that the aim of the
>> pattern matching is to detect changes of the shapes of the data,
>> it changes the usefulness of some features/patterns.
> OK, now that I see what argument you are really winding up for, I think I'm
> going to disagree. Yes, data-as-data is a huge benefit; it is something we
> were
> not so good at before, and something that has become more important over time.
> That has motivated us to *prioritize* the data-centric features of pattern
> matching over more general ones, because they deliver direct value the
> soonest.
> But if you're trying to leverage that into a "this is the only benefit" (or
> even the main benefit), I think that's taking it too far.
> The truly big picture here is that pattern matching is the dual of
> aggregation.
> Java gives us lots of ways to put things together (constructors, factories,
> builders, maybe some day collection literals), but the reverse of each of
> these
> is ad-hoc, different, and usually harder-to-use / more error-prone. The big
> picture here is that pattern matching *completes the object model*, by
> providing the missing reverse link. (In mathematical terms, a constructor and
> deconstructor (or factory and static pattern, or builder and "unbuilder", or
> collection literal and collection pattern) form an *embedding-projection
> pair*.)
> Much of this was laid out in Pattern Matching in the Java Object Model:
> [
> https://github.com/openjdk/amber-docs/blob/master/site/design-notes/patterns/pattern-match-object-model.md
> |
> https://github.com/openjdk/amber-docs/blob/master/site/design-notes/patterns/pattern-match-object-model.md
> ]
The problem is that what you propose is a leaky abstraction, because pattern
matching works on classes and not on types, so it's not a reverse link.
Let say we have a class with two shapes/deconstruction
class A {
deconstructor (B) { ... }
deconstructor (C) { ... }
}
With the pattern A(D d), D is a runtime class not a type, you have no idea if
it means
instanceof A a && B b = a.deconstructor() && b instanceof D
or
instanceof A a && C c = a.deconstructor() && c instanceof D
Unlike with a method call (constructor call) where the type of the arguments
are available, with the pattern matching, you do not have the types of the
arguments, only runtime classes to match.
so while a deconstructor can be seen as the inverse of a constructor, a type
pattern does not give you the information of the type that allow you to do the
method selection on the deconstructors at compile time.
>> it makes the varargs pattern a kind of harmful, because it matches data of
>> several shapes, so the code may still compile if the shape of the
>> record/data-type change.
> I think you've stretched your argument to the breaking point. No one said that
> each pattern can only match *one* structure of data.
I think it's a very good question, the reason we may want to match several
structures of data is backward compatibility, but it does not make a lot of
sense to offer backward compatibility on data if at the same time the data are
more important than the code i.e. if the data drive the code.
As i said, it's a question where OOP and DOD (data oriented design ?) disagree
one with the other.
And this is a problem specific to the deconstructor, for named pattern method,
there is no such problem, obviously a user can add as many pattern methods
he/she want.
> But for each way of putting together the data, there should be a corresponding
> way to take it apart.
if the pattern matching was a real inverse link, yes, maybe.
>> - the varargs pattern can be emulated by an array pattern and it's even
>> better
>> because an array pattern checks that the shape is an array and
> Well, we don't have array patterns yet either, but just as varargs invocation
> is
> shorthand for a manually created array, varargs patterns are shorthand for an
> explicit array pattern.
The problem is that varargs pattern can also recognizes a record with no record
or class with a deconstructor with no varargs.
>> The result is that i'm not sure the vararg pattern is a target worth
>> pursuing.
> I think its fine to be "not sure", and its doubly fine to say "I'm not sure
> the
> cost-benefit is so compelling, maybe there are other features that we
Re: It's the data, stupid !
> From: "Brian Goetz" > To: "Remi Forax" , "amber-spec-experts" > > Sent: Monday, May 30, 2022 4:31:19 PM > Subject: Re: It's the data, stupid ! > Indeed, this is a big part of the motivation. And it's not just pattern > matching; its the combination of records (representing data as data), sealed > classes (the other half of algebraic data types, enabling richer data-as-data > descriptions), and pattern matching (ad-hoc polymorphism, great for data). The > catchphrase we've been using in last few years has been make it easier to do > "data-oriented programming" in Java. This isn't a departure from OO, it's a > recognition that not everything is best modeled as an stateful entity that > communicates by sending and receiving messages. > Rolling back to the origin of these feature set (several years ago at this > point), we observed that programs are getting smaller; monoliths give way to > smaller services. And the smaller the unit of code is, the closer it is to the > boundary, at which it is exchanging messy untyped (or differently typed) data > with the messy real world -- JSON, database result sets, etc. (Gone are the > days where it was Java objects all the way down, including across the wire > boundary.) We needed a simpler way to represent strongly typed ad-hoc data in > Java, one that is easy to use, which can be easily mapped to and from the > external messy formats at the boundary. OO is great at defining and defending > boundaries (it totally shines at platform libraries), but when it comes to > modeling ordinary data, costs just as much but offers us less in return. And > pattern matching is key to being able to easily act on that data, take it > apart, put it back together differently, etc. The future installments of > pattern matching are aimed at simplifying the code at that boundary; using > pattern matching to mediate conversion from untyped, schema-free envelopes > like > JSON to illegal-states-are-unrepresentable data. > So, yes: records + sealed classes + pattern matching = embracing data as data. > I've got a piece I've been writing on this very topic, I'll send a link when > its up. > And yes, we've been talking a lot about the details, because that's what this > group is for. But I don't think we have lost sight of the big picture. > Is there something you think we've missed? First, i've overlook the importance of the record pattern as a check of the shape of the data. Then if we say that data are more important than code and that the aim of the pattern matching is to detect changes of the shapes of the data, it changes the usefulness of some features/patterns. it makes the varargs pattern a kind of harmful, because it matches data of several shapes, so the code may still compile if the shape of the record/data-type change. Given that we have already establish that - the varargs pattern can be emulated by an array pattern and it's even better because an array pattern checks that the shape is an array and - the varargs is dangerous because record with varargs are hard to get right. The result is that i'm not sure the vararg pattern is a target worth pursuing. Deconstructors of a class also becomes a kind of a war ground between the OOP and the pattern matching, OOP says that API is important and pattern matching says it's ok to change the data changing the API because the compiler will points where the code should be updated. We still want encapsulation because it's a class but we want to detect if its shape change so having a class with several shapes becomes not as useful as i first envision. And we have some pattern methods, that works more like extension methods because you can define them outside of the class you do the matching on (they are not instance pattern method) and how those are supposed to work when a shape is updated. Rémi > On 5/30/2022 8:33 AM, Remi Forax wrote: >> Hi all, >> i think the recent discussions about the pattern matching are too much about >> details of implementation and i fear we are losing the big picture, so let me >> explain why i (we ?) want to add pattern matching to Java. >> Java roots is OOP, encapsulation serves us well for the last 25+ years, it >> emphasis API above everything else, data are not important because it's just >> a >> possible implementation of the API. >> But OOP / encapsulation is really important for libraries, less for >> applications. For an application, data are more important, or at least as >> important as API. >> The goal of pattern matching is make data the center of the universe. Data >> are >> more important than code, if the data change because the busine
It's the data, stupid !
Hi all, i think the recent discussions about the pattern matching are too much about details of implementation and i fear we are losing the big picture, so let me explain why i (we ?) want to add pattern matching to Java. Java roots is OOP, encapsulation serves us well for the last 25+ years, it emphasis API above everything else, data are not important because it's just a possible implementation of the API. But OOP / encapsulation is really important for libraries, less for applications. For an application, data are more important, or at least as important as API. The goal of pattern matching is make data the center of the universe. Data are more important than code, if the data change because the business requirements change, the code should be updated accordingly. Pattern matching allows to write code depending on the data and it will fail to compile if the data change, indicating every places in the code where the code needs to be updated to take care of the new data shape. The data can change in different ways, 1) a new kind of a type (a subtype of an interface) can be introduced, we have added sealed types and make switch on type exhaustive so if a developer add a new subtype of an interface, the compiler will refuse to compile all patterns that are not exhaustive anymore, indicating that the code must be updated. 2) a data can have a new field/component, we have introduced record pattern that match the exact shape of a record, so if a developer add a new component, the compiler will refuse to compile the record pattern with a wrong shape indicating that the code must be updated. So experts, do you agree that this is what we want or did i miss something ? Rémi PS: the title is a nod to James Carville.
Re: Guard variable and being effectively final
- Original Message -
> From: "cay horstmann"
> To: "Remi Forax" , "amber-spec-experts"
>
> Sent: Saturday, May 21, 2022 7:50:44 PM
> Subject: Re: Guard variable and being effectively final
> Hi Rémy,
>
> it compiles with build 19-ea+23-1706 if you replace && with when.
sadly, assigning op also compiles :(
IntBinaryOperator op = null;
I think the new implementation is missing a check that the local variables
after a "when" has to be effectively final
>
> Also, remove "4" from the list or add an operator :-)
yes
>
> Cheers,
>
> Cay
Rémi
>
> Il 21/05/2022 13:55, Remi Forax ha scritto:
>> interface reverse_polish_notation {
>>static Map OPS =
>>Map.of("+", (a, b) -> a + b, "*", (a, b) -> a * b);
>>
>>static int eval(List expr) {
>> var stack = new ArrayDeque();
>> for(var token: expr) {
>>final IntBinaryOperator op;
>>stack.push(switch (token) {
>> case String __ && (op = OPS.get(token)) != null -> {
>>var value1 = stack.pop();
>>var value2 = stack.pop();
>>yield op.applyAsInt(value1, value2);
>> }
>> default -> Integer.parseInt(token);
>>});
>> }
>> return stack.pop();
>>}
>>
>>static void main(String[] args) {
>> var expr = List.of("1", "2", "+", "3", "*", "4");
>> System.out.println(eval(expr));
>>}
>> }
>
> --
>
> Cay S. Horstmann | http://horstmann.com | mailto:c...@horstmann.com
Guard variable and being effectively final
Not sure if it's an implementation bug (bad error message from the compiler) or
a spec bug,
hence this message to both amber-dev and amber-spec-experts.
If i try to compile this code with Java 19 (which currently still uses &&
instead of when for a guard)
interface reverse_polish_notation {
static Map OPS =
Map.of("+", (a, b) -> a + b, "*", (a, b) -> a * b);
static int eval(List expr) {
var stack = new ArrayDeque();
for(var token: expr) {
final IntBinaryOperator op;
stack.push(switch (token) {
case String __ && (op = OPS.get(token)) != null -> {
var value1 = stack.pop();
var value2 = stack.pop();
yield op.applyAsInt(value1, value2);
}
default -> Integer.parseInt(token);
});
}
return stack.pop();
}
static void main(String[] args) {
var expr = List.of("1", "2", "+", "3", "*", "4");
System.out.println(eval(expr));
}
}
I get the following error
java --enable-preview --source 19 reverse_polish_notation.java
reverse_polish_notation.java:17: error: local variables referenced from a guard
must be final or effectively final
case String __ && (op = OPS.get(token)) != null -> {
^
Note: reverse_polish_notation.java uses preview features of Java SE 19.
Note: Recompile with -Xlint:preview for details.
1 error
error: compilation failed
Obviously the error message is funny, IntBinaryOperator is declared final so it
is effectively final.
In case of a lambda,
final IntBinaryOperator op;
Supplier supplier = () -> op = null;
supplier.get() can be called several times so "op = null" does not compile.
But in the example above, "op" can not be assigned more than once so maybe it
should compile.
regards,
Rémi
Re: Pattern matching: next steps after JEP 405
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Wednesday, May 18, 2022 9:18:01 PM
> Subject: Pattern matching: next steps after JEP 405
> JEP 405 has been proposed to target for 19. But, it has some loose ends that
> I'd
> like to refine before it eventually becomes a final feature. These include:
[...]
> - Varargs patterns. Records can be varargs, but we have an asymmetry where we
> can use varargs in constructors but not in deconstruction. This should be
> rectified. The semantics of this is straightforward; given
> record Foo(int x, int y, int... zs) { }
> just as
> new Foo(x, y, z1, z2)
> is shorthand for
> new Foo(x, y, new int[] { z1, z2 })
> we also can express
> case Foo(var x, var y, var z1, var z2)
> as being shorthand for
> case Foo(var x, var y, int[] { var z1, var z2 })
> This means that varargs drags in array patterns.
Thinking a bit about the varargs pattern, introducing them is not a good idea
because a varargs record is not a safe construct by default,
- varargs are arrays, and arrays are mutable in Java, so varargs records are
not immutable by default
- equals() and hashCode() does not work as is too.
The record Foo should be written
record Foo(int x, int y, ... zs) {
Foo {
zs = zs.clone();
}
public int[] zs() {
return zs.clone();
}
public boolean equals(Object o) {
return o instanceof Foo foo && x == foo.x && y == foo.y && Arrays.equals(zs,
foo.zs);
}
public int hashCode() {
return hash(x, y, Arrays.hashCode(zs));
}
}
Given that most people will forget that the default behavior of a varargs
record is not the right one, introducing a specific pattern for varargs record
to mirror them is like giving a gentle nudge to somebody on a cliff.
Note that, it does not mean that we will not support varargs record, because
one can still write either
case Foo(int x, int y, int[] zs)
or
case Foo(int x, int y, int[] { int... zs }) // or a similar syntax that mix a
record pattern and an array pattern
but just that there will be no streamlined syntax for a varargs record.
Rémi
Re: Collections patterns
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Thursday, May 19, 2022 3:03:55 PM > Subject: Re: Collections patterns >> We may want to extract sub-parts of the array / collections by example, and i >> would prefer to have the same semantics and a similar syntax. > This is pretty vague, so I'll have to guess about what you might mean. > Maybe you mean: "I want to match a list if it contains the a subsequence that > matches this sequence of patterns", something like: > [ ... p1, p2, ... ] > There is surely room to have APIs that query lists like this, but I think this > is way out of scope for a pattern matching feature the language. Pattern > matching is about _destructuring_. (Destructuring can be conditional.) An > array > is a linear sequence of elments; it can be destructured by a linear sequence > of > patterns. > Maybe you mean: "I want to decompose a list into the head element and a tail > list". > In Haskell, we iterate a list by recursion: > len :: [a] -> Int > len [] = 0 > len x:xs = 1 + len xs > But again, this is *mere destructuring*, because the cons operator (:) is the > linguistic primitive for aggregation, and [ ... ] lists are just sugar over > cons. So matching to `x:xs` is again destructuring. We could try to apply this > to Java, but it gets very clunky (inefficient, no tail recursion, yada yada) > because our lists are *built differently*. Further further, arrays are another > step removed from lists even. > Or maybe you mean something else; if so, please share! The current proposal is more about matching and extracting the first arguments than matching/extracting the last arguments or the rest are also useful IMO. By example, if i want to parse command line arguments composed of options and a filename, i may want to write something like case [String... options, String filename] -> ... case [String...] -> help() Rémi
Re: Pattern matching: next steps after JEP 405
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Thursday, May 19, 2022 3:05:07 PM > Subject: Re: Pattern matching: next steps after JEP 405 >> When you have a type pattern X in a middle of a pattern *and* you have >> conversions, then there is an ambiguity, >> does instanceof Box(X x) means >> Box(var v) && v instanceof X x >> or >> Box(var v) && X x = (X) v; > This is not an ambiguity in the language, it is confusion on the part of the > reader :) > In any case, I'm not following your argument here. If you have both a type pattern and allow conversions, you have Box(X) is equivalent to Box(var v) && v instanceof Y y && X x = (X) y How do you find Y ? And yes, the bar is not only that Y has to be unique for the compiler, it has also to be obvious for the human reader too. Rémi
Re: Pattern matching: next steps after JEP 405
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Wednesday, May 18, 2022 11:08:41 PM > Subject: Re: Pattern matching: next steps after JEP 405 >>> - Primitive patterns. This is driven by another existing asymmetry; we can >>> use >>> conversions (boxing, widening) when constructing records, but not when >>> deconstructing them. There is a straightforward (and in hindsight, obvious) >>> interpretation for primitive patterns that is derived entirely from existing >>> cast conversion rules. >> When calling a method / constructing an object, you can have several >> overloads >> so you need conversions, those conversions are known at compile time. >> (Note that Java has overloads mostly because there is a rift between >> primitives >> and objects, if there was a common supertype, i'm not sure overloads will >> have >> carried their own weights.) >> When doing pattern matching, there is no overloads otherwise you will have to >> decide which conversions to do at runtime. > There are no overloads YET, because the only deconstruction patterns are in > records, and records have only one state description. But that is a > short-lived > state of affairs. When we do declared deconstruction patterns, we will need > overload selection, and it will surely need to dualize the existing > three-phase > overload selection for constructors (e.g., loose, string, and varargs > invocation contexts.) When you have a type pattern X in a middle of a pattern *and* you have conversions, then there is an ambiguity, does instanceof Box(X x) means Box(var v) && v instanceof X x or Box(var v) && X x = (X) v; For the deconstruction pattern, if we have overloads, there is the same kind of ambiguity does Box(X x) means calling Box.deconstructor(X) or calling Box.deconstructor(Y) with y instanceof X So if we want to support overloads, and we may have to, at least to support adding components but keeping the class backward compatible, we need to introduce new rules to solve that ambiguity. Rémi
Collections patterns
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Wednesday, May 18, 2022 11:08:41 PM > Subject: Re: Pattern matching: next steps after JEP 405 >> Inference is also something we will need for pattern assignment >> Box<>(var s) = box; > Yes, it would work the same in all pattern contexts -- instanceof as well. > Every > pattern context has a match target whose static type is known. >>> - Array patterns. The semantics of array patterns are a pretty simple >>> extension >>> to that of record patterns; the rules for exhaustiveness, applicability, >>> nesting, etc, are a relatively light transformation of the corresponding >>> rules >>> for record patterns. The only new wrinkle is the ability to say "exactly N >>> elements" or "N or more elements". >> I wonder if we should not at least work a little on patterns on collections, >> just to be sure that the syntax and semantics of the patterns on collections >> and patterns on arrays are not too dissimilar. > This is a big new area; collection patterns would have to be co-designed with > collection literals, and both almost surely depend on some sort of type class > mechanism if we want to avoid the feature being lame. I don't think its > realistic to wait this long, nor am I aiming at doing anything that looks like > a generic array query mechanism. Arrays have a first element, a second > element, > etc; the nesting semantics are very straightforward, and the only real > question > that needs additional support seems to be "match exactly N" or "match first > N". We may want to extract sub-parts of the array / collections by example, and i would prefer to have the same semantics and a similar syntax. And i don't think we need type classes here because we can use the target class mechanism instead, like we have done with lambdas, instead of Type x -> x + 1 or whatever the exact syntax, we have (Type) x -> x + 1 We may want [1, 2, 3] to have a type, so it's maybe a little more complicated than just using the target typing but i don't think type classes are needed for collection litterals. For operator overloading on numeric value classes, that's another story. Rémi
Re: Pattern matching: next steps after JEP 405
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Wednesday, May 18, 2022 9:18:01 PM
> Subject: Pattern matching: next steps after JEP 405
> JEP 405 has been proposed to target for 19. But, it has some loose ends that
> I'd
> like to refine before it eventually becomes a final feature. These include:
> - Inference for record patterns. Right now, we make you spell out the type
> parameters for a generic record pattern, such as:
> case Box(String s):
> We also don't allow parameterizations that are not consistent with the match
> target. While this is clear, it is verbose (and gets worse when there is
> nesting), and also, because of the consistency restriction, the
> parameterization is entirely inferrable. So we would like to allow the
> parameters to be inferred. (Further, inference is relevant to GADTs, because
> we
> may be able to gather constraints on the pattern from the nested patterns, if
> they provide some type parameter specialization.)
Inference is also something we will need for pattern assignment
Box<>(var s) = box;
> - Refined type checking for GADTs. Given a hierarchy like:
> sealed interface Node { }
> record IntNode(int i) implements Node { }
> record FloatNode(float f) implements Node { }
> we currently cannot type-check programs like:
> Node twice(Node n) {
> return switch (n) {
> case IntNode(int x) -> new IntNode(x*2);
> case FloatNode(float x) -> new FloatNode(x*2);
> }
> }
> because, while the match constraints the instantiation of T in each arm of the
> switch, the compiler doesn't know this yet.
> - Varargs patterns. Records can be varargs, but we have an asymmetry where we
> can use varargs in constructors but not in deconstruction. This should be
> rectified. The semantics of this is straightforward; given
> record Foo(int x, int y, int... zs) { }
> just as
> new Foo(x, y, z1, z2)
> is shorthand for
> new Foo(x, y, new int[] { z1, z2 })
> we also can express
> case Foo(var x, var y, var z1, var z2)
> as being shorthand for
> case Foo(var x, var y, int[] { var z1, var z2 })
> This means that varargs drags in array patterns.
> - Array patterns. The semantics of array patterns are a pretty simple
> extension
> to that of record patterns; the rules for exhaustiveness, applicability,
> nesting, etc, are a relatively light transformation of the corresponding rules
> for record patterns. The only new wrinkle is the ability to say "exactly N
> elements" or "N or more elements".
I wonder if we should not at least work a little on patterns on collections,
just to be sure that the syntax and semantics of the patterns on collections
and patterns on arrays are not too dissimilar.
> - Primitive patterns. This is driven by another existing asymmetry; we can use
> conversions (boxing, widening) when constructing records, but not when
> deconstructing them. There is a straightforward (and in hindsight, obvious)
> interpretation for primitive patterns that is derived entirely from existing
> cast conversion rules.
When calling a method / constructing an object, you can have several overloads
so you need conversions, those conversions are known at compile time.
(Note that Java has overloads mostly because there is a rift between primitives
and objects, if there was a common supertype, i'm not sure overloads will have
carried their own weights.)
When doing pattern matching, there is no overloads otherwise you will have to
decide which conversions to do at runtime.
Given that one mechanism decide which overload to call at compile time and the
other decide which patterns to call at runtime, they are not symmetric.
You can restrict the set of conversions so there is a bijection between the
classes checked at the runtime and the type the user will write, but it's a new
mechanism, not an existing symmetry.
In case of a pattern assignment, this is different because there is only one
pattern that have to cover all the type, so in that case, conversions are not
an issues because there is no runtime checks.
> Obviously there is more we will want to do, but this set feels like what we
> have
> to do to "complete" what we started in JEP 405. I'll post detailed summaries,
> in separate threads, of each over the next few days.
Rémi
Re: [External] : Re: Record pattern and side effects
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Friday, May 6, 2022 3:11:43 PM
> Subject: Re: [External] : Re: Record pattern and side effects
>> The accessor throws an exception and with the semantics you propose it will
>> happily be wrapped by as many MatchExceptions as possible.
> But this class is already so deeply questionable, because accessors should not
> throw exceptions, that we've lost the game before we started. Whether the
> exception comes wrapped or not is like asking "do you want whipped cream on
> your mud and axle-grease pie" :)
People makes mistakes and other ones have to debug it.
> (Also, I don't see where the exception is wrapped multiple times? So I'm not
> even sure you are clear on what is being propsed.)
"I don't see" -> that's exactly my point ! Nobody will see it.
Rémi
>> public sealed interface RecChain {
>> default int size() {
>> return switch (this) {
>> case Nil __ -> 0;
>> case Cons(var v, var s, var next) -> 1 + next.size();
>> };
>> }
>> record Nil() implements RecChain { }
>> record Cons(int value, int size, RecChain next) implements RecChain {
>> @Override
>> public int size() {
>> return size != -1? size: RecChain.super.size();
>> }
>> }
>> public static void main(String[] args){
>> RecChain chain = new Nil();
>> for (var i = 0; i < 100; i++) {
>> chain = new Cons(i, -1, chain);
>> }
>> System.out.println(chain.size());
>> }
>> }
>> Rémi
Re: [External] : Re: Record pattern and side effects
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Friday, April 22, 2022 3:34:29 PM
> Subject: Re: [External] : Re: Record pattern and side effects
>>> Let's imagine that dtor D throws. The wrapping happens when a dtor/accessor
>>> is
>>> invoked _implicitly_ as a result of evaluating a pattern match. In both
>>> cases,
>>> we will wrap the thrown exception and throw MatchException. In this way,
>>> both
>>> instanceof and switch are "clients of" pattern matching, and it is pattern
>>> matching that throws.
>>> I don't see any destruction here.
>> I'm thinking about the refactoring from a code using accessors to a code
>> using a
>> deconstructor.
>> By example, IDEs may propose to refactor this code
>> if (x instanceof D d) A(d.p()); else B;
>> to
>> if (x instanceof D(P p)) A(p); else B;
>> or vice versa
>> If you wraps deconstructor exceptions, but not accessor exceptions you have
>> mismatch.
> OK, sure. This bothers me zero. Having an accessor (or dtor) throw is already
> really^3 weird; having a program depend on which specific exception it throws
> is really^32 weird. (In both cases, they still throw an exception that you
> probably shouldn't be catching, with a clear stack trace explaining where it
> went wrong.) Not a case to design the language around.
> Still not seeing any "destruction" here.
Let's try with a puzzler, i have a recursive list with a slight twist, the Cons
can store the size of the list or not (using -1 if not).
The accessor throws an exception and with the semantics you propose it will
happily be wrapped by as many MatchExceptions as possible.
public sealed interface RecChain {
default int size() {
return switch (this) {
case Nil __ -> 0;
case Cons(var v, var s, var next) -> 1 + next.size();
};
}
record Nil() implements RecChain { }
record Cons(int value, int size, RecChain next) implements RecChain {
@Override
public int size() {
return size != -1? size: RecChain.super.size();
}
}
public static void main(String[] args){
RecChain chain = new Nil();
for (var i = 0; i < 100; i++) {
chain = new Cons(i, -1, chain);
}
System.out.println(chain.size());
}
}
Rémi
Re: [External] : Re: Record pattern and side effects
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Thursday, April 21, 2022 6:20:24 PM
> Subject: Re: [External] : Re: Record pattern and side effects
>>> We’ve already asked one of the questions on side effects (though not sure we
>>> agreed on the answer): what if the dtor throws? The working story is that
>>> the
>>> exception is wrapped in a MatchException. (I know you don’t like this, but
>>> let’s not rehash the same arguments.)
>> Wrapping exceptions into a MatchException destroy any idea of refactoring
>> from a
>> cascade of if ... instanceof to a switch.
>> I think refactoring is a use case we should support.
> Wrapping exceptions thrown from dtors does not affect refactoring.
> If I have:
> if (x instanceof D(P)) A;
> else if (x instanceof D(Q)) B;
> else C;
> and I refactor to
> switch (x) {
> case D(P): A; break;
> case D(Q): B; break;
> default: C
> }
> Let's imagine that dtor D throws. The wrapping happens when a dtor/accessor is
> invoked _implicitly_ as a result of evaluating a pattern match. In both cases,
> we will wrap the thrown exception and throw MatchException. In this way, both
> instanceof and switch are "clients of" pattern matching, and it is pattern
> matching that throws.
> I don't see any destruction here.
I'm thinking about the refactoring from a code using accessors to a code using
a deconstructor.
By example, IDEs may propose to refactor this code
if (x instanceof D d) A(d.p()); else B;
to
if (x instanceof D(P p)) A(p); else B;
or vice versa
If you wraps deconstructor exceptions, but not accessor exceptions you have
mismatch.
And as i said earlier, there is also the issue with a deconstructor calling
another deconstructor (like your example with super), you may wrap a
MatchException into a MatchException.
Rémi
Re: Record pattern and side effects
- Original Message - > From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Sunday, April 17, 2022 4:58:26 PM > Subject: Re: Record pattern and side effects > Yes, this is something we have to get “on the record”. > > Record patterns are a special case of deconstruction patterns; in general, we > will invoke the deconstructor (which is some sort of imperative code) as part > of the match, which may have side-effects or throw exceptions. With records, > we go right to the accessors, but its the same game, so I’ll just say “invoke > the deconstructor” to describe both. > > While we can do what we can to discourage side-effects in deconstructors, they > will happen. This raises all sorts of questions about what flexibility the > compiler has. > > Q: if we have > >case Foo(Bar(String s)): >case Foo(Bar(Integer i)): > > must we call the Foo and Bar deconstructors once, twice, or “dealer’s choice”? > (I know you like the trick of factoring a common head, and this is a good > trick, but it doesn’t answer the general question.) > > Q: To illustrate the limitations of the “common head” trick, if we have > >case Foo(P, Bar(String s)): >case Foo(Q, Bar(String s)): > > can we factor a common “tail”, where we invoke Foo and Bar just once, and then > use P and Q against the first binding? > > Q: What about reordering? If we have disjoint patterns, can we reorder: > >case Foo(Bar x): >case TypeDisjointWithFoo t: >case Foo(Baz x): > > into > >case Foo(Bar x): >case Foo(Baz x): >case TypeDisjointWithFoo t: > > and then fold the head, so we only invoke the Foo dtor once? > > Most of the papers about efficient pattern dispatch are relatively little help > on this front, because the come with the assumption of purity / > side-effect-freedom. But it seems obvious that if we were trying to optimize > dispatch, our cost model would be something like arithmetic op << type test << > dtor invocation, and so we’d want to optimize for minimizing dtor invocations > where we can. yes ! > > We’ve already asked one of the questions on side effects (though not sure we > agreed on the answer): what if the dtor throws? The working story is that the > exception is wrapped in a MatchException. (I know you don’t like this, but > let’s not rehash the same arguments.) Wrapping exceptions into a MatchException destroy any idea of refactoring from a cascade of if ... instanceof to a switch. I think refactoring is a use case we should support. Rémi
Re: case null / null pattern (v2)
> From: "Brian Goetz"
> To: "Remi Forax" , "amber-spec-experts"
>
> Sent: Tuesday, April 19, 2022 10:02:22 PM
> Subject: Re: case null / null pattern (v2)
> With the currently specified semantics, the second pattern is dead, because
> switches will only match null at the top level with a case null. This was an
> accommodation to clarify that that the null-hostility of switch is a property
> of switch, not patterns, and make it more clear when switch will NPE.
"case Foo fooButNull" is equivalent to "case null" but with a binding typed as
Foo that's why i ask if it should even compile,
the compiler should ask for an explicit "case null".
But because there is no null pattern, we can not do that if it appears as a
sub-pattern.
> Regardless, what you're asking for is a more precise remainder checking. The
> first pattern matches all non-null Foo; because no case matches null, you're
> asking that we recognize that there is a dominance relationship here. This is
> reasonable to consider (though harder, because null makes everything harder.)
I'm not sure it should be include in the dominance, because Foo(int) is a
subtype of Foo, so
case Foo foo ->
case Foo(int _) foo ->
should not compile.
The ideal solution for me is to disable that pattern (record pattern + type
pattern on the same type) and allow the null pattern, but as you said earlier,
if we allow case(null), it's opening the pandora box because people will want
to write case Foo(7).
Rémi
> On 4/18/2022 6:49 PM, Remi Forax wrote:
>> I've found a way to encode the null pattern if you have a record
>> record Foo(int x) { }
>> Foo foo = ...
>> return switch(foo) {
>> case Foo(int _) foo -> "i'm a foo not null here !";
>> case Foo fooButNull -> "i can be only null here !";
>> };
>> I wonder if allowing those two patterns, a record pattern and a type pattern
>> using the same type is a good idea or not, it seems a great way to obfuscate
>> thing.
>> Rémi
case null / null pattern (v2)
I've found a way to encode the null pattern if you have a record
record Foo(int x) { }
Foo foo = ...
return switch(foo) {
case Foo(int _) foo -> "i'm a foo not null here !";
case Foo fooButNull -> "i can be only null here !";
};
I wonder if allowing those two patterns, a record pattern and a type pattern
using the same type is a good idea or not, it seems a great way to obfuscate
thing.
Rémi
Record pattern and side effects
This is something i think we have no discussed, with a record pattern, the
switch has to call the record accessors, and those can do side effects,
revealing the order of the calls to the accessors.
So by example, with a code like this
record Foo(Object o1, Object o2) {
public Object o2() {
throw new AssertionError();
}
}
void int m(Foo foo) {
return switch(foo) {
case Foo(String s, Object o2) -> 1
case Foo foo -> 2
};
}
m(new Foo(3, 4)); // throw AssertionError ?
Do the call throw an AssertionError ?
I believe the answer is no, because 3 is not a String, so Foo::o2() is not
called.
Rémi
case null vs null pattern
Hi all,
i maybe wrong but it seems that the spec consider null as a kind of case
instead of as a kind of pattern, which disables the refactoring that should be
possible with the introduction of the record pattern.
Let suppose i have a sealed type with only one implementation declared like this
sealed interface I {
record A() implements I { }
}
if null is part of the set of possible values, i have switches like this
switch(i) {
case null, A a -> // do something with 'a' here
}
Now we are introducing record pattern into the mix, so i can have a Box of I,
record Box(I i) { }
the problem is that i can not write
switch(box) {
case Box(A|null a) -> // do something with 'a' here
}
because null is handled as a kind of case instead of as a kind of a null
pattern.
Should we introduce a null pattern instead of having a specific "case null" ?
(and disallow the null pattern in an instanceof ?)
regards,
Rémi
Re: Evolving past reference type patterns
> From: "Guy Steele"
> To: "Brian Goetz"
> Cc: "amber-spec-experts"
> Sent: Saturday, April 16, 2022 4:10:06 AM
> Subject: Re: Evolving past reference type patterns
> Yes, this is a clear improvement to the example code.
> That said, I am always (or at least now) a bit leery of language designers
> motivating a new language feature by pointing out that it would make a
> compiler
> easier to write. As I have learned the hard way on more than one language
> project, compilers are not always representative of typical application code.
> (Please consider this remark as only very minor pushback on the form of the
> argument.)
This is a very specific example due to the way integers are encoded in the
bytecode,
also you can simplify the code a bit more because ICONST_M1, ICONST_0, etc are
all subsequents.
Moreover, as i said earlier, it's more a work for method patterns.
If we suppose we have a static pattern method isByte in java.lang.Byte
class Byte {
static pattern (byte) isByte(int value) {
if (value >= -128 && value <= 127) {
return match (short) value;
}
return no-match;
}
}
the code becomes
return with(switch (value) {
case -1 .. 5 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_M1 +
value);
case Byte.isByte(byte _) -> ConstantInstruction.ofArgument(Opcode.BIPUSH,
value);
case Short.isShort(short _) ->
ConstantInstruction.ofArgument(Opcode.SIPUSH, value);
default -> ConstantInstruction.ofLoad(Opcode.LDC,
BytecodeHelpers.constantEntry(constantPool(), value));
});
Rémi
>> On Apr 15, 2022, at 5:36 PM, Brian Goetz < [ mailto:brian.go...@oracle.com |
>> brian.go...@oracle.com ] > wrote:
>>>* asking if something fits in the range of a byte or int; doing this by
>>> hand is
>>> annoying and error-prone
>>>* asking if casting from long to int would produce truncation; doing
>>> this by
>>> hand is annoying and error-prone
>> Here’s some real code I wrote recently that would benefit dramatically from
>> this:
>> default CodeBuilder constantInstruction(int value) {
>> return with(switch (value) {
>> case -1 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_M1);
>> case 0 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_0);
>> case 1 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_1);
>> case 2 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_2);
>> case 3 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_3);
>> case 4 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_4);
>> case 5 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_5);
>> default -> {
>> if (value >= -128 && value <= 127) {
>> yield ConstantInstruction.ofArgument(Opcode.BIPUSH, value);
>> }
>> else if (value >= -32768 && value <= 32767) {
>> yield ConstantInstruction.ofArgument(Opcode.SIPUSH, value);
>> }
>> else {
>>yield ConstantInstruction.ofLoad(Opcode.LDC,
>> BytecodeHelpers.constantEntry(constantPool(), value));
>> }
>> }
>> });
>> }
>> could become the less error-prone and uniform:
>> default CodeBuilder constantInstruction(int value) {
>> return with(switch (value) {
>> case -1 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_M1);
>> case 0 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_0);
>> case 1 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_1);
>> case 2 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_2);
>> case 3 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_3);
>> case 4 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_4);
>> case 5 -> ConstantInstruction.ofIntrinsic(Opcode.ICONST_5);
>> case byte value -> ConstantInstruction.ofArgument(Opcode.BIPUSH,
>> value);
>> case short value -> ConstantInstruction.ofArgument(Opcode.SIPUSH,
>> value);
>>default -> ConstantInstruction.ofLoad(Opcode.LDC,
>> BytecodeHelpers.constantEntry(constantPool(), value));
>> });
>> }
>>
Re: [External] : Re: Evolving past reference type patterns
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Saturday, April 16, 2022 12:25:20 AM > Subject: Re: [External] : Re: Evolving past reference type patterns >>> >>> Can you provides examples of such refactorings ? > Refactoring > int x = aShort; > foo(x, x); > to > let int x = aShort > in foo(x, x); We already agree that pattern assignment (the let =) should use assignment conversions. Rémi
Re: Evolving past reference type patterns
> From: "Brian Goetz" > To: "amber-spec-experts" > Sent: Friday, April 15, 2022 10:50:27 PM > Subject: Evolving past reference type patterns [...] > There are also some small differences between the safe cast conversions and > method invocation context. There is the same issue with unboxing null (throws > in (loose) invocation context), and method invocation context makes no attempt > to do narrowing, even for literals. This last seems mostly a historical wart, > which now can’t be changed because it would either potentially change (very > few) overload selection choices, or would require another stage of selection. It's not an historical wart, it's careful design (or co-design between the language and the VM). Assignments can be used for fields, that are stored on heap, while method arguments can only be stored on stack. Values on stack/in registers are either 32 bits or 64 bits while values on the heap are multiple of 8 bits. That's the same reason why the result of the addition of two shorts is an int and why assignment conversions in the context of a computation (the pattern matching helps to express computation) feels weird. > What are the arguments against this interpretation? They seem to be various > flavors of “ok, but, do we really need this?” and “yikes, new complexity.” > The first argument comes from a desire to treat pattern matching as a > “Coin”-like feature, strictly limiting its scope. (As an example of a similar > kind of pushback, in the early days, it was asked “but does pattern matching > have to be an expression, couldn’t we just have an “ifmatch” statement? (See > answer here: [ > http://mail.openjdk.java.net/pipermail/amber-dev/2018-December/003842.html | > http://mail.openjdk.java.net/pipermail/amber-dev/2018-December/003842.html ] ) > This is the sort of question we get a lot — there’s a natural tendency to try > to “scope down” features that seem unfamiliar. But I think it’s > counterproductive here. We all know that this is a big feature, don't worry. > The second argument is largely a red herring, in that this is not new > complexity, since these are exactly the rules for successful casts. In fact, > not doing it might well be perceived as new complexity, since it results in > more corner cases where refactorings that seem like they should work, do not, > because of conversions. > It is new complexity, the rules for primitive narrowing are brand new. Can you provides examples of such refactorings ? Rémi
Re: [External] : Re: Primitive type patterns
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Thursday, April 7, 2022 11:24:05 PM
> Subject: Re: [External] : Re: Primitive type patterns
>> We already discussed those rules when we discuss instanceof, it means that "x
>> instanceof primitive" has different meaning depending on the type of x
> No, it does not. It means "does x match the pattern P" everywhere. It is
> pattern
> P is that has different meanings depending on type. This may sound like a
> silly
> distinction, but it is not! Pattern matching is inherently polymorphic -- it
> is
> all about reflecting dynamic conversions statically -- and exhibits the *same*
> polymorphism regardless of where it occurs.
The switch is (runtime) polymorphic while each patterns does not have to be
(statically) polymorphic.
If you prefer, i'm okay with be a pattern overriding polymorphic but not with a
pattern being overloading polymorphic (sometimes called ad-hoc polymorphism).
Your are proposing overloading of patterns, i.e the same pattern having
different meaning depending on the static types.
> And, why would we not want duality with:
> record R(short s) { }
> ...
> new R(x)
because new R(x) is alone while case R(...) is part of a larger set of
patterns/sub-pattern of the pattern matching, if for each pattern/sub-pattern,
we need a double-entry table to understand the semantics, we are well past our
complexity budget.
Rémi
Re: Primitive type patterns
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Thursday, April 7, 2022 9:54:11 PM
> Subject: Re: Primitive type patterns
> There's another, probably stronger, reason why primitive patterns supporting
> widening, narrowing, boxing, unboxing, etc, are basically a forced move,
> besides alignment with `let` statements, discussed earlier:
>> There is another constraint on primitive type patterns: the let/bind
>> statement
>> coming down the road. Because a type pattern looks (not accidentally) like a
>> local variable declaration, a let/bind we will want to align the semantics of
>> "local variable declaration with initializer" and "let/bind with total type
>> pattern". Concretely:
>> let String s = "foo";
>> is a pattern match against the (total) pattern `String s`, which introduces
>> `s`
>> into the remainder of the block. Since let/bind is a generalization of local
>> variable declaration with initialization, let/bind should align with locals
>> where the two can express the same thing. This means that the set of
>> conversions allowed in assignment context (JLS 5.2) should also be supported
>> by
>> type patterns.
> The other reason is the duality with constructors (and eventually, methods).
> if
> we have a record:
> record R(int x, Long l) { }
> we can construct an R with
> new R(int, Long) // no adaptation
> new R(Integer, Long) // unbox x
> new R(short, Long) // widen x
> new R(int, long) // box y
> Deconstruction patterns are the dual of constructors; we should be able to
> deconstruct an R with:
> case R(int x, Long l) // no adaptation
> case R(Integer x, Long l) // box x
> case R(short s, Long l) // range check
> case R(int x, long l) // unbox y, null check
> So the set of adaptations in method invocation context should align with those
> in nested patterns, too.
We already discussed those rules when we discuss instanceof, it means that "x
instanceof primitive" has different meaning depending on the type of x
Object x = ...
if (x instanceof short) { ... } // <=> instanceof Short + unboxing
int x = ...
if (x instanceof short) { ... } // <=> range check
Byte x = ...
if (x instanceof short) { ... } // <=> nullcheck + unboxing + widening
You are overloading instanceof with different meanings, losing everybody apart
the compiler in the process.
It's also another creative way to have an action at distance,
var x = ...
f (x instanceof short) { ... } // <=> ???
We do not need all theses conversions to be part of the pattern, those
conversions are already done as part expression/instruction after the ':' or
'->' of the switch in an organic way.
// with op(int, Long)
case R(int x, Long l) -> op(x, l); // no adaptation
// with op(Integer, Long)
case R(int x, Long l) -> op(x, l); // box x
// with op(int, long)
case R(int x, Long l) -> op(x, l); // unbox l, nullcheck
And for the range check, as i said earlier, it's better to use a pattern method
with a good name so everybody will be able to read the code.
Rémi
Re: [External] : Re: Remainder in pattern matching
- Original Message - > From: "Dan Heidinga" > To: "Brian Goetz" > Cc: "amber-spec-experts" > Sent: Wednesday, March 30, 2022 8:43:42 PM > Subject: Re: [External] : Re: Remainder in pattern matching > On Wed, Mar 30, 2022 at 2:38 PM Brian Goetz wrote: >> [...] > > And from your other response: > >> Another thing it gains is that it discourages people >> from thinking they can use exceptions in dtors; having these laundered >> through MatchException discourages using this as a side channel, though >> that's a more minor thing. > > This is a stronger argument than you give it credit for being. > Wrapping the exception adds a bit of friction to doing the wrong thing > which will pay off in helping guide users to the intended behaviour. Wrapping exceptions into a MatchException seems a very bad idea to me. When you compute something on an AST, the pattern matching is recursive, so if an exception occurs, instead of having one exception with a long stacktrace, we will get a linked list of MatchException with each of them having a long stacktraces. > > --Dan Rémi > >> On 3/30/2022 2:12 PM, Dan Heidinga wrote: >> >> The rules regarding NPE, ICCE and MatchException look reasonable to me. >> >> >> As a separate but not-separate exception problem, we have to deal with at >> least >> two additional sources of exceptions: >> >> - A dtor / record acessor may throw an arbitrary exception in the course of >> evaluating whether a case matches. >> >> - User code in the switch may throw an arbitrary exception. >> >> For the latter, this has always been handled by having the switch terminate >> abruptly with the same exception, and we should continue to do this. >> >> For the former, we surely do not want to swallow this exception (such an >> exception indicates a bug). The choices here are to treat this the same way >> we >> do with user code, throwing it out of the switch, or to wrap with >> MatchException. >> >> I prefer the latter -- wrapping with MatchException -- because the exception >> is >> thrown from synthetic code between the user code and the ultimate thrower, >> which means the pattern matching feature is mediating access to the thrower. >> I >> think we should handle this as "if a pattern invoked from pattern matching >> completes abruptly by throwing X, pattern matching completes abruptly with >> MatchException", because the specific X is not a detail we want the user to >> bind to. (We don't want them to bind to anything, but if they do, we want >> them >> to bind to the logical action, not the implementation details.) >> >> My intuition (and maybe I have the wrong mental model?) is that the >> pattern matching calling a user written dtor / record accessor is akin >> to calling a method. We don't wrap the exceptions thrown by methods >> apart from some very narrow cases (ie: reflection), and I thought part >> of reflection's behaviour was related to needing to ensure exceptions >> (particularly checked ones) were converted to something explicitly >> handled by the caller. >> >> If the dtor / record accessor can declare they throw checked >> exceptions, then I can kind of see the rationale for wrapping them. >> Otherwise, it seems clearer to me to let them be thrown without >> wrapping. >> >> I don't think we expect users to explicitly handle MatchException when >> using pattern matching so what does wrapping gain us here? >> >> --Dan >>
Re: Patterns and GADTs (and type checking and inference and overload selection)
- Original Message - > From: "Brian Goetz" > To: "amber-spec-experts" > Sent: Wednesday, March 30, 2022 9:33:21 PM > Subject: Patterns and GADTs (and type checking and inference and overload > selection) > GADTs -- sealed families whose permitted subtypes specialize the type > variables of the base class -- pose some interesting challenges for > pattern matching. > > (Remi: this is a big, complex area. Off-the-cuff "this is wrong" or > "you should X instead" replies are not helpful. If in doubt, ask > questions. One comprehensive reply is more useful than many small > replies. Probably best to think about the whole thing for some time > before responding.) No disagreement here, it's a nice summary of where we are and what are the challenges ahead. Rémi
Re: [External] : Re: Remainder in pattern matching
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Wednesday, March 30, 2022 8:35:17 PM > Subject: Re: [External] : Re: Remainder in pattern matching >> It seems that what you are saying is that you think an Exception is better >> than >> an Error. > Not exactly; what I'm saying is that the attempt to separate stray nulls from > separate compilation issues here seems like a heroic effort for low value, and > I'd rather have one channel for "exhaustiveness failure" and let > implementations decide how heroic they want to get in sorting out the possible > causes. NPE is a developer issue, separate compilation failure/ICCE is a deployment issue, there is no point to have one channel. Rémi
Re: [External] : Re: Remainder in pattern matching
- Original Message - > From: "Brian Goetz" > To: "Dan Heidinga" > Cc: "amber-spec-experts" > Sent: Wednesday, March 30, 2022 8:26:53 PM > Subject: Re: [External] : Re: Remainder in pattern matching [...] > > One thing wrapping gains is that it gives us a place to centralize > "something failed in pattern matching", which includes exhaustiveness > failures as well as failures of invariants which PM assumes (e.g., dtors > don't throw.) but such centralization is a bad practice, that the reason why catch(Exception) is considered as a bad practice. BTW, i hope that with loom people will use virtual threads (they are cheap) to manage scenarios where you want to discard a computation if something fails, like in Erlang. Rémi
Re: [External] : Re: Remainder in pattern matching
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Wednesday, March 30, 2022 6:32:15 PM > Subject: Re: [External] : Re: Remainder in pattern matching >> For when the static world and the dynamic world disagree, i think your >> analysis >> has miss an important question, switching on an enum throw an ICCE very late >> when we discover an unknown value, but in the case of a sealed type, > Actually, I thought about that quite a bit before proposing this. And my > conclusion is: using ICCE was mostly a (well intentioned) mistake here, and > "doubling down" on that path is more trouble than it is worth. So we are > minimally consistent with the ICCE choice in the cases that were compilable in > 12, but for anything else, we follow the general rule. > The thought experiment that I did was: what if we had not done switch > expressions in 12. Then the only precedent we have to deal with is the null > case, which has a pretty obvious answer. So what would we do? Would we > introduce 10s of catch-all cases solely for the purpose of diagnosing the > source of remainder, or would we introduce a throwing default that throws > MatchException on everything but null? I concluded we would do the latter, so > what is proposed here is basically that, but carving out the 12-compatibility > case. We are discussing about what to do if a sealed types has more permitted subtypes at runtime than the one seen when the code was compiled. It's a separate compilation issue, hence the ICCE. It seems that what you are saying is that you think an Exception is better than an Error. If we follow that path, it means that it may make sense to recover from a MatchException but i fail to see how, we can not ask a developer of a code to change it while that code is executed, separate compilation errors are not recoverable. >> Remainders are dangling else in a cascade of if ... else, so yes, we have to >> care of them. > Yes, but we can care for all of them in one swoop with a synthetic default. >> So yes, it may a lot of bytecodes if we choose to add all branches but the >> benefit is not questionable, it's far better than the alternative which is >> GoodLuckFigureByYourselfException. > Yes, when you get a dynamic error here in a complex switch, the range of what > could have gone wrong is large. (The same will be true outside of switches > when > we have more kinds of patterns (list patterns, map patterns, etc) and more > ways > to compose patterns into bigger patterns; if we have a big complex pattern > that > matches the JSON document with the keys we want, if it doesn't match because > (say) some integer nested nine levels deep overflowed 32 bits, this is also > going to be hard to diagnose.) But you are proposing a new and significant > language requirement -- that the language should mandate an arbitrarily > complex > explanation of why something didn't match. I won't dispute that this has > benefit -- but I am not convinced this is necessarily the place for this, or > whether the cost is justified by the benefit. The explanation is not complex, there is a sealed type has more subtypes now than at a time the code was compiled. Rémi
Re: Remainder in pattern matching
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Wednesday, March 30, 2022 4:40:28 PM
> Subject: Remainder in pattern matching
> We should have wrapped this up a while ago, so I apologize for the late
> notice,
> but we really have to wrap up exceptions thrown from pattern contexts (today,
> switch) when an exhaustive context encounters a remainder. I think there's
> really one one sane choice, and the only thing to discuss is the spelling, but
> let's go through it.
> In the beginning, nulls were special in switch. The first thing is to evaluate
> the switch operand; if it is null, switch threw NPE. (I don't think this was
> motivated by any overt null hostility, at least not at first; it came from
> unboxing, where we said "if its a box, unbox it", and the unboxing throws NPE,
> and the same treatment was later added to enums (though that came out in the
> same version) and strings.)
> We have since refined switch so that some switches accept null. But for those
> that don't, I see no other move besides "if the operand is null and there is
> no
> null handling case, throw NPE." Null will always be a special remainder value
> (when it appears in the remainder.)
> In Java 12, when we did switch expressions, we had to confront the issue of
> novel enum constants. We considered a number of alternatives, and came up with
> throwing ICCE. This was a reasonable choice, though as it turns out is not one
> that scales as well as we had hoped it would at the time. The choice here is
> based on "the view of classfiles at compile time and run time has shifted in
> an
> incompatible way." ICCE is, as Kevin pointed out, a reliable signal that your
> classpath is borked.
> We now have two precedents from which to extrapolate, but as it turns out,
> neither is really very good for the general remainder case.
> Recall that we have a definition of _exhaustiveness_, which is, at some level,
> deliberately not exhaustive. We know that there are edge cases for which it is
> counterproductive to insist that the user explicitly cover, often for two
> reasons: one is that its annoying to the user (writing cases for things they
> believe should never happen), and the other that it undermines type checking
> (the most common way to do this is a default clause, which can sweep other
> errors under the rug.)
> If we have an exhaustive set of patterns on a type, the set of possible values
> for that type that are not covered by some pattern in the set is called the
> _remainder_. Computing the remainder exactly is hard, but computing an upper
> bound on the remainder is pretty easy. I'll say "x may be in the remainder of
> P* on T" to indicate that we're defining the upper bound.
> - If P* contains a deconstruction pattern P(Q*), null may be in the remainder
> of
> P*.
> - If T is sealed, instances of a novel subtype of T may be in the remainder of
> P*.
> - If T is an enum, novel enum constants of T may be in the remainder of P*.
> - If R(X x, Y y) is a record, and x is in the remainder of Q* on X, then `R(x,
> any)` may be in the remainder of { R(q) : q in Q*} on R.
> Examples:
> sealed interface X permits X1, X2 { }
> record X1(String s) implements X { }
> record X2(String s) implements X { }
> record R(X x1, X x2) { }
> switch (r) {
> case R(X1(String s), any):
> case R(X2(String s), X1(String s)):
> case R(X2(String s), X2(String s)):
> }
> This switch is exhaustive. Let N be a novel subtype of X. So the remainder
> includes:
> null, R(N, _), R(_, N), R(null, _), R(X2, null)
> It might be tempting to argue (in fact, someone has) that we should try to
> pick
> a "root cause" (null or novel) and throw that. But I think this is both
> excessive and unworkable.
[...] see below
> So what I propose is the following simple answer instead:
> - If the switch target is null and no case handles null, throw NPE. (We know
> statically whether any case handles null, so this is easy and similar to what
> we do today.)
> - If the switch is an exhaustive enum switch, and no case handles the target,
> throw ICCE. (Again, we know statically whether the switch is over an enum
> type.)
> - In any other case of an exhaustive switch for which no case handles the
> target, we throw a new exception type, java.lang.MatchException, with an error
> message indicating remainder.
I agree for the first rule, if null is not handled, let throw a NPE.
For when the static world and the dynamic world disagree, i think your analysis
has miss an important question, switching on an enum throw an ICCE very late
when we discover an unknown value, but in the case of a sealed type,
we can decide to reject the switch much sooner.
There is a spectrum of choices here, where to throw an ICCE, it can be
- when the class is verified (when the method is verified for OpenJ9)
- the first time the switch is reached (with an indy that validates once that
the different sealed types "permits" set have not changed).
- when we have exhausted
Re: [External] : Re: Declared patterns -- translation and reflection
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Wednesday, March 30, 2022 2:47:45 AM
> Subject: Re: [External] : Re: Declared patterns -- translation and reflection
>>> The mangling has to be stable across compilations with respect to any
>>> source-
>>> and binary-compatible changes to the pattern declaration. One mangling that
>>> works quite well is to use the "symbolic-freedom encoding" of the erasure of
>>> the pattern descriptor. Because the erasure of the descriptor is exactly as
>>> stable as any other method signature derived from source declarations, it
>>> will
>>> have the desired binary compatibility properties, overriding will work as
>>> expected, etc.
>> I think we need a least to use a special name like the same
>> way
>> we have .
> Yes. Instance/static patterns will have names, so for them, we'll use the name
> as declared in the source. Dtors have no names, just like ctors, so we have to
> invent something to stand in for that. or similar is fine.
Pattern methods (static or not) does not have a real name, so '<' and '>' are
here to signal that the name is in the Pattern attribute.
We do not want people to unmangle the name of pattern methods that why the name
is in the attribute, using '<' and '>' signal that idea.
As a war story, most of the IDEs try to decode nested class name trying to
making sense of the names in between the '$' and tend to throw exceptions when
they encounters classes a different patterns that the one generated by javac.
But we may not care given that not a lot of people read the bytecode directly.
I think John can help us here :)
>> I agree that we also need to encode the method type descriptor (the carrier
>> type) into the name, so the name of the method in the classfile should be
>> or (or perhaps ofr
>> the pattern methods).
> The key constraint is that the mangled name be stable with respect to
> compatible
> changes in the declaration. The rest is just "classfile syntax."
yes.
>>> Return value
>>> In an earlier design, we used a pattern object (which was a bundle of method
>>> handles) as the return value of the pattern. This enabled clients to invoke
>>> these via condy and bind method handles into the constant pool for
>>> deconstruction and static patterns.
>>> Either way, we make use of some sort of carrier object to carry the bindings
>>> from the pattern to the client; either we return the carrier from the
>>> pattern
>>> method, or there is a method on the pattern object that we invoke to get a
>>> carrier. We have a few preferences about the carrier; we'd like to be able
>>> to
>>> late-bind to the actual implementation (i.e., we don't want to freeze the
>>> name
>>> of a carrier class in the method descriptor), and at least for records, we'd
>>> like to let the record instance itself be the carrier (since it is immutable
>>> and we can just invoke the accessors to get the bindings.)
>> So the return type is either Object (too hide the type of the carrier) or a
>> lambda that returns an Object (PatternObject or PatternCarrier acting like a
>> glorified lambda).
> If the pattern method actually runs the match, then I think Object is right.
> If
> the method returns a constant bundle of method handles, then it can return
> something like PatternHandle or a matcher lambda. But I am no longer seeing
> the
> benefit in this extra layer of indirection, given how the other translation
> work has played out.
I agree, Object is enough.
>>> Pattern {
>>> u2 attr_name;
>>> u4 attr_length;
>>> u2 patternFlags; // bitmask
>>> u2 patternName; // index of UTF8 constant
>>> u2 patternDescr; // index of MethodType (or alternately UTF8) constant
>>> u2 attributes_count;
>>> attribute_info attributes[attributes_count];
>>> }
>>> This says that "this method is a pattern", reifies the name of the pattern
>>> (patternName), reifies the pattern descriptor (patternDescr) which encodes
>>> the
>>> types of the bindings as a method descriptor or MethodType, and has
>>> attributes
>>> which can carry annotations, parameter metadata, and signature metadata for
>>> the
>>> bindings. The existing attributes (e.g. Signature, ParameterNames, RVAA)
>>> can be
>>> reused as is, with the interpretation that this is the signature (or names,
>>> or
>>> annos) of the *bindings*, not the input parameters. Flags can carry things
>>> like
>>> "deconstructor pattern" or "partial pattern" as needed.
>> From the classfile POV, a constructor is a method with a funny name in
>> between
>> brackets, i think deconstructor and pattern methods should work the same way.
> Be careful of extrapolating from one data point. Dtor are only one form of
> declared patterns; we also have to accomodate static and instance patterns.
see above, it's about signaling that the name is mangled.
Rémi
Are binding types covariant ? Was: Declared patterns -- translation and reflection
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Tuesday, March 29, 2022 11:01:18 PM
> Subject: Declared patterns -- translation and reflection
> Time to take a peek ahead at _declared patterns_. Declared patterns come in
> three varieties -- deconstruction patterns, static patterns, and instance
> patterns (corresponding to constructors, static methods, and instance
> methods.)
> I'm going to start with deconstruction patterns, but the basic game is the
> same
> for all three.
Once we have pattern methods, we can have an interface that defines a pattern
method and a class that implement it,
something like
interface I {
foo() (Object, int); // fake syntax: the first parenthesis are the parameters,
the seconds are the binding types
}
class A implements I {
foo() (String, int) { ... }
}
Do we agree that a binding type can be covariant ? (before saying no, think
about generics that's the reason we have return type covariance in Java).
In that case, are we are in trouble with the translation strategy ?
Rémi
Re: [External] : Re: Declared patterns -- translation and reflection
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Wednesday, March 30, 2022 2:42:38 AM > Subject: Re: [External] : Re: Declared patterns -- translation and reflection >> 1/ conceptually there is a mismatch, the syntax introduce names for the >> bindings, but they have no names at that point, bindings only have names >> AFTER >> the pattern matching succeed. > I think you have missed the point here. The names serve the implementation of > the pattern, not the interface -- just as parameter names to methods do. As > you > see in the example, these are effectively blank final locals in the body of > the > pattern, which must be assigned to. (I'd have pointed this out if this were > actually a message on declaring deconstructors, but since the message is on > translation and reflection I didn't want to digress.) >> 2/ sending the value of the binding by name is alien to Java. In Java, >> sending >> values is by the position of the value. > It's not by name. I don't know where you got this idea. >> 3/ the conceptual mismatch also exists at runtime, you need to permute the >> value >> of bindings before creating the carrier because a carrier takes the value of >> the binding by position while the code will takes the value of the bindings >> by >> name (you need the equivalent of MethodHandles.permuteArguments() otherwise >> you >> will see the re-organisation of the code if they are side effects). > It's not by name. I don't know where you got this idea. I think i understand the underlying semantics of the syntax, i'm not 100% confident. You know that it's not about the syntax per se but what the syntax try to communicate to the users. The problem with the proposed syntax is that you invent a new kind of variable, until now, we had local variables and fields (and array cells but those have no name). Your binding is a new kind of variable. It means that - as a user, i need to learn new rules: can i use the value of a binding to compute the value of another one, can i declare a binding final ? can i capture a binding in a lambda/anonymous class ? etc - the JLS needs to artificially grows to cover all these rules - the JVMS needs to be updated, more questions: do we need an attribute LocalBindingTable like we have a LocalVariableTable - tools needs to be updated: how debuggers reflects bindings, is it another tables in the UI ? Do/Can the debugger allows to chain the value of the binding while a user steps into the code, etc All this pain, because you want to name bindings. Rémi
Re: Declared patterns -- translation and reflection
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Tuesday, March 29, 2022 11:01:18 PM
> Subject: Declared patterns -- translation and reflection
> Time to take a peek ahead at _declared patterns_. Declared patterns come in
> three varieties -- deconstruction patterns, static patterns, and instance
> patterns (corresponding to constructors, static methods, and instance
> methods.)
> I'm going to start with deconstruction patterns, but the basic game is the
> same
> for all three.
> Ignoring the trivial details, a deconstruction pattern looks like a
> "constructor
> in reverse":
> ```{.java}
> class Point {
> int x, y;
> Point(int x, int y) {
> this.x = x;
> this.y = y;
> }
[]
> }
> ```
> Deconstruction patterns share the weird behaviors that constructors have in
> that
> they are instance members, but are not inherited, and that rather having
> names,
> they are accessed via the class name.
> Deconstruction patterns differ from static/instance patterns in that they are
> by
> definition total; they cannot fail to match. (This is a somewhat arbitrary
> simplification in the object model, but a reasonable one.) They also cannot
> have any input parameters, other than the receiver.
> Patterns differ from their ctor/method counterparts in that they have what
> appear to be _two_ argument lists; a parameter list (like ctors and methods),
> and a _binding_ list. The parameter list is often empty (with the receiver as
> the match target). The binding list can be thought of as a "conditional
> multiple return". That they may return multiple values (and, for partial
> patterns, can return no values at all when they don't match) presents a
> challenge for translation to classfiles, and for the reflection model.
> Translation to methods
> Patterns contain imperative code, so surely we want to translate them to
> methods
> in some way. The pattern input parameters map cleanly to method parameters.
> The pattern bindings need to tunneled, somehow, through the method return (or
> some other mechanism). For our deconstructor, we might translate as:
> PatternCarrier ()
> (where the method applies the pattern, and PatternCarrier wraps and provides
> access to the bindings) or
> PatternObject ()
> (where PatternObject provides indirection to behavior to invoke the pattern,
> which in turn returns the carrier.)
> With either of these approaches, though, the pattern name is a problem,
> because
> patterns can be overloaded on their _bindings_, but both of these return types
> are insensitive to bindings.
> It is useful to characterize the "shape" of a pattern with a MethodType, where
> the parameters of the MethodType are the binding types. (The return type is
> less constrained, but it is sometimes useful to use the return type of the
> MethodType for the required type of the pattern.) Call this the "descriptor"
> of
> the pattern.
> If we do this, we can use some name mangling to encode the descriptor in the
> method name:
> PatternCarrier name$mangle()
> The mangling has to be stable across compilations with respect to any source-
> and binary-compatible changes to the pattern declaration. One mangling that
> works quite well is to use the "symbolic-freedom encoding" of the erasure of
> the pattern descriptor. Because the erasure of the descriptor is exactly as
> stable as any other method signature derived from source declarations, it will
> have the desired binary compatibility properties, overriding will work as
> expected, etc.
I think we need a least to use a special name like the same way
we have .
I agree that we also need to encode the method type descriptor (the carrier
type) into the name, so the name of the method in the classfile should be
or (or perhaps ofr
the pattern methods).
> Return value
> In an earlier design, we used a pattern object (which was a bundle of method
> handles) as the return value of the pattern. This enabled clients to invoke
> these via condy and bind method handles into the constant pool for
> deconstruction and static patterns.
> Either way, we make use of some sort of carrier object to carry the bindings
> from the pattern to the client; either we return the carrier from the pattern
> method, or there is a method on the pattern object that we invoke to get a
> carrier. We have a few preferences about the carrier; we'd like to be able to
> late-bind to the actual implementation (i.e., we don't want to freeze the name
> of a carrier class in the method descriptor), and at least for records, we'd
> like to let the record instance itself be the carrier (since it is immutable
> and we can just invoke the accessors to get the bindings.)
So the return type is either Object (too hide the type of the carrier) or a
lambda that returns an Object (PatternObject or PatternCarrier acting like a
glorified lambda).
> Carriers
> As part of the work on template strings, Jim has put back some code that was
> originally written f
Re: Declared patterns -- translation and reflection
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Tuesday, March 29, 2022 11:01:18 PM
> Subject: Declared patterns -- translation and reflection
> Time to take a peek ahead at _declared patterns_. Declared patterns come in
> three varieties -- deconstruction patterns, static patterns, and instance
> patterns (corresponding to constructors, static methods, and instance
> methods.)
> I'm going to start with deconstruction patterns, but the basic game is the
> same
> for all three.
I mostly agree with everything said apart from the syntax of a deconstructor
(see my next message about how small things can be improved).
I have several problems with the proposed syntax for a deconstructor.
I can see the appeal of having a code very similar to a constructor but it's a
trap, a constructor and a deconstructor do not have the same semantics, a
constructor initialize fields (which have a name) while a deconstructor (or a
pattern method) initialize bindings which does not have a name at that point
yet.
1/ conceptually there is a mismatch, the syntax introduce names for the
bindings, but they have no names at that point, bindings only have names AFTER
the pattern matching succeed.
2/ sending the value of the binding by name is alien to Java. In Java, sending
values is by the position of the value.
3/ the conceptual mismatch also exists at runtime, you need to permute the
value of bindings before creating the carrier because a carrier takes the value
of the binding by position while the code will takes the value of the bindings
by name (you need the equivalent of MethodHandles.permuteArguments() otherwise
you will see the re-organisation of the code if they are side effects).
Let's try to come with a syntax,
as i said, bindings have no names at that point so the deconstructor should
declare the bindings (int, int) and not (int x, int y),
so a syntax like
_deconstructor_ (int, int) {
_send_bindings_(this.x, this.y);
}
Here the syntax shows that the value of the bindings are assigned following the
position of the expression like usual in Java.
We can discuss if _send_bindings_ should be "return" or another keyword and if
the binding types should be declared before or after _deconstructor_.
By example, if you wan to maintain a kind of symmetry with the constructor, we
can reuse the name of the class instead of _deconstructor_ and move the binding
types in front of the name of the class to show that the bindings move from the
class to the pattern matching in the same direction like a return type of a
method.
Something like this:
(int, int) Point {
_send_bindings_(this.x, this.y);
}
To summarize, the proposed syntax does the convey the underlying semantics of
the bindings initialization and make things more confusing than it should.
> Ignoring the trivial details, a deconstruction pattern looks like a
> "constructor
> in reverse":
> ```{.java}
> class Point {
> int x, y;
> Point(int x, int y) {
> this.x = x;
> this.y = y;
> }
> deconstructor(int x, int y) {
> x = this.x;
> y = this.y;
> }
> }
> ```
Rémi
Re: [External] : Re: Pattern assignment
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Monday, March 28, 2022 5:41:10 PM
> Subject: Re: [External] : Re: Pattern assignment
>> There are another different between assignment and _let_, a _let_ creates new
>> fresh local variables (binding) while assignment is able to reuse an existing
>> local variable.
> Correct, the more precise analogy is not to _assignment_, but to _local
> variable
> declaration with initialization_ (whose semantics are derived from
> assignment.)
>> In Java, the if statement is used a lot (too much IMO but i don't think we
>> should fight to change that) so it may make sense to be able to reuse an
>> existing local variables.
> Yes, this has come up before. I agree that there are cases where we might want
> this (there's one distinguished case where we almost cannot avoid this), but
> in
> general, I am pretty reluctant to go there -- I think this is incremental
> complexity (and encouragement of more mutability) with not enough commensurate
> benefit.
My general point is that it's less complex to consider that the semantics
should be an _assignment pattern_ instead a _local variable declarations with
initialization semantics_ if most (not the let ... in) of semantics variants
you are proposing can be express as combinations of assignments + if/else.
And the "encouragement of more mutability" is a false dichotomy argument
because you are conflating the mutation of objects with the mutation of local
variables, mutation of objects are visible from the outside (from the user POV)
which make those objects harder to debug, mutation of local variables are not
visible from the outside, so those are very different beasts but you already
know that.
>>> ## Possible extensions
>>> There are a number of ways we can extend `let` statements to make it more
>>> useful; these could be added at the same time, or at a later time.
>>> What about partial patterns?
>>> There are times when it may be more convenient to use a `let` even when we
>>> know
>>> the pattern is partial. In most cases, we'll still want to complete
>>> abruptly if
>>> the
>>> pattern doesn't match, but we may want to control what happens. For example:
>>> ```
>>> let Optional.of(var contents) = optName
>>> else throw new IllegalArgumentException("name is empty");
>>> ```
>>> Having an `else` clause allows us to use a partial pattern, which receives
>>> control if the pattern does not match. The `else` clause could choose to
>>> throw,
>>> but could also choose to `break` or `return` to an enclosing context, or
>>> even
>>> recover by assigning the bindings.
>> I don't like that because in that case "let pattern else ..." is equivalent
>> of
>> "if instanceof pattern else ... " with the former being expression oriented
>> and
>> the later statement oriented.
>> As i said earlier, i don't think we should fight the fact that Java is
>> statement
>> oriented by adding expression oriented variations of existing constructs.
> We haven't talked about let expressions yet; this is still a statement.
Okay, i did not expect that. For me let was an expression because it's usually
the "raison d'être" of let, being an assignment expression.
Reusing 'let' here is really confusing.
> It's a fair point to say that the above example could be rewritten as an
> if-else, and when the else throws unconditionally, we still get the same
> scoping. Or that it can be rewritten as
> if (!(pattern match))
> throw blah
> On the other hand, people don't particularly like having to invert the match
> like this just to get the scoping they want.
If you really want
> In any case, the real value of the else block is where you want to continue
> (and
> merge the control flow) with default values of the bindings set in the else
> clause (next section). Dropping "else" makes this extremely messy. And once
> you
> have else, the rest comes for the ride.
But your proposal do the opposite, you are not dropping the "else" but you are
dropping the "then" which is also makes thing messy if you want to assign +
call a method.
One advantage of the "if" is that you can easily add more instructions inside
the then branch or the else branch.
With a let ... else, users will have to jungle between if instanceof/ else and
let ... else if they add/remove instruction in the then branch.
>>> What about recovery?
>>> If we're sup
Re: Pattern assignment
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Friday, March 25, 2022 4:38:52 PM
> Subject: Pattern assignment
> We still have a lot of work to do on the current round of pattern matching
> (record patterns), but let's take a quick peek down the road. Pattern
> assignment is a sensible next building block, not only because it is directly
> useful, but also because it will be required for _declaring_ deconstruction
> patterns in classes (that's how one pattern delegates to another.) What
> follows
> is a rambling sketch of all the things we _could_ do with pattern assignment,
> though we need not do all of them initially, or even ever.
And obviously (but let state the obvious hidden behind "directly useful") we
have introduced records as named tuples and _declaring_deconstruction_ is a way
deconstruct that tuple.
Construction:
var x = ...
var y = ...
var point = new Point(x, y);
De-construction
var point = ...
__let__ Point(var x, var y) = point;
> # Pattern assignment
> So far, we've got two contexts in the language that can accommodate patterns
> --
> `instanceof` and `switch`. Both of these are conditional contexts, designed
> for
> dealing with partial patterns -- test whether a pattern matches, and if so,
> conditionally extract some state and act on it.
> There are cases, though, when we know a pattern will always match, in which
> case
> we'd like to spare ourselves the ceremony of asking. If we have a 3d `Point`,
> asking if it is a `Point` is redundant and distracting:
> ```
> Point p = ...
> if (p instanceof Point(var x, var y, var z)) {
> // use x, y, z
> }
> ```
> In this situation, we're asking a question to which we know the answer, and
> we're distorting the structure of our code to do it. Further, we're depriving
> ourselves of the type checking the compiler would willingly do to validate
> that
> the pattern is total. Much better to have a way to _assert_ that the pattern
> matches.
> ## Let-bind statements
> In such a case, where we want to assert that the pattern matches, and forcibly
> bind it, we'd rather say so directly. We've experimented with a few ways to
> express this, and the best approach seems to be some sort of `let` statement:
> ```
> let Point(var x, var y, var z) p = ...;
> // can use x, y, z, p
> ```
> Other ways to surface this might be to call it `bind`:
> ```
> bind Point(var x, var y, var z) p = ...;
> ```
> or even use no keyword, and treat it as a generalization of assignment:
> ```
> Point(var x, var y, var z) p = ...;
> ```
> (Usual disclaimer: we discuss substance before syntax.)
> A `let` statement takes a pattern and an expression, and we statically verify
> that the pattern is exhaustive on the type of the expression; if it is not,
> this
> is a
> type error at compile time. Any bindings that appear in the pattern are
> definitely assigned and in scope in the remainder of the block that encloses
> the
> `let` statement.
> Let statements are also useful in _declaring_ patterns; just as a subclass
> constructor will delegate part of its job to a superclass constructor, a
> subclass deconstruction pattern will likely want to delegate part of its job
> to
> a superclass deconstruction pattern. Let statements are a natural way to
> invoke
> total patterns from other total patterns.
yes !
> Remainder
> Let statements require that the pattern be exhaustive on the type of the
> expression.
> For total patterns like type patterns, this means that every value is matched,
> including `null`:
> ```
> let Object o = x;
> ```
> Whatever the value of `x`, `o` will be assigned to `x` (even if `x` is null)
> because `Object o` is total on `Object`. Similarly, some patterns are clearly
> not total on some types:
> ```
> Object o = ...
> let String s = o; // compile error
> ```
> Here, `String s` is not total on `Object`, so the `let` statement is not
> valid.
> But as previously discussed, there is a middle ground -- patterns that are
> _total with remainder_ -- which are "total enough" to be allowed to be
> considered
> exhaustive, but which in fact do not match on certain "weird" values. An
> example is the record pattern `Box(var x)`; it matches all box instances, even
> those containing null, but does not match a `null` value itself (because to
> deconstruct a `Box`, we effectively have to invoke an instance member on the
> box, and we cannot invoke instance members on null receivers.) Similarly, the
> pattern `Box(Bag(String s))` is total on `Box>`, with remainder
> `null` and `Box(null)`.
> Because `let` statements guarantee that its bindings are definitely assigned
> after the `let` statement completes normally, the natural thing to do when
> presented with a remainder value is to complete abruptly by reason of
> exception.
> (This is what `switch` does as well.) So the following statement:
> ```
> Box> bbs = ...
> let Box(Bag(String s)) = bbs;
> ```
> would throw when encountering `null` or `Box(null)` (bu
Re: Pattern coverage
Thanks for sharing, in the text, they are several mentions of the default pattern but the default pattern is not defined. Rémi > From: "Brian Goetz" > To: "amber-spec-experts" > Sent: Thursday, March 24, 2022 6:39:21 PM > Subject: Pattern coverage > I've put a document at > [ http://cr.openjdk.java.net/~briangoetz/eg-attachments/Coverage.pdf | > http://cr.openjdk.java.net/~briangoetz/eg-attachments/Coverage.pdf ] > which outlines a formal model for pattern coverage, including record patterns > and the effects of sealing. This refines the work we did earlier. The document > may be a bit rough so please let me know if you spot any errors. The approach > here should be more amenable to specification than the previous approach.
Re: Record pattern, the runtime side
- Original Message - > From: "Brian Goetz" > To: "Remi Forax" , "amber-spec-experts" > > Sent: Wednesday, March 16, 2022 5:41:49 PM > Subject: Re: Record pattern, the runtime side >> It works in 3 steps: >> Step 1, at compile time, the compiler takes all the patterns and creates a >> tree >> of pattern from the list of patterns, >> pattern that starts with the same prefix are merged together. > > We can "normalize" a complex pattern into a sequence of simpler > conditionals. For example, matching the record pattern > > case Circle(Point(var x, var y), var r) > > can be unrolled (and type inference applied) as > > x matches Circle(Point(var x, var y), var r) > === x matches Circle(Point p, int r) && p matches Point(int x, int y) > > Deconstruction patterns are known to have only an `instanceof` > precondition; the deconstructor body won't ever fail (unlike more > general static or instance patterns like Optional::of.) If you define "matches" in term of instanceof this transformation does not work in the context of an assignment, because you want Point(var x, var y) = null to throw a NPE. But it's a very valid transformation if the pattern is not total and "matches" means instanceof in the context of a switch or instanceof and requireNonNull + cast in the context of an assignment. Also from the runtime POV, a deconstructor and a pattern methods (static or instance) are identical, if we follow the idea of John to use null for not match. Obviously, it does not preclude us to differentiate between the two at the language level. > So we can further rewrite this as: > > x matches Circle(Point(var x, var y), var r) > === x matches Circle(Point p, int r) && p matches Point(int x, int y) > === x instanceof Circle c && c.deconstruct(Point p, int r) && p > instanceof Point && p.deconstruct(int x, int y) > > (where the "deconstruct" term invokes the deconstructor and binds the > relevant values.) With this rewrite, we are moving from higher world of patterns to the lower world of matcher (it's how i've called it), were the exact semantics of a pattern is decomposed into different method handles. It's at that stage that depending on if the pattern is total or not, an instanceof/type check is used or not. > > If we are disciplined about the order in which we unroll (e.g., always > depth-first and always left-to-right), with a modest amount of > normalization, your "same pattern prefix" turns into the simpler "common > prefix of normalized operations". Yes ! > Record deconstructors can be further normalized, because the can be replaced > with calling the accessors: > > x matches Circle(Point(var x, var y), var r) > === x matches Circle(Point p, int r) && p matches Point(int x, int y) > === x instanceof Circle c && (Point p = c.center()) && (int r = > c.radius()) && p instanceof Point > && (int x = p.x()) && (int y = p.y()) I've unified the access to a record and the access to a carrier for that exact reason, so the translation for a deconstructor or a record is identical. A carrier being an anonymous record or it's dual, a record is a named carrier. > > Of course, this is all very implementation-centric; factoring matching > this way is somewhat unusual, since the arity and order of side-effects > might be surprising to Java developers. (Yes, having side-effects in > pattern definitions is bad, but it may still happen.) So the spec will > have to do some fast dancing to allow this. yes, this will require pedagogy to explain why a pattern method is called once and not as many time as present in the source code. But i believe we should not make the performance worst because few users may write pattern methods/deconstructors that side effect otherwise we will have people that rewrite switch to a cascade of if ... instanceof due to performance concern (as we have seen several examples of this in the past with people re-writing enhanced for loops to basic loops before escape analysis was added to the VM, or more recently with lambdas because before Java 17 each lambda was using its own metaspace). > >> In the end, the tree of patterns is encoded in the bytecode as a tree of >> constant dynamic (each Pattern is created only from constant and patterns). > > With record patterns, we don't even need pattern descriptions, because > we can translate it all down to instanceof tests and invoking record > component accessors. Of course, that ends when we have deconstruction > patterns, which correspond to imperative code; th
Record pattern: matching an empty record
Hi all,
while writing the prototype of the runtime,
i found a case i think we never discuss, can we match an empty record ?
record Empty() { }
switch(object) {
case Empty() -> ... // no binding here
I think the answer is yes because i don't see why we should do a special case
for that, but i may be wrong.
Rémi
Record pattern, the runtime side
Following the discussions we had, i've implemented a prototype of what can be
the runtime part of the pattern matching that supports record pattern.
It works in 3 steps:
Step 1, at compile time, the compiler takes all the patterns and creates a tree
of pattern from the list of patterns,
pattern that starts with the same prefix are merged together.
In the end, the tree of patterns is encoded in the bytecode as a tree of
constant dynamic (each Pattern is created only from constant and patterns).
sealed interface Pattern {}
record NullPattern() implements Pattern {}
record ConstantPattern(Object constant) implements Pattern {}
record TypePattern(Class type) implements Pattern {}
record RecordPattern(Class recordClass, Pattern... patterns) implements
Pattern {}
record OrPattern(Pattern pattern1, Pattern pattern2) implements Pattern {}
record ResultPattern(int index, Pattern pattern) implements Pattern {}
The last two patterns are less obvious, the OrPattern allows to create the tree
by saying that a pattern should be executed before another one.
Because the patterns are organized as a tree and not a list, we need a way to
associate which branch of the tree correspond to which pattern from the use
POV, the ResultPattern encodes in the carrier the index (the first pattern is
0, the second one is 1, etc) inside the first component of the carrier.
I've chosen to not represent the bindings (and their corresponding the
component index in the carrier) and to use the fact that binding slot are
numbered in the order of the tree traversal.
This is maybe too brittle because the compiler and the runtime has to agree on
the order of the traversal. But this avoids to encode too many information in
the bytecode.
Step 2,
at runtime, the first call to invokedynamic with the pattern tree as arguments,
creates a tree of method handles that will match the pattern.
During that phase, the runtime environment can be checked to see if the pattern
tree is invalid with the runtime classes, in that case a linkage error is
thrown.
In the prototype the method is called toMatcher and takes a Lookup (to find the
accessors of the record of the RecordPattern), a receiverClass the type of the
variable switch upon,
the carrier type (a method type as a tuple of the type of the binding in the
tree traversal order), the index of the first binding (in case of a switch the
first component in the matching index so the binding slots starts at 1) and
method handle (the null matcher) to call if a null is found (the two possible
semantics are doNotMatch/return null or throw a NPE).
pattern.toMatcher(lookup, receiverClass, carrierType, firstBinding (0 or 1),
nullMatcher);
Step 3,
during the execution,
- if it's an instanceof a carrier which is null means no match otherwise the
carrier contains the value of the bindings,
- if it's a switch, a carrier which is null means "default" otherwise the
component 0 or the carrier contains the index of the pattern matched and the
binding values
- if it's an assignment, the carrier can not be null because the nullMatcher
throws a NPE earlier, so the carrier contains the binding values
An example of instanceof
https://github.com/forax/switch-carrier/blob/master/src/main/java/com/github/forax/carrier/java/lang/runtime/InstanceOfExamples.java#L15
An example of switch
https://github.com/forax/switch-carrier/blob/master/src/main/java/com/github/forax/carrier/java/lang/runtime/SwitchExamples.java#L17
An example of assignment
https://github.com/forax/switch-carrier/blob/master/src/main/java/com/github/forax/carrier/java/lang/runtime/AssignmentExamples.java#L14
regards,
Rémi
Re: [External] : Re: Proposal: java.lang.runtime.Carrier
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "John Rose" , "Jim Laskey" > , "amber-spec-experts" > > Sent: Wednesday, March 9, 2022 2:31:33 PM > Subject: Re: [External] : Re: Proposal: java.lang.runtime.Carrier >> What i was proposing is for switch to cram "not match" and the index of the >> matching case into one int because using -1 seems natural and it will work >> well >> with the tableswitch. > There’s two levels here, and I think part of the confusion with regard to > pattern translation is we’re talking at different levels. yes, > The first level is: I’ve written a deconstructor for class Foo, and I want to > match it with instanceof, case, whatever. I need a way to “invoke” the pattern > and let it conditionally “return” multiple values. Carrier is a good tool for > this job. > The second level is: I want to use indy to choose which branch of a switch to > take, *and* at the same time, carry all the values needed to that branch. > Carrier could be applied to this as well. > Somewhere in between, there’s the question of how we roll up the values in a > compound pattern (e.g., Circle(Point(var x, var y) p, var r) c). This could > involve flattening all the bindings (x, y, p, r, c) into a fresh carrier, or > it > could involve a “carrier of carriers”. There are many degrees of freedom in > the > translation story. > What Jim is proposing here is a runtime for bootstraps to make decomposable > tuples that can be pass across boundaries that agree on a contract. This could > be a simple return-to-caller, or it could rely on sharing the carrier in the > heap between entities that have a shared static typing proof. >> To come back to the carrier API, does it means that the carrier class is >> always >> a nullable value type or does it mean that we need to knob to select between >> a >> primitive type or a value type ? > Probably the carrier can always be a *primitive* class type, and the null can > be > handled separately by boxing from QCarrier$33 to LCarrier$33. All the Carrier > API does is provide a constructor which takes N values and returns a carrier; > at that point, you already know you want a non-null value. Consumers higher up > the food chain can opt into nullity. Also, i wonder if the external Carrier API should have a way to wrap an existing record class to see it as a Carrier, so the destructuring pattern will behave the same way with a record or with the result of a de-constructor. Rémi
Re: [External] : Re: Proposal: java.lang.runtime.Carrier
> From: "John Rose" > To: "Remi Forax" > Cc: "Brian Goetz" , "Jim Laskey" > , "amber-spec-experts" > > Sent: Tuesday, March 8, 2022 9:52:00 PM > Subject: Re: [External] : Re: Proposal: java.lang.runtime.Carrier > On 7 Mar 2022, at 16:07, [ mailto:fo...@univ-mlv.fr | fo...@univ-mlv.fr ] > wrote: >> Not necessarily, while java.lang.Object appears in the bytecode a JIT like c2 >> propagates the real type of the constants (here empty() is typed with the >> type >> of the carrier not with java.lang.Object at runtime) so introducing null may >> introduce some stupid nullchecks. > Null checks are usually cheap, because they are usually done implicitly in > parallel with the first access that requires a non-null value. This works for > true references, but not scalarized values. > We want scalarized values, of course. So what I might worry more about here is > forcing the JVM to buffer the value object on the heap because of possibility > of null. (B3 primitives/bare values are designed to make that less likely.) I > think even here null is OK, because all B2 types are both nullable and > scalarizable (if not null), so it is very likely the JVM is going to do extra > work to adjoin (the technical term is “cram in”) a null representation into > the > scalarized representation, using an extra register if necessary. > Bottom line: Nullability doesn’t concern me here, since I assume we are going > to > continue to work hard (because of B2 value types like ju.Optional) to support > both nullability and scalarization. What i was proposing is for switch to cram "not match" and the index of the matching case into one int because using -1 seems natural and it will work well with the tableswitch. Anyway, as i said earlier, i don't think it's a bid deal to use "null" to mean "do not match" because "do not match" need to be tested at two different stages, inside the matching pipeline here using null is easier and in the end to nourish the tableswitch/instanceof, in that case null need to be transformed to -1 / false. The other thing is that for the deconstructor call, here i believe we have to go to the B2 representation anyway (a nullable record) because it is part of the public API, not necessary in the language but from the binary compatibility POV, so at least users will be able to see it with a debugger. So it may make sense to always use 'null' to mean "do not match", in the matching pipeline and as result of a deconstructor call. To come back to the carrier API, does it means that the carrier class is always a nullable value type or does it mean that we need to knob to select between a primitive type or a value type ? Rémi
Re: [External] : Re: Proposal: java.lang.runtime.Carrier
From: "Brian Goetz" To: "Remi Forax" Cc: "Jim Laskey" , "amber-spec-experts" Sent: Monday, March 7, 2022 4:08:00 PM Subject: Re: [External] : Re: Proposal: java.lang.runtime.Carrier BQ_BEGIN BQ_BEGIN Adding more information, we want the carrier to be a primitive type (to be able to optimize it away), which means that we can not use null to represent "do_not_match", we have to have a flag inside the carrier for that. BQ_END The alternate approach is to use a .ref class for partial patterns (using null for "no match") and a B3 class for total patterns (since it needs no failure channel.) BQ_END yes, we still need an int to indicate which case of the switch match and using B3 in Valhalla already give us the performance we want. Anyway changing from one representation to the other is not a big deal. BQ_BEGIN I think its pretty important that the static name of the carrier class not appear in generated bytecode. BQ_END Yes, that why i'm using java.lang.Object in the bytecode visible code. BQ_BEGIN As a result, we will have to use a reference type (object or interface), which means we get the null channel "for free". BQ_END Not necessarily, while java.lang.Object appears in the bytecode a JIT like c2 propagates the real type of the constants (here empty() is typed with the type of the carrier not with java.lang.Object at runtime) so introducing null may introduce some stupid nullchecks. Rémi
Re: [External] : Re: Proposal: java.lang.runtime.Carrier
> From: "Remi Forax"
> To: "Brian Goetz"
> Cc: "Jim Laskey" , "amber-spec-experts"
>
> Sent: Saturday, March 5, 2022 11:54:14 PM
> Subject: Re: [External] : Re: Proposal: java.lang.runtime.Carrier
>> From: "Brian Goetz"
>> To: "Remi Forax"
>> Cc: "Jim Laskey" , "amber-spec-experts"
>>
>> Sent: Friday, March 4, 2022 3:11:44 AM
>> Subject: Re: [External] : Re: Proposal: java.lang.runtime.Carrier
>>>> Either way, we don't need to mutate or replace carriers.
>>> You want the same carrier for the whole pattern matching:
>> I think you're going about this backwards. You seem to have a clear picture
>> of
>> how pattern matching "should" be translated. If so, you should share! Maybe
>> your way is better. But you keep making statements like "we need" and "we
>> want"
>> without explaining why.
>>> - if you have a logical OR between patterns (not something in the current
>>> Java
>>> spec but Python, C# or clojure core.match have it so we may want to add an
>>> OR
>>> in the future)
>> OR combinators are a good point, but they can be done without a with
>> operation.
>>> - if different cases starts with the same prefix of patterns, so you don't
>>> have
>>> to re-execute the de-constructors/pattern methods of the prefix several
>>> times
>> Agree that optimizing away multiple invocations is good, but again, I don't
>> see
>> that as being coupled to the pseudo-mutability of the carrier.
>> Perhaps you should start with how you see translation working?
> Sure,
> the idea is that to execute the pattern matching at runtime, each step is
> decomposed into few higher order functions, things like testing, projecting a
> value (deconstructing), etc
> each higher order manipulate one kind of function that takes two values, the
> value we are actually matching and the carrier, and returns a carrier.
> Obviously, each simple function is a method handle, so there is no boxing in
> the
> middle and everything is inlined.
> Here is a possible decomposition
> - MH of(Object carrier, MH pattern)
> which is equivalent to o -> pattern.apply(o, carrier)
> - MH match(int index)
> which is equivalent to (o, carrier) -> with(index, carrier, 0), i.e. return a
> new carrier with the component 0 updated with index
> - MH do_not_match()
> which is equivalent to match(-1)
> - MH is_instance(Class type)
> which is equivalent to (o, carrier) -> type.isInstance(o)
> - MH is_null()
> which is equivalent to (o, carrier) -> o == null
> - MH throw_NPE(String message)
> which is equivalent to (o, carrier) -> throw new NPE(message)
> - MH project(MH project, MH pattern)
> which is equivalent to (o, carrier) -> pattern.apply(project.apply(o),
> carrier)
> - MH bind(int binding, MH pattern)
> which is equivalent to (o, carrier) -> pattern.apply(with(o, carrier, binding)
> - MH test(MH test, MH target, MH fallback)
> which is equivalent to (o, carrier) -> test.test(o, carrier)? target.apply(o,
> carrier): fallback.apply(o, carrier)
> - MH or(MH pattern1, MH pattern2)
> which is equivalent to
> (o, carrier) -> {
> var carrier2 = pattern1.apply(o, carrier);
> if (carrier2.accessor[0] == -1) {
> return carrier2;
> }
> return pattern2.apply(o, carrier2);
> }
> For the carrier, the convention is that the component 0 is an int, -1 means
> "not
> match", and any positive index means the indexth case match.
> In the detail, it's a little more complex because we sometimes need to pass
> the
> type of the first parameter to correctly type the returned MH and we also need
> an object CarrierMetadata that keep track of the type of the carrier
> components
> (and provides an empty carrier and the accessors/withers).
> Here is a small example
> record Point( int x, int y) {}
> record Rectangle(Point p1, Point p2) {}
> // Object o = ...
> //switch(o) {
> // case Rectangle(Point p1, Point p2) -> ...
> //}
> var lookup = MethodHandles. lookup ();
> var carrierMetadata = new CarrierMetadata( methodType (Object. class , int .
> class , Point. class , Point. class ));
> var empty = carrierMetadata.empty();
> var op = of (empty,
> test ( is_instance (Object. class , Rectangle. class ),
> cast (Object. class ,
> or (carrierMetadata,
> project ( record_accessor (lookup, Rectangle. class , 0 ),
> test ( is_null (Point. class ),
> do_not_match (Point. class , carrierMetadata),
> bind ( 1 , carrierMetadata))),
> project (
Re: [External] : Re: Proposal: java.lang.runtime.Carrier
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "Jim Laskey" , "amber-spec-experts"
>
> Sent: Friday, March 4, 2022 3:11:44 AM
> Subject: Re: [External] : Re: Proposal: java.lang.runtime.Carrier
>>> Either way, we don't need to mutate or replace carriers.
>> You want the same carrier for the whole pattern matching:
> I think you're going about this backwards. You seem to have a clear picture of
> how pattern matching "should" be translated. If so, you should share! Maybe
> your way is better. But you keep making statements like "we need" and "we
> want"
> without explaining why.
>> - if you have a logical OR between patterns (not something in the current
>> Java
>> spec but Python, C# or clojure core.match have it so we may want to add an OR
>> in the future)
> OR combinators are a good point, but they can be done without a with
> operation.
>> - if different cases starts with the same prefix of patterns, so you don't
>> have
>> to re-execute the de-constructors/pattern methods of the prefix several times
> Agree that optimizing away multiple invocations is good, but again, I don't
> see
> that as being coupled to the pseudo-mutability of the carrier.
> Perhaps you should start with how you see translation working?
Sure,
the idea is that to execute the pattern matching at runtime, each step is
decomposed into few higher order functions, things like testing, projecting a
value (deconstructing), etc
each higher order manipulate one kind of function that takes two values, the
value we are actually matching and the carrier, and returns a carrier.
Obviously, each simple function is a method handle, so there is no boxing in
the middle and everything is inlined.
Here is a possible decomposition
- MH of(Object carrier, MH pattern)
which is equivalent to o -> pattern.apply(o, carrier)
- MH match(int index)
which is equivalent to (o, carrier) -> with(index, carrier, 0), i.e. return a
new carrier with the component 0 updated with index
- MH do_not_match()
which is equivalent to match(-1)
- MH is_instance(Class type)
which is equivalent to (o, carrier) -> type.isInstance(o)
- MH is_null()
which is equivalent to (o, carrier) -> o == null
- MH throw_NPE(String message)
which is equivalent to (o, carrier) -> throw new NPE(message)
- MH project(MH project, MH pattern)
which is equivalent to (o, carrier) -> pattern.apply(project.apply(o), carrier)
- MH bind(int binding, MH pattern)
which is equivalent to (o, carrier) -> pattern.apply(with(o, carrier, binding)
- MH test(MH test, MH target, MH fallback)
which is equivalent to (o, carrier) -> test.test(o, carrier)? target.apply(o,
carrier): fallback.apply(o, carrier)
- MH or(MH pattern1, MH pattern2)
which is equivalent to
(o, carrier) -> {
var carrier2 = pattern1.apply(o, carrier);
if (carrier2.accessor[0] == -1) {
return carrier2;
}
return pattern2.apply(o, carrier2);
}
For the carrier, the convention is that the component 0 is an int, -1 means
"not match", and any positive index means the indexth case match.
In the detail, it's a little more complex because we sometimes need to pass the
type of the first parameter to correctly type the returned MH and we also need
an object CarrierMetadata that keep track of the type of the carrier components
(and provides an empty carrier and the accessors/withers).
Here is a small example
record Point( int x, int y) {}
record Rectangle(Point p1, Point p2) {}
// Object o = ...
//switch(o) {
// case Rectangle(Point p1, Point p2) -> ...
//}
var lookup = MethodHandles. lookup ();
var carrierMetadata = new CarrierMetadata( methodType (Object. class , int .
class , Point. class , Point. class ));
var empty = carrierMetadata.empty();
var op = of (empty,
test ( is_instance (Object. class , Rectangle. class ),
cast (Object. class ,
or (carrierMetadata,
project ( record_accessor (lookup, Rectangle. class , 0 ),
test ( is_null (Point. class ),
do_not_match (Point. class , carrierMetadata),
bind ( 1 , carrierMetadata))),
project ( record_accessor (lookup, Rectangle. class , 1 ),
test ( is_null (Point. class ),
do_not_match (Point. class , carrierMetadata),
bind ( 2 , carrierMetadata,
match (Point. class , carrierMetadata, 0
)
),
throw_NPE (Object. class , "o is null" )
)
);
// match: new Rectangle(new Point(1, 2), new Point(3, 4))
var rectangle1 = (Object) new Rectangle( new Point( 1 , 2 ), new Point( 3 , 4
));
var carrier1 = op.invokeExact(rectangle1);
System. out .println( "result: " + ( int ) carrierMetadata.accessor( 0
).invokeExact(carrier1));
System. out .println( "binding 1 " + (Point) carrierMetadata.accessor( 1
).invokeExact(carrier1));
System. out .println
Re: [External] : Re: Telling the totality story
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Friday, March 4, 2022 3:23:58 AM
> Subject: Re: [External] : Re: Telling the totality story
>>> Let
>>> This is where it gets ugly. Let has no opinions about null, but the game
>>> here is
>>> to pretend it does. So in a let statement:
>>> let P = e
>>> - Evaluate e
>>> - If e is null
>>> - If P is a covering pattern, its binding is bound to null;
>>> - else we throws NPE
>>> - Otherwise, e is matched to P. If it does not match (its remainder), a
>>> MatchException is thrown (or the else clause is taken, if one is present.)
>> It's not clear to me why a MatchException should be thrown instead of not
>> compiling if not exhaustive.
> You're confusing "exhaustive" and "total". A let must be exhaustive, but
> exhaustiveness != totality.
>> It's mean that there are remainders that does not lead to either a NPE or an
>> error, do you have an example of such remainder ?
> Yes. Suppose we have records Box(T t) and Pair(T t, U u), and A is
> sealed to B|C. Then if we're matching on a Pair, A>, then
> Pair(null, B)
> Pair(Box(B), D) // D is a type from the future
> Pair(Box(D), B)
> Pair(Box(D), D)
> Pair(null, D)
> are all in the remainder. It is a big stretch to claim either NPE or ICCE is
> right for any of these, and completely arbitrary to pick one for Pair(null,
> D).
> Also, its much more expensive to try to distinguish between these, and pick
> the
> "right" error for each, rather than insert a default clause that throws
> MatchRemainderException.
The exception are here because the view at runtime and the view at compile time
are slightly different
- an NPE is raised when a value is null but the pattern ask for a
deconstruction
- an ICCE if the compiler has use a sealed type during it's analysis and the
sealed type has changed
if we have only one case Pair<>(Box<>(A a1), A a2) and i suppose a default then
Pair(null, B) and Pair(null, D) throw a NPE
and
Pair(Box(B), D), Pair(Box(D), B) and Pair(Box(D), D) all match
Now, if we have
sealed A permits B, C { }
and
a switch that relies on A when doing the exhaustiveness check
By example,
switch(pair) {
case Pair<>(Box<>(A a1), B a2) -> ...
case Pair<>(Box<>(A a1), C a2) -> ...
case Pair<>(Object o) -> ...
}
in that case, the compiler registers that at runtime before the first execution
of the switch, A should only permit B and C,
if it's not the case an ICCE should be thrown.
In term of implementation, the ICCE checks can be done by the bootstrap method
of the invokedynamic corresponding to the switch,
so once before the switch/invokedynamic is linked.
One open question, i think first asked by Tagir, is what if Object (in the
example above) is not a super-type of Box anymore, should we also checks that
at runtime and throw an ICCE.
Rémi
Re: Telling the totality story
> From: "Brian Goetz" > To: "amber-spec-experts" > Sent: Thursday, March 3, 2022 9:42:25 PM > Subject: Telling the totality story > Given the misconceptions about totality and whether "pattern matching means > the > same thing in all contexts", it is pretty clear that the story we're telling > is > not evoking the right concepts. It is important not only to have the right > answer, but also to have the story that helps people understand why its right, > so let's take another whack at that. > (The irony here is that it only makes a difference at one point -- null -- > which > is the least interesting part of the story. So it is much sound and fury that > signifies nothing.) > None of what we say below represents a change in the language; it is just > test-driving a new way to tell the same story. (From a mathematical and > specification perspective, it is a much worse story, since it is full of > accidental detail about nulls (among other sins), but it might make some > people > happier, and maybe we can learn something from the reaction.) > The key change here is to say that every pattern-matching construct needs > special rules for null. This is already true, so we're just doubling down on > it. None of this changes anything about how it works. [...] > Let > This is where it gets ugly. Let has no opinions about null, but the game here > is > to pretend it does. So in a let statement: > let P = e > - Evaluate e > - If e is null > - If P is a covering pattern, its binding is bound to null; > - else we throws NPE > - Otherwise, e is matched to P. If it does not match (its remainder), a > MatchException is thrown (or the else clause is taken, if one is present.) It's not clear to me why a MatchException should be thrown instead of not compiling if not exhaustive. It's mean that there are remainders that does not lead to either a NPE or an error, do you have an example of such remainder ? Rémi
Re: [External] : Re: Primitive type patterns
- Original Message -
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Wednesday, March 2, 2022 9:11:58 PM
> Subject: Re: [External] : Re: Primitive type patterns
> On 3/2/2022 2:36 PM, fo...@univ-mlv.fr wrote:
>
>> There are two ways to express "match non null Integer + unboxing",
>> this one
>>Integer value = ...
>>switch(value) {
>> case Integer(int i) -> ...
>>}
>>
>> And we already agree that we want that syntax.
>
> Wait, what? The above is not yet on the table; we will have to wait for
> deconstruction patterns on classes to be able to express that. When we
> get there, we'll have a choice of whether we want to add a deconstructor
> to the wrapper classes. (At which point, you might well say "we already
> have a way to do that"...)
It's a bad faith argument, we know since the beginning that we need
deconstructors.
Rémi
Re: [External] : Re: Proposal: java.lang.runtime.Carrier
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "Jim Laskey" , "amber-spec-experts"
>
> Sent: Thursday, March 3, 2022 7:59:01 PM
> Subject: Re: [External] : Re: Proposal: java.lang.runtime.Carrier
>>>> For the pattern matching,
>>>> we also need a 'with' method, that return a method handle that takes a
>>>> carrier
>>>> and a value and return a new carrier with the component value updated.
>>> It is not clear to me why we "need" this. Rather than jumping right to
>>> "Here is
>>> the solution", can you instead try to shine some light on the problem you
>>> are
>>> trying to solve?
>> When you have nested record patterns, each of these patterns contribute to
>> introduce bindings, so when executing the code of the pattern matching, the
>> code that match a nested pattern needs to add values into the carrier object.
>> Given that the Carrier API is non mutable, we need the equivalent of a
>> functional setter, a wither.
> I don't think we need to do this.
> Recall what nested patterns means: if R(T t) is a record, and Q is a pattern
> that is not total, then
> x matches R(Q)
> means
> x matches R(T t) && t matches Q
> So if we have
> record R(S s) { }
> record S(int a, int b) { }
> then
> case R(S(var a, var b))
> operates by matching the target to R, deconstructing with the R deconstructor,
> which yields a carrier of shape (S). Then we further match the first component
> of this carrier to the S deconstructor, which yields a carrier of shape (II).
> No mutation needed.
> Note that this unrolling can happen in one of two ways:
> - The compiler just unrolls it doing plain vanilla compiler stuff
> - A pattern runtime has a nesting combinator that takes a pattern description
> for an outer and an inner pattern, which when evaluated with R and S, yields a
> carrier of shape (R;S;II), the compiler evaluates this nesting combinator with
> condy, and uses that to do the match.
> Either way, we don't need to mutate or replace carriers.
You want the same carrier for the whole pattern matching:
- if you have a logical OR between patterns (not something in the current Java
spec but Python, C# or clojure core.match have it so we may want to add an OR
in the future)
- if different cases starts with the same prefix of patterns, so you don't have
to re-execute the de-constructors/pattern methods of the prefix several times
It's like when you create a value type, you start with the default value with
all the field initialized with their default value, and each time there is a
new binding, you do the equivalent of a withfield.
Rémi
Re: [External] : Re: Proposal: java.lang.runtime.Carrier
> From: "Brian Goetz" > To: "Remi Forax" , "Jim Laskey" > Cc: "amber-spec-experts" > Sent: Thursday, March 3, 2022 7:29:21 PM > Subject: Re: [External] : Re: Proposal: java.lang.runtime.Carrier >> For the pattern matching, >> we also need a 'with' method, that return a method handle that takes a >> carrier >> and a value and return a new carrier with the component value updated. > It is not clear to me why we "need" this. Rather than jumping right to "Here > is > the solution", can you instead try to shine some light on the problem you are > trying to solve? When you have nested record patterns, each of these patterns contribute to introduce bindings, so when executing the code of the pattern matching, the code that match a nested pattern needs to add values into the carrier object. Given that the Carrier API is non mutable, we need the equivalent of a functional setter, a wither. Rémi
Re: Proposal: java.lang.runtime.Carrier
For the pattern matching, we also need a 'with' method, that return a method handle that takes a carrier and a value and return a new carrier with the component value updated. static MethodHandle withComponent(MethodType methodType, int i) // returns a mh (Carrier;T) -> Carrier with T the type of the component It can be built on top of constructor() + component() but i think that i should be part of the API instead of every user of the Carrier API trying to re-implement it. In term of spec, Jim, can you rename "component getter" to "component accessor" which is the term used by records. Rémi > From: "Brian Goetz" > To: "Jim Laskey" , "amber-spec-experts" > > Sent: Thursday, March 3, 2022 4:29:51 PM > Subject: Re: Proposal: java.lang.runtime.Carrier > Thanks Jim. > As background, (some form of) this code originated in a prototype for pattern > matching, where we needed a carrier for a tuple (T, U, V) to carry the results > of a match from a deconstruction pattern (or other declared pattern) on the > stack as a return value. We didn't want to spin a custom class per pattern, > and > we didn't want to commit to the actual layout, because we wanted to preserve > the ability to switch later to a value class. So the idea is you describe the > carrier you want as a MethodType, and there's a condy that gives you an MH > that > maps that shape of arguments to an opaque carrier (the constructor), and other > condys that give you MHs that map from the carrier to the individual bindings. > So pattern matching will stick those MHs in CP slots. > The carrier might be some bespoke thing (e.g., record anon(T t, U u, V v)), or > something that holds an Object[], or something with three int fields and two > ref fields, or whatever the runtime decides to serve up. > The template mechanism wants almost exactly the same thing for bundling the > parameters for uninterprted template strings. > Think of it as a macro-box; instead of boxing primitives to Object and Objects > to varargs, there's a single boxing operation from a tuple to an opaque type. > On 3/3/2022 8:57 AM, Jim Laskey wrote: >> We propose to provide a runtime anonymous carrier class object generator ; >> java.lang.runtime.Carrier . This generator class is designed to share >> anonymous >> classes when shapes are similar. For example, if several clients require >> objects containing two integer fields, then Carrier will ensure that each >> client generates carrier objects using the same underlying anonymous class. >> Providing this mechanism decouples the strategy for carrier class generation >> from the client facility. One could implement one class per shape; one class >> for all shapes (with an Object[]), or something in the middle; having this >> decision behind a bootstrap means that it can be evolved at runtime, and >> optimized differently for different situations. Motivation >> The [ https://bugs.openjdk.java.net/browse/JDK-8273943 | String Templates JEP >> draft ] proposes the introduction of a TemplatedString object for the primary >> purpose of carrying the template and associated values derived from a >> template >> literal . To avoid value boxing, early prototypes described these carrier >> objects using per-callsite anonymous classes shaped by value types, The use >> of >> distinct anonymous classes here is overkill, especially considering that many >> of these classes are similar; containing one or two object fields and/or one >> or >> two integral fields. Pattern matching has a similar issue when carrying the >> values for the holes of a pattern. With potentially hundreds (thousands?) of >> template literals or patterns per application, we need to find an alternate >> approach for these value carriers . Description >> In general terms, the Carrier class simply caches anonymous classes keyed on >> shape. To further increase similarity in shape, the ordering of value types >> is >> handled by the API and not in the underlying anonymous class. If one client >> requires an object with one object value and one integer value and a second >> client requires an object with one integer value and one object value, then >> both clients will use the same underlying anonymous class. Further, types are >> folded as either integer (byte, short, int, boolean, char, float), long >> (long, >> double) or object. [We've seen that performance hit by folding the long group >> into the integer group is significant, hence the separate group.] >> The Carrier API uses MethodType parameter types to describe the shape of a >> carrier. This incorporates with the primary use case where bootstrap methods >> need to capture indy non-static arguments. The API has three static methods; >> // Return a constructor MethodHandle for a carrier with components >> // aligning with the parameter types of the supplied methodType. >> static MethodHandle constructor(MethodType methodType) >> // Return a component getter MethodHandle for component i. >> sta
Re: [External] : Re: Primitive type patterns
- Original Message -
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Monday, February 28, 2022 10:00:08 PM
> Subject: Re: [External] : Re: Primitive type patterns
> This is a valid generalized preference (and surely no one is going to
> say "no, I prefer to play to our weaknesses.") But at root, I think
> what you are saying is that you would prefer that pattern matching
> simply be a much smaller and less fundamental feature than what is being
> discussed here. And again, while I think that's a valid preference, I
> think the basis of your preference is that it is "simpler", but I do not
> think it actually delivers the simplicity dividend you are hoping for,
> because there will be subtle mismatches that impede composition and
> refactoring (e.g., new "null gates" and "box gates".)
There are two ways to express "match non null Integer + unboxing",
this one
Integer value = ...
switch(value) {
case Integer(int i) -> ...
}
And we already agree that we want that syntax.
You are proposing a new one
Integer value = ...
switch(value) {
case int i -> ...
}
Obviously, your proposal makes things less simple because we new have to ways
to say the same thing.
Moreover, introducing assignment conversions introduce more corner cases,
we already discussed about
- unboxing + widening being supported while than the widening + unboxing is not,
- pattern matching behaving differently from the visitor pattern/method
overriding rules
There is also issues IMO when you start mixing wrappers and primitive types.
Just ask yourself, can you predict if the following codes compiles and if it
compiles, which case is called depending on the value of value.
Integer value = ...
switch(value) {
case double d -> ...
case Integer i -> ...
}
int value = ...
switch(value) {
case float f -> ...
case Integer i -> ...
}
int value = ...
switch(value) {
case Float f -> ...
case Integer i -> ...
}
I think we do not need assignment conversions *and* that introducing them makes
the semantics harder to understand.
Rémi
Re: [External] : Re: Primitive type patterns
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Monday, February 28, 2022 7:53:07 PM
> Subject: Re: [External] : Re: Primitive type patterns
>>> Now, what if instead of Object, we start with Long?
>>> Long l = 0L
>>> if (l instanceof byte b) { ... }
>>> First, applicability: does Long unbox to a primitive type that can be
>>> narrowed
>>> to byte? Yes! Long unboxes to long, and long can be narrowed to byte.
>>> Then: matching: if the RHS is not null, we unbox, and do a range check. (The
>>> rules in my previous mail probably didn't get this case perfectly right),
>>> but
>>> 0L will match, and 0x will not -- as we would expect.
>> This is totally alien to me, when you have x instanceof Foo (note: this is
>> not
>> the pattern version) with X the type of x, then if x is declared with a super
>> type of X it works the exact same way, i.e i don't have to care to much about
>> the type on what i'm doing an instanceof / switching over it.
> Yes, I understand your discomfort. And I will admit, I don't love this
> particular corner-of-a-corner either. (But let's be clear: it is a corner. If
> you're seeking to throw out the whole scheme on the basis that corners exist,
> you'll find the judge to be unsympathetic.)
> So why have I proposed it this way? Because, unfortunately, of this existing
> line in JLS 5.2 (which I never liked):
> > an unboxing conversion followed by a widening primitive conversion
> This is what lets you say:
> long l = anInteger
> And, I never liked this rule, but we're stuck with it. The inverse, from which
> we would derive this rule, is that
> anInteger instanceof long l
> should be applicable, and in fact always match when the LHS is non-null. I
> would
> prefer to not allow this assignment conversion, and similarly not allow both
> unboxing and widening in one go in pattern matching, but I didn't get to write
> JLS 5.2.
> What's new here is going in the *other* direction:
> anInteger instanceof short s
> and I think what is making you uncomfortable is that you are processing two
> generalizations at once, and it's pushing your "OMG different! scary!"
> buttons:
> - that we're defining primitive type patterns in a way such that we can derive
> the existing assignment conversions;
> - that primitive type patterns can have dynamic checks that primitive
> assignments cannot, so we're including the value-range check.
> Each individually is not quite as scary, but I can understand why the two
> together would seem scary. (And, as I mentioned, I don't like the
> unbox-and-widen conversions either, but I didn't invent those.)
I think we should play on our strengths, destructuring instead of unboxing,
pattern methods instead of primitive conversions + rangecheck.
I prefer to have a few small well defined patterns (type pattern, type
destructuring, pattern method) each with a simple semantics (so keep the type
pattern to be about subtyping relationship only) and draw power from the
ability to compose them instead of having patterns with a huge semantic baggage
(type pattern with the assignment semantics) and the corner cases that come
with it.
We may still need assignment conversions when we mix pattern and assignment,
but because we want to be "backward compatibility" with the simple assignment
semantics.
Rémi
Re: [External] : Re: Primitive type patterns
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Monday, February 28, 2022 7:08:07 PM
> Subject: Re: [External] : Re: Primitive type patterns
>>> So *of course* there's an obvious definition of how `int x` matches against
>>> Integer, and its not a question of whether we "define" it that way, its a
>>> question of whether we expose the obvious meaning, or suppress it. I think
>>> the
>>> arguments in favor of suppression are pretty weak.
>> The strong argument is that instanceof/switch case is about subtyping
>> relationship while assignment is about assignment conversions, trying to blur
>> the lines between the two has already been tried in the past and it results
>> in
>> pain (see below).
> This is pretty much assuming your conclusion, and then stating it as
> justification :)
> I get it; you would prefer that pattern matching be *only* about subtyping. I
> understand why you prefer that. But I think this is mostly a "retreat to the
> comfort zone" thing.
>> What about ?
>> Object o =...
>> if (o instanceof byte) { ... }
>> Does it means that o can be a Long ?
> This is a good question. (But I know it's also a trap.) We first have to ask
> about (static) applicability: is the pattern `byte b` applicable to Object? If
> not, we'll get a compilation error.
> My earlier message said:
> - A primitive type pattern `P p` should be applicable to a reference target T
> if
> T unboxes to P, or T unboxes to a primitive type that can be widened to P [ or
> if T unboxes to a primitive type that can be narrowed to P. ]
> Does Object unbox to byte? No.
> Does Object unbox to a primitive type that can be widened to byte? No.
> [brackets] Does Object unbox to a primitive type than can be narrowed to byte?
> No.
> How does this square with assignments? I cannot assign
> byte b = anObject
> | incompatible types: java.lang.Object cannot be converted to byte
> If I try this with casting:
> Object o = 0L
> byte b = (byte) o
> I get a CCE, because the cast will only convert from Byte.
> Now, what if instead of Object, we start with Long?
> Long l = 0L
> if (l instanceof byte b) { ... }
> First, applicability: does Long unbox to a primitive type that can be narrowed
> to byte? Yes! Long unboxes to long, and long can be narrowed to byte.
> Then: matching: if the RHS is not null, we unbox, and do a range check. (The
> rules in my previous mail probably didn't get this case perfectly right), but
> 0L will match, and 0x will not -- as we would expect.
This is totally alien to me, when you have x instanceof Foo (note: this is not
the pattern version) with X the type of x, then if x is declared with a super
type of X it works the exact same way, i.e i don't have to care to much about
the type on what i'm doing an instanceof / switching over it.
The rule you propose breaks that, i've to take a very close look to the type of
'x'. As i said earlier, it's too smart for me.
I see no problem to have such semantics specified by a pattern method, but i
don't think we should make the type pattern too clever.
Rémi
Re: [External] : Re: Primitive type patterns
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Monday, February 28, 2022 3:34:02 PM
> Subject: Re: [External] : Re: Primitive type patterns
>> Nope,
>> i'm saying that inside a pattern if we let the unboxing to be possible with
>> the
>> semantics that if the value is null, instead of throwing a NPE if it does not
>> match, we are introducing the equivalent of the null-safe operator of Groovy
>> (the elvis operator), something we should not do (and that was rejected in
>> the
>> past, i think as part of Coin or Coin 2).
> Too silly to respond to. Let's stay serious, please.
>> Maybe it means that we should allow unboxing with the semantics that it
>> throw a
>> NPE i.e the exact semantics of the assignment conversion instead of
>> disallowing
>> unboxing as i proposed, I don't know, but I really dislike the semantics you
>> are proposing here.
> It seems more likely that you just *don't understand* what is being proposed
> here (and the fact that you keep repeating the incorrect claim that pattern
> matching somehow means different things in different constructs underscores
> that.)
[]
It's your view of the world you think that you can subsume all pattern usages
by saying that they obey to the assignment conversions and you think you can
decompose the unboxing as nullcheck + do not match.
I believe that you are wrong on both counts, the former results in an
over-generalization that will transfer part of the work the compiler does at
compile time at runtime and the later means you are choosing in place of the
user if it means Integer(int value) or Integer instead of asking.
Given that the first point subsumes the second one, i think we should focus on
it, first.
So with
record Atom(Object o) { ... }
Atom box = ...;
if (o instanceof Atom(byte b)) {
}
can box = new Atom(3); matches the pattern at runtime ?
Rémi
Re: Primitive type patterns
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Monday, February 28, 2022 5:50:25 PM
> Subject: Re: Primitive type patterns
> Let me put the central question here in the spotlight.
>> Boxing and unboxing
>> Suppose our match target is a box type, such as:
>> record IntegerBox(Integer i) { }
>> Clearly we can match it with:
>> case IntegerBox(Integer i):
> We could stop here, and say the pattern `int i` is not applicable to
> `Integer`,
> which means that `IntegerBox(int i)` is not applicable to IntegerBox. This is
> a
> legitimate choice (though, I think it would be a bad one, one that we would
> surely revisit sooner rather than later.)
> Users might well not understand why they could not say
> case IntegerBox(int i)
> because (a) you can unbox in other contexts and (b) this has a very clear
> meaning, and one analogous to `case ObjectBox(String s)` -- if the box
> contents
> fits in a String, then match, otherwise don't. They might get even more balky
> if you could say
> int x = anInteger
> but not
> let int x = anInteger
> And, if we went with both Remi's preferences -- the minimalist matching
> proposal
> and dropping "let" in the let/bind syntax (which we will NOT discuss further
> here) then the following would be ambiguous:
> int x = anInteger // is this an assignment, in which case we unbox, or a
> match,
> in which case we don't?
> and we would have to "arbitrarily" pick the legacy interpretation.
I said the opposite, if you use "let" you do not have to support assignment
conversions.
> But all of this "balking" is symptom, not disease.
> The reality is that pattern matching is more primitive than unboxing
> conversions, and if we lean into that, things get simpler. An unboxing
> conversion may seem like one thing, but is actually two: try to match against
> a
> partial pattern, and if there is no match, fail. In other words, unboxing is:
> int unbox(Integer n) ->
> switch(n) {
> case int i -> i;
> case null -> throw new NPE();
> }
> The unboxing we have jams together the pattern match and the throw-on-fail,
> because it has no choice; unboxing wants to be total, and there's no place to
> specify what to try if the pattern match fails. But pattern matching is
> inherently conditional, allowing us to build different constructs on it which
> handle failures differently.
> So *of course* there's an obvious definition of how `int x` matches against
> Integer, and its not a question of whether we "define" it that way, its a
> question of whether we expose the obvious meaning, or suppress it. I think the
> arguments in favor of suppression are pretty weak.
The strong argument is that instanceof/switch case is about subtyping
relationship while assignment is about assignment conversions, trying to blur
the lines between the two has already been tried in the past and it results in
pain (see below). But i suppose every generations need to re-discover it.
> There's a similar argument when it comes to narrowing from (say) long to int.
> There's a very natural interpretation of matching `int x` to a long: does the
> long value fit in an int. In assignment context, we do the best we currently
> can -- we allow the narrowing only if we can statically prove it will match,
> which mean if the match target is a constant. But again, conversions in
> assignment context are not the primitive. If we take the obvious definition of
> matching `int x` against long, then the current rules fall out naturally, and
> we can ask sensible questions like
> if (aLong instanceof byte) { ... }
> else if (aLong instanceof short) { ... }
> by *asking the primitive directly*, rather than appealing to some proxy (like
> manually unrolling the range check.)
What about ?
Object o =...
if (o instanceof byte) { ... }
Does it means that o can be a Long ?
Rémi
Re: [External] : Re: Primitive type patterns
From: "Brian Goetz"
To: "Remi Forax"
Cc: "amber-spec-experts"
Sent: Saturday, February 26, 2022 5:49:08 PM
Subject: Re: [External] : Re: Primitive type patterns
BQ_BEGIN
BQ_BEGIN
BQ_BEGIN
Relationship with assignment context
BQ_END
That's a huge leap, let's take a step back.
I see two questions that should be answered first.
1) do we really want pattern in case of assignment/declaration to support
assignment conversions ?
2) do we want patterns used by the switch or instanceof to follow the exact
same rules as patterns used in assignment/declaration ?
BQ_END
BQ_END
[...]
BQ_BEGIN
BQ_BEGIN
2) we already know that depending on the context (inside a switch, inside a
instanceof, inside an assignment) the rules for pattern are not exactly the
same.
BQ_END
OMG Remi, would you please stop repeating this incorrect claim. The rules for
pattern matching are exactly the same across contexts; the differences are that
the contexts get to choose when to try to match, and what to do if nothing
matches.
BQ_END
if (o instanceof Circle(Point(int x, int y), int radius))
means
if (o instanceof Circle circle) {
Point center = circle.center();
if (center == null) {
goto end
}
int x = center.x();
int y = center.y();
int radius = circle.radius();
end:
}
while
Circle(Point(int x, int y), int radius) = c;
means
Point center = circle.center();
if (center == null) {
throw new NPE()
}
int x = center.x();
int y = center.y();
int radius = circle.radius();
end:
In the context of switch or instanceof, a pattern does not have to be total,
while in case of a assignment the pattern has to be total,
so depending on the context, the semantics is different.
Rémi
Re: [External] : Re: Primitive type patterns
From: "Brian Goetz" To: "Remi Forax" Cc: "amber-spec-experts" Sent: Saturday, February 26, 2022 5:49:08 PM Subject: Re: [External] : Re: Primitive type patterns BQ_BEGIN BQ_BEGIN BQ_BEGIN Relationship with assignment context BQ_END That's a huge leap, let's take a step back. I see two questions that should be answered first. 1) do we really want pattern in case of assignment/declaration to support assignment conversions ? 2) do we want patterns used by the switch or instanceof to follow the exact same rules as patterns used in assignment/declaration ? BQ_END I agree we should take a step back, but let's take a step farther -- because I want to make an even bigger leap that you think :) Stepping way far back in the beginning ... Java had reference types with subtyping, and eight primitive types. Which raises an immediate question: what types can be assigned to what? Java chose a sensible guideline; assignment should be allowed if the value set on the left is "bigger" than that on the right. This gives us String => Object, int => long, int => double, etc. (At this point, note that we've gone beyond strict value set inclusion; an int is *not* a floating point number, but we chose (reasonably) to do the conversion because we can *embed* the ints in the value set of double. Java was already appealing to the notion of embedding-projection pair even then, in assignment conversions; assignment from A to B is OK if we have an embedding of A into B.) On the other hand, Java won't let you assign long => int, because it might be a lossy conversion. To opt into the loss, you have to cast, which acknowledges that the conversion may be information-losing. Except! If you can prove the conversion isn't information losing (because the thing on the right is a compile-time constant), then its OK, because we know its safe. JLS Ch5 had its share of ad-hoc-seeming complexity, but mostly stayed in its corner until you called it, and the rules all seemed justifiable. Then we added autoboxing. And boxing is not problematic; int embeds into Integer. So the conversion from int => Integer is fine. (It added more complexity to overload selection, brought in strict and loose conversion contexts, and we're still paying when methods like remove(int) merge with remove(T), but OK.) But the other direction is problematic; there is one value of Integer that doesn't correspond to any value of int, which is our favorite value, null. The decision made at the time was to allow the conversion from Integer => int, and throw on null. This was again a justifiable choice, and comes from the fact that the mapping from Integer to int is a _projection_, not an embedding. It was decided (reasonably, but we could have gone the other way too) that null was a "silly" enough value to justify not requiring a cast, and throwing if the silly value comes up. We could have required a cast from Integer to int, as we do from long to int, and I can imagine the discussion about why that was not chosen. Having set the stage, one can see all the concepts in pattern matching dancing on it, just with different names. Whether we can assign T to U with or without a cast, is something we needed a static rule for. So we took the set of type pairs (T, U) for which the pattern `T t` is strictly total on U, and said "these are the conversions allowed in assignment context" (with a special rule for when the target is an integer constant.) When we got to autoboxing, we made a subjective call that `int x` should be "total enough" on `Integer` that we're willing to throw in the one place it's not. That's exactly the concept of "P is exhaustive, but not total, on T" (i.e., there is a non-empty remainder.) All of this has happened before. All of this will happen again. So the bigger leap I've got in mind is: what would James et al have done, had they had pattern matching from day one? I believe that: - T t = u would be allowed if `T t` is exhaustive on the static type of u; - If there is remainder, assignment can throw (preserving the invariant that if the assignment completes normally, something was assigned). So it's not that I want to align assignment with pattern matching because we've got a syntactic construct on the whiteboard that operates by pattern matching but happens to looks like assignment; it's because assignment *is* a constrained case of pattern matching. We've found the missing primitive, and I want to put it under the edifice. If we define pattern matching correctly, we could rewrite JLS 5.2 entirely in terms of pattern matching (whether we want to actually rewrite it or not, that's a separate story.) The great thing about pattern matching as a generalization of assignment is
Re: [External] : Re: Primitive type patterns
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Saturday, February 26, 2022 5:49:08 PM
> Subject: Re: [External] : Re: Primitive type patterns
>>> Relationship with assignment context
>> That's a huge leap, let's take a step back.
>> I see two questions that should be answered first.
>> 1) do we really want pattern in case of assignment/declaration to support
>> assignment conversions ?
>> 2) do we want patterns used by the switch or instanceof to follow the exact
>> same
>> rules as patterns used in assignment/declaration ?
[...]
>> 2) we already know that depending on the context (inside a switch, inside a
>> instanceof, inside an assignment) the rules for pattern are not exactly the
>> same.
>> So we may consider that in the assignment context, assignment conversions
>> apply
>> while for a matching context, simpler rules apply.
> We could of course say that; we could say that `int x` is simply *not
> applicable* to a target of type Integer. We can discuss that, but I don't
> think
> its a simplification, though; I think its actually *more* complexity because
> it's yet another context with yet another subtly different set of rules. One
> obvious consequence of that restriction would be that users cannot refactor
> Foo f = e
> to
> let Foo f = e
> to
> if (e instanceof Foo f) { ... }
> for yet more accidental reasons. Is this really making things simpler?
>> Then the model you propose is too clever for me, the fact that
>> instanceof Point(double x, double y)
>> has a different meaning depending if Point is declared like
>> record Point(double x, double y) { }
>> or like this
>> record Point(Double x, Double y) { }
>> is too much.
> If this is your concern, then I think you are objecting to something much more
> fundamental than the semantics of primitive patterns; you're objecting to the
> concept of nesting partial patterns entirely.
Nope,
i'm saying that inside a pattern if we let the unboxing to be possible with the
semantics that if the value is null, instead of throwing a NPE if it does not
match, we are introducing the equivalent of the null-safe operator of Groovy
(the elvis operator), something we should not do (and that was rejected in the
past, i think as part of Coin or Coin 2).
let A(B(C c)) = a else null;
is equivalent to
var result = a.?getB().?getC();
Maybe it means that we should allow unboxing with the semantics that it throw a
NPE i.e the exact semantics of the assignment conversion instead of disallowing
unboxing as i proposed, I don't know, but I really dislike the semantics you
are proposing here.
Rémi
Re: Primitive type patterns
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Friday, February 25, 2022 10:45:44 PM
> Subject: Primitive type patterns
> As a consequence of doing record patterns, we also grapple with primitive type
> patterns. Until now, we've only supported reference type patterns, which are
> simple:
> - A reference type pattern `T t` is applicable to a match target of type M if
> M
> can be cast to T without an unchecked warning.
> - A reference type pattern `T t` covers a match type M iff M <: T
> - A reference type pattern `T t` matches a value m of type M if M <: T || m
> instanceof T
> Two of these three characterizations are static computations (applicability
> and
> coverage); the third is a runtime test (matching). For each kind of pattern,
> we
> have to define all three of these.
> Primitive type patterns in records
> Record patterns necessitate the ability to write type patterns for any type
> that
> can be a record component. If we have:
> record IntBox(int i) { }
> then we want to be able to write:
> case IntBox(int i):
> which means we need to be able to express type patterns for primitive types.
> Relationship with assignment context
> There is another constraint on primitive type patterns: the let/bind statement
> coming down the road. Because a type pattern looks (not accidentally) like a
> local variable declaration, a let/bind we will want to align the semantics of
> "local variable declaration with initializer" and "let/bind with total type
> pattern". Concretely:
> let String s = "foo";
> is a pattern match against the (total) pattern `String s`, which introduces
> `s`
> into the remainder of the block. Since let/bind is a generalization of local
> variable declaration with initialization, let/bind should align with locals
> where the two can express the same thing. This means that the set of
> conversions allowed in assignment context (JLS 5.2) should also be supported
> by
> type patterns.
> Of the conversions supported by 5.2, the only one that applies when both the
> initializer and local variable are of reference type is "widening reference",
> which the above match semantics (`T t` matches `m` when `M <: T`) support. So
> we need to fill in the other three boxes of the 2x2 matrix of { ref, primitive
> } x { ref, primitive }.
That's a huge leap, let's take a step back.
I see two questions that should be answered first.
1) do we really want pattern in case of assignment/declaration to support
assignment conversions ?
2) do we want patterns used by the switch or instanceof to follow the exact
same rules as patterns used in assignment/declaration ?
For 1, given that we are using pattern to do destructured assignment, we may
want to simplify the assignment rules to keep things simple avoid users
shooting themselves in the foot with implicit unboxing.
With an example,
record Box(T value) {}
Box box = ...
Box<>(int result) = box; // assignment of result may throw a NPE
I don't think we have to support that implicit unboxing given that we have a
way to ask for an unboxing explicitly (once java.lang.Integer have a
de-constructor)
Box<>(Integer(int result)) = box;
I think we should not jump with the shark too soon here and ask ourselves if we
really want assignment conversions in case of destructured assignment.
2) we already know that depending on the context (inside a switch, inside a
instanceof, inside an assignment) the rules for pattern are not exactly the
same.
So we may consider that in the assignment context, assignment conversions apply
while for a matching context, simpler rules apply.
Given that the syntax for switch reuse '->', i believe we should use the
overriding rules (the one we use for lambdas) instead of the assignment rules
(the one we use for method reference).
And yes, i know that the '->' of switch is not the same as the '->' of lambda,
but i don't think we should bother users to intuitively think that the same
rules apply.
Then the model you propose is too clever for me, the fact that
instanceof Point(double x, double y)
has a different meaning depending if Point is declared like
record Point(double x, double y) { }
or like this
record Point(Double x, Double y) { }
is too much.
The semantics of Java around null is already a giant landmine field, we should
restraint ourselves to add more null-funny behaviors.
regards,
Rémi
Re: [External] : Re: Record patterns (and beyond): exceptions
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Friday, February 18, 2022 3:34:45 PM
> Subject: Re: [External] : Re: Record patterns (and beyond): exceptions
>>> But this clearly does not fall into ICCE. ICCE means, basically, "your
>>> classpath
>>> is borked"; that things that were known to be true at compile time are not
>>> true
>>> at runtime. (Inconsistent separate compilation is the most common cause.)
>>> But
>>> Box(Bag(null)) is not an artifact of inconsistent separate compilation.
>> I think i've not understood the problem correctly, i was thinking the error
>> was
>> due to the erasure, Box being erased to Box, the problem with erasure is
>> that you see the problem late, in case of the switch after the phase that
>> does
>> instanceofs, so we end up with ClassCastException instead of ICCE.
> CCE is not the right thing either. Let's step back and go over the concepts.
> We want for the compiler to be able to do type checking that a switch is
> "total
> enough" to not require a default clause. We want this not just because writing
> a default clause when you think you've got things covered is annoying, but
> also, because once you have a default clause, you've given up on getting any
> better type checking for totality. In a switch over enum X {A, B}, having only
> cases for A and B means that, when someone adds C later, you'll find out about
> it, rather than sweeping it under the rug. Sealed class hierarchies have the
> same issues as enums; the possibility of novel values due to separate
> compilation. So far, all of these could be described by ICCE (and they are,
> currently.)
> We've already talked for several lifetimes about null; switches that reject
> null
> do so with NPE. That also makes sense. We had hoped that this covered the
> weird
> values that might leak out of otherwise-exhaustive switches, but that was
> wishful thinking.
> Having nested deconstruction patterns introduces an additional layer of
> weirdness. Suppose we have
> sealed interface A permits X, Y { }
> Box box;
> switch (box) {
> case Box(X x):
> case Box(Y y):
> }
> This should be exhaustive, but we have to deal with two additional bad values:
> Box(null), which is neither a Box(A) or a Box(B), and Box(C), for a novel
> subtype C. We don't want to disturb the user to deal with these by making them
> have a default clause.
> So we define exhaustiveness separately from totality, and remainder is the
> difference. (We can constructively characterize the upper bound on remainder.)
> And we can introduce a throwing default, as we did with expression switches
> over enums. But what should it throw?
> The obvious but naive answer is "well, Box(null) should throw NPE, and Box(C)
> should throw ICCE." But only a few minutes thinking shows this to be
> misleading, expensive, and arbitrary. When we encountered Box(null), it was
> not
> because anyone tried to dereference a null, so throwing NPE is misleading.
A NPE is not a problem if (the big if) the error message is "null neither match
Box(X) nor Box(Y)"
> If the shape of the remainder is complicated, this means generating tons of
> low-value, compiler-generated boilerplate to differentiate Box(Bag(null)) from
> Box(Container()). That's expensive. And, what about Foo(null, C)? Then
> we have to arbitrarily pick one. It's a silly scheme.
We already talked about that, the shape of the remainder is complex if you want
to generate all branches at compile time, it's not an issue if you generate the
branches at runtime, because you can generate them lazily.
For some checks, they can only be done at runtime anyway, like does this
pattern is still total ?
About Foo(null, C), i suppose you mean a case where you have both a null that
need to be deconstructed and a new subtype, the solution is to go left to
right, like usual in Java.
> So the logical thing to do is to say that these things fall into a different
> category from NPE and ICCE, which is that they are remainder, which gets its
> own label.
Nope, as a user i want a real error message, not something saying nope, sorry
too complex, i bailout.
[...]
>>> Some patterns are considered exhaustive, but not total. A deconstruction
>>> pattern
>>> D(E(total)) is one such example; it is exhaustive on D, but does not match
>>> D(null), because matching the nested E(total) requires invoking a
>>> deconstructor
>>> in E, and you can't invoke an instance member on a null receiver. Still, we
>>> consider D(E(total)) exhaustive on D,
Re: [External] : Re: Record patterns (and beyond): exceptions
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Thursday, February 17, 2022 3:43:08 PM > Subject: Re: [External] : Re: Record patterns (and beyond): exceptions >>> As we look ahead to record patterns, there is a new kind of remainder: the >>> "spine" of nested record patterns. This includes things like Box(null), >>> Box(novel), Box(Bag(null)), Box(Mapping(null, novel)), etc. It should be >>> clear >>> that there is no clean extrapolation from what we currently do, to what we >>> should do in these cases. But that's OK; both of the existing >>> remainder-rejection cases derive from "what does the context think" -- >>> switch >>> hates null (so, NPE), and enum switches are a thing (which makes ICCE on an >>> enum switch reasonable.) But in the general case, we'll want some sort of >>> MatchRemainderException. >> Nope, it can not be a runtime exception because people will write code to >> catch >> it and we will have a boat load of subtle bugs because exception are side >> effect so you can see in which order the de-constructors or the pattern >> methods >> are called. ICCE is fine. > But this clearly does not fall into ICCE. ICCE means, basically, "your > classpath > is borked"; that things that were known to be true at compile time are not > true > at runtime. (Inconsistent separate compilation is the most common cause.) But > Box(Bag(null)) is not an artifact of inconsistent separate compilation. I think i've not understood the problem correctly, i was thinking the error was due to the erasure, Box being erased to Box, the problem with erasure is that you see the problem late, in case of the switch after the phase that does instanceofs, so we end up with ClassCastException instead of ICCE. > In any case, I am not getting your point about "but people can catch it." So > what? People can catch OOME too, and try to parse the output of toString() > when > we tell them not to. But that's no reason to make all exceptions > "OpaqueError". > So what is your point here? You can catch OOME if you write the code by hand. People are using IDEs and when the IDE is lost or the user have click on the wrong button, catch(Exception) appears. That the reason why we have both IOError and UncheckedIOException in the JDK. >>> Note that throwing an exception from remainder is delayed until the last >>> possible moment. We could have: >>> case Box(Bag(var x)): ... >>> case Box(var x) when x == null: ... >>> and the reasonable treatment is to treat Box(Bag(var x)) as not matching >>> Box(null), even though it is exhuastive on Box>), and therefore fall >>> into the second case on Box(null). Only when we reach the end of the switch, >>> and we haven't matched any cases, do we throw MatchRemainderException. >> I really dislike that idea, it will be a burden in the future each time we >> want >> to change the implementation. >> I would like the semantics to make no promise about when the error will be >> thrown, the semantics should not be defined if a deconstructors/pattern >> method >> does a side effect, the same way the stream semantics is not defined if the >> lambda taken as parameter of Stream.map() does a side effect. >> I think the parallel between the pattern matching and a stream in term of >> execution semantics is important here. From the outside, those things are >> monads, they should work the same way. > I think this stems from the same misunderstanding you have about the boundary > between the pattern semantics and the construct semantics. I'm going to > test-drive some adjusted language here. > A total pattern is just that -- it matches everything. > Some patterns are considered exhaustive, but not total. A deconstruction > pattern > D(E(total)) is one such example; it is exhaustive on D, but does not match > D(null), because matching the nested E(total) requires invoking a > deconstructor > in E, and you can't invoke an instance member on a null receiver. Still, we > consider D(E(total)) exhaustive on D, which means it is enough to satisfy > the type checker that you've covered everything. Remainder is just the gap > between exhaustiveness and totality. The gap is due to E(...) not matching null, for me it's a NPE with an error message saying exactly that. > If we have the following switch: > case D(E(Object o)): > case D(var x) when x == null: > the semantics of D(E(Object)) are that *it matches all non-null D, except > D(null)*. So throwing when we ev
Re: Record patterns (and beyond): exceptions
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Wednesday, February 16, 2022 7:34:17 PM
> Subject: Record patterns (and beyond): exceptions
> As we move towards the next deliverable -- record patterns -- we have two new
> questions regarding exceptions to answer.
> Questions
> 1. When a dtor throws an exception. ( You might think we can kick this down
> the
> road, since records automatically acquire a synthetic dtor, and users can't
> write dtors yet, but since that synthetic dtor will invoke record component
> accessors, and users can override record component accessors and therefore
> they
> can throw, we actually need to deal with this now.)
> This has two sub-questions:
> 1a. Do we want to do any extra type checking on the bodies of dtors / record
> accessors to discourage explicitly throwing exceptions? Obviously we cannot
> prevent exceptions like NPEs arising out of dereference, but we could warn on
> an explicit throw statement in a record accessor / dtor declaration, to remind
> users that throwing from dtors is not the droid they are looking for.
For de-constructor, given that they does not exist yet, we can do like record
constructor, banned checked exceptions and for accessors, we can emit a warning
as you suggest and do not allow record pattern if one of the getters throws a
checked exception.
> 1b. When the dtor for Foo in the switch statement below throws E:
> switch (x) {
> case Box(Foo(var a)): ...
> case Box(Bar(var b)): ...
> }
> what should happen? Candidates include:
> - allow the switch to complete abruptly by throwing E?
> - same, but wrap E in some sort of ExceptionInMatcherException?
> - ignore the exception and treat the match as having failed, and move on to
> the
> next case?
The nice thing about the rules above is that a record pattern can never throw a
checked exception. So there is nothing to do here.
> 2. Exceptions for remainder. We've established that there is a difference
> between an _exhaustive_ set of patterns (one good enough to satisfy the
> compiler that the switch is complete enough) and a _total_ set of patterns
> (one
> that actually covers all input values.) The difference is called the
> _remainder_. For constructs that require totality, such as pattern switches
> and
> let/bind, we have invariants about what will have happened if the construct
> completes normally; for switches, this means exactly one of the cases was
> selected. The compiler must make up the difference by inserting a throwing
> catch-all, as we already do with expression switches over enums, and all
> switches over sealed types, that lack total/default clauses.
> So far, remainder is restricted to two kinds of values: null (about which
> switch
> already has a strong opinion) and "novel" enum constants / novel subtypes of
> sealed types. For the former, we throw NPE; for the latter, we throw ICCE.
> As we look ahead to record patterns, there is a new kind of remainder: the
> "spine" of nested record patterns. This includes things like Box(null),
> Box(novel), Box(Bag(null)), Box(Mapping(null, novel)), etc. It should be clear
> that there is no clean extrapolation from what we currently do, to what we
> should do in these cases. But that's OK; both of the existing
> remainder-rejection cases derive from "what does the context think" -- switch
> hates null (so, NPE), and enum switches are a thing (which makes ICCE on an
> enum switch reasonable.) But in the general case, we'll want some sort of
> MatchRemainderException.
Nope, it can not be a runtime exception because people will write code to catch
it and we will have a boat load of subtle bugs because exception are side
effect so you can see in which order the de-constructors or the pattern methods
are called. ICCE is fine.
> Note that throwing an exception from remainder is delayed until the last
> possible moment. We could have:
> case Box(Bag(var x)): ...
> case Box(var x) when x == null: ...
> and the reasonable treatment is to treat Box(Bag(var x)) as not matching
> Box(null), even though it is exhuastive on Box>), and therefore fall
> into the second case on Box(null). Only when we reach the end of the switch,
> and we haven't matched any cases, do we throw MatchRemainderException.
I really dislike that idea, it will be a burden in the future each time we want
to change the implementation.
I would like the semantics to make no promise about when the error will be
thrown, the semantics should not be defined if a deconstructors/pattern method
does a side effect, the same way the stream semantics is not defined if the
lambda taken as parameter of Stream.map() does a side effect.
I think the parallel between the pattern matching and a stream in term of
execution semantics is important here. From the outside, those things are
monads, they should work the same way.
Rémi
Re: [External] : Re: Reviewing feedback on patterns in switch
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Wednesday, February 16, 2022 5:03:19 PM
> Subject: Re: [External] : Re: Reviewing feedback on patterns in switch
> Of course, in an ecosystem as diverse as Java developers, one routinely
> expects
> to get complaints about both X and ~X. Which makes it notable that we have not
> gotten any complaints about "why do you force me to write an empty default".
> (I'm not complaining!)
> The case you raise -- legacy { switch type, labels, statement } switches -- is
> harder to fix. The things we've explored (like an opt-in to totality) are
> pretty poor fixes, since (a) they are noisy warts, and (b) people will forget
> them and still have the problem. So these are harder, longer-term problems.
> (For now, the best we can do is noisy warnings.)
The problem of the noisy warning is that there is no "right" way to fix the
warning now.
Adding a default is not what we want, we need a way to opt-in exhaustiveness.
But we never agree on the way to do that.
Rémi
> On 2/16/2022 11:00 AM, Remi Forax wrote:
>>> From: "Brian Goetz" [ mailto:brian.go...@oracle.com |
>>> ]
>>> To: "amber-spec-experts" [ mailto:amber-spec-experts@openjdk.java.net |
>>> ]
>>> Sent: Wednesday, February 16, 2022 4:49:19 PM
>>> Subject: Re: Reviewing feedback on patterns in switch
>>> One thing that we have, perhaps surprisingly, *not* gotten feedback on is
>>> forcing all non-legacy switches (legacy type, legacy labels, statement
>>> only) to
>>> be exhaustive. I would have thought people would complain about pattern
>>> switches needing to be exhaustive, but no one has! So either no one has
>>> tried
>>> it, or we got away with it...
>> Yes, we had several feedbacks about the opposite, why the switch statement
>> on an
>> enum is not exhaustive, i.e. why the following code does not compile
>> enum Color { RED , BLUE } int x;
>> Color color = null ; switch (color) { case RED -> x = 0 ; case BLUE -> x = 1
>> ;
>> }
>> System. out .println(x); // x may not be initialized
>> Rémi
>>> On 1/25/2022 2:46 PM, Brian Goetz wrote:
>>>> We’ve previewed patterns in switch for two rounds, and have received some
>>>> feedback. Overall, things work quite well, but there were a few items
>>>> which
>>>> received some nontrivial feedback, and I’m prepared to suggest some changes
>>>> based on them. I’ll summarize them here and create a new thread for each
>>>> with
>>>> a more detailed description.
>>>> I’ll make a call for additional items a little later; for now, let’s focus
>>>> on
>>>> these items before adding new things (or reopening old ones.)
>>>> 1. Treatment of total patterns in switch / instanceof
>>>> 2. Positioning of guards
>>>> 3. Type refinements for GADTs
>>>> 4. Diamond for type patterns (and record patterns)
Re: [External] : Re: Reviewing feedback on patterns in switch
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Wednesday, February 16, 2022 3:57:39 PM
> Subject: Re: [External] : Re: Reviewing feedback on patterns in switch
> OK, I'll make you a deal: I'll answer your question about let/bind, under the
> condition that we not divert the discussion on that right now -- there'll be a
> proper writeup soon. The answer here is entirely for context.
> If you don't agree, stop reading now :)
I think it's wiser to delay the discussion about let for later :)
Rémi
> On 2/15/2022 5:58 PM, Remi Forax wrote:
>>> - There are future constructs that may take patterns, and may (or may not)
>>> want
>>> to express guard-like behavior, such as `let` statements (e.g., let .. when
>>> ..
>>> else.) Expressing guards here with && is even less evocative of "guard
>>> condition" than it is with switches.
>> It's not clear to me how to use "let when else". Is it more like a ?: in C
>> than
>> the let in in Caml ?
> The simplest form of `let` is a statement that takes a total pattern:
> let Point(var x, var y) = aPoint;
> and introduces bindings x and y into the remainder of the block. When
> applicable, this is better than a conditional context because (a) you get type
> checking for totality, and (b) you don't indent the rest of your method inside
> a test that you know will always succeed.
> If the pattern is total but has some remainder, the construct must throw on
> the
> remainder, to preserve the invariant that when a `let` statement completes
> normally, all bindings are DA.
> What if I want to use a partial pattern, and then customize either the
> throwing
> part or provide default values? I can provide an else clause:
> Object o = ...
> let String s = o
> else throw new NotStringException();
> or
> Object o = ...
> let String s = o
> else { s = "no string"; }
> These are two ways to preserve the "DA on normal completion" invariant; either
> by not completing normally, or by ensuring the bindings are DA.
> Now, we are in a situation where we are with switch: patterns do not express
> all
> possible conditions. Which is why we introduced guards to switches. And we can
> use the same trick here:
> Object o = ...
> let String s = o
> when (!s.isEmpty())
> else { s = "no string"; }
> If we tried to use && here, it would look like
> Object o = ...
> let String s = o && (!s.isEmpty())
> else { s = "no string"; }
> which has the same problem as `case false && false`.
> Reminder: THIS EXPLANATION WAS PROVIDED SOLELY TO CLARIFY THE "FUTURE
> CONSTRUCT"
> COMMENT IN THE && DISCUSSION.
Re: [External] : Re: Reviewing feedback on patterns in switch
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Wednesday, February 16, 2022 3:48:14 PM > Subject: Re: [External] : Re: Reviewing feedback on patterns in switch >> Not sure it's a no-brainer. >> The question is more a question of consistency. There are two consistencies >> and >> we have to choose one, either switch never allows null by default and users >> have to opt-in with case null or we want patterns to behave the same way if >> they are declared at top-level or if they are nested. I would say that the >> semantics you propose is more like the current Java and the other semantics >> is >> more like the Java of a future (if we choose the second option). > You are right that any justification involving "for consistency" is mostly a > self-justification. But here's where I think this is a cleaner decomposition. > We define the semantics of the patterns in a vacuum; matching is a three-place > predicate involving a static target type, a target expression, and a pattern. > Null is not special here. (This is how we've done this all along.) > Pattern contexts (instanceof, switch, and in the future, nested patterns, > let/bind, catch, etc) on the other hand, may have pre-existing (and in some > cases reasonable) opinions about null. What's new here is to fully separate > the > construct opinions about special values from the pattern semantics -- the > construct makes its decision about the special values, before consulting the > pattern. > This lets instanceof treat null as valid but say "null is not an instance of > anything", past-switch treats null as always an error, and future-switch > treats > null as a value you can opt into matching with the `null` label. (Yes, this is > clunky; if we had non-nullable type patterns, we'd get there more directly.) > But the part that I think is more or less obvious-in-hindsight is that the > switch opinions are switches opinions, and the pattern opinions are pattern > opinions, and there is a well-defined order in which those opinions are acted > on -- the construct mediates between the target and the patterns. That is, we > compose the result from the construct semantics and-then the pattern > semantics. I think it will be more clear when we will introduce patterns on local variable declaration because those will only allow some patterns but not all. > None of this is really all that much about "how do people like it". But what I > do think people will like is that they get a simple rule out of switches: > "switches throw on null unless the letters n-u-l-l appear in the switch body". > And a simple rule for instanceof: "instanceof never evaluates to true on > null". > And that these rules are *independent of patterns*. So switch and instanceof > can be understood separately from patterns. It's not about how people like it but how people rationalize it. You can say "switches throw on null unless the letters n-u-l-l appear in the switch body" or "switches throw on null unless a null-friendly pattern appear in the switch body and this is also true for nested patterns". Both are valid approach. Rémi
Re: Reviewing feedback on patterns in switch
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Wednesday, February 16, 2022 4:49:19 PM
> Subject: Re: Reviewing feedback on patterns in switch
> One thing that we have, perhaps surprisingly, *not* gotten feedback on is
> forcing all non-legacy switches (legacy type, legacy labels, statement only)
> to
> be exhaustive. I would have thought people would complain about pattern
> switches needing to be exhaustive, but no one has! So either no one has tried
> it, or we got away with it...
Yes, we had several feedbacks about the opposite, why the switch statement on
an enum is not exhaustive, i.e. why the following code does not compile
enum Color { RED , BLUE }
int x;
Color color = null ;
switch (color) {
case RED -> x = 0 ;
case BLUE -> x = 1 ;
}
System. out .println(x); // x may not be initialized
Rémi
> On 1/25/2022 2:46 PM, Brian Goetz wrote:
>> We’ve previewed patterns in switch for two rounds, and have received some
>> feedback. Overall, things work quite well, but there were a few items which
>> received some nontrivial feedback, and I’m prepared to suggest some changes
>> based on them. I’ll summarize them here and create a new thread for each
>> with
>> a more detailed description.
>> I’ll make a call for additional items a little later; for now, let’s focus on
>> these items before adding new things (or reopening old ones.)
>> 1. Treatment of total patterns in switch / instanceof
>> 2. Positioning of guards
>> 3. Type refinements for GADTs
>> 4. Diamond for type patterns (and record patterns)
Re: Reviewing feedback on patterns in switch
> From: "Brian Goetz"
> To: "Guy Steele" , "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Wednesday, February 16, 2022 4:05:35 PM
> Subject: Re: Reviewing feedback on patterns in switch
>>> For me, && is more natural than "when" because i've written more switch that
>>> uses && than "when".
>>> And don't forget that unlike most of the code, with pattern matching the
>>> number
>>> of characters does matter, this is more similar to lambdas, if what you
>>> write
>>> is too verbose, you will not write it.
>> At the risk of premature bikeshedding, have we already discussed and
>> discarded
>> the idea of spelling “when” as “if”? It’s been a long time, and I forget.
> There was not extensive discussion on this, and its all very
> subjective/handwavy/"what we think people would think", but I remember a few
> comments on this:
> - The generality of "if" reminded people of the Perl-style "statement unless
> condition" postfix convention, and that people might see it as an
> "inconsistency" that they could not then say
> x = 3 if (condition);
> which is definitely somewhere we don't want to go.
> - We're use to seeing "if" with a consequence, and a "naked" if might have the
> effect of "lookahead pollution" in our mental parsers.
> - Keeping `if` for statements allows us to keep the "body" of case clauses
> visually distinct from the "envelope":
> case Foo(var x)
> if (x > 3) : if (x > 10) { ... }
> would make people's eyes go buggy. One could argue that "when" is not
> fantastically better:
> case Foo(var x)
> when (x > 3) : if (x > 10) { ... }
> but it doesn't take quite as long to de-bug oneself in that case.
And also the if stis followed by parenthesis and there is no need of
parenthesis for a guard.
So either people will always put parenthesis after if as a guard or be mystify
that parenthesis are not required for a guard but required for the if
statement.
Rémi
Re: Reviewing feedback on patterns in switch
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Tuesday, February 15, 2022 7:50:06 PM
> Subject: Re: Reviewing feedback on patterns in switch
> We're preparing a third preview of type patterns in switch. Normally we would
> release after a second preview, but (a) we're about to get record patterns,
> which may disclose additional issues with switch, so best to keep it open for
> at least another round, and (b) we're proposing some nontrivial changes which
> deserve another preview.
> Here's where we are on these.
>> 1. Treatment of total patterns in switch / instanceof
> Quite honestly, in hindsight, I don't know why we didn't see this sooner; the
> incremental evolution proposed here is more principled than where we were in
> the previous round; now the construct (instanceof, switch, etc) *always* gets
> first crack at enforcing its nullity (and exception) opinions, and *then*
> delegates to the matching semantics of the pattern if it decides to do so.
> This
> fully separates pattern semantics from conditional construct semantics, rather
> than complecting them (which in turn deprived users of seeing the model more
> clearly.) In hindsight, this is a no-brainer (which is why we preview things.)
> We'll be addressing this in the 3rd preview.
Not sure it's a no-brainer.
The question is more a question of consistency. There are two consistencies and
we have to choose one, either switch never allows null by default and users
have to opt-in with case null or we want patterns to behave the same way if
they are declared at top-level or if they are nested. I would say that the
semantics you propose is more like the current Java and the other semantics is
more like the Java of a future (if we choose the second option).
I think we should try the semantics you propose and see if people agree or not.
>> 2. Positioning of guards
> Making guards part of switch also feels like a better factoring than making
> them
> part of patterns; it simplifies patterns and totality, and puts switch on a
> more equal footing with our other conditional constructs. We did go back and
> forth a few times on this, but having given this a few weeks to settle, I'm
> pretty convinced we'd regret going the other way.
> There were two sub-points here: (a) is the guard part of the pattern or part
> of
> switch, and (b) the syntax. There was general agreement on (a), but some had
> preference for && on (b). I spent some more time thinking about this choice,
> and have come down firmly on the `when` side of the house as a result for a
> number of reasons.
Still agree on (a)
> - Possibility for ambiguity. If switching over booleans (which we will surely
> eventually be forced into), locutions like `case false && false` will be very
> confusing; it's pure puzzler territory.
> - && has a stronger precedence than keyword-based operators like
> `instanceof`';
> we want guards to be weakest here.
I don't understand your point, we want instanceof pattern && expression to be
equivalent to instanceof type && expression + cast, so the fact that && has a
stronger precedence makes that possible so it's not an issue.
> - Using && will confuse users about whether it is part of the expression, or
> part of the switch statement. If we're deciding it is part of the switch, this
> should be clear, and a `when` clause makes that clear.
I don't think it's that important, apart if we start to also want to combine
patterns with &&
> - There are future constructs that may take patterns, and may (or may not)
> want
> to express guard-like behavior, such as `let` statements (e.g., let .. when ..
> else.) Expressing guards here with && is even less evocative of "guard
> condition" than it is with switches.
It's not clear to me how to use "let when else". Is it more like a ?: in C than
the let in in Caml ?
> - Users coming from other languages will find `case...when` quite clear.
> - We've talked about "targetless" switches as a possible future feature, which
> express multi-way conditionals:
> switch {
> case when (today() == TUESDAY): ...
> case when (location() == GREENLAND): ...
> ...
> }
> This would look quite silly with &&.
For me, this is like cond in Lisp but more verbose. using "case" and "when"
here is sillly.
> Similarly, one could mix guards with a targeted switch:
> switch (x) {
> case Time t: ...
> case Place p: ...
> default when (today() == TUESDAY): ... tuesday-specific default
> default: ... regular default ...
default && today() == TUESDAY is fine for me.
> Expressing guards that are the whole condition with `when` is much more
> natural
> than with &&.
For me, && is more natural than "when" because i've written more switch that
uses && than "when".
And don't forget that unlike most of the code, with pattern matching the number
of characters does matter, this is more similar to lambdas, if what you write
is too verbose, you will not write it.
> tl;dr: inventing a `when` modifier on switch no
Re: [External] : Re: JEP 405 update
- Original Message - > From: "Brian Goetz" > To: "Remi Forax" , "Gavin Bierman" > > Cc: "amber-spec-experts" > Sent: Thursday, February 10, 2022 6:12:51 PM > Subject: Re: [External] : Re: JEP 405 update >>> That’s really what I am getting at. In that sense, record patterns are a >>> key, >>> because they >>> support nesting of patterns. You are correct that the construct within >>> which we >>> surface pattern matching can have a say in the top-level treatment of null. >> It's not what i've said, the treatment of null is due to the fact that a type >> pattern (at least the non total one) is semantically an instanceof thus >> reject >> null. > > We have been over this over and over again. This is not right, and you > are confusing people. I think you misunderstood me, it's my fault i've used the term "reject" instead of "not match", i was not talking about NPE here but the fact that both non total type pattern and record pattern does not match null. The fact that the switch can throw a NPE or not is a property of the record pattern, that was my point. Rémi
Re: [External] : Re: JEP 405 update
- Original Message -
> From: "Gavin Bierman"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Wednesday, February 9, 2022 11:11:56 PM
> Subject: Re: [External] : Re: JEP 405 update
> Thanks Remi. Yes, this text could be better (interestingly, it is unchanged
> from
> nearly a year ago!).
I believe it's a direct consequence of this JEP being more readable now that it
was before.
It was covering too mush ground at too high level.
> I think you’d agree that *pattern matching* with nested
> patterns provides a null-safe approach to data access, right?
Not more than the type pattern
Object o = ...
switch(o) {
case Bar bar -> // bar can not be null here
default -> ...
}
or
Object o = ...
if o instanceof Bar bar) {
// bar can not be null here
}
In a sense, the record pattern is an "extension" of the type pattern so like
the type pattern it does not allow null (if not total).
BTW, the record pattern is not an extension of the type pattern if the type is
total,
It's another clue that the type pattern and totality are two different concepts.
We would not have such issue if the type pattern was separated into two kind of
patterns, the type pattern (case Foo foo) that requires an explicit type and
the var pattern (case var foo) that is total (again the C# solution).
> That’s really what I am getting at. In that sense, record patterns are a key,
> because they
> support nesting of patterns. You are correct that the construct within which
> we
> surface pattern matching can have a say in the top-level treatment of null.
It's not what i've said, the treatment of null is due to the fact that a type
pattern (at least the non total one) is semantically an instanceof thus reject
null.
> Indeed, one of the things we are discussing at the moment is exactly whether
> switch should have a stronger opinion about top-level null!
yes i know, and i find that idea weird, we should try to make the patterns to
behave the same way whatever the container, to not bother users with to many
details.
>
> Let me see if I can work on the text some more.
>
> Thanks for the feedback,
> Gavin
Rémi
>
>> On 9 Feb 2022, at 16:40, Remi Forax wrote:
>>
>> Hi Gavin,
>> I don't buy the argument that record patterns promote a null-safe style of
>> programming as this is stated several times in the JEP.
>>
>> The ""null-safety"" (notice the air quotes) mostly comes from the instanceof
>> or
>> the switch (which semantics is equivalent of a cascade of if instanceof), not
>> from the record pattern by itself.
>>
>> You can argue that when a record pattern is nested a nullcheck appears, but
>> it's
>> more than the underlying semantics is a degenerated instanceof when the
>> declared type and the instanceof type are the same.
>>
>> The record pattern is about destructuring after the instanceof/nullcheck has
>> been done, so i find that argument counter productive because it does not
>> help
>> to understand the semantics.
>>
>> Also, we have talked several times to introduce the record pattern when
>> doing an
>> assignment
>> Point point = ...
>> Point(int x, int y) = point;
>> // can use x and y here !
>>
>> This will throw a NPE if point is null, similarly to an unboxing operation.
>>
>> The null-safety is not attached to the record pattern per se but by the
>> container that use it (instanceof, case of a switch, enclosing pattern).
>>
>> regards,
>> Rémi
>>
>> - Original Message -
>>> From: "Gavin Bierman"
>>> To: "amber-spec-experts"
>>> Sent: Wednesday, February 9, 2022 12:59:02 PM
>>> Subject: JEP 405 update
>>
>>> Dear experts,
>>>
>>> Just to let you know that I have updated JEP 405:
>>>
>>> https://openjdk.java.net/jeps/405
>>>
>>> You will see that we have removed the array patterns from this JEP (and it
>>> has
>>> been retitled accordingly). We're still committed to supporting a direct
>>> pattern
>>> form for arrays, but given our tight schedule and a number of queries about
>>> the
>>> exact syntactic form for array patterns, we think we'll be better off
>>> decoupling
>>> them from JEP 405, and releasing them in a future patterns JEP.
>>>
>>> Comments welcomed!
> >> Gavin
Re: JEP 405 update
Hi Gavin, I don't buy the argument that record patterns promote a null-safe style of programming as this is stated several times in the JEP. The ""null-safety"" (notice the air quotes) mostly comes from the instanceof or the switch (which semantics is equivalent of a cascade of if instanceof), not from the record pattern by itself. You can argue that when a record pattern is nested a nullcheck appears, but it's more than the underlying semantics is a degenerated instanceof when the declared type and the instanceof type are the same. The record pattern is about destructuring after the instanceof/nullcheck has been done, so i find that argument counter productive because it does not help to understand the semantics. Also, we have talked several times to introduce the record pattern when doing an assignment Point point = ... Point(int x, int y) = point; // can use x and y here ! This will throw a NPE if point is null, similarly to an unboxing operation. The null-safety is not attached to the record pattern per se but by the container that use it (instanceof, case of a switch, enclosing pattern). regards, Rémi - Original Message - > From: "Gavin Bierman" > To: "amber-spec-experts" > Sent: Wednesday, February 9, 2022 12:59:02 PM > Subject: JEP 405 update > Dear experts, > > Just to let you know that I have updated JEP 405: > > https://openjdk.java.net/jeps/405 > > You will see that we have removed the array patterns from this JEP (and it has > been retitled accordingly). We're still committed to supporting a direct > pattern > form for arrays, but given our tight schedule and a number of queries about > the > exact syntactic form for array patterns, we think we'll be better off > decoupling > them from JEP 405, and releasing them in a future patterns JEP. > > Comments welcomed! > Gavin
Re: Control flow analysis for exhaustive pattern-switch statement
> From: "Brian Goetz"
> To: "Tagir Valeev" , "amber-spec-experts"
>
> Sent: Thursday, February 3, 2022 5:35:13 PM
> Subject: Re: Control flow analysis for exhaustive pattern-switch statement
> Thanks Tagir for digging into this. This is good analysis.
> The decision to require completeness of enhanced switches was taken relatively
> late, so its quite possible we have left something out. And in the next round,
> when nested patterns come along, we need to formalize totality and remainder,
> which also intersects with this problem. So this is timely.
[...]
>> I understand that the same reasoning does not apply for switch over
>> enums, as for compatibility reasons, default behavior is to do
>> nothing. However, for patterns, uninitialized `x` cannot appear after
>> the switch, even if we recompile the sealed interface separately
>> adding one more inheritor.
> Slight correction: *statement* switches over enums. We've carved out a place
> where the only switches that are not total, are *statement* switches over the
> legacy types with the legacy switch labels. In that case, DU analysis picks up
> some of the slack.
> (It is still on our "to be considered" list whether it is worth it to allow
> statement switches to be explicitly marked as total to engage greater
> typechecking, or whether we want to embark on the decade-long path of warning
> increasingly loudly on non-total switches until we eventually make them
> illegal.)
Both options are not exclusive, we can have a switch marked as exhaustive AND
have a warning to nudge people to the "right" path.
I believe we should mark a way to mark switch as total but i would prefer to
have a syntax which is not something like total-switch which to me is a
regression given that we have decided to rehabilitate the switch instead of
using another keyword (what's Brian called the snitch approach).
If we try to summarize the approachs we have discussed, we have
- use a new keyword like "total-switch"
- use a new keyword inside the parenthesis of the switch, switch(total value) {
... }
- use default -> throw new AnException()
- use default -> throw; as a shortcut
The two later propositions are not good because adding a "default" make the
switch exhaustive but we are lose the typechecking when a new enum constant is
introduced.
The two former propositions are not good because it's a snitch in disguise.
I believe a good compromise might be to have a syntax inside the curly braces
like "default -> " but not using "default" because it's not the semantics we
want.
It just occurs to me that we can use an ellipsis (...) for that
switch(option) {
case READ -> { }
case WRITE -> { }
case READ_WRITE -> { }
... // this is a real syntax, it means that the switch is total
}
An ellipsis is an opt-in way to ask for an exhaustive checking, it will fail at
compile time if one of the constants is not listed, and in case of separate
compilation at runtime an error will be thrown.
[...]
> Thanks,
> -Brian
Rémi
Re: [External] : Re: Treatment of total patterns (was: Reviewing feedback on patterns in switch)
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "Tagir Valeev" , "amber-spec-experts" > > Sent: Thursday, January 27, 2022 4:41:27 PM > Subject: Re: [External] : Re: Treatment of total patterns (was: Reviewing > feedback on patterns in switch) >> In that case, i prefer the current semantics because it's the same if a >> pattern >> is a top-level or not. > I wish people could keep these things straight. We’re not talking about > changing > the semantics of how pattern matching works, which patterns match what, what > nesting means, etc. We’re simply talking about the *boundary* between a > specific pattern-accepting construct, which has pre-existing value filtering > opinions, and the patterns it accepts. > The current (preview) boundary says: > - If a switch has a `case null`, or a total pattern, a null value matches > that, > otherwise we throw NPE on null, and for non-null, it is matched to the > patterns > in the case labels. > The adjusted boundary says: > - If a switch has a `case null`, a null value matches that, otherwise we throw > NPE on null, and for non-null, it is matched to the patterns in the case > label. > So this adjusts *which* patterns the switch lets see null values. Previously, > it > was “none”; in the current preview, it is “case null and total patterns”, and > the adjustment proposed is “case null”. The latter is a tradeoff to avoid > confusing the users, who currently believe switch always throws on null, by > saying “switch accepts null if it says case null.” > We currently have a similar problem with `intsnaceof`, where we disallow total > patterns on the RHS of instanceof. We would adjust in the same way: instanceof > always says false on nulls, and tests against the RHS on non-null. > Nothing to do with the semantics of pattern matching. Total patterns are still > total. You can say you only change the semantics of switch not the semantics of pattern matching, but the idea that you can separate the two is confusing. For me, the semantics of pattern matching change because currently a total pattern always match null, whatever its position, as top-level or inside a record pattern (for example), with the semantics you propose a top-level pattern will not match null anymore but will match null if nested. So yes, i suppose you can say that the semantics of a total pattern is not changed because whatever the position it *can* match null, but 'm not sure this way of thinking helps. To make thing super clear, with the current semantics, "case Object o" always match null, with your proposal, the answer is it depends if it is nested or not. That's why i prefer the current semantics. regards, Rémi PS: the feedback about the fact that it's not clear if a switch allows null or not can also be seen as a symptom of the fact that the notion of total pattern is not obvious for everybody (and having no syntax hint does not help).
Re: Reviewing feedback on patterns in switch
- Original Message - > From: "mark" > To: "Brian Goetz" > Cc: "amber-spec-experts" > Sent: Friday, January 28, 2022 12:43:55 PM > Subject: Re: Reviewing feedback on patterns in switch > On 2022-01-25T19:46:09 + > Brian Goetz wrote: > >> We’ve previewed patterns in switch for two rounds, and have received some >> feedback. Overall, things work quite well, but there were a few items which >> received some nontrivial feedback, and I’m prepared to suggest some changes >> based on them. I’ll summarize them here and create a new thread for each >> with >> a more detailed description. >> >> I’ll make a call for additional items a little later; for now, let’s focus on >> these items before adding new things (or reopening old ones.) >> >> 1. Treatment of total patterns in switch / instanceof >> >> 2. Positioning of guards >> >> 3. Type refinements for GADTs >> >> 4. Diamond for type patterns (and record patterns) > > Hello! > > I'm a little late to the party, as ever, but is there a specific build I > should be looking at so that I can get a better idea of what the current > state of things are? Hi Mark, the last jdk18 build or any new jdk19 builds will do. > > -- > Mark Raynsford | https://www.io7m.com Rémi
Re: [External] : Re: Diamond in type patterns (was: Reviewing feedback on patterns in switch)
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Thursday, January 27, 2022 2:04:35 PM > Subject: Re: [External] : Re: Diamond in type patterns (was: Reviewing > feedback > on patterns in switch) >> It's more an engineering thing here, we have far more casts than switch + >> pattern in existing code, and given that we suppose (perhaps wrongly) that >> the >> semantics of the inference is not exactly one already existing, > I’d like to drill into this supposition. My supposition (maybe wrong) is that > we > already solved most of this when we did `var`, with upward projection. > To recap, we spent a lot of time with `var` on what to do about non-denotable > types. These included the null type (banned on the grounds of uselessness), > intersection types (allowed), and capture types (sanitized with upward > projection.) The basic idea of upward projection is that when we infer > List, we replace it with a super type that has no capture types, and get > List out. (There’s also a downward projection.) > Let’s start with your examples of where ordinary inference produces an > undesirable result, and then evaluate whether either or the projections solves > the problem? I don't think current projections are enough because we may want the inference to insert a wildcard by itsef, for example with Object o = ... var list = (List<>) o; or maybe we should not try to infer such code. Rémi
Re: [External] : Re: Diamond in type patterns (was: Reviewing feedback on patterns in switch)
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Wednesday, January 26, 2022 3:34:21 PM > Subject: Re: [External] : Re: Diamond in type patterns (was: Reviewing > feedback > on patterns in switch) >> I think we should figure out how it should work on cast and then we can >> happily >> applied it on patterns. > I’m happy to have the cast discussion happen concurrently, but right now, my > priority is on patterns, as we’re already two previews into > patterns-in-switch. remember, the move from preview to real feature should be only when we are ready > But I’m not ready to say “we can’t solve this for patterns unless we also > solve > it for cast RIGHT NOW. So I agree with the goal (solve it everywhere, > eventually) but not with the ordering constraint. It's more an engineering thing here, we have far more casts than switch + pattern in existing code, and given that we suppose (perhaps wrongly) that the semantics of the inference is not exactly one already existing, i think we will get better result if we try to automatically transforms all existing casts using a parametrized type to diamond casts and see when the inference fails and why. Also this feature is fully orthogonal with the rest of the patterns because the diamond syntax in type pattern is an invalid syntax, so this feature can have it's own tempo. >> despite the syntax being the same, the diamond syntax, i don't think we can >> reuse the same inference rules between the new diamond and the cast diamond. > Understood. (This is why, for example, we introduced upward and downward > projection when we did var, because the rules for inference were not what we > wanted for var.) But before we go on to the details, are we agreed on the > goal? Agree on the goal, but do you agree on the methodology i propose above. Rémi
Re: [External] : Re: Treatment of total patterns (was: Reviewing feedback on patterns in switch)
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "Tagir Valeev" , "amber-spec-experts"
>
> Sent: Wednesday, January 26, 2022 3:08:39 PM
> Subject: Re: [External] : Re: Treatment of total patterns (was: Reviewing
> feedback on patterns in switch)
> I don’t think its helpful to try and reopen these old and settled issues. I
> get
> that you think null should have a larger syntactic presence in the language,
> and you’ve made those points plenty of times, but we’re not reopening whether
> `Object o` is total, or whether `var` is more than type inference. We’re
> focused here on the interaction between switch and patterns, precisely because
> switch comes to the table with pre-existing null hostilities. We are not going
> to distort the semantics of pattern matching just so we can extrapolate from
> how C switch worked; we’ve been over this too many times.
In that case, i prefer the current semantics because it's the same if a pattern
is a top-level or not.
Rémi
>> On Jan 26, 2022, at 8:45 AM, [ mailto:fo...@univ-mlv.fr |
>> fo...@univ-mlv.fr ] wrote:
>>> From: "Brian Goetz" < [ mailto:brian.go...@oracle.com |
>>> brian.go...@oracle.com ]
>>> >
>>> To: "Remi Forax" < [ mailto:fo...@univ-mlv.fr | fo...@univ-mlv.fr ] >
>>> Cc: "Tagir Valeev" < [ mailto:amae...@gmail.com | amae...@gmail.com ] >,
>>> "amber-spec-experts" < [ mailto:amber-spec-experts@openjdk.java.net |
>>> amber-spec-experts@openjdk.java.net ] >
>>> Sent: Wednesday, January 26, 2022 1:47:38 PM
>>> Subject: Re: [External] : Re: Treatment of total patterns (was: Reviewing
>>> feedback on patterns in switch)
>>> Heh, you are incrementally rediscovering exactly why we chose the original
>>> “total is total” rule; of all the possible treatments, it is the most
>>> logically
>>> consistent. Welcome.
>>> In this case, however, switches must be total. So here, either D is total
>>> (perhaps with remainder), or B/C/D cover whatever the content of Box is, or
>>> it
>>> doesn’’t compile. If there is remainder (which is likely to be null, but
>>> which
>>> won’t happen with a type pattern, only when D is more complicated), and no
>>> later case handles Box(null), then the switch will NPE. We don’t know if
>>> Box(null) is matched by any of these cases, but we *do* know that we will
>>> not
>>> arrive at the statement after the switch if the target was Box(null).
>> It's true that if you can observe the different side effects when the code is
>> run, and from that you may have an idea if case Box(D d) matches or not (and
>> prey that there is not a catch() in the middle),
>> but the bar is very low if you say that to understand a code you have to run
>> it.
>>> The proposed change to top-level null hostility doesn’t affect that.
>> yes, that my point, having to run a code to understand it is a clue that the
>> semantics you propose or the Java 18 one are both equally bad.
>> Again, the C# semantics does not have such problem, if we suppose that the
>> code
>> compiles then with the code below, d can not be null
>> switch(box) {
>> case Box(B b) -> { }
>> case Box(C c) -> { }
>> case Box(D d) -> { } // does not accept null
>> }
>> while with this code, d can be null
>> switch(box) {
>> case Box(B b) -> { }
>> case Box(C c) -> { }
>> case Box(var d) -> { } // accept null
>> }
>> Rémi
>>>> On Jan 26, 2022, at 2:53 AM, Remi Forax < [ mailto:fo...@univ-mlv.fr |
>>>> fo...@univ-mlv.fr ] > wrote:
>>>> We should go a step further, this also means that with
>>>> switch(box) {
>>>> case Box(B b) -> {}
>>>> case Box(C c) -> {}
>>>> case Box(D d) -> {}
>>>> }
>>>> we have no idea if the switch will accept Box(null) or not.
>>>> So the idea that a type behave differently if nested inside a pattern or
>>>> not is
>>>> not a good one.
Re: [External] : Re: Patterns and GADTs (was: Reviewing feedback on patterns in switch)
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Wednesday, January 26, 2022 3:30:45 PM
> Subject: Re: [External] : Re: Patterns and GADTs (was: Reviewing feedback on
> patterns in switch)
> I don’t object to having something explicit in the code, but I do object to
> having that be unchecked. In the original example:
>> T unbox(Node n) {
>> return switch (n) {
>> case A n -> n.t;
>> case B n -> n.s;
>> };
>> }
> we could cast `n,s` to T, but the compiler would have no reason to believe
> that
> this is valid, so it would give us an unchecked warning. But the reality is,
> we
> do have enough information to validate this. Now, I’m not sure if users would
> be any happier about
> case B n -> (T) n.s
> even if they did not get an unchecked warning. Still, a cast is probably
> better
> than new, narrowly targeted syntax here.
> If we’re diffident about refining the type of T, we could consider an implicit
> conversion (String can be converted to T in a context where we’ve gathered the
> appropriate constraints on T), but this is more complicated, and I’m not sure
> users will find it any more understandable. Refining the type is something
> that
> will make more sense to the user (“I know T is String here!”) than complex
> rules about when we can funge T and String.
I agree, for the compiler, it should be like adding a constraint T = String.
The conversion is an equivalent of String <: T which is only half true.
Let start without requiring a cast and see if it's too magical or not.
Rémi
>> On Jan 26, 2022, at 9:16 AM, [ mailto:fo...@univ-mlv.fr |
>> fo...@univ-mlv.fr ] wrote:
>>> From: "Brian Goetz" < [ mailto:brian.go...@oracle.com |
>>> brian.go...@oracle.com ]
>>> >
>>> To: "Remi Forax" < [ mailto:fo...@univ-mlv.fr | fo...@univ-mlv.fr ] >
>>> Cc: "amber-spec-experts" < [ mailto:amber-spec-experts@openjdk.java.net |
>>> amber-spec-experts@openjdk.java.net ] >
>>> Sent: Wednesday, January 26, 2022 1:28:19 PM
>>> Subject: Re: [External] : Re: Patterns and GADTs (was: Reviewing feedback on
>>> patterns in switch)
>>>> The instanceof example is not a source backward compatible change,
>>>> remember that
>>>> instanceof is not a preview feature.
>>> I’m well aware, can you give an example where flow typing of *type variables
>>> only* might lead to incompatibility? (I’m aware that this is a possibility,
>>> but
>>> you’re stating it like its a fact already on the table.) For example, where
>>> it
>>> would change overload selection (this is where flow typing for variables
>>> falls
>>> down, among other places.)
>> sure,
>> sealed interface Node { }
>> record A(T t) implements Node { }
>> record B(String s) implements Node { }
>> void foo(Object o) { }
>> void foo(List list) { }
>> void m() {
>> if (node instanceof B b) {
>> foo(List.of());
>> }
>> }
>> Rémi
Feedback: refining the inference of yield type in a switch
Here are several examples showing that the inference does work as expected from
a beginner POV,
the first two examples come from one of my student.
Here, list is typed as List instead of List
var value = ...
var list = switch (value) {
case 0 -> List.of();
case 1 -> List.of("foo");
default -> throw new AssertionError();
};
if we add a cast in front of to try to help inference
var list = (List) switch (value) {
case 0 -> List.of();
case 1 -> List.of("foo");
default -> throw new AssertionError();
};
the inference does not care, so it becomes an unsafe cast.
Declaring the type explicitly works
List list = switch (value) {
case 0 -> List.of();
case 1 -> List.of("foo");
default -> throw new AssertionError();
};
Specifying the type argument for List.of() also works.
var list = switch (value) {
case 0 -> List.of();
case 1 -> List.of("foo");
default -> throw new AssertionError();
};
To summarize, the inference considers each yield type one by one instead of all
together (each one adding it's own constraints).
I think we should refine the way the inference work so "it just works" in this
kind of case.
regards,
Rémi
Re: [External] : Re: Diamond in type patterns (was: Reviewing feedback on patterns in switch)
> From: "Brian Goetz" > To: "Remi Forax" > Cc: "amber-spec-experts" > Sent: Wednesday, January 26, 2022 1:23:04 PM > Subject: Re: [External] : Re: Diamond in type patterns (was: Reviewing > feedback > on patterns in switch) >> The questions we did not answer the last time we talk about that subject >> - why should we allow raw types here ? > We already have a clear precedent with type patterns in instanceof — which is > not a preview feature any more. So for one, now you’re talking about making a > *change* to the existing language semantics. There are other concerns too. >> - given that this is equivalent to an instanceof + cast, why we can not use >> diamond inference on cast ? > You’re not being clear about what you’re saying, you could be saying either of > the following (or others): > - You’re proposing diamond here, but not there, then your proposal is > inconsistent, and therefore stupid. > - I love your proposal, but I think we should additionally talk about other > places to use diamond as well. > I can’t tell which of these you’re saying, or maybe its something else? I think we should figure out how it should work on cast and then we can happily applied it on patterns. >> - how this inference work ? Is is the same inference than with the diamond >> constructor ? > Again, I can’t tell whether you’re saying “this is dumb, it can’t work”, or > “this is great, but I can’t figure out the details.” The rules for what you can use as argument of a type parameter when doing a new and when doing a cast are not the same, for examples, new ArrayList() is not a valid code while (ArrayList) is a perfect valid cast, new ArrayList[3] is not a valid code while (ArrayList[]) may or may not be a valid cast, new ArrayList() is a valid code while (ArrayList) may or may not be a valid cast . despite the syntax being the same, the diamond syntax, i don't think we can reuse the same inference rules between the new diamond and the cast diamond. regards, Rémi
Re: [External] : Re: Patterns and GADTs (was: Reviewing feedback on patterns in switch)
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "amber-spec-experts"
> Sent: Wednesday, January 26, 2022 1:28:19 PM
> Subject: Re: [External] : Re: Patterns and GADTs (was: Reviewing feedback on
> patterns in switch)
>> The instanceof example is not a source backward compatible change, remember
>> that
>> instanceof is not a preview feature.
> I’m well aware, can you give an example where flow typing of *type variables
> only* might lead to incompatibility? (I’m aware that this is a possibility,
> but
> you’re stating it like its a fact already on the table.) For example, where it
> would change overload selection (this is where flow typing for variables falls
> down, among other places.)
sure,
sealed interface Node { }
record A(T t) implements Node { }
record B(String s) implements Node { }
void foo(Object o) { }
void foo(List list) { }
void m() {
if (node instanceof B b) {
foo(List.of());
}
}
Rémi
Re: [External] : Re: Treatment of total patterns (was: Reviewing feedback on patterns in switch)
> From: "Brian Goetz"
> To: "Remi Forax"
> Cc: "Tagir Valeev" , "amber-spec-experts"
>
> Sent: Wednesday, January 26, 2022 1:47:38 PM
> Subject: Re: [External] : Re: Treatment of total patterns (was: Reviewing
> feedback on patterns in switch)
> Heh, you are incrementally rediscovering exactly why we chose the original
> “total is total” rule; of all the possible treatments, it is the most
> logically
> consistent. Welcome.
> In this case, however, switches must be total. So here, either D is total
> (perhaps with remainder), or B/C/D cover whatever the content of Box is, or it
> doesn’’t compile. If there is remainder (which is likely to be null, but which
> won’t happen with a type pattern, only when D is more complicated), and no
> later case handles Box(null), then the switch will NPE. We don’t know if
> Box(null) is matched by any of these cases, but we *do* know that we will not
> arrive at the statement after the switch if the target was Box(null).
It's true that if you can observe the different side effects when the code is
run, and from that you may have an idea if case Box(D d) matches or not (and
prey that there is not a catch() in the middle),
but the bar is very low if you say that to understand a code you have to run
it.
> The proposed change to top-level null hostility doesn’t affect that.
yes, that my point, having to run a code to understand it is a clue that the
semantics you propose or the Java 18 one are both equally bad.
Again, the C# semantics does not have such problem, if we suppose that the code
compiles then with the code below, d can not be null
switch(box) {
case Box(B b) -> { }
case Box(C c) -> { }
case Box(D d) -> { } // does not accept null
}
while with this code, d can be null
switch(box) {
case Box(B b) -> { }
case Box(C c) -> { }
case Box(var d) -> { } // accept null
}
Rémi
>> On Jan 26, 2022, at 2:53 AM, Remi Forax < [ mailto:fo...@univ-mlv.fr |
>> fo...@univ-mlv.fr ] > wrote:
>> We should go a step further, this also means that with
>> switch(box) {
>> case Box(B b) -> {}
>> case Box(C c) -> {}
>> case Box(D d) -> {}
>> }
>> we have no idea if the switch will accept Box(null) or not.
>> So the idea that a type behave differently if nested inside a pattern or not
>> is
>> not a good one.
Re: Positioning of guards (was: Reviewing feedback on patterns in switch)
- Original Message -
> From: "Tagir Valeev"
> To: "Brian Goetz"
> Cc: "amber-spec-experts"
> Sent: Wednesday, January 26, 2022 4:50:05 AM
> Subject: Re: Positioning of guards (was: Reviewing feedback on patterns in
> switch)
> Hello!
>
> For the record: I like the current version with &&. It's short and
> easy to understand (as people already know what && means in Java). I
> see no reason in replacing it with `when`, which is more limiting.
>
>> because of the potential confusion should we ever choose to support switch
>> over
>> boolean
>
> It looks like any boolean expression that is a potentially constant
> differs syntactically from the guarded pattern, so we can distinguish
> between guarded pattern with && and boolean expression on AST level
> without resolving the references. End users will unlikely use anything
> other than explicit 'true' and 'false' constants, so it will add some
> complexity to the compiler but does not add any problems to real users
>
>> because the && did not stand out enough as turning a total pattern into a
>> partial one
>
> I think it's a matter of taste and habit. I, for one, already get used
> to it. It signals about partiality much more, compared to a simple
> type pattern. Looking at `CharSequence cs`, you cannot say whether
> it's total or not if you don't know the type of the selector
> expression. However, looking at `CharSequence cs && cs.length() > 0`
> you are completely sure it's not total. So if we need a clear signal
> to tell total and partial patterns apart, we have much bigger problems
> with type patterns.
>
>> Guarded patterns are never total
> Except when guard is a constant expression that evaluates to `true`:
>
> void test(Object obj) {
>switch (obj) { // compiles
>case Object s && true -> System.out.println(s);
>}
> }
I think we should separate the two ideas in Brian's mail,
one is should we allow a guard inside a pattern ? and the other is what is the
syntax for a guard ?
My position is that we should only allow guard in a switch, not as pattern.
And i see no problem to use "&&" instead of "when", as Tagir, i'm kind of used
to it too.
regards,
Rémi
>
> On Wed, Jan 26, 2022 at 2:49 AM Brian Goetz wrote:
>>
>> > 2. Positioning of guards
>>
>> We received several forms of feedback over the form and placement of guarded
>> patterns. Recall that we define a guarded pattern to be `P && g`, where P
>> is a
>> pattern and g is a boolean expression. Guarded patterns are never total.
>> Note
>> that we had a choice of the guard being part of the pattern, or being part of
>> the `case` label; the current status chooses the former. (Part of our
>> reasoning was that there might be other partial pattern contexts coming, and
>> we
>> didn’t want to solve this problem each time. (For instanceof, it makes no
>> difference.) )
>>
>> I am prepared to reconsider the association of the guard with the pattern,
>> and
>> instead treat it as part of the case. This is expressively weaker but may
>> have
>> other advantages.
>>
>> Additionally, people objected to the use of &&, not necessarily because
>> “keywords are better”, but because of the potential confusion should we ever
>> choose to support switch over boolean, and because the && did not stand out
>> enough as turning a total pattern into a partial one. What the alternative
>> looks like is something like:
>>
>> switch (x) {
>> case Foo(var x, var y)
>> when x == y -> A;
>> case Foo(var x, var y) -> B;
>> }
>>
>> Here, `when` (bike shed to be painted separately) is a qualifier on the case,
>> not the pattern. A total pattern with a `when` is considered a partial case.
>> This simplifies patterns, and moves the complexity of guards into switch,
>> where arguably it belongs.
>>
>> The loss of expressiveness is in not allowing nested patterns like:
>>
>> P(Q && guard)
>>
>> and instead having to move the guard to after the matching construct. Some
>> users recoiled at seeing guards inside pattern invocations; it seemed to some
>> like mixing two things that should stay separate. (For unrolling a nested
>> pattern, `case P(Q)` where Q is not total unrolls to `case Pvar alpha) when
> > alpha instanceof Q`.)
Re: Treatment of total patterns (was: Reviewing feedback on patterns in switch)
- Original Message -
> From: "Tagir Valeev"
> To: "Brian Goetz"
> Cc: "amber-spec-experts"
> Sent: Wednesday, January 26, 2022 5:20:24 AM
> Subject: Re: Treatment of total patterns (was: Reviewing feedback on patterns
> in switch)
>> Null is only matched by a switch case that includes `case null`. Switches
>> with
>> no `case null` are treated as if they have a `case null: throw NPE`. This
>> means that `case Object o` doesn’t match null; only `case null, Object o`
>> does.
>> Total patterns are re-allowed in instanceof expressions, and are consistent
>> with
>> their legacy form.
>
> I strongly support this change.
>
> In my experience, it's much more important to have automatic
> refactorings between switch and chains of 'if' than between nested and
> flat switches. People have chains of 'if's very often and they are not
> legacy. Sometimes, you want to add conditions unrelated to the
> selector expression, so it could be natural to convert 'switch' to
> 'if'. In other cases, you simplify the chain of 'if' statements and
> see that the new set of conditions nicely fits into a pattern switch.
> These if<->switch conversions will be an everyday tool for developers.
> In contrast, destructuring with a switch will be a comparatively rare
> thing, and it's even more rare when you need to convert nested
> switches to flat ones or vice versa. I'm saying this from my Kotlin
> programming experience where you can have when-is and sort of
> destructuring of data classes which are roughly similar to what we are
> doing for Java. One level 'when' is common, two-level 'when' or
> conditions on destructuring components are more rare.
>
> We already implemented some kind of switch<->if conversion in IntelliJ
> IDEA. And it already has a number of corner cases to handle in order
> to support total patterns that match null. In particular, we cannot
> convert `case Object obj` to `if (x instanceof Object obj), as total
> patterns are prohibited for instanceof and null won't be matched
> anyway. We cannot just omit a condition, as `obj` could be used
> afterwards, so we have to explicitly declare a variable (and I
> believe, this part is still buggy and may produce incompilable code).
> The proposed change will make switch<->if refactorings more mechanical
> and predictable.
>
> Another thing I mentioned before and want to stress again is that this
> change will allow us to infer required nullity for the variable used
> in the switch selector from AST only. No need to use resolution or
> type inference. This will make interprocedural analysis stronger.
> E.g., consider:
> // Test.java
> class Test {
> static void test(A a) {
>switch(a) {
>case B b -> {}
>case C c -> {}
>case D d -> {}
>}
> }
> }
>
> There are two possibilities:
> 1. D is a superclass of A, thus the last pattern is total, and null is
> accepted here:
>
> interface D {}
> interface A extends D {}
> interface B extends A {}
> interface C extends A {}
>
> 2. A is a sealed type with B, C, and D inheritors, switch is
> exhaustive, and null is not accepted here:
>
> sealed interface A {}
> non-sealed interface B extends A {}
> non-sealed interface C extends A {}
> non-sealed interface D extends A {}
>
> So without looking at A definition (which might be anywhere) we cannot
> say whether test(null) will throw NPE or not. We cannot cache the
> knowledge about 'test' method parameter nullability within the
> Test.java, because its nullability might change if we change the
> hierarchy of A, even if Test.java content is untouched. Currently, we
> are conservative and not infer nullability when any unguarded pattern
> appears in switch cases. With the required `case null`, we can perform
> more precise analysis.
We should go a step further, this also means that with
switch(box) {
case Box(B b) -> {}
case Box(C c) -> {}
case Box(D d) -> {}
}
we have no idea if the switch will accept Box(null) or not.
So the idea that a type behave differently if nested inside a pattern or not is
not a good one.
>
> With best regards,
> Tagir Valeev.
regards,
Rémi
>
> On Wed, Jan 26, 2022 at 2:47 AM Brian Goetz wrote:
>>
>>
>> 1. Treatment of total patterns in switch / instanceof
>>
>>
>> The handling of totality has been a long and painful discussion, trying to
>> balance between where we want this feature to land in the long term, and
>> people’s existing mental models of what switch and instanceof are supposed to
>> do. Because we’ve made the conscious decision to rehabilitate switch rather
>> than make a new construct (which would live side by side with the old
>> construct
>> forever), we have to take into account the preconceived mental models to a
>> greater degree.
>>
>> Totality is a predicate on a pattern and the static type of its match target;
>> for a pattern P to be total on T, it means that all values of T are matched
>> by
>> P. Note that when I say “matched by”, I am appealing not necessarily t
Re: Treatment of total patterns (was: Reviewing feedback on patterns in switch)
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Tuesday, January 25, 2022 8:47:09 PM
> Subject: Treatment of total patterns (was: Reviewing feedback on patterns in
> switch)
>> 1. Treatment of total patterns in switch / instanceof
> The handling of totality has been a long and painful discussion, trying to
> balance between where we want this feature to land in the long term, and
> people’s existing mental models of what switch and instanceof are supposed to
> do. Because we’ve made the conscious decision to rehabilitate switch rather
> than make a new construct (which would live side by side with the old
> construct
> forever), we have to take into account the preconceived mental models to a
> greater degree.
> Totality is a predicate on a pattern and the static type of its match target;
> for a pattern P to be total on T, it means that all values of T are matched by
> P. Note that when I say “matched by”, I am appealing not necessarily to “what
> does switch do” or “what does instanceof do”, but to an abstract notion of
> matching.
> The main place where there is a gap between pattern totality and whether a
> pattern matches in a switch has to do with null. We’ve done a nice job
> retrofitting “case null” onto switch (especially with `case null, Foo f` which
> allows the null to be bound to f), but people are still uncomfortable with
> `case Object o` binding null to o.
> (Another place there is a gap is with nested patterns; Box(Bag(String s))
> should
> be total on Box>, but can’t match Box(null). We don’t want to
> force
> users to add default cases, but a switch on Box> would need an
> implicit throwing case to deal with the remainder.)
> I am not inclined to reopen the “should `Object o` be total” discussion; I
> really don’t think there’s anything new to say there. But we can refine the
> interaction between a total pattern and what the switch and instanceof
> constructs might do. Just because `Object o` is total on Object, doesn’t mean
> `case Object o` has to match it. It is the latter I am suggesting we might
> reopen.
> The motivation for treating total patterns as total (and therefore nullable)
> in
> switch comes in part from the desire to avoid introducing sharp edges into
> refactoring. Specifically, we had two invariants in mind:
> x matches P(Q) === x matches P(var alpha) && alpha matches Q:
> and
> switch (x) {
> case P(Q): A
> case P(T): B
> }
> where T is total on the type of x, should be equivalent to
> switch (x) {
> case P(var alpha):
> switch(alpha) {
> case Q: A
> case T: B
> }
> }
> }
> These invariants are powerful both for linguistic transformation and for
> refactoring.
> The refinements I propose are:
> - Null is only matched by a switch case that includes `case null`. Switches
> with
> no `case null` are treated as if they have a `case null: throw NPE`. This
> means
> that `case Object o` doesn’t match null; only `case null, Object o` does.
> - Total patterns are re-allowed in instanceof expressions, and are consistent
> with their legacy form.
> Essentially, this gives switch and instanceof a chance to treat null specially
> with their existing semantics, which takes precedence over the pattern match.
> The consequences of this for our refactoring rules are:
> - When unrolling a nested pattern P(Q), we can only do so when Q is not total.
> - When unrolling a switch over nested patterns to a nested switch, `case P(T)`
> must be unrolled not to `case T`, but `case null, T`.
> These changes entail no changes to the semantics of pattern matching; they are
> changes to the semantics of instanceof/switch with regard to null.
I have a slight preference for the C# syntax, the only way to have a total
pattern is to use "var" so case P(T) is equivalent to instanceof P p && p.t
instanceof T t.
Yes, it's not great because it means that "var" is not just inference but i
think i prefer that compromise than having a type in a pattern means something
different if it is nested or not.
The semantics you are proposing (or the one currently implemented in Java 18)
is objectively neither worst nor better than the C# one, it's just different.
Pragmatically, we should choose the C# semantics, just because there are
already thousands of people who knows it.
Rémi
Re: Positioning of guards (was: Reviewing feedback on patterns in switch)
- Original Message -
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Tuesday, January 25, 2022 8:49:02 PM
> Subject: Positioning of guards (was: Reviewing feedback on patterns in switch)
>> 2. Positioning of guards
>
> We received several forms of feedback over the form and placement of guarded
> patterns. Recall that we define a guarded pattern to be `P && g`, where P is
> a
> pattern and g is a boolean expression. Guarded patterns are never total.
> Note
> that we had a choice of the guard being part of the pattern, or being part of
> the `case` label; the current status chooses the former. (Part of our
> reasoning was that there might be other partial pattern contexts coming, and
> we
> didn’t want to solve this problem each time. (For intsanceof, it makes no
> difference.) )
>
> I am prepared to reconsider the association of the guard with the pattern, and
> instead treat it as part of the case. This is expressively weaker but may
> have
> other advantages.
>
> Additionally, people objected to the use of &&, not necessarily because
> “keywords are better”, but because of the potential confusion should we ever
> choose to support switch over boolean, and because the && did not stand out
> enough as turning a total pattern into a partial one. What the alternative
> looks like is something like:
>
>switch (x) {
>case Foo(var x, var y)
>when x == y -> A;
>case Foo(var x, var y) -> B;
>}
>
> Here, `when` (bike shed to be painted separately) is a qualifier on the case,
> not the pattern. A total pattern with a `when` is considered a partial case.
> This simplifies patterns, and moves the complexity of guards into switch,
> where arguably it belongs.
>
> The loss of expressiveness is in not allowing nested patterns like:
>
>P(Q && guard)
>
> and instead having to move the guard to after the matching construct. Some
> users recoiled at seeing guards inside pattern invocations; it seemed to some
> like mixing two things that should stay separate. (For unrolling a nested
> pattern, `case P(Q)` where Q is not total unrolls to `case Pvar alpha) when
> alpha instanceof Q`.)
I think it's a good simplification.
Rémi
Re: Patterns and GADTs (was: Reviewing feedback on patterns in switch)
- Original Message -
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Tuesday, January 25, 2022 8:49:08 PM
> Subject: Patterns and GADTs (was: Reviewing feedback on patterns in switch)
>> 3. Type refinements for GADTs
>
> There are a number of unsatisfying aspects to how we currently handle GADTs;
> specifically, we are missing the type refinement process outlined in "Simple
> unification-based type inference for GADTs” (SPJ et al, 2005). Here are some
> examples of where we fall down.
>
> Suppose we have
>
>sealed interface Node { }
>record A(T t) extends Node { }
>record B(String s) extends Node { }
>
> and we want to write:
>
> T unbox(Node n) {
>return switch (n) {
>case A n -> n.t;
>case B n -> n.s;
>};
>}
>
> The RHS of all the arrows must be compatible with the return type, which is T.
> Clearly that is true for the first case, but not for the second; the compiler
> doesn’t know that T has to be String if we’ve matched a Node to B. What is
> missing is to refine the type of T based on matches. Here, we would gather an
> additional constraint on `case B` for T=String; if we had a case which was
> covariant in T:
>
>record C(T t)
>
> then a `case C` would gather an additional constraint of T <: B for its
> RHS.
>
> More generally, any code dominated by a match that provides additional bounds
> for type variables could benefit from those bounds. For example, we’d
> probably
> want the same in:
>
>if (n instanceof B b) { /* T is String here */ }
>
> and (as with flow scoping):
>
>if (! (n instanceof B b))
>throw …
>// T is String here
>
> We can probably piggy back the specification of this on flow scoping,
> appealing
> to “wherever a binding introduced by this pattern would be in scope.”
I agree that we should do something to support GADTs
The instanceof example is not a source backward compatible change, remember
that instanceof is not a preview feature.
The main objection to that is that we do not have flow scoping for local
variables but we have it for type variables which is weird.
I wonder if we can come with an explicit syntax for it, the same way instanceof
String s is an explicit syntax for local variables.
By example, something like
return switch (n) {
case A n -> n.t;
case B n -> n.s as T=String;
};
but maybe it's too much ceremony.
regards,
Rémi
Re: Diamond in type patterns (was: Reviewing feedback on patterns in switch)
- Original Message -
> From: "Brian Goetz"
> To: "amber-spec-experts"
> Sent: Tuesday, January 25, 2022 8:49:12 PM
> Subject: Diamond in type patterns (was: Reviewing feedback on patterns in
> switch)
>> 4. Diamond for type patterns (and record patterns)
>
>
> The type pattern `T t` declares `t`, if the pattern matches, with the type T.
> If T is a generic type, then we do a consistency check to ensure soundness:
>
>List list = …
>switch (list) {
>case ArrayList a: A // ok
>case ArrayList a: B // ok
>case ArrayList a: C// ok, raw type
>case ArrayList a: // error, would require unchecked conversion
> }
>
> All of these make sense, but users are going to be tempted to use `case
> ArrayList a` rather than the full `case ArrayList a`, and then be sad
> (or confused) when they get type errors. Since the type can be precisely
> defined by inference, this seems a place for allowing diamond:
>
>case ArrayList<> a: B
>
> (And the same when we have record patterns.)
The questions we did not answer the last time we talk about that subject
- why should we allow raw types here ?
- given that this is equivalent to an instanceof + cast, why we can not use
diamond inference on cast ?
- how this inference work ? Is is the same inference than with the diamond
constructor ?
By example, if we have
List list = ...
switch(list) {
case ArrayList<> a:
}
Do we really want to infer ArrayList to then rejects it because it's
an unsafe cast.
regards,
Rémi
GADT and type variable specialization during inference
Moved to amber-spec-experts,
I've got a very similar issue, if we support patter matching on Class.
public Stream createPrimitiveStrem(Class type) {
return switch(type) {
case int.class -> new SpecializedStream();
case double.class -> new SpecializedStream();
...
};
}
A cast and a pinky swear @SuppressWarnings("unchecked") solve the issue but we
may do better.
Rémi
- Original Message -
> From: "Brian Goetz"
> To: "Jan Lahoda" , "amber-dev"
>
> Sent: Lundi 8 Novembre 2021 21:37:24
> Subject: Re: RFR: 8273328: Compiler implementation for Pattern Matching for
> switch (Second Preview)
> This is more of a spec issue than a compiler issue. I finally got
> around to running some of my favorite GADT examples on this.
>
> I started with this Haskell type:
>
> data Term t where
> Lit :: t -> Term t
> Succ :: Term Int -> Term Int
> IsZero :: Term Int -> Term Bool
> If :: Term Bool -> Term a -> Term a -> Term a
>
> I can map it to this Java hierarchy:
>
> sealed interface Term { }
>
> record Lit(T val) implements Term { }
> record Succ(Term a) implements Term { }
> record IsZero(Term a) implements Term { }
> record If(Term cond, Term a, Term b) implements
> Term { }
>
> We correctly eliminate the impossible cases in:
>
> String foo(Term t) {
> return switch (t) {
> case Lit -> "Lit";
> case If -> "If";
> }
> }
>
> And the compiler correctly tells me that the switch is exhaustive. But
> if I try to write an evaluator:
>
> static T eval(Term term) {
> return switch (term) {
> case Lit t -> t.val;
> case Succ t -> eval(t.a) + 1;
> case IsZero t -> eval(t.a) == 0;
> case If t -> eval(t.cond) ? eval(t.a) : eval(t.b);
> };
> }
>
>
> I get errors on the Succ and IsZero cases. In Haskell, the equivalent:
>
> eval :: Term t -> t
> eval (Lit t) = t
> eval (Succ i) = (eval i) + 1
> ...
>
> works correctly because when we match on `eval (Succ i)`, we unify t
> with Int, since that's the only instantiation for t that would work in
> this case. That's what allows us to return an Int when we're expecting
> a t, because we've already figured out they are the same.
>
> So in order to make our eval work, we would have to know to _refine_ the
> bounds of T on the RHS of
>
> case Succ t -> eval(t.a) + 1;
>
> where we would say "Succ is Term, and Succ extends Term, so
> T=Integer here".
>
> In the absence of this, I have to do an explicit boxing plus an
> unchecked cast:
>
> static T eval(Term term) {
> return switch (term) {
> case Lit t -> t.val;
> case Succ t -> (T) (Integer) (eval(t.a) + 1);
> case IsZero t -> (T) (Boolean) (eval(t.a) == 0);
> case If t -> eval(t.cond) ? eval(t.a) : eval(t.b);
> };
> }
>
> That we need both the boxing and the T cast is probably a bug.
>
Re: Translation musings (was: Are templated string embedded expressions "method parameters" or "lambdas"?)
> From: "Brian Goetz"
> To: "John Rose"
> Cc: "amber-spec-experts"
> Sent: Mardi 2 Novembre 2021 18:43:29
> Subject: Translation musings (was: Are templated string embedded expressions
> "method parameters" or "lambdas"?)
> I think we need to redirect a bit.
>> I’m pointing out a different proposition: *If* you need static validation of
>> a
>> string, *then* you need something constant about the receiver. By constant I
>> mean, very specifically, “fixed no later than just before the first execution
>> of of the string template expression”. The “something constant” might be the
>> type (some subtype of a known “ST-Handler”), or it might be a constant value
>> (e.g., a static final field value).
> And I'm saying something different:
> - We never *need* static validation of the string.
> - We never *need* an indy-based, multi-step translation.
> Both of these are nice to haves, not must-haves; the goal is to extract these
> when we can have them, not to restrict the feature so that we can't use the
> feature if we can't produce them.
> We are in deep danger of way over-rotating towards the translation. I would go
> as far as to say that if we always, now and for the forever future, just
> translated with "invokeinterface", it would *still* be a successful and
> popular
> feature. Do we want to do better when we can? Of course. Are we willing to
> restrict the feature to the cases where can do so? Absolutely not.
> It's really easy to get wrapped around the axle over cases like
> String::format,
> and convince ourselves that this is the 99% use case. But this represents only
> a slice (yes, an important slice) of the benefits we get from this feature. In
> many cases, the work we'll do to translate the template (e.g., a CLASS."public
> class Foo { ... }" policy) or after we translate the template (execute a SQL
> query) completely dominates the performance profile.
> So yes, let's optimize for the best-case translation when we can, but let's
> not
> let that constrain what feature we deliver.
Templated strings can have a awful performance model if we are not careful, and
i don't think that relying on the user being careful aka the C++ strategy is a
good strategy.
They have been enough lengthy discussions about the performance of
String.format() vs StringBuilder, i think it's reasonable to avoid to create
new performance pot holes when designing this feature.
So static validation and being indy-based are a little more than just nice to
have, they are here to provide an easy to understand performance model.
regards,
Rémi
