[Python-Dev] Re: The semantics of pattern matching for Python

[Guido van Rossum]

On Sun, Nov 22, 2020 at 5:03 PM Jim J. Jewett  wrote:

> I think your changed constructor:
>
> class Car:
> def __init__(self, manufacturer, variant):
> self.brand = manufacturer
> self.model = variant
>
> is a particularly good example, and the PEP should specify whether:
> Car("Chrysler", "PT Cruiser")
>
> is matched by:
> Car(brand="Chrysler", mod:=model)
> or:
> Car(manufacturer="Chrysler", mod:=variant)
> or both, or possibly even Frankenstein combinations like:
> Car(brand="Chrysler", mod:=variant)
>

(a) We got rid of the walrus and now spell it using ' as
' instead of ' := ".

(b) All your examples have a positional argument following a keyword
argument and that's invalid syntax, in patterns as in expressions. (This
was the only bug in Steven's example that I found, but I didn't bother to
call it out.)

(c) What Ethan said -- after fixing the syntax this does show that the
keywords in class patterns correspond to attributes, not to constructor
arguments (unless those are the same, as they often are, and always for
dataclasses and named tuples).

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YEVCM6Z4YN2VI6CVBOEQKIXIJIELM2B5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[Ethan Furman]

On 11/22/20 5:00 PM, Jim J. Jewett wrote:

I think your changed constructor:

class Car:
 def __init__(self, manufacturer, variant):
 self.brand = manufacturer
 self.model = variant

is a particularly good example, and the PEP should specify whether:
 Car("Chrysler", "PT Cruiser")

is matched by:
 Car(brand="Chrysler", mod:=model)
or:
 Car(manufacturer="Chrysler", mod:=variant)
or both, or possibly even Frankenstein combinations like:
 Car(brand="Chrysler", mod:=variant)


It could be a good example to include to emphasize that pattern matching is based on attributes, not function headers -- 
so `brand` and `model` would work, whilst `manufacturer` and `variant` would not.


--
~Ethan~
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EO2XDZT52MCLNNJUV3PC2HFNC5BWJMUW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[Jim J. Jewett]

I think your changed constructor:

class Car:
def __init__(self, manufacturer, variant):
self.brand = manufacturer
self.model = variant

is a particularly good example, and the PEP should specify whether: 
Car("Chrysler", "PT Cruiser")

is matched by:
Car(brand="Chrysler", mod:=model)
or:
Car(manufacturer="Chrysler", mod:=variant)
or both, or possibly even Frankenstein combinations like:
Car(brand="Chrysler", mod:=variant)

-jJ
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HBHBYOG56E75ET732WM7ZBDRTWGBVA25/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[Daniel Moisset]

Others have replied with most of this covering my opinion but there's a
point I'd like to highlight here

On Fri, 20 Nov 2020 at 14:23, Mark Shannon  wrote:

> Hi Daniel,
>
> On 20/11/2020 10:50 am, Daniel Moisset wrote:
> >  (... snipping for brevity ...)
> >
> >  1. You mention a goal about "erroneous patterns" (which I'm still not
> > sure I agree on), and your proposal addresses that by forcing
> > classes to be explicit (via __atributes__ and __deconstruct__) about
> > what attributes are accepted as matches. This is against one design
> > principle that's not in your list but it was (at least implicitly)
> > in PEP622 and 634: "pattern matching must allow matching objects on
> > types not specifically designed for it"; this will allow to apply
> > this feature to classes that you can not modify (like instances
> > created by a 3rd party library ). That's why PEP634 relies on
> > getattr() ; that could be extended in the feature (providing some
> > override using attributes like yours) but that wouldn't be required
> > by default
>
> Why force pattern matching onto library code that was not designed for
> pattern matching? It seems risky.
>
> Fishing arbitrary attributes out of an object and assuming that the
> values returned by attribute lookup are equivalent to the internal
> structure breaks abstraction and data-hiding.
>
> An object's API may consist of methods only. Pulling arbitrary
> attributes out of that object may have all sorts of unintended
> side-effects.
>
> PEP 634 and the DLS paper assert that deconstruction, by accessing
> attributes of an object, is the opposite of construction.
> This assertion seems false in OOP.
>
>
I think your description about what you and I call OOP lies at the center
of why we're unlikely to agree on what's the best approach on class
patterns.

The Python I write and tend to use allows and encourages the use of the dot
operator (aka "Fishing arbitrary attributes out of an object") and the
libraries I use expose through it important parts of the API which
doesn't follow the style you mention where  "An object's API may consist of
methods only.". Of course I rely on properly documented attributes, like
datetime.date.year, Fractional.denominator, JSONDEcodeError.message (in the
stdlib) or fields I defined in my django models, or things like
requests.response.status_code (in 3rd party libraries that I used as is or
by extending). Those are all types of objects that I'd like the proposal to
allow as subjects of a match statement and attributes that I'd like to have
in lookups. Given that I can not force a lot of 3rd party libraries to add
a new special attribute `__attributes__`, our proposal relies on the
standard `getattr()` API (which is the core operation on Python object
model to doing most things with an object, even calling methods).

I've used other OOP languages within that style were attributes have to be
explicitly exposed, and even so that's disencouraged and they follow the
principle of "API=only methods". And I can imagine some people could choose
to write Python like that, even if I don't and most of the libraries I use
don't either. For people with that philosophy about OOP, our pattern
matching proposal will be horrible and useless and they can choose not to
use the match statement, ever. The argumentation behind the proposal is
implicitly based on the assumption that *most* Python developers consider
the dot operator one of the "natural" ways to expose object interfaces in
Python, even if it can pull attributes and use non-method APIs. If you
think that assumption is false, we'll have to agree to disagree but I don't
see how we could get to a common vision on how class patterns should behave.

(...)
> When we added the "with" statement, there was no attempt to force
> existing code to support it. We made the standard library support it,
> and let the community add support as and when it suited them.
>
> We should do the same with pattern matching.
>

With our proposal, nothing forces the existing code to support it, they
support it naturally using their existing attribute access API (through the
dot operator). One advantage that the with statement had is that it's quite
easy to wrap an object that wasn't designed to be used as context manager
in another that can (the stdlib even provides a common example of that with
contextlib.closing). Trying to use something like that (writing "match
wrapper(obj): ..." where obj doesn't support some match protocol and the
wrapper adds it) is non viable in pattern matching, because there are many
cases where the object that needs support might be many levels of
indirection (through item/key/attribute access) behind the subject object.

The current proposals using getattr is useful as is (Even if limited in
some cases). Again that doesn't discard the opportunity of adding an
extended protocol that allows decomposing using other APIs beyond attribute
access (I would 

[Python-Dev] Re: The semantics of pattern matching for Python

[Brian Coleman]

Regarding the difficulty which some people have respecting class patterns and 
dictionary patterns, I would like to draw attention to a similar feature in 
JavaScript, object destructuring. JavaScript does not have pattern matching but 
object destructuring is closely related. Take the example of a function to 
compute the distance between two points.

```
function distance(p1, p2) {
  // This is an object destructuring assignment that extracts the x field of p1 
into x1 and the y field of p1 into y1
  const { x: x1, y: y1 } = p1;
  // This is an object destructuring assignment that extracts the x field of p2 
into x2 and the y field of p2 into y2
  const { x: x2, y: y2 } = p2;

  const dx = x2 - x1;
  const dy = y2 - y1;
  return Math.sqrt(dx*dx + dy*dy);
}

// An object literal is assigned to p1 here
const p1 = { x: 0, y: 0 };
// An object literal is assigned to p2 here
const p2 = { x: 1, y: 1 };
const d = distance(p1, p2);
```

Similarly to dictionary patterns in Python, object destructuring in JavaScript 
places the variable names that are the targets of the assignments in the same 
position where a value would be expected in an object literal. This feature has 
existed in JavaScript for several years now and I would like to draw attention 
to it as a counterpoint to the argument that pattern matching is not a good fit 
for languages that are not statically typed. Destructuring can also be found in 
Common Lisp, which is not statically typed. Pattern matching is also a core 
part of Erlang, which is not statically typed..

I would also like to draw attention to the fact that object destructuring was 
added to JavaScript years into it's development. I do not believe this to be a 
reasonable argument against adopting pattern matching in Python.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6XDXHVT3ZFOK66GVU5UWYGSHJX4UF2CW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[Tobias Kohn]

 Hi David and Steve,

There is hardly anything that needs to be added to your comments, of  
course.  However, I consider these explicitly named attributes in the  
class pattern to be one of the most difficult aspects of our pattern  
matching proposal, which is something I just want to briefly  
acknowledge.  Although there are good reasons for using the syntax as  
proposed, understanding that ``Spam(a=x)`` assigns the attribute ``a``  
to the variable ``x`` is not quite as intuitive and straight forward.


Using the curly braces for that purpose might help in that we  
instantly think differently and more in line with dictionaries. This  
small clue could potentially have quite an impact on readability.   
This idea has thus a huge advantage over the square brackets not  
because of PEP 637, but because it might change the way we look at the  
code at hand.


The reason why I would not favour this specific syntax is because we  
expect that the predominant case would be to just write ``Spam(1, 2)``  
without the attribute names.  It is even possible to mix and match the  
two, which would then lead to something like ``Spam{1, b=2}``.  In  
that case, the advantage of the dictionary-like notation might just  
evaporate I assume, leaving behind a notation that turns out to be  
unusual and somewhat annoying in the overwhelming majority of cases.


There are a few other concepts and ideas behind it all, which I do not  
want to go into unless there is demand for it, but part of it is the  
idea that we want to be as much as possible in line with the syntax  
already used in iterable unpacking, say.


Kind regards,
Tobias

Quoting David Mertz :


On Sat, Nov 21, 2020 at 12:23 PM Steven D'Aprano  wrote:


Clearly Spam(a=1, b=2) does not necessarily result in an instance with
attributes a and b. But the pattern `Spam(a=1, b=2)` is intended to be
equivalent to (roughly):

    if (instance(obj, Spam)
        and getattr(obj, a) == 1
        and getattr(obj, b) == 2)

it doesn't imply that obj was *literally* created by a call to
the constructor `Spam(a=1, b=2)`, or even that this call would be
possible.


 
I think this explanation makes me not worry about the fact that  
`Spam(a=1, b=2)` in a pattern looks a lot like a constructor.  Like  
some other commenters, I was vaguely bothered that the identical  
spelling might have these different meanings in different contexts.   
But I think a match case just clearly enough IS a different context  
that using slightly different intuitions is no real conceptual  
stretch for remembering or teaching it.

 
As a strawman, we could use different syntax for "match the  
thing of class Spam that has attributes with these values:

 
match eggs:
    case Spam[a=1, b=2]: ...
 
Or:
 
match eggs:
    case Spam{a=1, b=2}: ...
 
Well, the square brackets COULD mean something different if PEP  
637 is adopted.  But even supposing the curly braces could be fit  
into the grammar.  Yes, it sort of suggests the connection between  
dictionaries and Spam.__dict__.  But it still reads as "this is  
something special that I have to think about a little differently."

 
Even where there are capture variables, I think I'd be  
completely comfortable thinking about the different context for:

 
match eggs:
    case Spam(a=x, b=2): ...
 

--
   The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/J4GVSQN4UPOCN3EXZASQOIWN7MO723A4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[Tobias Kohn]

 Hi Steve,

Thank you very much for your comments here.  This is certainly not the  
first time I feel that you not only have an excellent insight into a  
topic, but also manage to make your points very clearly and  
succinctly.  Your car example highlights the background of the  
proposed syntax very nicely, indeed, and the for-in example strikes me  
as quite astute.


Kind regards,
Tobias

Quoting Steven D'Aprano :


On Fri, Nov 20, 2020 at 02:23:45PM +, Mark Shannon wrote:


Why force pattern matching onto library code that was not designed for
pattern matching? It seems risky.


Can you give a concrete example of how this will "force" pattern
matching onto library code? I don't think that anyone has suggested that
we go around to third-party libraries and insert pattern matching in
them, so I'm having trouble understanding your fear here.


Fishing arbitrary attributes out of an object and assuming that the
values returned by attribute lookup are equivalent to the internal
structure breaks abstraction and data-hiding.


Again, can we have a concrete example of what you fear?

Python is not really big on data-hiding. It's pretty close to impossible
to hide data in anything written in pure Python.


An object's API may consist of methods only. Pulling arbitrary
attributes out of that object may have all sorts of unintended side-effects.


Well, sure, but merely calling print() on an object might have all sorts
of unintended side-effects. I think that almost the only operation
guaranteed to be provably side-effect free in Python is the `is`
operator. So I'm not sure what you fear here?

If I have a case like:

   match obj:
       case Spam(eggs=x):

I presume any sensible implementation is going to short-cut the
attempted pattern match for x if obj is *not* an instance of Spam. So
it's not going to be attempting to pull out arbitrary attributes of
arbitrary objects, but only specific attributes of Spam objects.

To the degree that your objection here has any validity at all, surely
it has been true since Python 1.5 or older that we can pull arbitrary
attributes out of unknown objects? That's what duck-typing does, whether
you guard it with a LBYL call to hasattr or an EAFP try...except block.

   if hasattr(obj, 'eggs'):
       result = obj.eggs + 1

Not only could obj.eggs have side-effects, but so could the call to
hasattr. Can you explain how pattern matching is worse than what we
already do?


PEP 634 and the DLS paper assert that deconstruction, by accessing
attributes of an object, is the opposite of construction.
This assertion seems false in OOP.


Okay. Does it matter?

Clearly Spam(a=1, b=2) does not necessarily result in an instance with
attributes a and b. But the pattern `Spam(a=1, b=2)` is intended to be
equivalent to (roughly):

   if (instance(obj, Spam)
       and getattr(obj, a) == 1
       and getattr(obj, b) == 2)

it doesn't imply that obj was *literally* created by a call to
the constructor `Spam(a=1, b=2)`, or even that this call would be
possible.

I think that it will certainly be true that for many objects, there is a
very close (possibly even exact) correspondence between the constructor
parameters and the instance attributes, i.e. deconstruction via
attribute access is the opposite of construction.

But for the exceptions, why does it matter that they are exceptions?

Let me be concrete for the sake of those who may not be following these
abstract arguments. Suppose I have a class:

   class Car:
       def __init__(self, brand, model):
           self.brand = brand
           self.model = model

and an instance:

   obj = Car("Suzuki", "Swift")

For this class, deconstruction by attribute access is exactly the
opposite of construction, and I can match any Suzuki like this:

   match obj:
       case Car(brand="Suzuki", model)

which is roughly equivalent to:

   if isinstance(obj, Car) and getattr(obj, "brand") == "Suzuki":
       model = getattr(obj, "model")

It's not actually asserting that the instance *was* constructed with a
call to `Car(brand="Suzuki", model="Swift")`, only that for the purposes
of deconstruction it might as well have been.

If the constructor changes, leaving the internal structure the same:

   class Car:
       def __init__(self, manufacturer, variant):
           self.brand = manufacturer
           self.model = variant

the case statement need not change.

Remember that non-underscore attributes are public in Python, so a
change to the internal structure:

   class Car:
       def __init__(self, brand, model):
           self.brand_name = brand
           self.model_id = model

is already a breaking change, whether we have pattern matching or not.


When we added the "with" statement, there was no attempt to force
existing code to support it. We made the standard library support it,
and let the community add support as and when it suited them.

We should do the same with pattern matching.


That's a terrible analogy. Pattern matching is 

[Python-Dev] Re: The semantics of pattern matching for Python

[Guido van Rossum]

On Mon, Nov 16, 2020 at 6:41 AM Mark Shannon  wrote:

> I believe that a pattern matching implementation must have the following
> properties:
>
> * The semantics must be precisely defined.
> * It must be implemented efficiently.
> * Failed matches must not pollute the enclosing namespace.
> * Objects should be able determine which patterns they match.
> * It should be able to handle erroneous patterns, beyond just syntax
> errors.
>
> PEP 634 and PEP 642 don't have *any* of these properties.
>

Mark, I trust that those are your beliefs. But your words are just
sufficiently vague that anyone can read into them what they want. (IOW, the
semantics of your words are not precisely defined. :-)

1.
When drafting PEP 634,  I spent a *lot* of time making the description of
the semantics as clear and precise as they could be, using carefully
defined terms like matching, subpattern, binding and so on. The syntax of
patterns is defined recursively, and so are the semantics. If you have
specific concerns about ambiguities in the description, please quote the
relevant text of the PEP and ask what the intention is or just suggest an
improvement. Otherwise I cannot do anything with your claim "the PEP does
not describe the semantics precisely" except disagreeing. Note that
specifying semantics in terms of a translation to "equivalent" Python code
is also fraught with difficulties, and is in danger of over-specifying
irrelevant details.

2.
Regarding efficient implementation, I would phrase it differently. I think
that it must be *possible* to implement it efficiently. During the design
phase we spent a lot of time thinking about the implementation, and the
specification leaves plenty of room for improving upon Brandt's initial
implementation. Your demand that the initial implementation be "efficient"
is baseless, and your claim that the reference implementation isn't
sufficiently efficient is insulting to Brandt. I also see no reason to
believe it. Historically, everything in Python starts out relatively
inefficient and over time we speed up those parts of the implementation
that need it.

3.
Your claim "Failed matches must not pollute the enclosing namespace" is
entirely subjective. Since succeeding matches store captured values in the
enclosing namespace, those variables that may be overwritten by the
matching process are liable to be overwritten anyway. I've seen your
counter-proposals for this (both save/restore and $-number variables), and
I won't object if you implement something like that after the initial
implementation lands, but I *would* object if I couldn't enter e.g. set a
breakpoint on a guard expression in a debugger and inspect the capture
variables using their regular names. I don't see your statement as a
requirement for an initial implementation, and I think the compiler will be
simpler without it.

4.
"Objects should be able [to] determine which patterns they match" is again
a subjective claim. We debated adding a `__match__` method to objects to
enable this but couldn't get agreement on how it should be called and what
it should return. We even came up with a proposal that made it into the
first draft of PEP 622, after a long debate about the efficiency of the
interface. But in the end it added complexity and we didn't have any
sufficiently strong use cases, so we decided that we would punt on this
one. This is the kind of thing that can easily be added in a future
revision without breaking backwards compatibility, so we took it out to
limit the scope of the proposal.

5.
Your bullet about erroneous patterns seems to be meant to call out one
particular quirk of the proposal, which is that class patterns can specify
as keywords anything that is a valid attribute, which includes for example
methods. My attitude towards this can be characterized as "so what?" It has
already been pointed out that the same mistake can occur in
non-pattern-matching code (e.g. `if nt.count == 1`), and I don't think this
kind of mistake is going to be any more prevalent in class patterns than it
is in other code.

Finally, recalling your past comments on PEP 622, PEP 634, and other PEPs,
I feel hurt by the apparent disrespect you express for the authors and
their designs (the current thread being just one example). If I compare
Larry Hastings' post to discourse

where he explains that he doesn't want pattern matching I see none of that,
but in your posts it shines through again and again. Can I ask you to
please take a step back and think about why this may be, and try to change
something in your writing style? We all will need to get along for a long
time after this particular topic is put to rest, but each time I see one of
your posts like this I find it harder to respond rationally rather than
emotionally. (It is no accident that it took me 5 days to respond here.)

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my 

[Python-Dev] Re: The semantics of pattern matching for Python

[Andrew Svetlov]

If you don't want the overlapping with the existing syntax angle brackets
can be used:

match eggs:
case Spam: ...


On Sat, Nov 21, 2020 at 7:31 PM David Mertz  wrote:

> On Sat, Nov 21, 2020 at 12:23 PM Steven D'Aprano 
> wrote:
>
>> Clearly Spam(a=1, b=2) does not necessarily result in an instance with
>> attributes a and b. But the pattern `Spam(a=1, b=2)` is intended to be
>> equivalent to (roughly):
>>
>> if (instance(obj, Spam)
>> and getattr(obj, a) == 1
>> and getattr(obj, b) == 2)
>>
>> it doesn't imply that obj was *literally* created by a call to
>> the constructor `Spam(a=1, b=2)`, or even that this call would be
>> possible.
>>
>
> I think this explanation makes me not worry about the fact that `Spam(a=1,
> b=2)` in a pattern looks a lot like a constructor.  Like some other
> commenters, I was vaguely bothered that the identical spelling might have
> these different meanings in different contexts.  But I think a match case
> just clearly enough IS a different context that using slightly different
> intuitions is no real conceptual stretch for remembering or teaching it.
>
> As a strawman, we could use different syntax for "match the thing of class
> Spam that has attributes with these values:
>
> match eggs:
> case Spam[a=1, b=2]: ...
>
> Or:
>
> match eggs:
> case Spam{a=1, b=2}: ...
>
> Well, the square brackets COULD mean something different if PEP 637 is
> adopted.  But even supposing the curly braces could be fit into the
> grammar.  Yes, it sort of suggests the connection between dictionaries and
> Spam.__dict__.  But it still reads as "this is something special that I
> have to think about a little differently."
>
> Even where there are capture variables, I think I'd be completely
> comfortable thinking about the different context for:
>
> match eggs:
> case Spam(a=x, b=2): ...
>
> --
> The dead increasingly dominate and strangle both the living and the
> not-yet born.  Vampiric capital and undead corporate persons abuse
> the lives and control the thoughts of homo faber. Ideas, once born,
> become abortifacients against new conceptions.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/XSRVX2NTDGUF7CWTPR5SHIOAQPWNXXYZ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
Thanks,
Andrew Svetlov
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/45JLJCSQRZ3SCVV6EP546W4P5LBYNUKM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[David Mertz]

On Sat, Nov 21, 2020 at 12:23 PM Steven D'Aprano 
wrote:

> Clearly Spam(a=1, b=2) does not necessarily result in an instance with
> attributes a and b. But the pattern `Spam(a=1, b=2)` is intended to be
> equivalent to (roughly):
>
> if (instance(obj, Spam)
> and getattr(obj, a) == 1
> and getattr(obj, b) == 2)
>
> it doesn't imply that obj was *literally* created by a call to
> the constructor `Spam(a=1, b=2)`, or even that this call would be
> possible.
>

I think this explanation makes me not worry about the fact that `Spam(a=1,
b=2)` in a pattern looks a lot like a constructor.  Like some other
commenters, I was vaguely bothered that the identical spelling might have
these different meanings in different contexts.  But I think a match case
just clearly enough IS a different context that using slightly different
intuitions is no real conceptual stretch for remembering or teaching it.

As a strawman, we could use different syntax for "match the thing of class
Spam that has attributes with these values:

match eggs:
case Spam[a=1, b=2]: ...

Or:

match eggs:
case Spam{a=1, b=2}: ...

Well, the square brackets COULD mean something different if PEP 637 is
adopted.  But even supposing the curly braces could be fit into the
grammar.  Yes, it sort of suggests the connection between dictionaries and
Spam.__dict__.  But it still reads as "this is something special that I
have to think about a little differently."

Even where there are capture variables, I think I'd be completely
comfortable thinking about the different context for:

match eggs:
case Spam(a=x, b=2): ...

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XSRVX2NTDGUF7CWTPR5SHIOAQPWNXXYZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[Steven D'Aprano]

On Fri, Nov 20, 2020 at 02:23:45PM +, Mark Shannon wrote:

> Why force pattern matching onto library code that was not designed for 
> pattern matching? It seems risky.

Can you give a concrete example of how this will "force" pattern 
matching onto library code? I don't think that anyone has suggested that 
we go around to third-party libraries and insert pattern matching in 
them, so I'm having trouble understanding your fear here.



> Fishing arbitrary attributes out of an object and assuming that the 
> values returned by attribute lookup are equivalent to the internal 
> structure breaks abstraction and data-hiding.

Again, can we have a concrete example of what you fear?

Python is not really big on data-hiding. It's pretty close to impossible 
to hide data in anything written in pure Python.



> An object's API may consist of methods only. Pulling arbitrary 
> attributes out of that object may have all sorts of unintended side-effects.

Well, sure, but merely calling print() on an object might have all sorts 
of unintended side-effects. I think that almost the only operation 
guaranteed to be provably side-effect free in Python is the `is` 
operator. So I'm not sure what you fear here?

If I have a case like:

match obj:
case Spam(eggs=x):

I presume any sensible implementation is going to short-cut the 
attempted pattern match for x if obj is *not* an instance of Spam. So 
it's not going to be attempting to pull out arbitrary attributes of 
arbitrary objects, but only specific attributes of Spam objects.

To the degree that your objection here has any validity at all, surely 
it has been true since Python 1.5 or older that we can pull arbitrary 
attributes out of unknown objects? That's what duck-typing does, whether 
you guard it with a LBYL call to hasattr or an EAFP try...except block.

if hasattr(obj, 'eggs'):
result = obj.eggs + 1

Not only could obj.eggs have side-effects, but so could the call to 
hasattr. Can you explain how pattern matching is worse than what we 
already do?


> PEP 634 and the DLS paper assert that deconstruction, by accessing 
> attributes of an object, is the opposite of construction.
> This assertion seems false in OOP.


Okay. Does it matter?

Clearly Spam(a=1, b=2) does not necessarily result in an instance with 
attributes a and b. But the pattern `Spam(a=1, b=2)` is intended to be 
equivalent to (roughly):

if (instance(obj, Spam)
and getattr(obj, a) == 1
and getattr(obj, b) == 2)

it doesn't imply that obj was *literally* created by a call to 
the constructor `Spam(a=1, b=2)`, or even that this call would be 
possible.

I think that it will certainly be true that for many objects, there is a 
very close (possibly even exact) correspondence between the constructor 
parameters and the instance attributes, i.e. deconstruction via 
attribute access is the opposite of construction.

But for the exceptions, why does it matter that they are exceptions?

Let me be concrete for the sake of those who may not be following these 
abstract arguments. Suppose I have a class:


class Car:
def __init__(self, brand, model):
self.brand = brand
self.model = model


and an instance:


obj = Car("Suzuki", "Swift")


For this class, deconstruction by attribute access is exactly the 
opposite of construction, and I can match any Suzuki like this:


match obj:
case Car(brand="Suzuki", model)


which is roughly equivalent to:


if isinstance(obj, Car) and getattr(obj, "brand") == "Suzuki":
model = getattr(obj, "model")


It's not actually asserting that the instance *was* constructed with a 
call to `Car(brand="Suzuki", model="Swift")`, only that for the purposes 
of deconstruction it might as well have been.

If the constructor changes, leaving the internal structure the same:


class Car:
def __init__(self, manufacturer, variant):
self.brand = manufacturer
self.model = variant


the case statement need not change.

Remember that non-underscore attributes are public in Python, so a 
change to the internal structure:


class Car:
def __init__(self, brand, model):
self.brand_name = brand
self.model_id = model


is already a breaking change, whether we have pattern matching or not.


> When we added the "with" statement, there was no attempt to force 
> existing code to support it. We made the standard library support it, 
> and let the community add support as and when it suited them.
>
> We should do the same with pattern matching.

That's a terrible analogy. Pattern matching is sugar for things that we 
can already do:

- isinstance
- getattr
- sequence unpacking
- equality
- dict key testing

etc. Pattern matching doesn't "force" objects to support anything they 
don't already support.

As far as I can tell, the only thing that the community will need to add 
support for is the mapping between 

[Python-Dev] Re: The semantics of pattern matching for Python

[Mark Shannon]

On 20/11/2020 6:57 pm, Tobias Kohn wrote:

Hi Daniel and Mark,

Sorry for being slightly late to the party, but please let me add a few 
remarks of my own to the discussion here.


*1. Must have precisely defined semantics*

Yes, there are some aspects that we left open intentionally.  Most


Why? Daniel has written up a formal semantics, why not use that or 
something like it?


prominently the question of how often the pattern matching engine will 
check whether the subject is an instance of a particular class.  Take 
the following trivial example::


   match some_data:
   case Pair(12, 34):
   ...
   case Triple(12, 34, z):
   ...
   case Pair(12, y):
   ...
   case Pair(x, y):
   ...

In a perfect world, the compiler discovers that it must check whether 
``some_data`` is an instance of ``Pair`` exactly once and not three 
times.  This, of course, plays right into Mark's second point on 
efficiency and seems obvious enough.  Yet, as soon as we are considering 
nested patterns, it turns much less obvious whether the compiler is 
supposed to cache repeated isinstance-checks.  Can we really expect that 
the compiler must discover that in both case clauses the first element 
is checked against the same class?  Or would it make more sense to 
simply expect the compiler to potentially perform this ``Num`` instance 
check twice::


   match some_data:
   case [ Num(), 12 ]:
   ...
   case [ Num(), y, *z ]:
   ...

It is easy to think of cases where we accidentally end up calling an 
isinstance check more than once because the compiler could not prove 
that they are equal.  Still, whenever possible we want to give the 
compiler the freedom to optimise the pattern matching statement by caching.


An optimization (in the CS sense, not the math sense) is a 
performance-improving, semantic-preserving transformation.

How can I preserve semantics if you won't tell me what the semantics are?



In a static language, all of this would not be an issue at all, of 
course.  In Python, however, we end up being caught between its dynamic 
features and the desire to make pattern matching reasonably efficient.  
So, we ended up leaving the question open as how often the pattern 
matching engine is allowed or supposed to check instances.  Naturally, 
if you go and write some isinstance-check on a class with side-effects, 
you can break it.



*2. Users should not have to pay an unnecessary performance penalty to 
use pattern matching*


To quote Mark [1] here:

/> Users should not have to pay an unnecessary performance penalty to 
use pattern matching./


Alright, what does this even mean?  What is an unnecessary performance 
penalty?  How should that be measured or compared?


If we shipped Python compiled with -O1, instead of -O3. It would be 
slower and unnecessarily so. That would be an unnecessary performance 
penalty.


In this context, I mean that there is an obvious performance improvement 
available and it hasn't been implemented. Or that the specification 
prevents an optimization without any compensating benefit.


Without a well defined semantics, I can't optimize at all. That is an 
unnecessary performance penalty.




Pattern matching is not just fancy syntax for an if-elif-statement, but 
a new way of writing and expressing structure.  There is currently 
nothing in Python that fully compares to pattern matching (which is 
obviously why we propose to add in the first place).  So, do you want to 
compare a pattern matching structure to an if-elif-chain or rather an 
implementation using reflection and/or the visitor pattern?  When 
implementing pattern matching, would we be allowed to trade off a little 
speed handling the first pattern for moving faster to patterns further down?


Do our PEPs really read to you like we went out of our ways to make it 
slow or inefficient?  Sure, we said let's start with an implementation 
that is correct and worry about optimising it later.  But I thought this 
is 101 of software engineering, anyway, and am thus rather surprised to 
find this item on the list.



*3. Failed matches should not pollute the enclosing namespace*

This is a slightly wider issue that has obviously sparked an entire 
discussion on this mailing list on scopes.


If there is a good solution that only assigns variables once the entire 
pattern matched, I would be very happy with that.  However, I think that 
variables should be assigned in full before evaluating any guards---even 
at the risk of the guard failing and variables being assigned that are 
not used later on.  Anything else would obviously introduce a mini-scope 
and lead to shadowing, which hardly improves anything with respect to 
legibility.


How would it introduce shadowing? Since `var` is a capture pattern, it 
can't read and thus can't be shadowed.





*4. Objects should be able determine which patterns they match*

Short version: no!


Yes! ;)



Class 

[Python-Dev] Re: The semantics of pattern matching for Python

[Terry Reedy]

Mark, did you get the response I sent to hotpy.org 4 days ago?  Is that 
a real address?  I ask because the typos I reported are still there and 
trying to visit hotpy.org fails.


--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TX7W5TVIJT5BAMP3ZEIS6QSOVJNHFDQA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[Tobias Kohn]

 Hi Daniel and Mark,

Sorry for being slightly late to the party, but please let me add a  
few remarks of my own to the discussion here.


1. MUST HAVE PRECISELY DEFINED SEMANTICS

Yes, there are some aspects that we left open intentionally.  Most  
prominently the question of how often the pattern matching engine will  
check whether the subject is an instance of a particular class.  Take  
the following trivial example::


  match some_data:
  case Pair(12, 34):
  ...
  case Triple(12, 34, z):
  ...
  case Pair(12, y):
  ...
  case Pair(x, y):
  ...

In a perfect world, the compiler discovers that it must check whether  
``some_data`` is an instance of ``Pair`` exactly once and not three  
times.  This, of course, plays right into Mark's second point on  
efficiency and seems obvious enough.  Yet, as soon as we are  
considering nested patterns, it turns much less obvious whether the  
compiler is supposed to cache repeated isinstance-checks.  Can we  
really expect that the compiler must discover that in both case  
clauses the first element is checked against the same class?  Or would  
it make more sense to simply expect the compiler to potentially  
perform this ``Num`` instance check twice::


  match some_data:
  case [ Num(), 12 ]:
  ...
  case [ Num(), y, *z ]:
  ...

It is easy to think of cases where we accidentally end up calling an  
isinstance check more than once because the compiler could not prove  
that they are equal.  Still, whenever possible we want to give the  
compiler the freedom to optimise the pattern matching statement by  
caching.


In a static language, all of this would not be an issue at all, of  
course.  In Python, however, we end up being caught between its  
dynamic features and the desire to make pattern matching reasonably  
efficient.  So, we ended up leaving the question open as how often the  
pattern matching engine is allowed or supposed to check instances.   
Naturally, if you go and write some isinstance-check on a class with  
side-effects, you can break it.


2. USERS SHOULD NOT HAVE TO PAY AN UNNECESSARY PERFORMANCE PENALTY TO  
USE PATTERN MATCHING


To quote Mark [1] here:

/> Users should not have to pay an unnecessary performance penalty to  
use pattern matching./


Alright, what does this even mean?  What is an unnecessary performance  
penalty?  How should that be measured or compared?


Pattern matching is not just fancy syntax for an if-elif-statement,  
but a new way of writing and expressing structure.  There is currently  
nothing in Python that fully compares to pattern matching (which is  
obviously why we propose to add in the first place).  So, do you want  
to compare a pattern matching structure to an if-elif-chain or rather  
an implementation using reflection and/or the visitor pattern?  When  
implementing pattern matching, would we be allowed to trade off a  
little speed handling the first pattern for moving faster to patterns  
further down?


Do our PEPs really read to you like we went out of our ways to make it  
slow or inefficient?  Sure, we said let's start with an implementation  
that is correct and worry about optimising it later.  But I thought  
this is 101 of software engineering, anyway, and am thus rather  
surprised to find this item on the list.


3. FAILED MATCHES SHOULD NOT POLLUTE THE ENCLOSING NAMESPACE

 This is a slightly wider issue that has obviously sparked an entire  
discussion on this mailing list on scopes.


If there is a good solution that only assigns variables once the  
entire pattern matched, I would be very happy with that.  However, I  
think that variables should be assigned in full before evaluating any  
guards---even at the risk of the guard failing and variables being  
assigned that are not used later on.  Anything else would obviously  
introduce a mini-scope and lead to shadowing, which hardly improves  
anything with respect to legibility.


4. OBJECTS SHOULD BE ABLE DETERMINE WHICH PATTERNS THEY MATCH

Short version: no!

Class patterns are an extension of instance checks.  Leaving out the  
meta-classes at this point, it is basically the class that is  
responsible for determining if an object is an instance of it.   
Pattern matching follows the same logic, whereas Mark suggests to put  
that upside-down.  Since you certainly do not want to define the  
machinery in each instance, you end up delegating the entire thing to  
the class, anyway.


I find this suggestion also somewhat strange in light of the history  
of our PEPs.  We started with a more complex protocol that would allow  
for customised patterns, which was then ditched because it was felt as  
being too complicated.  There is still a possibility to add it later  
on, of course.  But here we are with Mark proposing to introduce a  
complex protocol again.  It would obviously also mean that we could  
not rely as much on Python's existing 

[Python-Dev] Re: The semantics of pattern matching for Python

[Mark Shannon]

Hi Daniel,

On 20/11/2020 10:50 am, Daniel Moisset wrote:
Hello again Mark, I took some time looking in more detail at your 
proposal, and these are my thoughts. I'm writing with the assumption 
that this proposal describes some "internal" representation of match 
statements which is never exposed to the users (so I'd mostly steer away 
from lexical/syntactic preferences).


My first general thought is that this is a useful way to describe and 
discuss implementation, although I wouldn't wait on refinement of this 
to choose whether to accept/reject PEP 634 (or 642, or your favourite 
alternative), work can be done on parallel. It may be a good idea to 
wait for your proposal to be refined before landing a specific 
implementation into CPython (including how the chosen implementation, 
assuming it's accepted, desugars into your semantics).


Going into more details, here's a list of thoughts on more specific points:

 1. You mention a goal about "erroneous patterns" (which I'm still not
sure I agree on), and your proposal addresses that by forcing
classes to be explicit (via __atributes__ and __deconstruct__) about
what attributes are accepted as matches. This is against one design
principle that's not in your list but it was (at least implicitly)
in PEP622 and 634: "pattern matching must allow matching objects on
types not specifically designed for it"; this will allow to apply
this feature to classes that you can not modify (like instances
created by a 3rd party library ). That's why PEP634 relies on
getattr() ; that could be extended in the feature (providing some
override using attributes like yours) but that wouldn't be required
by default


Why force pattern matching onto library code that was not designed for 
pattern matching? It seems risky.


Fishing arbitrary attributes out of an object and assuming that the 
values returned by attribute lookup are equivalent to the internal 
structure breaks abstraction and data-hiding.


An object's API may consist of methods only. Pulling arbitrary 
attributes out of that object may have all sorts of unintended side-effects.


PEP 634 and the DLS paper assert that deconstruction, by accessing 
attributes of an object, is the opposite of construction.

This assertion seems false in OOP.

You might consider OOP as a hangover of the 1990s and that FP is the way 
forward, and you may well be right, but the vast majority of Python 
libraries have a distinctly OO flavor, and we need to respect that.


When we added the "with" statement, there was no attempt to force 
existing code to support it. We made the standard library support it, 
and let the community add support as and when it suited them.


We should do the same with pattern matching.


 2. We considered and discussed something similar to the __deconstruct__
approach (as an override for getattr()), and also noted is
potentially expensive, requiring to allocate an object and evaluate
*all* attributes, even if only one is required. That is against the
principle of " it should perform at least as well as an equivalent
sequence of if, elif statements."


The majority of cases will want more than one attribute, and evaluating 
all attributes at once may well be more efficient than doing so one at a 
time, especially for builtin classes.


I assure you that this is not going to be a performance bottleneck until 
the implementation is a lot slicker than the PEP 634 "reference 
implementation".



 3. You removed or patterns, and added multiple "case P" for each case
body. This easily covers cases where the multiple options are at top
level in the pattern, but if the or-pattern is in a subpattern you
have to duplicate much of the outer context. And this "duplication"
is actually exponential on the number of or patterns, so for
matching the pattern "[0|1, 0|1, 0|1, 'foo'|'bar'|'baz']" you need
to desugar this into 24 case clauses.


Why is this a problem? The compiler can handle millions of cases without 
it being a problem. The number of tests to distinguish N cases is 
log(N), so the number of tests remains the same.



 4. Something else about decomposing patterns into multiple case clauses
is that it makes it harder to express the constraint that all
alternatives must bind the same variable.


Why? It is a mechanical transformation.


 5. I found a bit confusing to read the multiple cases on top. It looks
like C switch statements, which fall-through by default, but in a
context where some case clauses do fall-through (if empty) and
others aren't (if they have a body, even if it's "pass"). I know
this is not user exposed syntax so it's not that important, but this
made the description harder to read for me.


You'll just have to let go of your C-based preconceptions ;)


 6. Given the previous points, added to not seeing the gains of not
putting the or patterns into the desugared version, I'd 

[Python-Dev] Re: The semantics of pattern matching for Python

[Steve Holden]

Fair enough.


On Fri, Nov 20, 2020 at 11:45 AM Thomas Wouters  wrote:

>
>
> On Fri, Nov 20, 2020 at 8:38 AM Steve Holden  wrote:
>
>> On Thu, Nov 19, 2020 at 8:08 PM Brett Cannon  wrote:
>>
>> All I will say is just because we aren't *required* to implement it in
>> __future__ that doesn't mean we can't or shouldn't. Everything should be
>> done to underline the tentative nature of these developments, or we risk
>> the potential of functionality frozen because "we're already using it in
>> production."
>>
>
> On the other hand, __future__ imports have never been provisional, and it
> was never the *intent *of __future__ imports to be provisional. PEP 236,
> which introduced __future__ imports after a vigorous debate on backward
> compatibility, explicitly states "fully backward- compatible additions
> can-- and should --be introduced without a corresponding future_statement".
> I think it would be very surprising if a feature introduced under a future
> import turned out to go away rather than become the default.
>
> --
> Thomas Wouters 
>
> Hi! I'm an email virus! Think twice before sending your email to help me
> spread!
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OD7YGKBLJFFMXYFMYWQZIX7UPHZG5ZR3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[Thomas Wouters]

On Fri, Nov 20, 2020 at 8:38 AM Steve Holden  wrote:

> On Thu, Nov 19, 2020 at 8:08 PM Brett Cannon  wrote:
>
> All I will say is just because we aren't *required* to implement it in
> __future__ that doesn't mean we can't or shouldn't. Everything should be
> done to underline the tentative nature of these developments, or we risk
> the potential of functionality frozen because "we're already using it in
> production."
>

On the other hand, __future__ imports have never been provisional, and it
was never the *intent *of __future__ imports to be provisional. PEP 236,
which introduced __future__ imports after a vigorous debate on backward
compatibility, explicitly states "fully backward- compatible additions
can-- and should --be introduced without a corresponding future_statement".
I think it would be very surprising if a feature introduced under a future
import turned out to go away rather than become the default.

-- 
Thomas Wouters 

Hi! I'm an email virus! Think twice before sending your email to help me
spread!
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/D7HWCZR2VXU3QWBPSCOP5PMW4E2XFA7D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[Daniel Moisset]

Hello again Mark, I took some time looking in more detail at your proposal,
and these are my thoughts. I'm writing with the assumption that this
proposal describes some "internal" representation of match statements which
is never exposed to the users (so I'd mostly steer away from
lexical/syntactic preferences).

My first general thought is that this is a useful way to describe and
discuss implementation, although I wouldn't wait on refinement of this to
choose whether to accept/reject PEP 634 (or 642, or your favourite
alternative), work can be done on parallel. It may be a good idea to wait
for your proposal to be refined before landing a specific implementation
into CPython (including how the chosen implementation, assuming it's
accepted, desugars into your semantics).

Going into more details, here's a list of thoughts on more specific points:


   1. You mention a goal about "erroneous patterns" (which I'm still not
   sure I agree on), and your proposal addresses that by forcing classes to be
   explicit (via __atributes__ and __deconstruct__) about what attributes are
   accepted as matches. This is against one design principle that's not in
   your list but it was (at least implicitly) in PEP622 and 634: "pattern
   matching must allow matching objects on types not specifically designed for
   it"; this will allow to apply this feature to classes that you can not
   modify (like instances created by a 3rd party library ). That's why PEP634
   relies on getattr() ; that could be extended in the feature (providing some
   override using attributes like yours) but that wouldn't be required by
   default
   2. We considered and discussed something similar to the __deconstruct__
   approach (as an override for getattr()), and also noted is potentially
   expensive, requiring to allocate an object and evaluate *all* attributes,
   even if only one is required. That is against the principle of " it should
   perform at least as well as an equivalent sequence of if, elif statements."
   3. You removed or patterns, and added multiple "case P" for each case
   body. This easily covers cases where the multiple options are at top level
   in the pattern, but if the or-pattern is in a subpattern you have to
   duplicate much of the outer context. And this "duplication" is actually
   exponential on the number of or patterns, so for matching the pattern
   "[0|1, 0|1, 0|1, 'foo'|'bar'|'baz']" you need to desugar this into 24 case
   clauses.
   4. Something else about decomposing patterns into multiple case clauses
   is that it makes it harder to express the constraint that all alternatives
   must bind the same variable.
   5. I found a bit confusing to read the multiple cases on top. It looks
   like C switch statements, which fall-through by default, but in a context
   where some case clauses do fall-through (if empty) and others aren't (if
   they have a body, even if it's "pass"). I know this is not user exposed
   syntax so it's not that important, but this made the description harder to
   read for me.
   6. Given the previous points, added to not seeing the gains of not
   putting the or patterns into the desugared version, I'd prefer it to be
   included in the desugaring.
   7. I think there's some surprising behaviour in the assignments being
   done after a successful match but before the guard is evaluated. In your
   proposal the guard has no access to the variables, so it has to be compiled
   differently (using $0, $1, ... rather than the actual names that appear in
   the expression). And if this guard calls a function which exposes those
   variables in any way (for example if the variable is in a closure) I think
   the behaviour may be unexpected /surprising; same if I stop a debugger
   inside that function and try to inspect the frame where the matching
   statement is.
   8. I like your implementation approach to capture on the stack and then
   assign. I was curious if you considered, rather than using a variable
   number of stack cells using a single object/dict to store those values. The
   compiler and the generated bytecode could end up being simpler, and you
   need less stack juggling and possibly no PEEK operation. a small list/array
   would suffice, but a dict may provide further debugging opportunities (and
   it's likely that a split table dict could make the representation quite
   compact). I know this is less performant but I'm also thinking of
   simplicity.
   9. I think your optimisation approaches are great, the spec was made lax
   expecting for people like you to come up with a proposal of this kind :) I
   don't think the first implementation of this should be required to
   optimize/implement things in a certain way, but if the spec is turned into
   implementation dependent and then fixed, it shouldn't break anything (it's
   like the change in dictionary order moving to "undefined/arbitrary" to
   "preserving insertion order") and can be done later one

Thanks 

[Python-Dev] Re: The semantics of pattern matching for Python

[Steve Holden]

On Thu, Nov 19, 2020 at 8:08 PM Brett Cannon  wrote:

>
>
> On Thu, Nov 19, 2020 at 5:16 AM Steve Holden  wrote:
>
>> On Wed, Nov 18, 2020 at 7:04 PM Daniel Moisset 
>> wrote:
>>
>>> [sorry for the duplicate, meant to reply-all]
>>>
>>> Thank you for this approach, I find it really helpful to put the
>>> conversation in these terms (semantics and guiding principles).
>>>
>>> It does help to rationalise discussion ;-)
>>
>>> [...]
>>> 4. "Objects should be able determine which patterns they match."
>>> This is something that you and I, and most of the authors of 622 agree
>>> on. What we found out when discussing this is that we didn't have clear
>>> what and how to open that customization. Some customization options added a
>>> lot of complexity at the cost of performance, some others were very simple
>>> but it wasn't clear that they would be actually useful, or extensible in
>>> the future. This has a lot to do with this being a somewhat new paradigm in
>>> Python, and our lack of knowledge on what the user community may do with it
>>> beyond what we imagined. So the decision was "pattern matching as it is
>>> presented without extensibility is useful, let's get this in, and once we
>>> see how it is used in the wild we'll understand better what kind of
>>> extensibility is valuable". For a crude analogy, imagine trying to get the
>>> descriptor protocol right when the basic python object model was being
>>> implemented. These things happened as different times as the use of the
>>> language evolved, and my opinion is that customization of the match
>>> protocol must follow a similar path.
>>>
>>>
>> I haven't so far understood whether the change proposed is intended to
>> start out in __future__. Given the preliminary nature of this development,
>> it would seem wise to retain the option to withdraw it or modify it
>> radically before production code depends on it.
>>
>
> The current proposal is not for it to be put behind a __future__ statement
> as it doesn't conflict with or break any pre-existing source code.
>

Ok, thanks for the clarification.

All I will say is just because we aren't *required* to implement it in
__future__ that doesn't mean we can't or shouldn't. Everything should be
done to underline the tentative nature of these developments, or we risk
the potential of functionality frozen because "we're already using it in
production."

The discussion to date suggests that such a breathing space would be
useful.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TUO5S5UZJ3SLNCM2DNCEJ5KTGYNDCAFD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[Brett Cannon]

On Thu, Nov 19, 2020 at 5:16 AM Steve Holden  wrote:

> On Wed, Nov 18, 2020 at 7:04 PM Daniel Moisset 
> wrote:
>
>> [sorry for the duplicate, meant to reply-all]
>>
>> Thank you for this approach, I find it really helpful to put the
>> conversation in these terms (semantics and guiding principles).
>>
>> It does help to rationalise discussion ;-)
>
>> [...]
>> 4. "Objects should be able determine which patterns they match."
>> This is something that you and I, and most of the authors of 622 agree
>> on. What we found out when discussing this is that we didn't have clear
>> what and how to open that customization. Some customization options added a
>> lot of complexity at the cost of performance, some others were very simple
>> but it wasn't clear that they would be actually useful, or extensible in
>> the future. This has a lot to do with this being a somewhat new paradigm in
>> Python, and our lack of knowledge on what the user community may do with it
>> beyond what we imagined. So the decision was "pattern matching as it is
>> presented without extensibility is useful, let's get this in, and once we
>> see how it is used in the wild we'll understand better what kind of
>> extensibility is valuable". For a crude analogy, imagine trying to get the
>> descriptor protocol right when the basic python object model was being
>> implemented. These things happened as different times as the use of the
>> language evolved, and my opinion is that customization of the match
>> protocol must follow a similar path.
>>
>>
> I haven't so far understood whether the change proposed is intended to
> start out in __future__. Given the preliminary nature of this development,
> it would seem wise to retain the option to withdraw it or modify it
> radically before production code depends on it.
>

The current proposal is not for it to be put behind a __future__ statement
as it doesn't conflict with or break any pre-existing source code.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BN4HQBX72L5GMVNEBGQDOSF6F3L4VRTS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[Steve Holden]

On Wed, Nov 18, 2020 at 7:04 PM Daniel Moisset  wrote:

> [sorry for the duplicate, meant to reply-all]
>
> Thank you for this approach, I find it really helpful to put the
> conversation in these terms (semantics and guiding principles).
>
> It does help to rationalise discussion ;-)

> [...]
> 4. "Objects should be able determine which patterns they match."
> This is something that you and I, and most of the authors of 622 agree on.
> What we found out when discussing this is that we didn't have clear what
> and how to open that customization. Some customization options added a lot
> of complexity at the cost of performance, some others were very simple but
> it wasn't clear that they would be actually useful, or extensible in the
> future. This has a lot to do with this being a somewhat new paradigm in
> Python, and our lack of knowledge on what the user community may do with it
> beyond what we imagined. So the decision was "pattern matching as it is
> presented without extensibility is useful, let's get this in, and once we
> see how it is used in the wild we'll understand better what kind of
> extensibility is valuable". For a crude analogy, imagine trying to get the
> descriptor protocol right when the basic python object model was being
> implemented. These things happened as different times as the use of the
> language evolved, and my opinion is that customization of the match
> protocol must follow a similar path.
>
>
I haven't so far understood whether the change proposed is intended to
start out in __future__. Given the preliminary nature of this development,
it would seem wise to retain the option to withdraw it or modify it
radically before production code depends on it.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EBFOBAFOJPMKDFEQGT7ZN46R6QVM5UH3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[Daniel Moisset]

[sorry for the duplicate, meant to reply-all]

Thank you for this approach, I find it really helpful to put the
conversation in these terms (semantics and guiding principles).

This is not an answer to the proposal (which I've read and helps me
contextualize) but to your points below and how they apply to PEP-634. I'm
also answering personally, with a reasonable guess about what the other
authors of 634-636 would agree, but they may correct me if I'm wrong.

On Mon, 16 Nov 2020 at 14:44, Mark Shannon  wrote:

> (...)
> I believe that a pattern matching implementation must have the following
> properties:
>
> * The semantics must be precisely defined.
> * It must be implemented efficiently.
> * Failed matches must not pollute the enclosing namespace.
> * Objects should be able determine which patterns they match.
> * It should be able to handle erroneous patterns, beyond just syntax
> errors.
>
> PEP 634 and PEP 642 don't have *any* of these properties.
>

Let me answer this one by one:

1. "The semantics must be precisely defined":
If this happens in PEP634 I don't think it was intentional, and I'm pretty
sure the authors would be happy to complete any incompleteness that it has.
I would happily have a more accurate description (I drafted a non-official
one for a much earlier version of PEP-622,
https://github.com/dmoisset/notebook/blob/master/python/pep622/semantic-specs.md
).
Can you clarify where you see these imprecisions?

2. "It must be implemented efficiently":
I don't think "efficient implementation" was a priority in PEP634,
although I saw your proposal defines this as "same performance as the
equivalent if statement", and I'm quite sure that level of performance can
be achieved (if it isn't already by Brandt's implementation). Finding the
best way to optimise wasn't a priority, but I think if there was anything
in our implementation that would make optimisations harder we would
consider them as a change. Do you think anything like that has been
presented?

3. "Failed matches must not pollute the enclosing namespace":
This for me is one of the less-desirable parts of the proposal, and was
agreed more as a matter of practicality and an engineering tradeoff. If you
have a reasonable way of solving this (like putting matched variables in
the stack and popping it later) reasonably I'd be much happier putting that
in.

4. "Objects should be able determine which patterns they match."
This is something that you and I, and most of the authors of 622 agree on.
What we found out when discussing this is that we didn't have clear what
and how to open that customization. Some customization options added a lot
of complexity at the cost of performance, some others were very simple but
it wasn't clear that they would be actually useful, or extensible in the
future. This has a lot to do with this being a somewhat new paradigm in
Python, and our lack of knowledge on what the user community may do with it
beyond what we imagined. So the decision was "pattern matching as it is
presented without extensibility is useful, let's get this in, and once we
see how it is used in the wild we'll understand better what kind of
extensibility is valuable". For a crude analogy, imagine trying to get the
descriptor protocol right when the basic python object model was being
implemented. These things happened as different times as the use of the
language evolved, and my opinion is that customization of the match
protocol must follow a similar path.

5. "It should be able to handle erroneous patterns, beyond just syntax
errors."
I'll be answering this based on the example in your document, matching
RemoteCount(success=True,
count=count) where RemoteCount is a namedtuple. The code is likely an
error, and I'm in general all for reporting errors early, but the kind of
error detection proposed here is the kind of errors that python normally
ignore. I find that example really similar to the kind of error you could
make writing "if remcount.count == 3: ..." or going beyond this example
"maxelems = configfile.read(); if len(elems) == maxelems: ...". There are
many type errors that python ignores (especially related to equality), and
Python has already made the decision of allowing mixed type equality
comparisons everywhere, so why do you think pattern matching should be
different with respect to this? In my opinion trying to "fix" this (or even
agreeing if this is a bug or not) is a much more general issue unrelated to
pattern matching. Given the current status-quo I normally trust python type
checkers to help me with these errors, and I'd expect them to do the same
with the "erroneous" match statement.
If there are other examples you had in mind when you wrote this I'd also be
happy to discuss those.

I'll try to get some time to review your specific counterproposal later,
thanks for it.

Best,
Daniel

>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to 

[Python-Dev] Re: The semantics of pattern matching for Python

[Mark Shannon]

Hi Guido,

On 16/11/2020 4:41 pm, Guido van Rossum wrote:

Thanks Mark, this is a helpful and valuable contribution.

I will try to understand and review it in the coming weeks (there is no 
hurry since the decision is up to the next SC) but I encourage you to 
just put it in PEP form and check it into the PEP repo.


Because I only skimmed very briefly, I don't have an answer to one 
question: does your PEP also define a precise mapping from the PEP 634 
syntax to your "desugared" syntax? I think that ought to be part of your 
PEP.


No it doesn't define a precise mapping, and I don't think it will.
I'm not familiar enough with ever corner of PEP 634 to do that.
I could add a general "how to" guide though. It fairly straightforward 
conceptually but, as you know, the devil is in the details.


Cheers,
Mark.



--Guido

On Mon, Nov 16, 2020 at 6:41 AM Mark Shannon > wrote:


Hi everyone,

There has been much discussion on the syntax of pattern matching for
Python (in case you hadn't noticed ;)

Unfortunately the semantics seem to have been somewhat overlooked.
What pattern matching actually does seems at least as important as the
syntax.


I believe that a pattern matching implementation must have the
following
properties:

* The semantics must be precisely defined.
* It must be implemented efficiently.
* Failed matches must not pollute the enclosing namespace.
* Objects should be able determine which patterns they match.
* It should be able to handle erroneous patterns, beyond just syntax
errors.

PEP 634 and PEP 642 don't have *any* of these properties.


I've written up a document to specify a possible semantics of pattern
matching for Python that has the above properties, and includes reasons
why they are necessary.


https://github.com/markshannon/pattern-matching/blob/master/precise_semantics.rst

It's in the format of a PEP, but it isn't a complete PEP as it lacks
surface syntax.

Please, let me know what you think.

Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org

To unsubscribe send an email to python-dev-le...@python.org

https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at

https://mail.python.org/archives/list/python-dev@python.org/message/BTPODVYLPKY5IHWFKYQJICONTNTRNDB2/
Code of Conduct: http://python.org/psf/codeofconduct/



--
--Guido van Rossum (python.org/~guido )
/Pronouns: he/him //(why is my pronoun here?)/ 


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EDHCIYYAGW4YRLWD6BKLTQY7FRNNTZH7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The semantics of pattern matching for Python

[Guido van Rossum]

Thanks Mark, this is a helpful and valuable contribution.

I will try to understand and review it in the coming weeks (there is no
hurry since the decision is up to the next SC) but I encourage you to just
put it in PEP form and check it into the PEP repo.

Because I only skimmed very briefly, I don't have an answer to one
question: does your PEP also define a precise mapping from the PEP 634
syntax to your "desugared" syntax? I think that ought to be part of your
PEP.

--Guido

On Mon, Nov 16, 2020 at 6:41 AM Mark Shannon  wrote:

> Hi everyone,
>
> There has been much discussion on the syntax of pattern matching for
> Python (in case you hadn't noticed ;)
>
> Unfortunately the semantics seem to have been somewhat overlooked.
> What pattern matching actually does seems at least as important as the
> syntax.
>
>
> I believe that a pattern matching implementation must have the following
> properties:
>
> * The semantics must be precisely defined.
> * It must be implemented efficiently.
> * Failed matches must not pollute the enclosing namespace.
> * Objects should be able determine which patterns they match.
> * It should be able to handle erroneous patterns, beyond just syntax
> errors.
>
> PEP 634 and PEP 642 don't have *any* of these properties.
>
>
> I've written up a document to specify a possible semantics of pattern
> matching for Python that has the above properties, and includes reasons
> why they are necessary.
>
>
> https://github.com/markshannon/pattern-matching/blob/master/precise_semantics.rst
>
> It's in the format of a PEP, but it isn't a complete PEP as it lacks
> surface syntax.
>
> Please, let me know what you think.
>
> Cheers,
> Mark.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/BTPODVYLPKY5IHWFKYQJICONTNTRNDB2/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WNOYRSRU6HNSZIQGO3YLQ6H2GJK6GGHP/
Code of Conduct: http://python.org/psf/codeofconduct/