[Python-Dev] Re: PEP 622: Structural Pattern Matching (version 2)
Mark Shannon wrote: > In future, could you avoid editing emails when replying to them? > A lot of context can get lost. I'll add another voice to Ethan's saying that I appreciate having as much as possible trimmed. As long as people are arguing in good faith (and I assume that they are here), the loss of context is usually pretty small, and outweighed by being able to find the newly important part more quickly. (I say this as someone reading and replying through the archives, in a way that seems to mangle quoted portions -- but they are usually still good enough to be useful.) > ... Yes, the pattern matching syntax is more concise in some > circumstances, but there still needs to be justification why a > combination of simpler language changes is insufficient. Code that fits on a single small (24x80) screen (without getting too dense) is much easier to understand, because I can see it all at once, instead of needing to remember things while I flip back and forth. A combination of simpler changes might be fine, but shorter really is very valuable, all by itself. > Saves two lines of code, but introduces two bugs! > (Assuming that the original behavior should be preserved) That is a pretty big assumption. I'm guessing that at least one of the "bugs" is that sequences other than the built-in list and tuple are now also accepted. Maybe there are good reasons to exclude other sequences, but ... in my experience, the reason is usually that someone didn't think of it, and no one felt strongly enough to fix it yet. That would mean pattern matching led to a bug *fix*. Closed source code tends to be even more fragile, though it also tends to not see as many surprising input types in the first place. > If you don't have access to the original source, then it can be made a > function, not a method. Even when I have read access, I may not have write access. Creating and naming a separate match function every place I need to match isn't quite boilerplate, but it adds enough code to feel that way. Trying to use a single match function with parameters gets ugly in a different way. I'm not quite convinced that this PEP has found the magic solution, but the goal is clearly worthy. > On 14/07/2020 5:25 pm, Tobias Kohn wrote: > > match response.status: > > case HTTP_RESPONSE.UPGRADE_REQUIRED: > Are you suggesting that all constants live in a separate module? The limit to dotted names is a possibly temporary wart. But even with that restriction, there is no reason you can't gather constants on an object first, and use its attributes. > For a PEP to succeed it needs to show ... > That the proposed change is the best known solution for the > problem being addressed. I think the bigger barrier is "although never is often better than right now," and hope that a better solution will be found later, and fear that backwards compatibility with this would block that better solution. (To be very explicit, I personally abstain on this, because I am not sure whether this is "good enough", nor am I confident a better solution can ever be found.) > I worry that the PEP is treating pattern matching as an ideal which we > should be striving towards. That is a bad thing, IMO. Fair. Like annotations as a typing system, you can personally ignore it if it isn't helpful, but there is still some ecosystem cost. I would like to see more old/new comparisons to judge how intrusive this will be, but eventually there will be (or not be) a leap of faith. In the past, most of those have worked out. > Pattern matching is well suited to statically typed functional languages. and statically typed data domains and communications protocols -jJ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4S75QY4FVVXW7YGTSFRHEXBK5OXXJM6Y/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: PEP 622: Structural Pattern Matching (version 2)
On 07/14/2020 11:05 AM, Mark Shannon wrote: On 14/07/2020 5:25 pm, Tobias Kohn wrote: On 07/14/2020, Mark Shannon wrote: In future, could you avoid editing emails when replying to them? A lot of context can get lost. I appreciate posters who take the time to trim the parts of an email that they are not replying to. Why the use of "shape" in "scare quotes"? Because the `shape' of an object is not something that is a well-defined term, but rather addresses the intuitive understanding to explain what we are talking about. It is usually the intention of the abstract to give a rough overview, before delving into the details and more formal descriptions. You ignore my point about explaining PEP 622 directly. Sure, the abstract must give a high level overview, but it should be able to do that without so much hand waving. I find the PEP abstract concrete enough. Perhaps you could write a replacement that you think does a better job? Actually, no. Pattern matching is not about _avoiding_ anything, it is about a more concise syntax. The suggestion that we wanted to avoid calls to `len` or `isinstance` is similar to the idea of using a lambda to avoid a function. The objective of pattern matching is to introduce a more succinct and readable way to express structure. It's all still there but we want to focus on the structure rather than `isinstance` calls. I should have been clearer. Avoiding *explicit* calls to `len` and `isinstance`. Yes, the pattern matching syntax is more concise in some circumstances, but there still needs to be justification why a combination of simpler language changes is insufficient. My apologies if I missed it, but do you have some ideas of these simpler language changes that would do the job? Pattern matching already exists in some limited way in Python. Even today you can write: `a, b = value` I think most of us are aware of unpacking in Python ;) Yes, but not everyone recognizes that unpacking is a form of pattern matching. instead of: `a = value[0] b = value[1]` The idea of this is, of course, not so much about avoiding item access, but about having a more concise syntax for it. Moreover, saying this only saves a single line and is therefore rather useless is quite beside the point. It is about a better representation of what the code is supposed to do and not about saving lines. How do you quantify "better representation" then? You mean, without large study groups and lots of money and/or time? Not easily. I can tell you that a, b = value is much more informative, and easier to read and process, than the alternative of using item assignment. Saves two lines of code, but introduces two bugs! (Assuming that the original behavior should be preserved) Thank you for pointing out the bug in our example. This highlights, however, rather the difficulty of refactoring (did I already mention the issue of the 'context' that real code is embedded in?), than any shortcoming of pattern matching as a tool. And by the way: constructive critisism would perhaps include explicitly naming the two bugs. I'm not sure it would be more constructive, this is not a bug report or code review after all. It should be. If the PEP is to be decided on it should be as correct as possible. The point I am making is this: It is far too easy to make mistakes using the constructs proposed in PEP 622. You're point is false. They weren't crafting new code, they were refactoring. For example, by adding a simple "matches" method to Node and Leaf, `is_tuple` can be rewritten as something like: This assumes that you have full control over the entire code. But if you are using some third-party library, you cannot "simply add a `matches` method" (without some substantial trickery). Moreover, there is quite some work needed to define such a `matches` function as you propose it (including support for the ellipsis as wildcard). In effect, you would have to implement large parts of the pattern matching for just this simple example. Whereas we believe that it has merit enough to have the compiler do it for a wide range of possible use cases. You wouldn't have to implement large parts of the pattern matching, because this would be specific to one class. There are only two or three cases to deal with. If you don't have access to the original source, then it can be made a function, not a method. So, a few lines for that example, another few for a different example, some more for that example over there -- how many times do we have to write similar code before we let Python do the grunt work for us? Preventing the use of symbolic constants or other complex rvalues is a impairment to usability. First, let me quote what the PEP has to say on that exact example: Although this will work, it's not necessarily what the proposal is focused on. So why does the PEP include it as one of just three examples? To
[Python-Dev] Re: PEP 622: Structural Pattern Matching (version 2)
Hi Tobias, In future, could you avoid editing emails when replying to them? A lot of context can get lost. On 14/07/2020 5:25 pm, Tobias Kohn wrote: Hi Mark, Thank you for your message. I might be able to answer some of the questions and also address some issues with the underlying assumptions in your email---after all, we would most certainly want to avoid discussing and reasoning about straw men, as you yourself have repeatedly pointed out. > Why the use of "shape" in "scare quotes"? Because the `shape' of an object is not something that is a well-defined term, but rather addresses the intuitive understanding to explain what we are talking about. It is usually the intention of the abstract to give a rough overview, before delving into the details and more formal descriptions. You ignore my point about explaining PEP 622 directly. Sure, the abstract must give a high level overview, but it should be able to do that without so much hand waving. > Using a contrived example like this seems like a straw man. It feels like it is constructed to favour the PEP, whilst being unlike any real code. It is a trait of many (good) tutorials to present a new structure with an example that highlights its feature in an approachable way rather than coming up with `real code'---particularly given that real code is always surrounded by a larger context, which rather clouds any understanding. And it would only be a straw man had we constructed a contrived example of unrealistic code that we then showed to be simplified by pattern matching. That's not the case: it really is just a simple educational example to present the idea. The example given in the PEP does seems like a "contrived example of unrealistic code", as you put it. Is there real code like this? BTW: as a matter of course, it is constructed in favour of the PEP! > There seem to be three things you want to enhance here: > Unpacking; to avoid calls to `len` > Type checking; to avoid calls to `isinstance`. > To avoiding nesting by using complex lvalues. Actually, no. Pattern matching is not about _avoiding_ anything, it is about a more concise syntax. The suggestion that we wanted to avoid calls to `len` or `isinstance` is similar to the idea of using a lambda to avoid a function. The objective of pattern matching is to introduce a more succinct and readable way to express structure. It's all still there but we want to focus on the structure rather than `isinstance` calls. I should have been clearer. Avoiding *explicit* calls to `len` and `isinstance`. Yes, the pattern matching syntax is more concise in some circumstances, but there still needs to be justification why a combination of simpler language changes is insufficient. Pattern matching already exists in some limited way in Python. Even today you can write: `a, b = value` I think most of us are aware of unpacking in Python ;) instead of: `a = value[0] b = value[1]` The idea of this is, of course, not so much about avoiding item access, but about having a more concise syntax for it. Moreover, saying this only saves a single line and is therefore rather useless is quite beside the point. It is about a better representation of what the code is supposed to do and not about saving lines. How do you quantify "better representation" then? > Saves two lines of code, but introduces two bugs! > (Assuming that the original behavior should be preserved) Thank you for pointing out the bug in our example. This highlights, however, rather the difficulty of refactoring (did I already mention the issue of the 'context' that real code is embedded in?), than any shortcoming of pattern matching as a tool. And by the way: constructive critisism would perhaps include explicitly naming the two bugs. I'm not sure it would be more constructive, this is not a bug report or code review after all. The point I am making is this: It is far too easy to make mistakes using the constructs proposed in PEP 622. > For example, by adding a simple "matches" method to Node and Leaf, `is_tuple` can be rewritten as something like: This assumes that you have full control over the entire code. But if you are using some third-party library, you cannot "simply add a `matches` method" (without some substantial trickery). Moreover, there is quite some work needed to define such a `matches` function as you propose it (including support for the ellipsis as wildcard). In effect, you would have to implement large parts of the pattern matching for just this simple example. Whereas we believe that it has merit enough to have the compiler do it for a wide range of possible use cases. You wouldn't have to implement large parts of the pattern matching, because this would be specific to one class. There are only two or three cases to deal with. If you don't have access to the original source, then it can be made a function,
[Python-Dev] Re: PEP 622: Structural Pattern Matching (version 2)
Hi Mark, Thank you for your message. I might be able to answer some of the questions and also address some issues with the underlying assumptions in your email---after all, we would most certainly want to avoid discussing and reasoning about straw men, as you yourself have repeatedly pointed out. Why the use of "shape" in "scare quotes"? Because the `shape' of an object is not something that is a well-defined term, but rather addresses the intuitive understanding to explain what we are talking about. It is usually the intention of the abstract to give a rough overview, before delving into the details and more formal descriptions. Using a contrived example like this seems like a straw man. It feels like it is constructed to favour the PEP, whilst being unlike any real code. It is a trait of many (good) tutorials to present a new structure with an example that highlights its feature in an approachable way rather than coming up with `real code'---particularly given that real code is always surrounded by a larger context, which rather clouds any understanding. And it would only be a straw man had we constructed a contrived example of unrealistic code that we then showed to be simplified by pattern matching. That's not the case: it really is just a simple educational example to present the idea. BTW: as a matter of course, it is constructed in favour of the PEP! There seem to be three things you want to enhance here: Unpacking; to avoid calls to `len` Type checking; to avoid calls to `isinstance`. To avoiding nesting by using complex lvalues. Actually, no. Pattern matching is not about _avoiding_ anything, it is about a more concise syntax. The suggestion that we wanted to avoid calls to `len` or `isinstance` is similar to the idea of using a lambda to avoid a function. The objective of pattern matching is to introduce a more succinct and readable way to express structure. It's all still there but we want to focus on the structure rather than `isinstance` calls. Pattern matching already exists in some limited way in Python. Even today you can write: `a, b = value` instead of: `a = value[0] b = value[1]` The idea of this is, of course, not so much about avoiding item access, but about having a more concise syntax for it. Moreover, saying this only saves a single line and is therefore rather useless is quite beside the point. It is about a better representation of what the code is supposed to do and not about saving lines. Saves two lines of code, but introduces two bugs! (Assuming that the original behavior should be preserved) Thank you for pointing out the bug in our example. This highlights, however, rather the difficulty of refactoring (did I already mention the issue of the 'context' that real code is embedded in?), than any shortcoming of pattern matching as a tool. And by the way: constructive critisism would perhaps include explicitly naming the two bugs. For example, by adding a simple "matches" method to Node and Leaf, `is_tuple` can be rewritten as something like: This assumes that you have full control over the entire code. But if you are using some third-party library, you cannot "simply add a `matches` method" (without some substantial trickery). Moreover, there is quite some work needed to define such a `matches` function as you propose it (including support for the ellipsis as wildcard). In effect, you would have to implement large parts of the pattern matching for just this simple example. Whereas we believe that it has merit enough to have the compiler do it for a wide range of possible use cases. I really don't see you how you can claim that `case 406:` is more readable than `elif response == HTTP_UPGRADE_REQUIRED:` ? Preventing the use of symbolic constants or other complex rvalues is a impairment to usability. First, let me quote what the PEP has to say on that exact example: Although this will work, it's not necessarily what the proposal is focused on. Moreover, it is usually a good idea to put constants into a separate namespace that further describes their meaning and intended use. And that is fully supported by the syntax as proposed in the PEP. ``` match response.status: case HTTP_RESPONSE.UPGRADE_REQUIRED: ... ``` For a PEP to succeed it needs to show two things. 1. Exactly what problem is being solved, or need is to be fulfilled, and that is a sufficiently large problem, or need, to merit the proposed change. 2. That the proposed change is the best known solution for the problem being addressed. IMO, PEP 622 fails on both counts. This seems fair enough as far as the two issues are concerned. It might seem indeed as if pattern matching does not add anything that could not be done already in Python. Part of the problem is that pattern matching starts to really shine when the objects and data
[Python-Dev] Re: PEP 622: Structural Pattern Matching (version 2)
Hi,
As I wrote in an earlier email:
For a PEP to succeed it needs to show two things.
1. Exactly what problem is being solved, or need is to be fulfilled, and that
is a sufficiently large problem, or need, to merit the proposed change.
2. That the proposed change is the best known solution for the problem being
addressed.
IMO, PEP 622 fails on both counts.
This email addresses point 1 (for version 2).
Abstract
Why the use of "shape" in "scare quotes"?
It worries me that the abstract can't explain what PEP 622 does directly.
Could you use a real example in the abstract?
Using a contrived example like this seems like a straw man.
It feels like it is constructed to favour the PEP, whilst being unlike
any real code.
Rationale and Goals
---
Python programs frequently need to handle data which varies in type, presence of attributes/keys, or number of elements. Typical examples are operating on nodes of a mixed structure like an AST, handling UI events of different types, processing structured input (like structured files or network messages), or “parsing” arguments for a function that can accept different combinations of types and numbers of parameters.
AST, and UI objects usually (pretty much always) form a class hierarchy,
making is easy to add utility matching methods to the base class.
Where matching *might* have some value is when unrelated types are
involved, but you need to show that enhanced destructuring would be
insufficient.
In fact, the classic 'visitor' pattern is an example of this, done in an OOP
style -- but matching makes it much less tedious to write.
This might be true of the "classic" visitor pattern, but that's a straw man.
The Python visitor pattern is a joy to use.
Each case gets its own self contained method, clearly named, which is
much better than a giant `match` statement.
Much of the code to do so tends to consist of complex chains of nested if/elif
statements, including multiple calls to len(), isinstance() and
index/key/attribute access. Inside those branches users sometimes need to
destructure the data further to extract the required component values, which
may be nested several objects deep.
There seem to be three things you want to enhance here:
Unpacking; to avoid calls to `len`
Type checking; to avoid calls to `isinstance`.
To avoiding nesting by using complex lvalues.
You fail to justify why the first two cannot be handled separately with
much simpler extensions to the language and why complex lvalues are
better than nesting.
The examples
There are only three examples, none of which are compelling.
In fact, two of them serve as a warning against the additional
complexity this PEP entails.
Django example:
'''
Saves two lines of code, but introduces two bugs!
(Assuming that the original behavior should be preserved)
If the authors of the PEP cannot use this feature correctly in the first
example they give, what chance do the rest of us have?
is_tuple example:
'
(Repeating my earlier email)
Python's support for OOP provides an alternative to ADTs.
For example, by adding a simple "matches" method to Node and Leaf,
`is_tuple` can be rewritten as something like:
def is_tuple(node):
if not isinstance(node, Node):
return False
return node.matches("(", ")") or node.matches("(", ..., ")")
The "switch on http response codes" example.
This clearly demonstrates the value of symbolic constants for
readability and the serious flaw in the PEP that I cannot use them.
I really don't see you how you can claim that
`case 406:`
is more readable than
`elif response == HTTP_UPGRADE_REQUIRED:`
?
Preventing the use of symbolic constants or other complex rvalues is a
impairment to usability.
Silently failing when symbolic constants are used is terrible.
I do see the value of a `switch` statement for repeated tests against
the same value; its intent is clearer than repeated `elif`s. But there
is no need for it to be so hard to use.
Cheers,
Mark.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/IPRBLWDE64FACRUKORC32QLHRT3NFBVI/
Code of Conduct: http://python.org/psf/codeofconduct/
