[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Greg Ewing]

On 26/06/20 1:08 pm, Paul Svensson wrote:

We already allow (x, x) = (1, 2)
So, why do we need to disallow binding several values to the same name ?


I think it was done because people might expect that to match
only if the *same* value appears in both places. Some other
languages have pattern matching that works that way. I think
the intention is to leave open the possibility of implementing
it in the future.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RFBU4JCJACOZSJI6AVDUAIM7ACVQNOV7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Greg Ewing]

On 26/06/20 2:10 pm, Gregory P. Smith wrote:

match get_shape() as shape:
   case start, end := Line@(shape):


This looks just as inscrutable to me in its own way.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DIBSDVYS2JURMXSGL3EXWSIDCGJFQTYN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The Anti-PEP

[Greg Ewing]

On 26/06/20 2:18 pm, Gregory P. Smith wrote:
Regardless i don't see how an anti-pep would work much better, but I 
also don't see anything stopping anyone from trying one.  I worry that 
it'll fragment conversation even more and separate discussions so that 
everyone is even more confused about overall opinion tallies?  one way 
to find out...


Maybe there should be a section in each PEP for counter-arguments
that is maintained by another appointed person, independent of the
PEP author and its sponsor. The position could be called "Devil's
Advocate".

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PEUKMXKJGG3YCY2WOCFVIUB3IJPAUHH2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Greg Ewing]

On 26/06/20 5:45 pm, Ned Deily wrote:

Hasn't "|" been used in a similar way for decades in Unix/POSIX shell patterns, 
including with the shell case statement?


Yes, but I regard Unix shell syntaxes as just plain weird
and not a good model for designing a language on.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/C4INFHUOJDPG3D34QDWXFCUXOGXS4AJ2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching -- followup

[Greg Ewing]

On 26/06/20 6:21 am, Pablo Galindo Salgado wrote:
Which means that users can do a positional match against the proxy with 
a name pattern:


match input:
     case datetime.date(dt):
         print(f"The date {dt.isoformat()}"


I think that would be an incorrect way for matching on datetime
to behave. Since datetime has a constructor that takes positional
arguments, it should have a __match__ and/or __match_args__
that agrees.

This suggests that there will be a burden on many existing types
to ensure they implement appropriate matching behaviour, as the
default behaviour provided by object will be wrong for them.

I'm wondering whether the default "single positional match"
behaviour is a bad idea. I.e. the only thing that should work
by default is

   case someclass():

and not

   case someclass(instance):

Classes such as int with constructors that can take a single
argument should be required to implement the corresponding
match behaviour explicitly.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TWLPWBFQHYPRORJYN5YPV3WNYMYSJXMS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Rob Cliffe via Python-Dev]

Some proof-reading of the PEP.  I apologise if this is out of date.

1) In the beginning of the "Mapping Pattern" section:
         "{" (pattern ":" pattern)+ "}"
    This is spelt inconsistently: there is a `+` before the closing `}` 
but not after the opening `{`.


2) The second code snippet in the "Guards" section:

values = [0]

match value:
case [x] if x:
...  # This is not executed
case _:
...
print(x)  # This will print "0"

   Inconsistent spelling: `values` and `value`

3) At the end of the "Named sub-patterns" section:
"PEP 572"
   It would be more helpful to say "PEP 572 (Assignment Expressions)"

Rob Cliffe

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YZILPUKNGK4DX5YPEFETNJ7RCELXLCCP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Emily Bowman]

On Fri, Jun 26, 2020 at 12:42 AM Rob Cliffe via Python-Dev <
python-dev@python.org> wrote:

> 1) In the beginning of the "Mapping Pattern" section:
>  "{" (pattern ":" pattern)+ "}"
> This is spelt inconsistently: there is a `+` before the closing `}`
> but not after the opening `{`.
>

That grammar is more like a regex: + means "accept one or more of the
previous" here.

Maybe grammar inserts should be their own subsections to avoid confusion,
or at least highlighted, instead of mixing into the text.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/G2HL7KKWSZRQQ5TL5ZD5M3K2BPEUMLJK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The Anti-PEP

[Paul Moore]

On Fri, 26 Jun 2020 at 02:42, Raymond Hettinger
 wrote:
>
> > it is hard to make a decision between the pros and cons,
> > when the pros are in a single formal document and the
> > cons are scattered across the internet.
>
> Mark, I support your idea.  It is natural for PEP authors to not fully 
> articulate the voices of opposition or counter-proposals.
> The current process doesn't make it likely that a balanced document is 
> created for decision making purposes.

I agree that the case against a PEP can often be fragmented and harder
to follow than the case *for* the PEP. My impression, though, is that
this more often results in PEPs *failing* to get accepted, because
there's a general sense of "consensus hasn't been reached" rather than
a specific set of problems with the proposal.

One point that's not clear - would an Anti-PEP be *required*? If not,
what should be the implication of a PEP not having one? That the
objections shouldn't matter? That the PEP delegate can skip the work
of reviewing discussions? In that case, fear of no-one stepping up to
write an Anti-PEP could easily make the debate more contentious,
rather than more reasoned.

I have two particular issues:

1. Fragmented lack of consensus *can be* a valid problem with a PEP.
Expecting a focused document against a PEP may be unreasonable in such
cases - no one person may be sufficiently motivated to collect
opinions, summarise objections and write a document, but nevertheless
the *consensus* may well be against the PEP. Think of this as a
variant of "status quo wins" - the onus should not be on the people
arguing for "no change" to present a compelling, structured argument.
You can of course object to the principle of "status quo wins" - that
itself isn't set in stone, but it's a much bigger question about how
we want Python to develop.
2. In practice, PEPs that spawn large debates and general, unfocused
objections, have a mixed track record. Some get accepted, others get
rejected. Unfortunately, it's not immediately obvious that the
distinguishing factor is *solely* technical merit. There's a large
element of how good authors are at presenting their case, influencing
key community members, etc. This proposal seems to me to take that
situation and make it worse - now we have both the PEP and its
Anti-PEP, each competing not only on technical points, but also on the
"people skills" of the respective authors.

So while I think we should look at ways of improving this aspect of
the PEP process, both to ease the workload of the PEP delegate and to
ensure that PEP authors with weaker "people skills" don't get
under-represented, I'm not sure the Anti-PEP is the way to go.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JLSBHHH33Y2RB6O5DWZYGH2675SGTHZH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Paul Moore]

On Fri, 26 Jun 2020 at 03:37, Gregory P. Smith  wrote:
>
> Litmus test: Give someone who does not know Python this code example from the 
> PEP and ask them what it does and why it does what it does:
>
> match get_shape():
> case Line(start := Point(x, y), end) if start == end:
> print(f"Zero length line at {x}, {y}")
>
> I expect confusion to be the result.  If they don't blindly assume the 
> variables come from somewhere not shown to stop their anguish.
>
> With Python experience, my own reading is:

With Python experience *and limited use of other languages with match
constructs*, this reads naturally to me and makes instant sense.
Clearly it's unreasonable to expect all users to have multi-language
experience, but I'd argue that my experience demonstrates that
"limited use of constructs like this" is enough to understand the
proposed syntax. And therefore, that once it's added to Python, it
won't take long for people to become sufficiently familiar with it to
handle fairly complex examples.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3SZXT3GULG6NX55T4HAQM3TBURGR3PUZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching -- followup

[Daniel Moisset]

[apologies for the duplicate to Guido, used reply instead of reply to all]

To summarize my previous unanswered post, I posted a +1 to the "defaulting
to binding vs interpreting NAME as a constant" is a dangerous default. And
I submitted a couple of alternate syntactic ways to denote "capture is
desired" (the angle brackets, and the capture object). I think both are
reasonably readable, and one of them doesn't even add any unusual syntax
(not even the "dot prefix")

As an elaboration on that, after reading the discussion and trying to not
repeat what has previously been said:


   - I think there's a mismatch between an assumption made by the authors
   vs what many of us are posting here, which is explicitly stated in "
   
https://www.python.org/dev/peps/pep-0622/#alternatives-for-constant-value-pattern";
   : Quoting the PEP: «the name patterns are more common in typical code, so
   having special syntax for common case would be weird». Even if I think a
   popular use case would be analysing and deconstructing complex nested
   structures (like ASTs), I think roughly half of the uses will actually be
   for "the switch statement we never had", where all branches would be
   constants. I've been writing a lot of code like that these last couple of
   weeks, so I may be biased (although the PEP authors may have been writing
   AST visitors this last week and may be biased the other way ;-) )
  - As a sub point, I can understand if the PEP authors argue "match is
  not for that, use if/elif/dicts of functions in that case like you did
  before and ignore this PEP", but if that's the case, that should be
  explicit in the PEP.
   - I am fairly sure (as much as one can be of the future in these things)
   that with this PEP approved as is, linters will add new rules like "you
   have more than a top level name pattern, only the first one will match.
   Perhaps you wanted constant patterns?" and "your pattern captures shadow an
   existing name" and "a name you bound in a pattern isn't used inside the
   pattern". These will definitely help, but for me "how many new linter rules
   will be needed if this language change is introduced" is a good measure of
   how unelegant it is.
  - Perhaps arguing against myself, I know that, thanks to my regular
  usage of linters, I personally won't suffer much from this problem (once
  they get updated). But I also teach Python to people, and I feel that I'd
  have to add this to the list of "gotchas to avoid" if the PEP
passes as is.


Again, I can't write this email without saying that this feature is great,
that the effort put in this PEP is really palpable, that I'd love to find
the way to get it accepted, and that even if I'm normally conservative
upgrading python versions and waiting my environment to support it fully,
this will likely be the first time that I upgrade just for a language
feature :)


On Wed, 24 Jun 2020 at 20:44, Guido van Rossum  wrote:

> Everyone,
>
> If you've commented and you're worried you haven't been heard, please add
> your issue *concisely* to this new thread. Note that the following issues
> are already open and will be responded to separately; please don't bother
> commenting on these until we've done so:
>
> - Alternative spellings for '|'
> - Whether to add an 'else' clause (and how to indent it)
> - A different token for wildcards instead of '_'
> - What to do about the footgun of 'case foo' vs. 'case .foo'
>
> (Note that the last two could be combined, e.g. '?foo' or 'foo?' to mark a
> variable binding and '?' for a wildcard.)
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/STJSSAETMTUY7FK5AE53IM73Z2WORNYN/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NZAQQOXM4K2G4ID4FNWZ6KRDEWFSWDMJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching -- followup

[Paul Moore]

On Fri, 26 Jun 2020 at 11:29, Daniel Moisset  wrote:
>
> I think roughly half of the uses will actually be for "the switch statement 
> we never had", where all branches would be constants. I've been writing a lot 
> of code like that these last couple of weeks, so I may be biased (although 
> the PEP authors may have been writing AST visitors this last week and may be 
> biased the other way ;-) )
>
> As a sub point, I can understand if the PEP authors argue "match is not for 
> that, use if/elif/dicts of functions in that case like you did before and 
> ignore this PEP", but if that's the case, that should be explicit in the PEP.

For me, this prompts the question (which I appreciate is more about
implementation than design) - would there be any (significant)
performance difference between

match var:
case 1:
print("Got 1")
case 2:
print("Got 2")
case _:
print("Got another value")

and

if var == 1:
print("Got 1")
elif var == 2:
print("Got 2")
else:
print("Got another value")

?

In C, the switch statement was explicitly intended to be faster by
means of doing a computed branch. In a higher level language, I can
see the added features of match meaning that it's *slower* than a
series of if tests for simple cases. But I have no intuition about the
performance of this proposal. I'd like to believe that the choice
between the 2 alternatives above is purely a matter of preferred
style, but I don't know. If match is significantly slower, that could
make it a bit of an attractive nuisance.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GNTYPDKJFJDHHFAAA5ATI7RFVNEPOHQ3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Daniel Moisset]

Just a minor editorial thing on the PEP text:

The section https://www.python.org/dev/peps/pep-0622/#case-clauses presents
a simplified syntax. That one mentions "group_pattern", but the document
never mentions (in prose) what a group pattern is. It confused me until I
found the definition in the full grammar, which seems to refer to those
sequence patterns using () rather than []. Probably it makes more sense for
a quick read to remove the "| group_pattern" from the simplified grammar,
it looks more like an intermediate construct.

On Tue, 23 Jun 2020 at 17:04, Guido van Rossum  wrote:

> I'm happy to present a new PEP for the python-dev community to review.
> This is joint work with Brandt Bucher, Tobias Kohn, Ivan Levkivskyi and
> Talin.
>
> Many people have thought about extending Python with a form of pattern
> matching similar to that found in Scala, Rust, F#, Haskell and other
> languages with a functional flavor. The topic has come up regularly on
> python-ideas (most recently yesterday :-).
>
> I'll mostly let the PEP speak for itself:
> - Published: https://www.python.org/dev/peps/pep-0622/ (*)
> - Source: https://github.com/python/peps/blob/master/pep-0622.rst
>
> (*) The published version will hopefully be available soon.
>
> I want to clarify that the design space for such a match statement is
> enormous. For many key decisions the authors have clashed, in some cases we
> have gone back and forth several times, and a few uncomfortable compromises
> were struck. It is quite possible that some major design decisions will
> have to be revisited before this PEP can be accepted. Nevertheless, we're
> happy with the current proposal, and we have provided ample discussion in
> the PEP under the headings of Rejected Ideas and Deferred Ideas. Please
> read those before proposing changes!
>
> I'd like to end with the contents of the README of the repo where we've
> worked on the draft, which is shorter and gives a gentler introduction than
> the PEP itself:
>
>
> # Pattern Matching
>
> This repo contains a draft PEP proposing a `match` statement.
>
> Origins
> ---
>
> The work has several origins:
>
> - Many statically compiled languages (especially functional ones) have
>   a `match` expression, for example
>   [Scala](
> http://www.scala-lang.org/files/archive/spec/2.11/08-pattern-matching.html
> ),
>   [Rust](https://doc.rust-lang.org/reference/expressions/match-expr.html),
>   [F#](
> https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/pattern-matching
> );
> - Several extensive discussions on python-ideas, culminating in a
>   summarizing
>   [blog post](
> https://tobiaskohn.ch/index.php/2018/09/18/pattern-matching-syntax-in-python/
> )
>   by Tobias Kohn;
> - An independently developed [draft
>   PEP](
> https://github.com/ilevkivskyi/peps/blob/pattern-matching/pep-.rst)
>   by Ivan Levkivskyi.
>
> Implementation
> --
>
> A full reference implementation written by Brandt Bucher is available
> as a [fork]((https://github.com/brandtbucher/cpython/tree/patma)) of
> the CPython repo.  This is readily converted to a [pull
> request](https://github.com/brandtbucher/cpython/pull/2)).
>
> Examples
> 
>
> Some [example code](
> https://github.com/gvanrossum/patma/tree/master/examples/) is available
> from this repo.
>
> Tutorial
> 
>
> A `match` statement takes an expression and compares it to successive
> patterns given as one or more `case` blocks.  This is superficially
> similar to a `switch` statement in C, Java or JavaScript (an many
> other languages), but much more powerful.
>
> The simplest form compares a target value against one or more literals:
>
> ```py
> def http_error(status):
> match status:
> case 400:
> return "Bad request"
> case 401:
> return "Unauthorized"
> case 403:
> return "Forbidden"
> case 404:
> return "Not found"
> case 418:
> return "I'm a teapot"
> case _:
> return "Something else"
> ```
>
> Note the last block: the "variable name" `_` acts as a *wildcard* and
> never fails to match.
>
> You can combine several literals in a single pattern using `|` ("or"):
>
> ```py
> case 401|403|404:
> return "Not allowed"
> ```
>
> Patterns can look like unpacking assignments, and can be used to bind
> variables:
>
> ```py
> # The target is an (x, y) tuple
> match point:
> case (0, 0):
> print("Origin")
> case (0, y):
> print(f"Y={y}")
> case (x, 0):
> print(f"X={x}")
> case (x, y):
> print(f"X={x}, Y={y}")
> case _:
> raise ValueError("Not a point")
> ```
>
> Study that one carefully!  The first pattern has two literals, and can
> be thought of as an extension of the literal pattern shown above.  But
> the next two patterns combine a literal and a variable, and the
> variable is *extracted* from the target value (`point`).  The fourth
>

[Python-Dev] Re: PEP 622: Structural Pattern Matching -- followup

[Emily Bowman]

On Fri, Jun 26, 2020 at 3:47 AM Paul Moore  wrote:

> For me, this prompts the question (which I appreciate is more about
> implementation than design) - would there be any (significant)
> performance difference between [...]


> In C, the switch statement was explicitly intended to be faster by
> means of doing a computed branch. In a higher level language, I can
> see the added features of match meaning that it's *slower* than a
> series of if tests for simple cases. But I have no intuition about the
> performance of this proposal. I'd like to believe that the choice
> between the 2 alternatives above is purely a matter of preferred
> style, but I don't know. If match is significantly slower, that could
> make it a bit of an attractive nuisance.
>

Each case essentially compiles down to an equivalent if structure, already.
There's no penalty that I'm seeing. There's actually much more room for
optimizing eventually, since the test is bound to a single element instead
of any arbitrary if expression, and a Cython or PyPy could work some magic
to detect a small set of primitive types and optimize for that.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7U7VAG2NVWZFBIOH36KVEJC5PCSHZ2R7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Rhodri James]

On 25/06/2020 23:20, Emily Bowman wrote:

On my personal "potentially inscrutable uses of a tool" this still rates
well below list comprehensions, so there's that; the biggest pet peeve I


Clearly YMMV.  To me list comprehensions like "[f(x) for x in l]" were 
obviously related to the "f(x) ∀ x ∊ l" familiar from my maths degree.




have anymore is understanding at a glance what is and isn't an assignment.


Yes, this does seem to be a lot of people's problem.  My point is we get 
to that position one step at a time, so maybe we should be examining the 
first steps in that chain and re-evaluating whether they were in fact 
the right ones, given where we end up.  I accept the PEP's general point 
that name patterns will be common, but I don't think something like 
"case int as x:" is hard to write and it brings in the idea that we are 
talking about types right at the start.




--
Rhodri James *-* Kynesim Ltd
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UV6P5TV4WAQHTB47WJBLCPV4JDZO24AR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching -- followup

[Daniel Moisset]

This one is new but I think unrelated and unmentioned:

Why is the mapping match semantics non-strict about keys? Besides the
"asymmetry" with sequence matches, I think a strict match should be useful
sometimes (quickly deconstructing JSON data comes to my mind, where I want
to know that I didn't get unexpected keys). I cannot get that behaviour
with the current pep. But if we make key strictness the default, I can
always add **_ to my mapping pattern and make it non strict (that's
currently forbidden but the restriction can be lifted). Is there an
assumption (or even better, data evidence) that non-strict checks are much
much more common?

A similar but weaker argument can be made for class patterns (although I
can imagine non-strict matches *are* more common in that case).

Mostly but not completely unrelated to the above, and purely syntactic
sugar bikeshedding, but I think having "..." as an alias for "*_" or "**_"
(depending on context, and I'd say it's *both* inside a class pattern)
could make these patterns slightly more readable.

Best,
D.

On Wed, 24 Jun 2020 at 20:44, Guido van Rossum  wrote:

> Everyone,
>
> If you've commented and you're worried you haven't been heard, please add
> your issue *concisely* to this new thread. Note that the following issues
> are already open and will be responded to separately; please don't bother
> commenting on these until we've done so:
>
> - Alternative spellings for '|'
> - Whether to add an 'else' clause (and how to indent it)
> - A different token for wildcards instead of '_'
> - What to do about the footgun of 'case foo' vs. 'case .foo'
>
> (Note that the last two could be combined, e.g. '?foo' or 'foo?' to mark a
> variable binding and '?' for a wildcard.)
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/STJSSAETMTUY7FK5AE53IM73Z2WORNYN/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DS26FCVJ2FCQBRK6OQEBIKYBZPXIIY5P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 620: Hide implementation details from the C API

[Victor Stinner]

,Hi Carl,

Le ven. 26 juin 2020 à 07:36, Carl Shapiro  a écrit :
> It would be very helpful to broaden the objective of avoiding functions 
> returning PyObject** to other types of pointers.  I have in mind several 
> functions in the C-API that return a char* pointer to the contents of an 
> object.  While these functions are easy to implement on top of the CPython 
> object model they are challenging for alternative Python implementations.
>
> Consider PyBytes_AsString: it returns a mutable char* pointing to the 
> contents of a byte instance.  This presents several obvious problems.  For 
> starters, it burdens a relocating garbage collector to pin objects or create 
> a temporary copy of an object's contents in non-moving memory.  It also has 
> implications for treating PyObejct* as a handle, using tagged pointers (and 
> tagged immediates), and multi-threading.
>
> To eliminate C-API functions such as PyBytes_AsString, PyUnicode_AsUTF8, 
> etc., new functions should be added to the C-API that copy the contents of 
> objects out into a buffer, similar to PyUnicode_AsUCS4 or to return the 
> contents in an dynamically allocated buffer like PyUnicode_AsUCS4Copy.

Well, the general problem is to track when the caller ends using a
resource. Borrowed references are a variant of this problem,
PySequence_Fast_ITEMS() is another variant.

For PyUnicode_AsUTF8, INADA-san added PyUnicode_GetUTF8Buffer() which
should be used wit PyBuffer_Release():

* 
https://github.com/python/cpython/commit/c7ad974d341d3edb6b9d2a2dcae4d3d4794ada6b
* https://github.com/python/cpython/pull/17659
* 
https://discuss.python.org/t/better-api-for-encoding-unicode-objects-with-utf-8/2909

... but it was reverted soon after its addition:

* 
https://github.com/python/cpython/commit/3a8c56295d6272ad2177d2de8af4c3f824f3ef92
* https://github.com/python/cpython/pull/18985
* https://bugs.python.org/issue39087

See also the "(PEP 620) C API for efficient loop iterating on a
sequence of PyObject** or other C types" thread.

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PGXUHWZDDP3M27SAP4XR6YSI3D2Q6TWS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Rob Cliffe via Python-Dev]

On 23/06/2020 20:35, Chris Angelico wrote:

On Wed, Jun 24, 2020 at 5:30 AM Rob Cliffe via Python-Dev
 wrote:

The PEP is great, but this strikes me as horribly confusing, given that 
401|403|404 is already legal syntax.
IIUC any legal expression can come between `case` and `:`, but expressions that 
contain `|` at their outermost level are interpreted differently than from in 
other contexts.
Presumably adding parentheses:
 case (401|403|404):
would make it equivalent to
 case 407:

Is a separator (other than whitespace) actually needed?  Can the parser cope 
with
 case 401 403 404:

Failing that IMO preferable, albeit not ideal, possibilities would be
   1) Use colon as the separator.
   2) Use comma as the separator - this is already legal syntax too, but IMO it 
reads more naturally.
   (And IIRC there are already contexts where brackets are necessary to 
indicate a tuple.)
Perhaps someone can think of something better.

I also (with others) prefer `else:` or perhaps `case else:` to using the`_` 
variable.
The latter is obscure, and woudn't sit well with code that already uses that 
variable for its own purposes.


It's not really arbitrary expressions, though. It's more like an
assignment target list, but with some handling of constants.

case (x, y):

is very similar to

(x, y) = ...



If arbitrary expressions are not allowed
    - the power of this new feature is reduced
    - we have to remember another set of  rules about what is allowed 
and what isn't.  Just as we did with decorator syntax - until that 
restriction was done away with.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YJUS4ATNWTMCCTVQBOYQXPUGTYKNIIDK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Mark Shannon]

Hi,

For a PEP to succeed it needs to show two things.

1. Exactly what problem is being solved, or need is to be fulfilled, and 
that is a sufficiently large problem, or need, to merit the proposed change.


2. That the proposed change is the best known solution for the problem 
being addressed.


IMO, PEP 622 fails on both counts.

This email addresses point 1.

Given the positive response to the PEP, it may well be that it does 
address a need. However, the PEP itself fails to show that.



Abstract



This PEP proposes adding pattern matching statements [1] to Python in order to 
create more expressive ways of handling structured heterogeneous data. The 
authors take a holistic approach, providing both static and runtime 
specifications.


What does "static and dynamic specifications" mean? Surely, there are 
just specifications.
Python does not have a static checking phase, so static analysis tools 
need to understand the dynamic behaviour of the program, not have their 
own alternate semantics. There is no "static specification" of 
`isinstance()`, yet static analysis tools understand it.




PEP 275 and PEP 3103 previously proposed similar constructs, and were rejected. 
Instead of targeting the optimization of if ... elif ... else statements (as 
those PEPs did), this design focuses on generalizing sequence, mapping, and 
object destructuring. It uses syntactic features made possible by PEP 617, 
which introduced a more powerful method of parsing Python source code.


Why couple the choice part (a sort of enhanced elif) with destructing (a 
sort of enhanced unpacking)?
We could have a "switch" statement that chooses according to value, and 
we could have "destructuring" that pulls values apart. Why do they need 
to be coupled?


Rationale and Goals
---


Let us start from some anecdotal evidence: isinstance() is one of the most 
called functions in large scale Python code-bases (by static call count). In 
particular, when analyzing some multi-million line production code base, it was 
discovered that isinstance() is the second most called builtin function (after 
len()). Even taking into account builtin classes, it is still in the top ten. 
Most of such calls are followed by specific attribute access.


Why use anecdotal evidence? I don't doubt the numbers, but it would be 
better to use the standard library, or the top N most popular packages 
from GitHub.




There are two possible conclusions that can be drawn from this information:

Handling of heterogeneous data (i.e. situations where a variable can take 
values of multiple types) is common in real world code.
Python doesn't have expressive ways of destructuring object data (i.e. 
separating the content of an object into multiple variables).


I don't see how the second conclusion can be drawn.
How does the prevalence of `isinstance()` suggest that Python doesn't 
have expressive ways of destructuring object data?


That `len()` is also common, does suggests that some more expressive 
unpacking syntax might be useful. However, since `len()` only applies to 
sequences, it suggests to me that unpacking of non-sequences isn't 
generally useful.




This is in contrast with the opposite sides of both aspects:


This sentence makes no sense. What is "this"? Both aspects of what?



Its success in the numeric world indicates that Python is good when working 
with homogeneous data. It also has builtin support for homogeneous data 
structures such as e.g. lists and arrays, and semantic constructs such as 
iterators and generators.
Python is expressive and flexible at constructing objects. It has syntactic 
support for collection literals and comprehensions. Custom objects can be 
created using positional and keyword calls that are customized by special 
__init__() method.

This PEP aims at improving the support for destructuring heterogeneous data by 
adding a dedicated syntactic support for it in the form of pattern matching. On 
a very high level it is similar to regular expressions, but instead of matching 
strings, it will be possible to match arbitrary Python objects.


An explanation is needed of why "destructuring" needs to be so tightly 
coupled with matching by class or value.




We believe this will improve both readability and reliability of relevant code. 
To illustrate the readability improvement, let us consider an actual example 
from the Python standard library:

def is_tuple(node):
if isinstance(node, Node) and node.children == [LParen(), RParen()]:
return True
return (isinstance(node, Node)
and len(node.children) == 3
and isinstance(node.children[0], Leaf)
and isinstance(node.children[1], Node)
and isinstance(node.children[2], Leaf)
and node.children[0].value == "("
and node.children[2].value == ")")



Just one example?
The PEP needs to show that this sort of pattern is widespread.


With the syntax propo

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Rob Cliffe via Python-Dev]

Yes, my brain went through the same path.
Another minor nitpick: It would be kinda nice if the various types of 
pattern were listed in the grammar in the same order as the 
corresponding paragraphs subsequently appear.


On 26/06/2020 11:53, Daniel Moisset wrote:

Just a minor editorial thing on the PEP text:

The section 
https://www.python.org/dev/peps/pep-0622/#case-clauses presents a 
simplified syntax. That one mentions "group_pattern", but the document 
never mentions (in prose) what a group pattern is. It confused me 
until I found the definition in the full grammar, which seems to refer 
to those sequence patterns using () rather than []. Probably it makes 
more sense for a quick read to remove the "| group_pattern" from the 
simplified grammar, it looks more like an intermediate construct.



___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CIXWWJP47PGWVYJJLIVW5PRQ6XW3IRC5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Arthur Darcet]

On Fri, 26 Jun 2020 at 09:07, Greg Ewing 
wrote:

> On 26/06/20 2:10 pm, Gregory P. Smith wrote:
> > match get_shape() as shape:
> >case start, end := Line@(shape):
>
> This looks just as inscrutable to me in its own way.
>

Absolutely, but that's kind of the point I think: no possible way to
understand it for something else that what it means.


Arthur
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5AN52CT5BYARJHO3BL3OMXI4A27DHJCH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Stéfane Fermigier]

On Fri, Jun 26, 2020 at 3:38 PM Mark Shannon  wrote:

>
> What does "static and dynamic specifications" mean? Surely, there are
> just specifications.
>

There are specifications for both the runtime and the static aspects of the
Python programming language.


> Python does not have a static checking phase,


The (C)Python *interpreter* doesn't. Other Python implementations (existing
or hypothetical) may or may not have a static checking phase.

But static tools need specifications beyond (ie. additionally to) runtime
specifications, which are defined in PEP 483, 484, 585 and others.

> Let us start from some anecdotal evidence: isinstance() is one of the
> most called functions in large scale Python code-bases (by static call
> count). In particular, when analyzing some multi-million line production
> code base, it was discovered that isinstance() is the second most called
> builtin function (after len()). Even taking into account builtin classes,
> it is still in the top ten. Most of such calls are followed by specific
> attribute access.
>
> Why use anecdotal evidence? I don't doubt the numbers, but it would be
> better to use the standard library, or the top N most popular packages
> from GitHub.
>

Maybe a scientific paper could be written on this subject. I'm guessing
the "multi-million line production code base" in question is the Dropbox
code base, and maybe Dropbox has an idiomatic way of writing Python with
lots of "isinstance()"s.

> In general, we believe that pattern matching has been proved to be a
> useful and expressive tool in various modern languages. In particular, many
> aspects of this PEP were inspired by how pattern matching works in Rust [3]
> and Scala [4].
>
> Both those languages are statically typed, which allows the compiler to
> perform the much of the pattern matching at compile time.
>
> You should give examples from dynamic typed languages instead, e.g.
> clojure.
>

Here's one example:

https://github.com/clojure/core.match (in particular:
https://github.com/clojure/core.match/wiki/Understanding-the-algorithm ).

Alson some insights from
https://softwareengineering.stackexchange.com/questions/237023/pattern-matching-in-clojure-vs-scala


In this video  I watched
recently, Rich Hickey comments that he likes the destructuring part of
languages like Scala, but not so much the pattern matching part, and he
designed Clojure accordingly. That probably explains why the pattern
matching is in a library and not as robust, although the kind of problems
seen in the post you mentioned are clearly bugs.

What Rich Hickey mentions as an alternative to pattern matching is
multimethods . Most languages let you do
polymorphic dispatch based on type. Some languages let you also do it based
on a value. Using multimethods, Clojure lets you do it based on any
arbitrary function. That's a pretty powerful concept.

It comes down to the principle that programmers using a language should use
the language's own best idioms. Trying to write Scala-like code in Clojure
is going to have its difficulties, and vice versa.


  S.

-- 
Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier -
http://linkedin.com/in/sfermigier
Founder & CEO, Abilian - Enterprise Social Software -
http://www.abilian.com/
Chairman, National Council for Free & Open Source Software (CNLL) -
http://cnll.fr/
Founder & Organiser, PyParis & PyData Paris - http://pyparis.org/ &
http://pydata.fr/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JWBAB4EC57NCTYREKPF63K6J347TCQXJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Petr Viktorin]

On 2020-06-26 16:54, Stéfane Fermigier wrote:
[...]



Here's one example:

https://github.com/clojure/core.match (in particular: 
https://github.com/clojure/core.match/wiki/Understanding-the-algorithm ).


Alson some insights from 
https://softwareengineering.stackexchange.com/questions/237023/pattern-matching-in-clojure-vs-scala



In this video  I
watched recently, Rich Hickey comments that he likes the
destructuring part of languages like Scala, but not so much the
pattern matching part, and he designed Clojure accordingly. That
probably explains why the pattern matching is in a library and not
as robust, although the kind of problems seen in the post you
mentioned are clearly bugs.

What Rich Hickey mentions as an alternative to pattern matching is
multimethods . Most languages let
you do polymorphic dispatch based on type. Some languages let you
also do it based on a value. Using multimethods, Clojure lets you do
it based on any arbitrary function. That's a pretty powerful concept.

It comes down to the principle that programmers using a language
should use the language's own best idioms. Trying to write
Scala-like code in Clojure is going to have its difficulties, and
vice versa.


It does look like the PEP tries to do two different things: "switch" 
instead of if/elif, and destructuring.


Would it be useful to introduce an operator for "isinstance", if it's so 
commonly used? Are the calls to it (in the relevant codebase) actually 
used in complex code that needs destructuring, or could we live with 
this (IS_A being a placeholder for bikeshedding, of course):


if shape IS_A Point:
x, y = shape
...
elif shape IS_A Rectangle:
x, y, w, h = shape
...
elif shape IS_A Line:
x, y = line.start
if line.start == line.end:
print(f"Zero length line at {x}, {y}")

or:

queue: Union[Queue[int], Queue[str]]
if queue IS_A IntQueue:
# Type-checker detects unreachable code
...


There aren't many convincing examples for destructuring in the PEP, IMO.

The "mapping pattern" one could be rewritten as:

if route := config.get('route'):
process_route(route)
if subconfig := config.pop(constants.DEFAULT_PORT):
process_config(sub_config, config)

Sequence destructuring examples ([_] for "short sequence") don't seem 
too useful. Would they actually improve lots of existing code?


Complex object /tree destructuring (like the is_tuple) is painful in 
Python, but then again, the new syntax also becomes quite inscrutable 
for complex cases.

Is code like the is_tuple example in the Rationale actually common?

The "Sealed classes as algebraic data types" example looks like a good 
candidate for a dump() method or PEP 443 single dispatch, both of which 
should be amenable to static analysis.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/57RTPCDQX56WZKQC42KXYS2EHQWLALCN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The Anti-PEP

[Brett Cannon]

On Thu, Jun 25, 2020 at 7:29 PM Gregory P. Smith  wrote:

>
> On Thu, Jun 25, 2020 at 6:49 PM Raymond Hettinger <
> raymond.hettin...@gmail.com> wrote:
>
>> > it is hard to make a decision between the pros and cons,
>> > when the pros are in a single formal document and the
>> > cons are scattered across the internet.
>>
>> Mark, I support your idea.  It is natural for PEP authors to not fully
>> articulate the voices of opposition or counter-proposals.
>> The current process doesn't make it likely that a balanced document is
>> created for decision making purposes.
>>
>>
> On some PEPs in the past I seem to recall we've had the PEP author, or at
> least editor after the initial draft kicked things off _not_ be among those
> invested in seeing the PEP be approved.
>
> Or maybe I'm conflating the old role of the PEP delegate with the editor?
>

Nope, we have had some PEPs written to explicitly reject them just for the
recordkeeping of the idea.

-Brett


>
> Regardless i don't see how an anti-pep would work much better, but I also
> don't see anything stopping anyone from trying one.  I worry that it'll
> fragment conversation even more and separate discussions so that everyone
> is even more confused about overall opinion tallies?  one way to find out...
>
> -gps
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/JYJ2GB2LKX7ELWKURAQMOA7Z52DHE3B6/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XAG22S66OSQ6X2M3ZVWT7JUZBLHCM4NB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: The Anti-PEP

[Brett Cannon]

On Thu, Jun 25, 2020 at 1:37 PM Chris Jerdonek 
wrote:

> On Thu, Jun 25, 2020 at 11:52 AM Brett Cannon  wrote:
>
>> On Thu, Jun 25, 2020 at 5:45 AM Antoine Pitrou 
>> wrote:
>>
>>> I don't think this really works.  A PEP has to present a consistent
>>> view of the world, and works as a cohesive whole.  Arguments against a
>>> PEP don't form a PEP in themselves, they don't need to be consistent
>>> with each other; they merely oppose a particular set of propositions.
>>> So an "anti-PEP" would not be anything like a PEP; it would just be a
>>> list of assorted arguments.
>>>
>>
>> I agree, and that's what the Rejected Ideas section is supposed to
>> capture.
>>
>
> When I read the description of Rejected Ideas in PEP 1, it seems like it's
> more for ideas that have been rejected that are still in line with the
> overall PEP / motivation.
>

We can change PEP 1 if necessary to make people feel more comfortable in
using the Rejected Ideas section to record objections.


> It seems like what Mark is suggesting would fit better in a separate
> "Arguments Against" section. I guess it would be possible to include
> "reject the PEP" as a rejected idea or each individual argument against as
> its own rejected "idea," but it would seem a little weird to me to organize
> it that way.
>

I personally don't think so. "Don't do this PEP", to me, is still a
rejected idea in the eyes of the PEP. 😉

-Brett


>
> I do see that PEP 1 says about the Rationale section:
>
> The rationale should provide evidence of consensus within the community
>> and discuss important objections or concerns raised during discussion.
>
>
> But what Mark is suggesting might be too large for the Rationale section.
>
> --Chris
>
>
>> If a PEP is not keeping a record of what is discussed, including opposing
>> views which the PEP is choosing not to accept, then that's a deficiency in
>> the PEP and should be fixed. And if people feel their opposing view was not
>> recorded properly, then that should be brought up.
>>
>>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DVN56WOVOXHH63LX4WJXJ5KNQIADPITB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2020-06-19 - 2020-06-26)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open7490 (+14)
  closed 45380 (+75)
  total  52870 (+89)

Open issues with patches: 3035 


Issues opened (62)
==

#15849: PEP 3121, 384 Refactoring applied to xx module
https://bugs.python.org/issue15849  reopened by vstinner

#35975: Put back the ability to parse files where async/await aren't k
https://bugs.python.org/issue35975  reopened by gvanrossum

#41041: Multiprocesing Pool borken on macOS REPL
https://bugs.python.org/issue41041  opened by mbussonn

#41043: Escape the literal part of the path for glob()
https://bugs.python.org/issue41043  opened by serhiy.storchaka

#41045: f-string's "debug" feature is undocumented
https://bugs.python.org/issue41045  opened by eric.smith

#41046: unittest: make skipTest a classmethod
https://bugs.python.org/issue41046  opened by jameshcorbett

#41048: read_mime_types() should read the rule file using UTF-8, not t
https://bugs.python.org/issue41048  opened by serhiy.storchaka

#41050: class multiprocessing.Value calls set_start_method
https://bugs.python.org/issue41050  opened by Kernel Plevitsky

#41051: Flush file after warning is written
https://bugs.python.org/issue41051  opened by mjacob

#41052: Opt out serialization/deserialization for heap type
https://bugs.python.org/issue41052  opened by corona10

#41053: open() fails to read app exec links
https://bugs.python.org/issue41053  opened by saschanaz

#41056: minor NULL pointer and sign issues reported by Coverity
https://bugs.python.org/issue41056  opened by gregory.p.smith

#41059: Large number of Coverity reports for parser.c
https://bugs.python.org/issue41059  opened by gregory.p.smith

#41061: Incorrect expressions / assert with side effect in hashtable
https://bugs.python.org/issue41061  opened by christian.heimes

#41062: Advanced Debugger Support C-API is useless without HEAD_LOCK()
https://bugs.python.org/issue41062  opened by pashkin

#41063: Avoid using the locale encoding for open() in tests
https://bugs.python.org/issue41063  opened by serhiy.storchaka

#41064: f-string SyntaxError gives offset within fake line, other issu
https://bugs.python.org/issue41064  opened by JNCressey

#41065: Use zip-strict in zoneinfo
https://bugs.python.org/issue41065  opened by cool-RR

#41066: Update documentation for Pathlib
https://bugs.python.org/issue41066  opened by thatiparthy

#41067: Haiku build fix - posix module
https://bugs.python.org/issue41067  opened by devnexen

#41073: [C API] PyType_GetSlot() should accept static types
https://bugs.python.org/issue41073  opened by vstinner

#41075: Make support of navigating through prev. commands in IDLE more
https://bugs.python.org/issue41075  opened by wyz23x2

#41076: Pre-feed the parser with the f-string expression location
https://bugs.python.org/issue41076  opened by lys.nikolaou

#41078: [C API] Convert PyTuple_GET_ITEM() macro to a static inline fu
https://bugs.python.org/issue41078  opened by vstinner

#41079: _PyAsyncGenWrappedValue_Type is never Readied
https://bugs.python.org/issue41079  opened by Tomasz Pytel

#41081: Exclude __pycache__ directories from backups using CACHEDIR.TA
https://bugs.python.org/issue41081  opened by jstasiak

#41082: Error handling and documentation of Path.home()
https://bugs.python.org/issue41082  opened by timhoffm

#41083: plistlib can't decode date from year 0
https://bugs.python.org/issue41083  opened by shields-fn

#41084: Signify that a SyntaxError comes from an fstring in the error 
https://bugs.python.org/issue41084  opened by lys.nikolaou

#41086: Exception for uninstantiated interpolation (configparser)
https://bugs.python.org/issue41086  opened by scrummyin

#41088: Extend the AST Validator to validate all identifiers
https://bugs.python.org/issue41088  opened by BTaskaya

#41090: Support for "Universal 2" binary builds
https://bugs.python.org/issue41090  opened by ronaldoussoren

#41091: Remove recommendation in curses module documentation to initia
https://bugs.python.org/issue41091  opened by mjacob

#41092: Report actual size from 'os.path.getsize'
https://bugs.python.org/issue41092  opened by stephenfin

#41093: TCPServer's server_forever() shutdown immediately when calling
https://bugs.python.org/issue41093  opened by tontinton

#41095: inspect.signature() doesn't parse __text_signature__ containin
https://bugs.python.org/issue41095  opened by vstinner

#41096: Need command to exit PDB interactive shell
https://bugs.python.org/issue41096  opened by Kerrick Staley

#41097: confusing BufferError: Existing exports of data: object cannot
https://bugs.python.org/issue41097  opened by arigo

#41098: Deprecating PyUnicodeEncodeError_Create
https://bugs.python.org/issue41098  opened by inada.naoki

#41099: Deprecating PyUnicodeTranslateError_Create
https://bugs.python.org/iss

[Python-Dev] Re: (PEP 620) C API for efficient loop iterating on a sequence of PyObject** or other C types

[Carl Shapiro]

On Tue, Jun 23, 2020 at 7:52 AM Stefan Behnel  wrote:

> I agree that this is more explicit when it comes to resource management,
> but there is nothing that beats direct native data structure access when it
> comes to speed.


>From the perspective of the function that wants to get access to the
contents of an object, direct access will be the fastest.  However, more
globally, from the perspective of the runtime as a whole, supporting direct
access in all situations typically makes other things slower.  So, unless
there is hot code doing a lot of direct structure access, there can be a
big net loss.

The JNI has a good compromise, at least for primitive types like the
various classes of floats, and ints.  When requesting the contents of an
array, the runtime might be able to give you a direct pointer, but, if not,
what you get back might be a copy of the contents of the array.  To know
what happened, the API gives you a signal about the ownership of the
pointer in an out parameter.

https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#Get_PrimitiveType_ArrayElements_routines

You have to release these when you're done, kind of like what Python's
buffer protocol does today.  For small to medium sized stuff, there is an
API for just copying things out into a user provided buffer, which might be
faster.

https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#Get_PrimitiveType_ArrayRegion_routines

The C-API is currently inconsistent in what kind of access you can get to
the contents of an object.  As I mentioned in the other thread, it would be
beneficial to alternative implementations of Python to have more uniformity
in what the C-API provides.

If a "PyObject*[]" is not what the runtime uses internally
> as data structure, then why hand it out as an interface to users who
> require performance? There's PyIter_Next() already for those who don't.
>

For arrays of pointers to objects that may be under the management of a
moving garbage collector, what looks like direct access would actually be
emulated and dramatically slower than doing PyIter_Next.

If the intention is to switch to a more efficient internal data structure
> inside of CPython (or expose in PyPy whatever that uses), then I would look
> more at PEP-393 for a good interface here, or "array.array". It's perfectly
> fine to have 20 different internal array types, as long as they are
> explicitly and safely exposed to users.
>

The internal data structure might be exactly the same, but, for example, a
moving GC might make efficient direct access to the data structure from C
code impossible.  If you are not doing with reference types, I agree, we
can do much better.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FUGK3ECIXVQ5B6J6G7GUFCSHAWBBOTAI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 620: Hide implementation details from the C API

[Stefan Behnel]

Victor Stinner schrieb am 26.06.20 um 14:39:
> Well, the general problem is to track when the caller ends using a resource.

Although that is less of a problem if you only allow exposing the internal
data representation and nothing else. In that case, you can tie the
lifetime of the data access to the lifetime of the object.

Minus moving GCs, as Carl also pointed out. But even there, you could get
away (probably for quite a while) with pinning the data if someone asked
for it.

Stefan
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NS5MTQFCD7TRZCXS4ZSI3PCPEA5OL6PJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 622: Structural Pattern Matching

[Gregory P. Smith]

On Fri, Jun 26, 2020 at 6:42 AM Mark Shannon  wrote:

>
> > Let us start from some anecdotal evidence: isinstance() is one of the
> most called functions in large scale Python code-bases (by static call
> count). In particular, when analyzing some multi-million line production
> code base, it was discovered that isinstance() is the second most called
> builtin function (after len()). Even taking into account builtin classes,
> it is still in the top ten. Most of such calls are followed by specific
> attribute access.
>
> Why use anecdotal evidence? I don't doubt the numbers, but it would be
> better to use the standard library, or the top N most popular packages
> from GitHub.
>

Agreed.  This anecdote felt off to me and made for a bad introductory
feeling.  I know enough of who is involved to read it as likely "within the
internal Dropbox code base we found isinstance() to be the second most
called built-in by static call counts".  It'd be better worded as such
instead of left opaque if you are going to use this example at all.  [but
read on below, i'm not sure the anecdotal evidence is even relevant to
state]

Also if using this, please include text explaining what "static call count
means".  Was that "number of grep 'isinstance[(]' matches in all .py files
which we reasonably assume are calls"?  Or was that "measuring a running
application and counting cumulative calls of every built-in for the
lifetime of the large application"?  Include a footnote of if you have you
removed all use of six and py2->py3-isms?  Both six and manual py2->3
porting often wound up adding isinstance in places where they'll rightfully
be refactored out when cleaning up the py2 dead code legacy becomes anyones
priority.

A very rough grep of our much larger Python codebase within Google shows
isinstance *call site counts* to likely be lower than int or len and
similar to print.  With a notable percentage of isinstance usage clearly
related to py2 -> py3 compatibility, suggesting many can now go away.  I'm
not going to spend much time looking further as I don't think actual
numbers matter:  *Confirmed, isinstance gets used a lot.*  We can simply
state that as a truth and move on without needing a lot of justification.

>
> > There are two possible conclusions that can be drawn from this
> information:
> >
> > Handling of heterogeneous data (i.e. situations where a variable can
> take values of multiple types) is common in real world code.
> > Python doesn't have expressive ways of destructuring object data
> (i.e. separating the content of an object into multiple variables).
>
> I don't see how the second conclusion can be drawn.
> How does the prevalence of `isinstance()` suggest that Python doesn't
> have expressive ways of destructuring object data?

...

> >
> > We believe this will improve both readability and reliability of
> relevant code. To illustrate the readability improvement, let us consider
> an actual example from the Python standard library:
> >
> > def is_tuple(node):
> > if isinstance(node, Node) and node.children == [LParen(), RParen()]:
> > return True
> > return (isinstance(node, Node)
> > and len(node.children) == 3
> > and isinstance(node.children[0], Leaf)
> > and isinstance(node.children[1], Node)
> > and isinstance(node.children[2], Leaf)
> > and node.children[0].value == "("
> > and node.children[2].value == ")")
> >
>
> Just one example?
> The PEP needs to show that this sort of pattern is widespread.
>

Agreed.  I don't find application code following this pattern to be
common.  Yes it exists, but I would not expect to encounter it frequently
if I were doing random people's Python code reviews.

The supplied "stdlib" code example is lib2to3.fixer_util.is_tuple.  Using
that as an example of code "in the standard library" is *technically*
correct. But lib2to3 is an undocumented deprecated library that we have
slated for removal.  That makes it a bit weak to cite.

Better practical examples don't have to be within the stdlib.

Randomly perusing some projects I know that I expect to have such
constructs, here's a possible example:
https://github.com/PyCQA/pylint/blob/master/pylint/checkers/logging.py#L231.

There are also code patterns in pytype such as
https://github.com/google/pytype/blob/master/pytype/vm.py#L480  and
https://github.com/google/pytype/blob/master/pytype/vm.py#L1088 that might
make sense.

Though I realize you were probably in search of a simple one for the PEP in
order to write a before and after example.

-gps
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/X7XWT4YZFRFJJYJFHZY6X5LYLBZ7LH52/
Code of Conduct: http://python.org/psf/codeofc