[Python-Dev] Re: walrus operator and expression

[MRAB]

On 2022-03-28 23:59, Chris Angelico wrote:

On Tue, 29 Mar 2022 at 09:53, Ethan Furman  wrote:


In the following bit of code:


 while s := input.read(MAXBINSIZE):
 while len(s) < MAXBINSIZE and ns := input.read(MAXBINSIZE-len(s)):
 s += ns
 line = binascii.b2a_base64(s)
 output.write(line)

I'm getting this error on the second line:

 cannot use assignment expressions with expression

Can somebody explain why that isn't working?



I'm getting a good hint from the exception in 3.11:


while len(s) < MAXBINSIZE and ns := input.read(MAXBINSIZE-len(s)):

   File "", line 1
 while len(s) < MAXBINSIZE and ns := input.read(MAXBINSIZE-len(s)):
   ^^
SyntaxError: cannot use assignment expressions with expression

Looks like it's counting "len(s) < MAXBINSIZE and ns" as the
assignment target. Parens around the second half would solve that.

I think that's because assignment has low precedence, lower than that of 
the RHS, because the RHS needs to be evaluated before the assignment, 
but the consequence of that it that the preceding expression will also 
get evaluated first, consuming the identifier, leaving a trailing ':='. 
It's easier just to complain and require on parentheses than to back up 
while parsing.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Q6XOY2KCSZP2VZTKFBDF2OT7MG7EB7AP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Toshio Kuratomi]

On Sun, Mar 27, 2022, 11:07 AM Paul Moore  wrote:

> On Sun, 27 Mar 2022 at 17:11, Christopher Barker 
> wrote:
> > Back to the topic at hand, rather than remove urllib, maybe it could be
> made better -- an as-easy-to-use-as-requests package in the stdlib would be
> really great.
>
> I think that's where the mistake happens, though. Someone who needs
> "best of breed" is motivated (and likely knowledgeable enough) to make
> informed decisions about what's on PyPI. But someone who just wants to
> get the job done probably doesn't - and that's the audience for the
> stdlib. A stdlib module needs to be a good, reliable set of basic
> functionality that non-experts can use successfully. There can be
> better libraries on PyPI, but that doesn't mean the stdlib module is
> unnecessary, nor does it mean that the stdlib has to match the PyPI
> library feature for feature.
>
> So here, specifically, I'd rather see urlllib be the best urlllib it
> can be, and not demand that it turn into requests. Requests is there
> if people need/want it (as is httpx, and urllib3, and aiohttp). But
> urllib is for people who want to get a file from the web, and *not*
> have to deal with dependencies, 3rd party libraries, etc.
>


One thing about talking about "make urllib more like requests" that is
different than any of the other libs, though, is that requests aims to be
easier to use than anything else (which I note Chris Barker called out as
why he wanted urllib to be more like it).  I think that's important to
think about because i think ease of use is also the number one thing that
the audience you talk about is also looking for.

Of course, figuring out whether an api like request's is actually easier to
use than urllib or merely more featureful is open to debate.

-toshio
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4ZQC4H7HD3UXFT3CONU64YPOQBSPUTVY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Enhancing generic type documentation in the standard library

[Luciano Ramalho]

On Wed, Mar 23, 2022 at 5:01 PM Brett Cannon  wrote:
> The SC somewhat agrees!  See 
> https://mail.python.org/archives/list/typing-...@python.org/thread/TVMQJXOJFOYFPDMQDFG6G4B6J3MLRYKB/
>  where we have asked for at least the specs to get consolidated into proper 
> documentation instead of spread out across various PEPs.

That makes me somewhat happy.

Just kidding. I totally agree that the typing PEPs need to be
consolidated into specs, which then need to be kept up-to-date.

The issue I raised is related but different: we now have dozens of
generic types in the standard library that are not fully documented in
PEPs or hardly anywhere else.

For example, PEP 585–Type Hinting Generics In Standard Collections
merely lists the types which, back then, were changed to accept type
parameters within [ ]. That PEP does not document the number, meaning,
or variance of the accepted type parameters.

When Guido and I refactored the `typing` module docs for Python 3.9,
we marked dozens of types as deprecated. The entries for those
deprecated types, such as `typing.Generator`, are the only places
where the parameters accepted by the generic type are
documented—albeit in a way that's not user-friendly:

class typing.Generator(Iterator[T_co], Generic[T_co, T_contra, V_co])

> My personal hope is this will also lead to better docs overall in a central 
> location instead of spread out among modules, tools, etc.

We share that hope and I am willing to help.

Documenting the generic types in the standard library is a much
smaller task than turning the typing PEPs into specs. It seems like a
good next step on the way to better typing docs.

Cheers,

Luciano



>
> -Brett
>
>>
>>
>> We now have lots of generic types in the standard library, but their
>> formal type parameters are poorly documented—or not documented at
>> all—in the standard library documentation.
>>
>> More importantly: the documentation we have about specific
>> covariant/contravariant types is now in entries in the `typing` module
>> that are all deprecated since PEP 585 was implemented in Python 3.9.
>>
>> Below I present two of many examples where the documentation of a
>> generic type is not great.
>>
>> However, if people agree this is a problem, we need to discuss where
>> and how to put the documentation in a way that is not too disruptive
>> to users of Python who don't know or don't care about type hints, for
>> many reasons that we should not judge.
>>
>> For example, where do we document the fact that `dict` accepts two
>> invariant formal type parameters, and that `frozenset` accepts one
>> contravariant type parameter?
>>
>> What do you think?
>>
>> Cheers,
>>
>> Luciano
>>
>> _
>> EXAMPLE 1: `Callable` variance is not documented
>>
>> For example, in the `Callable` type, the `ReturnType` parameter is
>> covariant, and the `ParameterType` parameters are all contravariant.
>>
>> But there is no mention of variance in this entry:
>> https://docs.python.org/3/library/typing.html?highlight=typing#typing.Callable
>>
>> Also, no mention of the fact that `collections.abc.Callable` is generic here:
>> https://docs.python.org/3/library/collections.abc.html#collections.abc.Callable
>>
>> PEP 483—The Theory of Type Hints—is the only official Python doc where
>> I found the information about the variance of the formal type
>> parameters of `Callable`. The explanation there is brilliant [0].
>>
>> [0] https://peps.python.org/pep-0483/#covariance-and-contravariance
>>
>> Regardless, the intended audience of PEPs is "core developers"—which
>> is neither a superset nor a subset of "Python devs now using type
>> hints". We should not rely on PEPs to document features for Python
>> users in general.
>>
>> _
>> EXAMPLE 2: `Generator` variance could be better documented
>>
>> The entry for `typing.Generator` [1] has this heading:
>>
>> class typing.Generator(Iterator[T_co], Generic[T_co, T_contra, V_co])
>>
>> Answer quickly: how many formal type parameters does `Generator`
>> require? Which are covariant? Which are contravariant?
>>
>> [1] 
>> https://docs.python.org/3/library/typing.html?highlight=typing#typing.Generator
>>
>> Nowhere in that page [1] there's an explanation of the `*_co` and
>> `*_contra` naming convention, much less their semantics.
>>
>> Fortunately, the text of the `typing.Generator` entry says: "A
>> generator can be annotated by the generic type `Generator[YieldType,
>> SendType, ReturnType]".
>>
>> Unfortunately, `typing.Generator` is deprecated and will be gone in 5
>> years or so...
>>
>> The same issue applies to all the other generic types: builtins
>> (`dict`, `frozenset`), ABCs, etc.
>> Their formal type parameters they accept as generics are either
>> undocumented, or documented in parts of the `typing` module that are
>> already deprecated.
>>
>> Thoughts?
>>
>> --
>> Luciano Ramalho
>> |  Author of Fluent Python (O'Reilly, 2015)
>> | 

[Python-Dev] Re: walrus operator and expression

[Guido van Rossum]

Parenthesize.

On Mon, Mar 28, 2022 at 3:50 PM Ethan Furman  wrote:

> In the following bit of code:
>
>
>  while s := input.read(MAXBINSIZE):
>  while len(s) < MAXBINSIZE and ns := input.read(MAXBINSIZE-len(s)):
>  s += ns
>  line = binascii.b2a_base64(s)
>  output.write(line)
>
> I'm getting this error on the second line:
>
>  cannot use assignment expressions with expression
>
> Can somebody explain why that isn't working?
>
> Many thanks.
>
> --
> ~Ethan~
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/3NONJDUTXANBTINCOZITLHOAUDMJ3H66/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Q3O55IVWRYCNQT7NIG6X3XSOIGLZCZ6P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: walrus operator and expression

[Chris Angelico]

On Tue, 29 Mar 2022 at 09:53, Ethan Furman  wrote:
>
> In the following bit of code:
>
>
>  while s := input.read(MAXBINSIZE):
>  while len(s) < MAXBINSIZE and ns := input.read(MAXBINSIZE-len(s)):
>  s += ns
>  line = binascii.b2a_base64(s)
>  output.write(line)
>
> I'm getting this error on the second line:
>
>  cannot use assignment expressions with expression
>
> Can somebody explain why that isn't working?
>

I'm getting a good hint from the exception in 3.11:

>>> while len(s) < MAXBINSIZE and ns := input.read(MAXBINSIZE-len(s)):
  File "", line 1
while len(s) < MAXBINSIZE and ns := input.read(MAXBINSIZE-len(s)):
  ^^
SyntaxError: cannot use assignment expressions with expression

Looks like it's counting "len(s) < MAXBINSIZE and ns" as the
assignment target. Parens around the second half would solve that.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/E5DAPQX3CODJTU4PTTEDCD6NSMZSTAOB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] walrus operator and expression

[Ethan Furman]

In the following bit of code:


while s := input.read(MAXBINSIZE):
while len(s) < MAXBINSIZE and ns := input.read(MAXBINSIZE-len(s)):
s += ns
line = binascii.b2a_base64(s)
output.write(line)

I'm getting this error on the second line:

cannot use assignment expressions with expression

Can somebody explain why that isn't working?

Many thanks.

--
~Ethan~
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3NONJDUTXANBTINCOZITLHOAUDMJ3H66/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Brett Cannon]

On Mon, Mar 28, 2022 at 11:52 AM Christopher Barker 
wrote:

> On Mon, Mar 28, 2022 at 11:29 AM Paul Moore  wrote:
>
>> To be honest, I feel like I'm just reiterating stuff I've said before
>> here, and I think the same is true of the points I'm responding to
>
>  ...
>
>>  (I'm not *against* going over the debate again,
>> it helps make sure people haven't changed their minds, but it's
>> important to be clear that none of the practical facts have changed,
>> if that is the case).
>>
>
> Maybe there's a way to make this discussion (it feels more like a
> discussion than debate at the moment) more productive by writing some
> things down. I'm not sure it's a PEP, but some kind of:
>
> "policy for the stdlib" document in which we could capture the primary
> points of view, places where there's consensus, etc. would be helpful to
> keep us retreading this over and over again.
>
> I suggest this without the bandwidth to actually shepherd the project, but
> if someone wants to, I think it would be a great idea.
>

Once
https://mail.python.org/archives/list/python-committ...@python.org/thread/5EUZLT5PNA4HT42NGB5WVN5YWW5ASTT5/
is considered resolved, the next part of my "what *is* the stdlib" plan is
to finally try to suss all of this out and more-or-less write a stdlib
policy PEP so we stop talking about this. My guess it will be more of
guidance about what we want the stdlib to be and thus guide what things
belong in it. No ETA on that work since I also have four other big Python
projects on the go right now whose work I am constantly alternating between.

-Brett


>
> -CHB
>
> --
> Christopher Barker, PhD (Chris)
>
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/GAZAFRFVJVQZMEIHTQUJASP7VRAKA5RR/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PC67DOLDEQXIAGXEB2QXCGS3C4B6PTCY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

[Jason Ansel via Python-Dev]

The PyTorch team plans to use PEP 523 as a part of PyTorch 2.0, so this 
proposal may break the next major release of PyTorch.

The related project is TorchDynamo, which can be found here:
https://github.com/facebookresearch/torchdynamo

We will likely move this into the core of PyTorch closer to release.

If the changed happens, would PyTorch still be able to use the eval frame API?  
Or would it prevent from being used entirely?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RVQ7LDIJ2OYAN4QMIPTI3A3PODGBLNN7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Paul Moore]

On Mon, 28 Mar 2022 at 20:37, Steve Dower  wrote:

> Please let us know
> *publicly* if you want to become the maintainer for a stdlib module and
> then we can support them, but if nobody is willing/able/ready to care
> for them it's irresponsible for us to keep shipping them to users.

I'm sorry for picking on a single point you made and continuing to
simply state my contrary view, but I think it's important.

IMO, it's irresponsible of us to *remove* functionality if there's no
suitable alternative (and remember, my definition of "suitable"
includes "appropriate for cases where a PyPI library isn't acceptable
for some reason"). Of course, it's not OK for us to promise a level of
support that we can't or won't provide, but *we don't provide such
promises*. We say that we support the stdlib, but there's no actual
statement of what that means - so it's not clear to me how we're
failing to deliver "enough". Nor is it at all clear to me that dumping
the responsibility on an external maintainer will be any better - most
likely it just lets *us* feel better about the situation while not
actually being better for the end user.

If we were to define some sort of actual support guarantees, we might
be having a different conversation. But I'm unclear how a volunteer
organisation can provide meaningful promises that we wouldn't be able
to deliver for the stdlib as it stands now.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OSYFNLIULB56EVSG57QXKIW4NRWI4CNZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Skip Montanaro]

> What happens when the new maintainer puts malware in the next release of
> a package in sumo.txt?
> Will core devs be blamed for listing it?
> As a user, how do I determine if I can trust the packages there? (This
> is easily the hardest part of finding and installing a package from
> PyPI, though many people seem to skip it.)

I will point out that you quoted my entire post except for the most
important line, the one which reads:

> Just thinking out loud...

I am still thinking out loud, so keep that in mind. (Perhaps this belongs
on one of the two ideas groups, but this is where things started, so I will
continue here for now.)

Consider a hypothetical situation.

   - Suppose Python v. X is the last version delivered with batteries, so
   v. Y lacks a json module (assuming that's not determined to be so
   important that it must be delivered as part of the CPython installer proper)
   .
   - Downstream, some Linux distro realizes their users still want
   batteries (as do their Python admin tools), so recreates a Python
   package with them. Their package of v. Y *does* include (at least) the
   json battery.
   - One of the maintainers of the now externally maintained json battery
   takes it upon themselves to protest an incursion into East WhatsItStan by
   West WhatsItStan in the worst manner possible and inserts code to scrub
   disks of all West WhatsItStan-ian homed computers (as the primary
   maintainer of some widely installed (Node? JS?) package package did).
   - Some poor West WhatsItStan-ian upgrades their Linux distro, only to
   find when they next run their favorite Python application that their disk
   drive is wiped clean.

Note that I haven't postulated the existence of a sumo.txt file. Whether
the CPython distribution contains batteries or not, someone will recreate
the batteries, if not in toto, at least piecemeal (some application or
package depends on json and will blindly pull it in). Who gets the blame
here? The core Python devs (because json "always came with Python")? The
maintainers of the Linux distro for recreating the batteries but not having
any West WhatsItStanian-based testers? The new maintainers of the now
external (and corrupted) json package? Maybe the several levels of
indirection would serve to insulate the Python devs, but maybe not.

> If they do exert control, why not keep it in stdlib?

Perhaps that is the best route. Someone here (Stéfane Fermigier, it seems)
questioned whether batteries included are such a good idea. It's quite
possible that enough core devs disagree with that sentiment that the
batteries will stay (and perhaps grow in number, though more slowly than in
the past). That said, if enough devs agree with Stéfane, then what's the
best route forward? Discard all batteries and let the chips fall where they
may? Extract the dead batteries (and presumably their docs and test bits)
into separate github.com/python repositories? Ask others to extract them
into their own non-github.com/python repositories?

One of the common reasons old platform support is dropped from CPython
(OS/2 or AmigaOS anyone?) is that the maintenance load for the core devs is
too high relative to the community benefit derived from supporting small
minority platforms. The unlucky modules named by PEP 594 are about to
suffer the same fate. (I'm not arguing that they shouldn't be deleted.)

Once PEP 594 has been implemented, all the low-hanging fruit will have been
picked. At that point, it's keep everything or keep (almost) nothing. I
think that will largely depend on the willingness of the core devs to keep
maintaining modules they may well never use in their own work (like getopt
and wave). One thing I think is obvious. If you remove all the batteries
not deemed absolutely essential to build and test CPython, I think you have
to somehow symbolically say, "these correspond to the various batteries
which used to be in the CPython distribution." Maybe it's text in a README
file, a new PEP, a sumo.txt file, or a Linux distro's packaging.

Skip

P.S. Personally, I was never fond of shutil or the logging package (as two
examples — though for different reasons). We might have something better in
both realms by now if they hadn't been added to the stdlib long ago. This
ability of presence in the stdlib to forestall further development might
well be the strongest argument to remove most batteries. On the other hand,
we seem to have had little trouble cycling through several completely
different regular expression modules. Maybe I'm just imagining a barrier
where none exists.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PPGY7Q2OWFJ6JYUDAXWVW2HDGEK47C2I/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Steve Dower]

On 3/28/2022 7:26 PM, Paul Moore wrote:

To be honest, I feel like I'm just reiterating stuff I've said before
here, and I think the same is true of the points I'm responding to. Is
there actually any new development here, or is it just a repeat of the
same positions people have expressed in the past, based on the same
underlying realities? (I'm not *against* going over the debate again,
it helps make sure people haven't changed their minds, but it's
important to be clear that none of the practical facts have changed,
if that is the case).


Yep, same. Nothing much has changed, but if those of us who know that 
nothing has changed fail to reiterate those points, everyone new to the 
discussion will assume that everyone agrees with them.


Just deleted a long rant to the discussion in general (not directed at 
Paul at all ;) ) that would have achieved nothing, but I'll summarise to 
this: email doesn't fix bugs; maintainers fix bugs. Please let us know 
*publicly* if you want to become the maintainer for a stdlib module and 
then we can support them, but if nobody is willing/able/ready to care 
for them it's irresponsible for us to keep shipping them to users.

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I37FCHGNGCKACI4C4S3OA6MO4MZLRABY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Christopher Barker]

On Mon, Mar 28, 2022 at 11:29 AM Paul Moore  wrote:

> To be honest, I feel like I'm just reiterating stuff I've said before
> here, and I think the same is true of the points I'm responding to

 ...

>  (I'm not *against* going over the debate again,
> it helps make sure people haven't changed their minds, but it's
> important to be clear that none of the practical facts have changed,
> if that is the case).
>

Maybe there's a way to make this discussion (it feels more like a
discussion than debate at the moment) more productive by writing some
things down. I'm not sure it's a PEP, but some kind of:

"policy for the stdlib" document in which we could capture the primary
points of view, places where there's consensus, etc. would be helpful to
keep us retreading this over and over again.

I suggest this without the bandwidth to actually shepherd the project, but
if someone wants to, I think it would be a great idea.

-CHB

-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GAZAFRFVJVQZMEIHTQUJASP7VRAKA5RR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Christopher Barker]

On Sun, Mar 27, 2022 at 11:04 AM Paul Moore  wrote:

> > In fact, this is an example, I think, of where we should put some effort
> into making the included batteries better -- it's great to have a JSON lib
> built in, but it's too bad that it's not best-of-bread by pretty much any
> definition (speed, memory performance, flexibility) -- there are quite a
> few feature requests open for it -- it would be nice to actually implement
> some of those. (but yes, that's a lot of work that someone(s) would have to
> do)
> >
> > Back to the topic at hand, rather than remove urllib, maybe it could be
> made better -- an as-easy-to-use-as-requests package in the stdlib would be
> really great.
>
> I think that's where the mistake happens, though. Someone who needs
> "best of breed" is motivated (and likely knowledgeable enough) to make
> informed decisions about what's on PyPI. But someone who just wants to
> get the job done probably doesn't - and that's the audience for the
> stdlib. A stdlib module needs to be a good, reliable set of basic
> functionality that non-experts can use successfully. There can be
> better libraries on PyPI, but that doesn't mean the stdlib module is
> unnecessary, nor does it mean that the stdlib has to match the PyPI
> library feature for feature.
>
> So here, specifically, I'd rather see urlllib be the best urlllib it
> can be, and not demand that it turn into requests. Requests is there
> if people need/want it.


Actually, I think this is very hard to talk about in generalities. If. you
look at the two examples I used, json and urllib, we have two different
"problems".

I would like to see a requests-like interface in the stdlib not because
it's "best of breed", and has, e.g. more features or better performance, or
, I would like to see it because it has a cleaner, simpler interface
(the easy things should be easy) -- so I'm not necessarily advocating that
the entirely of requests be brought in to the stdlib, or that request be
made obsolete, but that we borrow a bit of its API. Which would also make
the transition from stdlib to requests easier -- write you scripts with the
stdlib, and when you find you need some more "advanced" features, you simpy
drop requests in and move along.

json is actually the opposite- most of the third-party json libs mimic (and
maybe extend) the json API -- so you can, in most cases, start out with the
stdlib, and drop in another app when you need more performance, or an extra
feature.

The issue I have with the stdlib json lib is that  I think it could have a
somewhat better API, and there's a few features I'd like to see it grow, so
that third party libs are less necessary. But it seems there's not much
motivation to do so, because "folks can always use a third party lib". Why
is this an problem? two reasons:

1) There are quite a few json libs out there, and they tend to be a fourth
party dependency (I just coined that term for a dependency of a dependency)
. I literally have some of my code requiring four different json libs to
run. It works, so maybe not a huge deal, but it would be really nice to
simplify that!

2) I don't think any of the third party libs provide a full menu -- JSON5
support, performance, etc, so sometimes there's no single best of breed for
even a particular application. Can/should that be solved by the stdlib?
maybe not, but it would be nice to see it addressed somewhere -- a grand
unification of JSON libs.

In short: I think I agree with most folks here that we should still include
the batteries, and they should be updated / maintained to some extent. What
exactly could/should be done is going to have to be worked on on a case by
case basis.

-CHB

-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WTBHIJKVU32OUJ7BGSBZP4HNTZAKT2TG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Paul Moore]

On Mon, 28 Mar 2022 at 17:45, Steve Dower  wrote:
>
> I think to most people "batteries included" doesn't necessarily mean
> "standard library," with all that implies. It just means "it came with
> the first thing that I installed" (or alternatively, "at no additional
> charge").

I think that for users it means "came as part of the official
installer" where "official" means "whoever I chose to trust" - which
in turn often means "python.org".

For package maintainers it means "the bunch of stuff I can assume
exists when someone says they have python" - which is why distros that
debundle parts of the stdlib get so much grief. Also note that "just
add a dependency" is not a simple request for a library maintainer -
if I depend on requests, and someone using my library uses httpx, then
I'm imposing a potential transitive dependency conflict on them. So
using stdlib functionality is a non-trivial advantage for library
maintainers (of course, sometimes the benefits of using a 3rd party
library outweigh this, but not always).

> Given there are *plenty* of existing distros out there that install more
> than just the standard library for the benefit of their users, and yet
> the people asking for a fatter stdlib don't seem to want them, perhaps
> we just need to officially endorse one?

But people don't use those existing options presumably for a reason.
Having the core devs say "distro X is approved" won't remove those
reasons. And removing the python.org distro because X is now approved
will create a problem for those users.

And none of this alters the fact that "stdlib" means "can be assumed
to be present in all Python installations" for some use cases.

> I've got no issue preinstalling a curated sumo.txt into the Windows
> installers, provided we're clear about support (i.e. none from us) and
> someone besides me is tracking everything in that curated list. That
> probably requires new volunteers who are somehow already trustworthy,
> but that should be easier than people to actually write and maintain new
> functionality on stdlib timelines.

"Support" is a somewhat ambiguous matter in the context of
volunteer-supported software. The point of having a stdlib is that
people know where to report issues, not that we're necessarily any
more responsive than a 3rd party maintainer. There's a value in
knowing that if you find an issue with "anything you installed as part
of the basic Python install", you report it in the same place. And
having the documentation all in one place. Etc.

Yes, people come along and make a big fuss about how they expect more
in the way of support ("this bug is urgent, my production system is
failing because of it") and yes, people with that attitude have a
negative effect on core developer morale. But we don't promise fixes,
or solutions. At best there's an implication that the core devs care
about the stdlib in some sort of vague, generalised sense. (I care
about the health of the stdlib, but I'm not likely to try to fix a bug
in ossaudiodev, for example).

If we did try to mandate something more concrete, then maybe the
conversation would be different. But I'm not aware of anyone proposing
anything like that.

> We would still have to have the "light" distribution, as many of our
> users wouldn't be able to use a "this is full of unsupported stuff"
> package, but at least most users could be pointed to the bigger install
> by default and not even care that the batteries just came in the same
> package and aren't actually an integral part of the core product.

To be honest, I feel like I'm just reiterating stuff I've said before
here, and I think the same is true of the points I'm responding to. Is
there actually any new development here, or is it just a repeat of the
same positions people have expressed in the past, based on the same
underlying realities? (I'm not *against* going over the debate again,
it helps make sure people haven't changed their minds, but it's
important to be clear that none of the practical facts have changed,
if that is the case).

I should also point out (as someone heavily involved in Python's
packaging ecosystem) that I don't believe that the existing packaging
infrastructure is sufficiently robust to support a world where the
Python stdlib was all delivered as external packages. And even if it
were, bootstrapping something like pip would be a nightmare - for a
start, we don't vendor libraries that include C extensions, so we
could only handle pure Python stdlib modules - and even vendoring all
of them (and all of our vendored tools' stdlib dependencies) would be
a nightmare.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SKIKUC4U3L74G43BNC5SDCXFYUY3G447/
Code of Conduct: 

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Damian Shaw]

> If urllib is removed, I would very much like to preserve at least the
functionality of urlparse /somewhere/.

Given every alternative library to urllib relies on urllib.parse (and some
rely on urllib.request), as well as popular libraries like pip, in this
hypothetical it would definitely need to be maintained somewhere.

Damian (he/him)

On Mon, Mar 28, 2022 at 1:38 PM lincoln auster [they/them] <
lincolnaus...@gmail.com> wrote:

>
> "Coyot Linden (Glenn Glazer)"  writes:
> > P.S. There are uses for urllib outside of standard web programming. I
> > recently needed to handle strings in query parameter format
> > (`?...&...&...`) and found urllib parse() the easiest way of doing
> > that even though I wasn't taking them in as a webservice. Deleting the
> > library would break this code. :(
>
> +1. If urllib is removed, I would very much like to preserve at least
> the functionality of urlparse /somewhere/.
>
> --
> Lincoln Auster
> they/them
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/C3JZDCWLCMED32Q4MFHMITI2QQGK2ME5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4FL6DXXIHEC44MQUXTRU4T5E3MIDM4T4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[lincoln auster [they/them]]

"Coyot Linden (Glenn Glazer)"  writes:
> P.S. There are uses for urllib outside of standard web programming. I
> recently needed to handle strings in query parameter format
> (`?...&...&...`) and found urllib parse() the easiest way of doing
> that even though I wasn't taking them in as a webservice. Deleting the
> library would break this code. :(

+1. If urllib is removed, I would very much like to preserve at least
the functionality of urlparse /somewhere/.

-- 
Lincoln Auster
they/them
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/C3JZDCWLCMED32Q4MFHMITI2QQGK2ME5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Steve Dower]

I think to most people "batteries included" doesn't necessarily mean 
"standard library," with all that implies. It just means "it came with 
the first thing that I installed" (or alternatively, "at no additional 
charge").


Given there are *plenty* of existing distros out there that install more 
than just the standard library for the benefit of their users, and yet 
the people asking for a fatter stdlib don't seem to want them, perhaps 
we just need to officially endorse one?


I've got no issue preinstalling a curated sumo.txt into the Windows 
installers, provided we're clear about support (i.e. none from us) and 
someone besides me is tracking everything in that curated list. That 
probably requires new volunteers who are somehow already trustworthy, 
but that should be easier than people to actually write and maintain new 
functionality on stdlib timelines.


We would still have to have the "light" distribution, as many of our 
users wouldn't be able to use a "this is full of unsupported stuff" 
package, but at least most users could be pointed to the bigger install 
by default and not even care that the batteries just came in the same 
package and aren't actually an integral part of the core product.


Cheers,
Steve

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TC5EEDFNSZJBR22GQDJ32UWVTQNILU2P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Petr Viktorin]

On 28. 03. 22 17:34, Skip Montanaro wrote:

Barry writes (in part):


We could still distribute “sumo” releases which include all the
batteries, but develop and maintain them outside the cpython repo,
and even release them separately on PyPI.  It’s *possible* but I
don’t know if it’s *practical*.


to which Stephen responds (in part):


[Emacs really didn't change much in 20 years.]  [I]n
Python, every new release makes the Mailman crew want to stop
supporting all previous releases of Python because there's some
feature that can't be emulated that we really love: genexps or async
or walrus operator or 


It seems to me that what we have is the possibility that, say, package
P migrates to PyPI in concert with the release of Python version X
where maintenance can be picked up by the broader Python community.
Whether or not it would proceed to track ongoing changes to the
language isn't clear, and it's not obvious to me that less effort
would be required other than perhaps by core devs (though some of them
would likely pitch in to maintain packages with which they are
currently involved). If you go with the "discard the batteries"
approach, I think it would at least be worthwhile distributing a
requirements.txt file ("sumo.txt" or "batteries.txt"?) which would
tell someone installing Python how to reclaim the batteries of their
Python youth, and for other Python implementations to track a set of
packages which would make them plausibly compatible with CPython. CI
could still rely on that to provide as much test coverage as current
today (I think). If nothing else, it might alert the core devs to the
potential for breakage of presumably widely used packages by changes
to CPython.



What happens when the new maintainer puts malware in the next release of 
a package in sumo.txt?

Will core devs be blamed for listing it?
As a user, how do I determine if I can trust the packages there? (This 
is easily the hardest part of finding and installing a package from 
PyPI, though many people seem to skip it.)




What happens to P when Python version Y grows a new syntactic feature?
Do P's maintainers fork to both continue feature growth as well as
syntactic modernization? If something like batteries.txt is created to
tie a slimmed-down CPython distribution to the batteries it once
contained, would the core group exert any control over unit testing,
documentation, package variations and such?


If they do exert control, why not keep it in stdlib?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YLK5UMSWYXX6JT2PL23F26DGQ5XSOVV2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: import * and __future__ imports

[Guido van Rossum]

On Mon, Mar 28, 2022 at 8:52 AM Irit Katriel 
wrote:

>
>
> On Mon, Mar 28, 2022 at 4:44 PM Guido van Rossum  wrote:
>
>>
>> "Future" imports are special to the parser, and they may also set a flag
>> for the runtime to alter its behavior, but they are intentionally not
>> treated specially by code generation, so they are still properly imported.
>> However the presence of the imported thing is not used by the runtime to
>> determine its behavior; those flags are stored elsewhere guided by the code
>> generator.
>>
>
> Right, this is why it's confusing when the object is there for no reason
> (the future import did not have any impact on the module, but the object
> showed up via an import *).
>

As Petr said, it's no more confusing than any other imported thing showing
up as a global.


> I don't think there's anything to do here.
>>
>
> How about the suggestion in https://bugs.python.org/issue26120 ? It's
> about these objects showing up in pydoc (both when the __future__ had an
> impact and when it didn't - in either case it's not an interesting part of
> the module's API).
>

If pydoc wants to filter that's up to pydoc (I'm not maintaining that).
Though really this should be keyed off `__all__`.


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UPFIY4DCIELDR6M5KAWLCRCKGG7G5MWM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: import * and __future__ imports

[Irit Katriel via Python-Dev]

On Mon, Mar 28, 2022 at 4:44 PM Guido van Rossum  wrote:

>
> "Future" imports are special to the parser, and they may also set a flag
> for the runtime to alter its behavior, but they are intentionally not
> treated specially by code generation, so they are still properly imported.
> However the presence of the imported thing is not used by the runtime to
> determine its behavior; those flags are stored elsewhere guided by the code
> generator.
>

Right, this is why it's confusing when the object is there for no reason
(the future import did not have any impact on the module, but the object
showed up via an import *).


>
I don't think there's anything to do here.
>

How about the suggestion in https://bugs.python.org/issue26120 ? It's about
these objects showing up in pydoc (both when the __future__ had an impact
and when it didn't - in either case it's not an interesting part of the
module's API).
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JVHTGPOZX72J73SUVKGEWBXHKK4KIFAV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: is __self__ an implementation detail

[Guido van Rossum]

It's documented in stdtypes.rst, so I think it's part of the standard
library.

I think it's unnecessarily obfuscated though.

On Mon, Mar 28, 2022 at 8:44 AM Robin Becker  wrote:

> A gentoo developer reported a symlink loop problem in reportlab's setup.py
> where we search for specific headers.
>
> The 'fixed' function looks like this
>
> def findFile(root, wanted, followlinks=True):
>  visited = set()
>  for p, D, F in os.walk(root,followlinks=followlinks):
>  #scan directories to check for prior visits
>  #use dev/inode to make unique key
>  SD = [].append
>  for d in D:
>  dk = os.stat(pjoin(p,d))
>  dk = dk.st_dev, dk.st_ino
>  if dk not in visited:
>  visited.add(dk)
>  SD(d)
>  D[:] = SD.__self__  #set the dirs to be scanned
>  for fn in F:
>  if fn==wanted:
>  return abspath(pjoin(p,fn))
>
> the fix is reported to have worked, but the developer is querying the
> lifting of  SD.append using the construct
>
> SD = [].append
> loop involving
>appends to the list
> .
> D[:] = SD.__self__
>
> his objection was that using __self__ might be implementation sensitive.
>
> I cannot tell from the documentation (eg
> https://docs.python.org/3.10/reference/datamodel.html) if these magic
> methods
> are part of python or of the implementation.
>
> On the other hand as a longtime python programmer I wonder if such simple
> cases are already optimized out in the
> produced bytecode; for years I have been avoiding s += 'string' and only
> recently found out that it was handled as a
> special case.
> --
> Robin Becker
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/3TAF3LR35HRRE6LD7XQ7O4BXPNLVXFVX/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6IMBFHOGAYJDDGJVPVYJNQJS4MICGJ7Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Coyot Linden (Glenn Glazer)]

  
  

  On Sun, Mar 27, 2022 at 3:08 AM Paul Moore p.f.mo...@gmail.com wrote:
  ...
  exactly - let's say someone needs to write some JSON for the
first time.
  With the json package included, all they need to do is `import
json`. If
that wasn't there, they's look in PyPi for a JSON
implementation, and find
an absolutely astonishing number of options. I just did a search
for "JSON" on PYPI, and it's HUGE -- most of them are for
specialized JSON-using
protocols of some sort. I was actually really surprised that
couple I know
about of the top of my head (ujson, orjson) are actually hard to
find.
  "You can just pip install it" is really not a good solution.
  In fact, this is an example, I think, of where we should put
some effort
into making the included batteries better -- it's great to have
a JSON lib
built in, but it's too bad that it's not best-of-bread by pretty
much any
definition (speed, memory performance, flexibility) -- there are
quite a
few feature requests open for it -- it would be nice to
actually implement some of those. (but yes, that's a lot of work
that
someone(s) would have to do)
  Back to the topic at hand, rather than remove urllib, maybe it
could be
made better -- an as-easy-to-use-as-requests package in the
stdlib would be
really great.
  -CHB


Strong plus one to this. Part of Python's huge attraction and
widespread adoption is that one doesn't need a lot of formal
training or engineering experience to just dive right in. Having a
rich set of libraries only helps this, forcing newbies to pip/PYPI
is a great way to lose them. 

I also completely agree that the right answer is to make the
standard libraries better, not throw them away.

And for those that don't use them, ignore them. 

Best,

coyot
P.S. There are uses for urllib outside of standard web programming.
I recently needed to handle strings in query parameter format
(`?...&...&...`) and found urllib parse() the easiest way of
doing that even though I wasn't taking them in as a webservice.
Deleting the library would break this code. :(
  

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Z7YNNVZKXBQO4QL2W2RYPDSTLEZZXNVV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: import * and __future__ imports

[Guido van Rossum]

On Mon, Mar 28, 2022 at 6:29 AM Irit Katriel via Python-Dev <
python-dev@python.org> wrote:

> If you have a __future__ import in a script, and you import * from it in
> another script, the object for this future appears in the dir() of the
> other script, even though the __future__ import has no effect there.
>
> % cat x.py
> from __future__ import annotations
>
>
> % cat y.py
> from x import *
>
> print(dir())
>
> class D:
> def f(self, a: D):
> return 42
>
> % ./python.exe y.py
> ['__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', 
> '__loader__', '__name__', '__package__', '__spec__', '*annotations*']
> Traceback (most recent call last):
>   File "/Users/iritkatriel/src/cpython-654/y.py", line 5, in 
> class D:
> 
>   File "/Users/iritkatriel/src/cpython-654/y.py", line 6, in D
> def f(self, a: D):
>^
>
> NameError: name 'D' is not defined
>
>
> I think we should change import * to exclude the __future__ import
> objects, and perhaps also to not show them in dir(x).   Any objections?
>
> This came up in the discussion about https://bugs.python.org/issue26120 .
> See the attached PR for a technique we can use to identify those objects.
>

"Future" imports are special to the parser, and they may also set a flag
for the runtime to alter its behavior, but they are intentionally not
treated specially by code generation, so they are still properly imported.
However the presence of the imported thing is not used by the runtime to
determine its behavior; those flags are stored elsewhere guided by the code
generator.

I don't think there's anything to do here.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GRZFVFLEGSXHLUHWHEV75AMMBCEJLVUE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] is __self__ an implementation detail

[Robin Becker]

A gentoo developer reported a symlink loop problem in reportlab's setup.py 
where we search for specific headers.

The 'fixed' function looks like this

def findFile(root, wanted, followlinks=True):
visited = set()
for p, D, F in os.walk(root,followlinks=followlinks):
#scan directories to check for prior visits
#use dev/inode to make unique key
SD = [].append
for d in D:
dk = os.stat(pjoin(p,d))
dk = dk.st_dev, dk.st_ino
if dk not in visited:
visited.add(dk)
SD(d)
D[:] = SD.__self__  #set the dirs to be scanned
for fn in F:
if fn==wanted:
return abspath(pjoin(p,fn))

the fix is reported to have worked, but the developer is querying the lifting 
of  SD.append using the construct

SD = [].append
loop involving
  appends to the list
.
D[:] = SD.__self__

his objection was that using __self__ might be implementation sensitive.

I cannot tell from the documentation (eg https://docs.python.org/3.10/reference/datamodel.html) if these magic methods 
are part of python or of the implementation.


On the other hand as a longtime python programmer I wonder if such simple cases are already optimized out in the 
produced bytecode; for years I have been avoiding s += 'string' and only recently found out that it was handled as a 
special case.

--
Robin Becker
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3TAF3LR35HRRE6LD7XQ7O4BXPNLVXFVX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Skip Montanaro]

Barry writes (in part):

> We could still distribute “sumo” releases which include all the
> batteries, but develop and maintain them outside the cpython repo,
> and even release them separately on PyPI.  It’s *possible* but I
> don’t know if it’s *practical*.

to which Stephen responds (in part):

> [Emacs really didn't change much in 20 years.]  [I]n
> Python, every new release makes the Mailman crew want to stop
> supporting all previous releases of Python because there's some
> feature that can't be emulated that we really love: genexps or async
> or walrus operator or 

It seems to me that what we have is the possibility that, say, package
P migrates to PyPI in concert with the release of Python version X
where maintenance can be picked up by the broader Python community.
Whether or not it would proceed to track ongoing changes to the
language isn't clear, and it's not obvious to me that less effort
would be required other than perhaps by core devs (though some of them
would likely pitch in to maintain packages with which they are
currently involved). If you go with the "discard the batteries"
approach, I think it would at least be worthwhile distributing a
requirements.txt file ("sumo.txt" or "batteries.txt"?) which would
tell someone installing Python how to reclaim the batteries of their
Python youth, and for other Python implementations to track a set of
packages which would make them plausibly compatible with CPython. CI
could still rely on that to provide as much test coverage as current
today (I think). If nothing else, it might alert the core devs to the
potential for breakage of presumably widely used packages by changes
to CPython.

What happens to P when Python version Y grows a new syntactic feature?
Do P's maintainers fork to both continue feature growth as well as
syntactic modernization? If something like batteries.txt is created to
tie a slimmed-down CPython distribution to the batteries it once
contained, would the core group exert any control over unit testing,
documentation, package variations and such?

Just thinking out loud...

Skip
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/76ZTGO2PD6Q5NZOO5JWJRTTR2E2JY2H6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: import * and __future__ imports

[Petr Viktorin]

On 28. 03. 22 15:25, Irit Katriel via Python-Dev wrote:
If you have a __future__ import in a script, and you import * from it in 
another script, the object for this future appears in the dir() of the 
other script, even though the __future__ import has no effect there.


% cat x.py
from __future__ import annotations
  


% cat y.py
from x import *

print(dir())

class D:
 def f(self, a: D):
 return 42

% ./python.exe y.py
['__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', 
'__loader__', '__name__', '__package__', '__spec__', '*annotations*']
Traceback (most recent call last):
   File "/Users/iritkatriel/src/cpython-654/y.py", line 5, in 
 class D:
 
   File "/Users/iritkatriel/src/cpython-654/y.py", line 6, in D
 def f(self, a: D):
^

NameError: name 'D' is not defined


I think we should change import * to exclude the __future__ import 
objects, and perhaps also to not show them in dir(x).   Any objections?


This came up in the discussion about https://bugs.python.org/issue26120 
 . See the attached PR for a 
technique we can use to identify those objects.



__future__ imports are just one way to put garbage in the global 
namespace. I don't think they should be handled specially.


IMO, it would be better to document the current best practice that 
`import *` should only be used with modules designed for `import *` (by 
setting `__all__`; carefully `del`-ing what they don't want to export 
might work too but it's fragile), or when you don't care about namespace 
pollution (e.g. in interactive sessions).



___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PTYURIZ4KHKTRDKOVATUYQW55EG6YGLK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] import * and __future__ imports

[Irit Katriel via Python-Dev]

If you have a __future__ import in a script, and you import * from it in 
another script, the object for this future appears in the dir() of the other 
script, even though the __future__ import has no effect there.
% cat x.py
from __future__ import annotations
 % cat y.py
from x import *

print(dir())

class D:
def f(self, a: D):
return 42

% ./python.exe y.py
['__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', 
'__loader__', '__name__', '__package__', '__spec__', 'annotations']
Traceback (most recent call last):
  File "/Users/iritkatriel/src/cpython-654/y.py", line 5, in 
class D:

  File "/Users/iritkatriel/src/cpython-654/y.py", line 6, in D
def f(self, a: D):
   ^
NameError: name 'D' is not defined
I think we should change import * to exclude the __future__ import objects, and 
perhaps also to not show them in dir(x).   Any objections?

This came up in the discussion about https://bugs.python.org/issue26120 . See 
the attached PR for a technique we can use to identify those objects.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3BUUCY2OHZILBJLOTS6I33M5YW4INDW3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

[Petr Viktorin]

On 24. 03. 22 20:06, Fabio Zadrozny wrote:


Em qui., 24 de mar. de 2022 às 15:39, Fabio Zadrozny > escreveu:


PEP 523 API added more private functions for code objects:

* _PyEval_RequestCodeExtraIndex()
* _PyCode_GetExtra()
* _PyCode_SetExtra()

The _PyEval_RequestCodeExtraIndex() function seems to be used by the
pydevd debugger. The two others seem to be unused in the wild.
I'm not
sure if these ones should be moved to the internal C API. They
can be
left unchanged, since they don't use a type only defined by the
internal C API.

Just to note, the pydevd/debugpy debuggers actually uses all of
those APIs.

i.e.:


https://github.com/fabioz/PyDev.Debugger/blob/main/_pydevd_frame_eval/pydevd_frame_evaluator.template.pyx#L187



https://github.com/fabioz/PyDev.Debugger/blob/main/_pydevd_frame_eval/pydevd_frame_evaluator.template.pyx#L232



https://github.com/fabioz/PyDev.Debugger/blob/main/_pydevd_frame_eval/pydevd_frame_evaluator.template.pyx#L311



The debugger already has workarounds because of changes to
evaluation api over time (see:

https://github.com/fabioz/PyDev.Debugger/blob/main/_pydevd_frame_eval/pydevd_frame_evaluator.template.pyx#L491

)
and I know 3.11 won't be different.

I'm ok with changes as I understand that this is a special API -- as
long as there's still a way to use it and get the information needed
(the debugger already goes through many hops because it needs to use
many internals of CPython -- in every new release it's a **really**
big task to update to the latest version as almost everything that
the debugger relies to make debugging fast changes across versions
and I never really know if it'll be possible to support it until I
really try to do the port -- I appreciate having less things in a
public API so it's easier to have extensions work in other
interpreters/not recompiling on newer versions, but please keep it
possible to use private APIs which provides the same access that
CPython has to access things internally for special cases such as
the debugger).

Maybe later on that PEP from mark which allows a better debugger API
could alleviate that (but until then, if possible I appreciate it if
there's some effort not to break things unless really needed --
ideally with instructions on how to port).

Anyways, to wrap up, the debugger already needs to be built with
`Py_BUILD_CORE_MODULE=1` anyways, so, I guess having it in a private
API (as long as it's still accessible in that case) is probably not
a big issue for the debugger and having setters/getters to set it
instead of relying on `state.interp.eval_frame` seems good to me.

Cheers,

Fabio


I think the main issue here is the compatibility across the same version 
though... is it possible to have some kind of guarantee on private APIs 
that something won't change across micro-releases?


Currently we don't really have that (except that in practice, bugfix 
releases tend to not need internal API changes.)



I.e.: having the frame evaluation function change across major releases 
and having them be reworked seems reasonable, but then having the frame 
evaluation be changed across micro-releases wouldn't be.


So, I'm ok in pushing things to the internal API, but then I still would 
like guarantees about the compatibility of that API in the same major 
release (otherwise those setters/getters/frame evaluation should 
probably remain on the public API if the related structure was moved to 
the internal API).


Perhaps we need a new "tier" of C API for debuggers -- API that's 
guaranteed stable for a major release, and if it's changed it should 
always break with compile errors (e.g. the function gets a new 
argument), rather than silently change semantics.

The internal API Cython & greenlet need might go it this category too.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/OF4UNE22IDDJDJVQZMOUXHELAYEFCJ45/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Stephen J. Turnbull]

Barry Warsaw writes:

 > While I don’t underestimate the work and complexity, we can do
 > both.  I.e. separate the stdlib development cycle from the
 > interpreter (for all but a handful of required packages perhaps).
 > We could still distribute “sumo” releases which include all the
 > batteries, but develop and maintain them outside the cpython repo,
 > and even release them separately on PyPI.  It’s *possible* but I
 > don’t know if it’s *practical*. 

After a couple decades maintaining XEmacs, I'd have to say that one of
the things that made our sumos (you heard that word here first,
folks!) practical was that about 80% of the code was developed in GNU
Emacs.  That is, developed in concert with core Emacs.  This meant
that dependencies (think "hybrid of hydra, kitsune-yokai, and
ouroboros" :-) evolve together, as they do in Python.  This is
probably more important in Emacs because of the big-ball-of-string
nature of big Lisp systems (at least with a relatively undisciplined
dev process), but I'd still say there would be *substantial*
coordination costs to be paid if you split the repos.

Also, the only feature in Emacs Lisp that really changed the way
people code since 1990 (and likely earlier) is lexical scope.
Everything else is merely a new callable for practical purposes, and
almost all of it can be implemented in Lisp either as a function or a
macro (although usually at great expense in performance).  But in
Python, every new release makes the Mailman crew want to stop
supporting all previous releases of Python because there's some
feature that can't be emulated that we really love: genexps or async
or walrus operator or   It was certainly quite costly in XEmacs
trying to most packages usable in 21.4, mostly in frustration about
not being able to use 21.5 features. ;-)  But Python has like 4 or 5
generations of stdlib in some form of development or maintenance at
any given time.  It's really not attractive to think either about
creating a separate Sumo Dev Core to maintain several generations of
stdlib corresponding to in-maintenance versions of Python, or about a
lowest-common-denominator stdlib.

Until XEmacs split off the stdlib into packages, Emacs didn't have
anything like a package repository, certainly nothing like PyPI.  I
don't have experience with the Emacs package repos ELPA and MELPA for,
uh, hysterical raisins, so season the next comment to taste.  My
feeling is that both splitting out the stdlib and increasing the rate
at which new packages are added are excessively expensive non-
solutions to the problem that Paul has described, that finding the
right package on PyPI is pretty hard, especially for people new to a
technology.  The extra effort that would be needed to achieve either,
as well as the increased on-going maintenance burden on the core team,
would not really resolve the problem, given that there are now whole
large ecosystems like Django and SciPy outside of Python core
development.

Although it's a different kind of effort from package incorporation
and maintenance, to the extent that effort is fungible, I think it
would be better devoted to creating an annotated catalog of PyPI.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZKODGJ3I6XQNX7ISE4EVCI5RVRCTTOA4/
Code of Conduct: http://python.org/psf/codeofconduct/