Re: [Python-Dev] Fuzzing the Python standard library

[Damian Shaw]

I'm not a core Python Dev, but quick question, why would you expect "
fractions.Fraction("1.64E664644")" not to take 100s of megabytes and
hours to run?

Simply evaluating: 164 * 10**66464 will take hundreds of megabytes by
definition.

Regards
Damian


On Tue, Jul 17, 2018, 12:54 Jussi Judin  wrote:

> Hi,
>
> I have been fuzzing[1] various parts of Python standard library for Python
> 3.7 with python-afl[2] to find out internal implementation issues that
> exist in the library. What I have been looking for are mainly following:
>
> * Exceptions that are something else than the documented ones. These
> usually indicate an internal implementation issue. For example one would
> not expect an UnicodeDecodeError from netrc.netrc() function when the
> documentation[3] promises netrc.NetrcParseError and there is no way to pass
> properly sanitized file object to the netrc.netrc().
> * Differences between values returned by C and Python versions of some
> functions. quopri module may have these.
> * Unexpected performance and memory allocation issues. These can be
> somewhat controversial to fix, if at all, but at least in some cases from
> end-user perspective it can be really nasty if for example
> fractions.Fraction("1.64E664644") results in hundreds of megabytes of
> memory allocated and takes very long to calculate. I gave up waiting for
> that function call to finish after 5 minutes.
>
> As this is going to result in a decent amount of bug reports (currently I
> only filed one[4], although that audio processing area has much more issues
> to file), I would like to ask your opinion on filing these bug reports.
> Should I report all issues regarding some specific module in one bug
> report, or try to further split them into more fine grained reports that
> may be related? These different types of errors are specifically noticeable
> in zipfile module that includes a lot of different exception and behavioral
> types on invalid data <
> https://github.com/Barro/python-stdlib-fuzzers/tree/master/zipfile/crashes>
> . And in case of sndhdr module, there are multiple modules with issues
> (aifc, sunau, wave) that then show up also in sndhdr when they are used. Or
> are some of you willing to go through the crashes that pop up and help with
> the report filing?
>
> The code and more verbose description for this is available from <
> https://github.com/Barro/python-stdlib-fuzzers>. It works by default on
> some GNU/Linux systems only (I use Debian testing), as it relies on
> /dev/shm/ being available and uses shell scripts as wrappers that rely on
> various tools that may not be installed on all systems by default.
>
> As a bonus, as this uses coverage based fuzzing, it also opens up the
> possibility of automatically creating a regression test suite for each of
> the fuzzed modules to ensure that the existing functionality (input files
> under /corpus/ directory) does not suddenly result in
> additional exceptions and that it is more easy to test potential bug fixes
> (crash inducing files under /crashes/ directory).
>
> As a downside, this uses two quite specific tools (afl, python-afl) that
> have further dependencies (Cython) inside them, I doubt the viability of
> integrating this type of testing as part of normal Python verification
> process. As a difference to libFuzzer based fuzzing that is already
> integrated in Python[5], this instruments the actual (and only the) Python
> code and not the actions that the interpreter does in the background. So
> this should result in better fuzzer coverage for Python code that is used
> with the downside that when C functions are called, they are complete black
> boxes to the fuzzer.
>
> I have mainly run these fuzzer instances at most for several hours per
> module with 4 instances and stopped running no-issue modules after there
> have been no new coverage discovered after more than 10 minutes. Also I
> have not really created high quality initial input files, so I wouldn't be
> surprised if there are more issues lurking around that could be found with
> throwing more CPU and higher quality fuzzers at the problem.
>
> [1]: https://en.wikipedia.org/wiki/Fuzzing
> [2]: https://github.com/jwilk/python-afl
> [3]: https://docs.python.org/3/library/netrc.html
> [4]: https://bugs.python.org/issue34088
> [5]: https://github.com/python/cpython/tree/3.7/Modules/_xxtestfuzz
>
> --
> Jussi Judin
> https://jjudin.iki.fi/
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/damian.peter.shaw%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

[Damian Shaw]

Apologies I'm not a core dev so this might be the wrong place to ask but I
have 2 small clarifying questions about the PEP.

Firstly, if I have a library which supports multiple versions of Python and
I need to catch all standard exceptions, what is considered the best
practise after this PEP is introduced?

Currently I might have code like this right now:
try:
... # Code
except Exception as e:
... # Logic to handle exception

Reading through the PEP, particularly on backwards compatibility it looks
like I would now need to do this:
try:
ExceptionGroup
except NameError:
standard_exceptions = Exception
else:
standard_exceptions = (Exception, ExceptionGroup)

try:
... # Code
except standard_exceptions as e:
... # Logic to handle exception

Secondly, once I only support versions of Python that support Exception
groups, what replaces the best practise for generally catching standard
exceptions? Is it this:
try:
... # Code
except *Exception as e:
... # Logic to handle exception

Thanks,
Damian (He/Him)

On Mon, Feb 22, 2021 at 7:48 PM Irit Katriel via Python-Dev <
python-dev@python.org> wrote:

>
> Hi all,
>
> We would like to request feedback on PEP 654 -- Exception Groups and
> except*.
>
> https://www.python.org/dev/peps/pep-0654/
>
> It proposes language extensions that allow programs to raise and handle
> multiple unrelated
> exceptions simultaneously, motivated by the needs of asyncio and other
> concurrency libraries,
> but with other use cases as well.
>
> * A new standard exception type,  ExceptionGroup, to represent multiple
> exceptions with
>   shared traceback.
> * Updates to the traceback printing code to display (possibly nested)
> ExceptionGroups.
> * A new syntax except* for handling ExceptionGroups.
>
> A reference implementation (unreviewed) can be found at:
> https://github.com/iritkatriel/cpython/pull/10
>
> Thank you for your help
>
> Kind regards
> Irit, Yury & Guido
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/L5Q27DVKOKZCDNCAWRIQVOZ5DZCZHLRM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IZFZOASIEUABFOJPUHXPZ3HRYKENE6VT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

[Damian Shaw]

Hi Irit,

Catching exceptions like this is an extremely common pattern in the real
world, e.g. this pattern has over 5 million GitHub matches:
https://github.com/search?l=&q=%22except+Exception%22+language%3APython&type=code

How common it is aside there are also many valid use cases for this
pattern, e.g. logging a final unhandled exception of a script, catching
exceptions inside an orchestration framework, exploring code where the list
of exceptions is unknown, etc.

A description from the PEP on how to handle this kind of pattern,
especially for code that must support multiple versions of Python, would be
extremely helpful.

Damian

On Tue, Feb 23, 2021 at 2:35 PM Irit Katriel 
wrote:

> On Tue, Feb 23, 2021 at 3:49 PM Damian Shaw 
> wrote:
>
>>
>> Firstly, if I have a library which supports multiple versions of Python
>> and I need to catch all standard exceptions, what is considered the best
>> practise after this PEP is introduced?
>>
>> Currently I might have code like this right now:
>> try:
>> ... # Code
>> except Exception as e:
>> ... # Logic to handle exception
>>
>>
> Hi Damian,
>
> Catching all exceptions in this way is not a common pattern. Typically you
> have an idea what kind of exceptions you expect to get from an operation,
> and which of those exceptions you are interested in handling. Most of your
> try/except blocks will be targeted, like handling KeyError from d[k], an
> operation that will never raise an ExceptionGroup.
>
> ExceptionGroups will only be raised by specific APIs which will advertise
> themselves as such. We don't expect that there will be a need for blanket
> migrations of "except T" to "except (T, ExceptionGroup)" to "except *T".
>
> Irit
>
>
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XAPO475TVFICD77M3VRLC7OKA5O7WTOT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 -- Exception Groups and except* : request for feedback for SC submission

[Damian Shaw]

Hi Irit,

A common example I see from the GitHub search is a catch all exception on
some third party API. If the user wants to catch all exceptions from
calling a third party API why would they want to let an ExceptionGroup go
unhandled in the future? Once ExceptionGroups are introduced then libraries
will surely implement them where appropriate and users would want to
continue having their board exception handling working, would they not?

I don't actually know if the code I wrote in the first email is correct
though, would it catch all standard Exception / ExceptionGroups across
Python 3.x if this PEP was implemented in the future? And would it be best
practice for this pattern? If so it seems it would require many users to
rewrite their code as it adds boilerplate and nuance for a pattern that is
a relatively simple and very commonly used right now.

I guess summarized my commentary is please don't dismiss this pattern out
of hand, which this PEP currently disrupts, it is used widely in Python
right now and has many valid use cases. I don't have anything further to
add so I won't continue this discussion and I assume anything further you
have to add to be considered and more informed than my own opinion.

Thanks,
Damian



On Tue, Feb 23, 2021 at 4:41 PM Irit Katriel 
wrote:

>
> Hi Damian,
>
> While I agree that there are a handful of use cases for catching all
> Exceptions, if you look at some of the 5 million hits you found in github,
> the vast majority are not such cases. They are mostly try/except blocks
> that wrap a particular operation (a dict access, a raw_input call, etc).
> You don't want to automatically translate "except Exception" to something
> that involves ExceptionGroups, because most of the time it is not needed --
> the code in the body can't raise ExceptionGroups.
>
> Your first email on this thread showed that you were able to quite
> quickly, just from what's already in the PEP, think of how to do what you
> need for your valid "except Exception" use case. My feeling is that people
> who need it will be able to do it quite easily. I'm more worried about
> people assuming that they need it when they don't.
>
> Irit
>
>
> On Tue, Feb 23, 2021 at 8:00 PM Damian Shaw 
> wrote:
>
>> Hi Irit,
>>
>> Catching exceptions like this is an extremely common pattern in the real
>> world, e.g. this pattern has over 5 million GitHub matches:
>> https://github.com/search?l=&q=%22except+Exception%22+language%3APython&type=code
>>
>> How common it is aside there are also many valid use cases for this
>> pattern, e.g. logging a final unhandled exception of a script, catching
>> exceptions inside an orchestration framework, exploring code where the list
>> of exceptions is unknown, etc.
>>
>> A description from the PEP on how to handle this kind of pattern,
>> especially for code that must support multiple versions of Python, would be
>> extremely helpful.
>>
>> Damian
>>
>>>
>>>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4MSMXURT72BQCISBQVVBW3TZWXF5QC7P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Steering Council update for February

[Damian Shaw]

> Does 'master' confuse people?

There's a general movement to replace language from common programming
practises that derive from, or are associated with, the dehumanization of
people. Such as master and slave, as well as whitelist and blacklist.

Github has decided on "main" being a suitable replacement:
https://github.com/github/renaming . I have no inside information on the
Python steering council, but I assume they are following Githubs guidance.

Hope this info helps,
(He / Him)
Damian

On Tue, Mar 9, 2021 at 7:31 PM Terry Reedy  wrote:

> On 3/9/2021 3:27 PM, Pablo Galindo Salgado wrote:
>
> > The Steering Council just published the community update for February:
>
> Thank you for posting this.
>
> > The Steering Council discussed renaming the master branch to main
> > and the consensus was that we should do that.
>
> I did not that this was an issue.  This will break 4 of my .bat files
> (update repository and workspaces, build python.exes, run sphinx on
> docs,  run coverage.) and, I presume, many other workflow scripts and
> bots.  What is the upside?  ('next (version)' might be more accurately
> descriptive, but 'main' works as the 'main branch we are working on'.)
> Does 'master' confuse people?
>
> --
> Terry Jan Reedy
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EMXFMHSFF64VCDFPWK7PXHC27CI2J5SM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RE3V6Y7CPHOL6LGPPYSVS3XQFTIQRZ3J/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Steering Council update for February

[Damian Shaw]

It is NOT a general convention. It is a push by Microsoft (owners of
GitHub). Outside of GitHub, the git command still uses "master" as the
default name.


> I agree, not yet. But I think the writing is on the wall that this will
be the new convention.

FYI for accuracy the git installer includes an option to change the default
branch name with a notice that there will be an upcoming change to have the
default name be something more inclusive:
https://sfconservancy.org/news/2020/jun/23/gitbranchname/ . This notice
appears to predate Github's announcement of the change of name of the
default branch: https://github.com/github/renaming

Damian
(he/him)


On Wed, Mar 10, 2021 at 12:56 PM Paul Bryan  wrote:

> On Thu, 2021-03-11 at 02:20 +1100, Chris Angelico wrote:
>
> It is NOT a general convention. It is a push by Microsoft (owners of
> GitHub). Outside of GitHub, the git command still uses "master" as the
> default name.
>
>
> I agree, not yet. But I think the writing is on the wall that this will be
> the new convention.
>
> This is a *political* move made for *political* reasons, and has
> consequences downstream. Why is it so important to cause actual real
> problems for no reason other than to feel good about one insignificant
> piece of language - and, as Steve pointed out, not even the most
> significant one?
>
>
> I don't see this so much as "feel good" as much as "fit in". I do not
> appreciate the cost of fitting-in though in this case.
>
> To elaborate, there are words we would clearly not use when naming
> branches, modules, classes, etc. I don't think this list is fixed; I think
> it changes with the times, and if we value our community, then our
> community's (changing) standards should figure into our choices.
>
> Paul
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/LYMQXOKK6Y4KOBQPO6QUR5SIQY53CGUW/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XY3ESG5QIJLSPKN5A5Q5ZTTIPVONIMKE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Boundaries between numbers and identifiers

[Damian Shaw]

This isn't a "professional" or probably even "valid" use case for Python
but one area this behavior is heavily used is code golf. For those not
familiar with code golf is a type of puzzle where the objective is to
complete a set of requirements in the least number of source code
characters as possible. Out of mainstream languages Python is surprisingly
good code golf.

This is just for fun puzzle solving and not a reason to keep or change
syntax in any particular way, in fact succeeding at code golf may even be
loosely correlated to bad syntax rules as puzzles tend to be completed in
one of the least readable ways a language can be written in. But at least
be aware if this becomes forbidden syntax that's likely the most affected
area of Python usage.

But it also made me think it could affect code minifiers, which is
apparently a real use case in Python:
https://github.com/dflook/python-minifier (Seems this minifier doesn't
actually remove the spaces between numbers and keywords where is could but
fascinating niche of Python I did not know about)

Regards
Damian
(he/him)



On Wed, Apr 14, 2021 at 7:56 AM Victor Stinner  wrote:

> Also, would it be possible to enhance to tokenizer to report a
> SyntaxWarning, rather than a SyntaxError?
>
> Victor
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/CH7SLXKIKX47KVCWEJEMOB35BCIM7Y5U/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JBSUS2VIEQASY45SR4YKYTZ5PLU7HQ5X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 563 and 649: The Great Compromise

[Damian Shaw]

Hi Larry, all, I was thinking also of a compromise but a slightly different
approach:

Store annotations as a subclass of string but with the required frames
attached to evaluate them as though they were in their local context. Then
have a function "get_annotation_values" that knows how to evaluate these
string subclasses with the attached frames.

This would allow those who use runtime annotations to access local scope
like PEP 649, and allow those who use static type checking to relax the
syntax (as long as they don't try and evaluate the syntax at runtime) as
per PEP 563.

E.g this would work:

def f():
class X: pass
def inner() -> X: pass
return innfer

f().__annotations__ # Similar to PEP 563, values of dict look like strings
get_annotation_values(f()) # Gets the same values as you would from
accessing __annotations__ in PEP 649

Please ignore this idea if it doesn't make any sense, I'm working on
limited technical knowledge of CPython.


On Sun, Apr 18, 2021 at 10:09 AM Jelle Zijlstra 
wrote:

>
>
> El sáb, 17 abr 2021 a las 20:45, Larry Hastings ()
> escribió:
>
>>
>> The heart of the debate between PEPs 563 and 649 is the question: what
>> should an annotation be?  Should it be a string or a Python value?  It
>> seems people who are pro-PEP 563 want it to be a string, and people who are
>> pro-PEP 649 want it to be a value.
>>
>> Actually, let me amend that slightly.  Most people who are pro-PEP 563
>> don't actually care that annotations are strings, per se.  What they want
>> are specific runtime behaviors, and they get those behaviors when PEP 563
>> turns their annotations into strings.
>>
>> I have an idea--a rough proposal--on how we can mix together aspects of
>> PEP 563 and PEP 649.  I think it satisfies everyone's use cases for both
>> PEPs.  The behavior it gets us:
>>
>>- annotations can be stored as strings
>>- annotations stored as strings can be examined as strings
>>- annotations can be examined as values
>>
>>
>> The idea:
>>
>> We add a new type of compile-time flag, akin to a "from __future__"
>> import, but not from the future.  Let's not call it "from __present__", for
>> now how about "from __behavior__".
>>
>> In this specific case, we call it "from __behavior__ import
>> str_annotations".  It behaves much like Python 3.9 does when you say "from
>> __future__ import annotations", except: it stores the dictionary with
>> stringized values in a new member on the function/class/module called
>> "__str_annotations__".
>>
>> If an object "o" has "__str_annotations__", set, you can access it and
>> see the stringized values.
>>
>> If you access "o.__annotations__", and the object has
>> "o.__str_annotations__" set but "o.__annotations__" is not set, it builds
>> (and caches) a new dict by iterating over o.__str_annotations__, calling
>> eval() on each value in "o.__str_annotations__".  It gets the globals()
>> dict the same way that PEP 649 does (including, if you compile a class with
>> str_annotations, it sets __globals__ on the class).  It does *not* unset
>> "o.__str_annotations__" unless someone explicitly sets
>> "o.__annotations__".  This is so you can write your code assuming that
>> "o.__str_annotations__" is set, and it doesn't explode if somebody
>> somewhere ever looks at "o.__annotations__".  (This could lead to them
>> getting out of sync, if someone modified "o.__annotations__".  But I
>> suspect practicality beats purity here.)
>>
> How would this work with annotations that access a local scope?
>
> def f():
> class X: pass
> def inner() -> X: pass
> return innfer
>
> f().__annotations__
>
> From your description it sounds like it would fail, just like calling
> typing.get_type_hints() would fail on it today.
>
> If so I don't see this as much better than the current situation: all it
> does is provide a builtin way of calling get_type_hints().
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/SZA3BUYWYVYS45PZT5NS4TOEZH6SLZIP/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CSM3YKL5Y46B3TCXL5LGJJXKFDCVHNNW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

[Damian Shaw]

> In this case PEP 649 doesn't help.

Sorry for the naive question but why doesn't PEP 649 help here? Is there
something fundamental about the dataclass that needs to inspect the type of
C.a to create the dataclass?

- Damian (he/him)

On Tue, Aug 10, 2021 at 1:10 PM Łukasz Langa  wrote:

>
> > On 10 Aug 2021, at 13:05, Eric V. Smith  wrote:
> >
> > If 649 is accepted, there will be few places where stringified
> annotations will be needed. For example, forward references that are
> defined before __annotations__ is examined will not need to be specified as
> strings. From the PEP: "Actually, annotations would become much easier to
> use, as they would now also handle forward references.
>
> In general, yes, agreed. However, a popular special case that you're
> intimately familiar with are class decorators. Since those are executed
> quite eagerly, you'll have to account for failing evaluation of forward
> references. Example:
>
> from dataclasses import dataclass
>
> @dataclass
> class C:
> a: A
>
> class A:
> ...
>
>
> In this case PEP 649 doesn't help. I don't see evaluation failures
> discussed explicitly in the PEP but I assume it would be a NameError. It
> would also be interesting to see if we can somehow make it handle
> annotations that refer to the class they're used in. Example:
>
> @dataclass
> class Node:
> parent: Node
>
> - Ł
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/FVJAUJ4TOYLAFJJS7UUPLFETXIBFBBEE/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6S5VOQQGEL74O7WCBAJECIZTQ5KNCTJD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

[Damian Shaw]

> dataclasses need to check for ClassVar

Interesting, so the use case we are talking about is: 1) You are using
annotations to mean actual types, 2) But you also have to inspect them at
runtime, 3) For some of the types the name might not be defined at runtime
yet

In this example doesn't the current behavior, PEP 649, and PEP 563 (using
get_type_hints) all throw an exception?

Could PEP 649 be modified to say that if a NameError is raised the result
is not cached and therefore you can inspect it later at runtime to get the
real type once it is defined? Wouldn't that then allow users to write code
that allows for all use cases under this scenario?

- Damian (he/him)


On Tue, Aug 10, 2021 at 1:55 PM Thomas Grainger  wrote:

> dataclasses need to check for ClassVar
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/WOTCUBAO62WRRCCQAO6ZJNASSMK36MDW/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/M3XEKV7KKAGG5QYR2XZSC35S7V4OALJS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 654 except* formatting

[Damian Shaw]

I'm confused, if you can't do that then what is Irit asking? I thought that:

> At the moment * is a separate token so both are allowed, but we could
change that (e.g., make except* a token), and in any case we need to settle
on a convention that we use in documentation, etc.

Meant exactly that was the question being asked.

Damian (he/him)

On Mon, Oct 4, 2021 at 1:30 AM Guido van Rossum  wrote:

> On Sun, Oct 3, 2021 at 9:20 PM Jonathan Goble  wrote:
>
>> Therefore my vote is for requiring `except* E` and keeping `except *E` as
>> a SyntaxError.
>>
>
> You can't do that with our current lexer+parser.
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/F2JUI7SWTQE6RJ4YYKQHJ233BERZHYWR/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RC267HMZNQ5GPMQXIC5TQBSYQIXHBG34/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Type annotations, PEP 649 and PEP 563

[Damian Shaw]

Sorry for the naive question but why doesn't "TYPE_CHECKING" work under PEP
649?

I think I've seen others mention this but as the code object isn't executed
until inspected then if you are just using annotations for type hints it
should work fine?

Is the use case wanting to use annotations for type hints and real time
inspection but you also don't want to import the objects at run time?

If that's really such a strong use cause couldn't PEP 649 be modified to
return a repr of the code object when it gets a NameError? Either by
attaching it to the NameError exception or as part of a ForwardRef style
object if that's how PEP 649 ends up getting implemented?

Damian (he/him)

On Thu, Oct 21, 2021, 00:05 Henry Fredrick Schreiner <
henry.fredrick.schrei...@cern.ch> wrote:

> > typing features from future Python versions
>
> I second both of these uses, but especially this (which seems to be
> missing from the original post), it’s been by far the main reason I’ve used
> this mode and I’ve seen this used, and is the main feature to look forward
> to when dropping Python 3.7 support. The new features coming to typing make
> static typing much easier, but no libraries can drop Python 3.7/3.8/3.9
> support; but static typing doesn’t need old versions.
>
> When talking about speed, one of the important things to consider here is
> the difference between the two proposals. PEP 649 was about the same as the
> current performance, but PEP 563 was significantly faster, since it doesn’t
> instantiate or deal with objects at all, which both the current default and
> PEP 563 do. You could even protect imports with TYPE_CHECKING with PEP 563,
> and further reduce the runtime cost of adding types - which could be seen
> as a reason to avoid adding types. To the best of my knowledge, it hasn’t
> been a blocker for packages, but something to include.
>
> Also, one of the original points for static typing is that strings can be
> substituted for objects. “Type” is identical, from a static typing
> perspective, to Type. You can swap one for the other, and for a Python 3.6+
> codebase, using something like “A | B” (with the quotes) is a valid way
> to have static types in 3.6 that pass MyPy (though are not usable at
> runtime, obviously, but that’s often not a requirement). NumPy, for
> example, makes heavy usage of unions and other newer additions in static
> typing.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/L3IKBO5YZNQ2B5Y6VA7KX352VCOCQEBB/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YZYND5RYBZRRCZY7J3HSE5RXP4JTUENC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Having Sorted Containers in stdlib?

[Damian Shaw]

Yeah, a datapoint I didn't see mentioned is searching on public github
repos for import sortedcontainers (which includes from sortedcontainers
import).

Obviously it's just one datapoint but it shows a very small count compared
to other packages mentioned when looking at many of the stats available:

https://github.com/search?q=import+sortedcontainers+language%3APython&type=code
https://github.com/search?q=import+pylint+language%3APython&type=code
https://github.com/search?q=import+black+language%3APython&type=code
https://github.com/search?q=import+mypy+language%3APython&type=code
https://github.com/search?q=import+django+language%3APython&type=code

Damian


On Fri, Nov 12, 2021 at 11:39 AM Marc-Andre Lemburg  wrote:

> On 12.11.2021 17:10, Steven D'Aprano wrote:
> > On Fri, Nov 12, 2021 at 10:07:13AM -0500, Paul Ganssle wrote:
> >
> >> I knew about sortedcontainers and I also don't remember ever seeing a
> >> situation where I needed one or recommended its use.
> >
> > We have a very odd situation where apparently sortedcontainers is one
> > of the most well-known, popular, most heavily downloaded libraries on
> > PyPI. According to here:
> >
> > https://hugovk.github.io/top-pypi-packages/
> >
> > sortedcontainers is the 290th most popular package on PyPI, ahead of
> > such luminaries as pylint, black, selenium, mypy, django and nose.
> >
> > And yet, nobody(?) admits to either using it or knowing what it could be
> > used for. How very curious :-/
>
> Those download stats can be misleading. Packages are often pulled in
> as a dependency of other packages which are popular or used a lot
> in CI/CD setups.
>
> Perhaps there's a reverse dependency graph we could use to find out
> why the package is downloaded this often. I remember having seen
> a project which does this, but have lost the URL.
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Experts (#1, Nov 12 2021)
> >>> Python Projects, Coaching and Support ...https://www.egenix.com/
> >>> Python Product Development ...https://consulting.egenix.com/
> 
>
> ::: We implement business ideas - efficiently in both time and costs :::
>
>eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
> D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>Registered at Amtsgericht Duesseldorf: HRB 46611
>https://www.egenix.com/company/contact/
>  https://www.malemburg.com/
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/FJT4SSWISWWRFB5FGRPXSLDLDZVMUIXF/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KIKXWW2LLRT2YCNE2OOW7ZJ64N6OHOML/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: It's now time to deprecate the stdlib urllib module

[Damian Shaw]

Speaking from anecdotal experience, "urllib.parse" is a very popular and
highly depended on module, I would be shocked if removing it wouldn't be
very disruptive.

In fact a quick search of the replacement modules you mention see that they
all rely it on it, here is an example from each:
* requests:
https://github.com/psf/requests/blob/99b3b492418d0751ca960178d274f89805095e4c/requests/sessions.py#L121
* aiohttp:
https://github.com/aio-libs/aiohttp/blob/7d78fd01dbe983d119141d7f2775aefd42494f99/aiohttp/formdata.py#L129
* httpx:
https://github.com/encode/httpx/blob/b7dc0c3df68279ce89f016a69a41b27a2346d54d/httpx/_content.py#L144

As for "urllib.request" I know that the philosophy of Python being a
"batteries included language" is going away, but having no way to make any
http call without importing Python definitely has a lot of situations where
it makes Python more difficult to use. Could it not always emit a warning
that this library should not be used in a production environment? Much in
the same way that Flask's default web server does.

Damian (he/hm)


On Sun, Feb 6, 2022 at 9:16 AM Victor Stinner  wrote:

> Hi,
>
> I propose to deprecate the urllib module in Python 3.11. It would emit
> a DeprecationWarning which warn users, so users should consider better
> alternatives like urllib3 or httpx: well known modules, better
> maintained, more secure, support HTTP/2 (httpx), etc.
>
> I don't propose to schedule its removal. Let's discuss the removal in
> 1 or 2 years.
>
> --
>
> urllib has many abstraction to support a wide range of protocols with
> "handlers": HTTP, HTTPS, FTP, "local file", proxy, HTTP
> authentication, HTTP Cookie, etc. A simple HTTP request using Basic
> Authentication requires 10-20 lines of code, whereas it should be a
> single line.
>
> Users (me included) don't like urllib API which was too complicated
> for common tasks.
>
> --
>
> Unhappy users created multiple better alternatives to the stdlib urllib
> module.
>
> In 2008, the "urllib3" module was created to provide an API designed
> to be as simple as possible for the most common HTTP and HTTPS
> requests. Example:
>
>req = http.request('GET', 'http://httpbin.org/robots.txt').
>
> In 2011, the "requests" module based on urllib3 was created.
>
> In 2013, the "aiohttp" module based on asyncio was created.
>
> In 2015, new "httpx" module was created:
>
> req = httpx.get('https://www.example.org/')
>
> Not only httpx has a regular "synchronous" API (blocking function
> calls), but it also has an asynchronous API!
>
> Sadly, while HTTP/3 is being developed, it seems like in this list,
> httpx is the only HTTP client library supporting HTTP/2 currently :-(
>
> For HTTP/2, I also found the "httplib2" module.
>
> For HTTP/3, I found the "http3" and "aioquic" modules.
>
> --
>
> Let's come back to urllib:
>
> * It's API is too complicated
> * It doesn't support HTTP/2 nor HTTP/3
> * It's barely maintained: there are 121 open issues including 3 security
> issues!
>
> The 3 open security issues:
>
> * bpo-33661 open 2018;
> * bpo-36338 open in 2019;
> * bpo-45795 open in 2021.
>
> Usually, it's bad when you refer to an open security issue by its
> creation year :-(
>
> The urllib module has long history of security vulnerabilities. List
> of *fixed* vulnerabilities:
>
> * 2011 (bpo-11662):
> https://python-security.readthedocs.io/vuln/urllib-redirect.html
> * 2017 (bpo-30119):
>
> https://python-security.readthedocs.io/vuln/urllib-ftp-stream-injection.html
> * 2017 (bpo-30500):
> https://python-security.readthedocs.io/vuln/urllib-connects-wrong-host.html
> * 2019 (bpo-35907):
> https://python-security.readthedocs.io/vuln/urllib-local-file-scheme.html
> * 2019 (bpo-38826):
> https://python-security.readthedocs.io/vuln/urllib-basic-auth-regex.html
> * 2021 (bpo-42967):
>
> https://python-security.readthedocs.io/vuln/urllib-query-string-semicolon-separator.html
> * 2021 (bpo-43075):
> https://python-security.readthedocs.io/vuln/urllib-basic-auth-regex2.html
> * 2021 (bpo-44022):
> https://python-security.readthedocs.io/vuln/urllib-100-continue-loop.html
>
> urllib is a package made of 4 parts:
>
> * urllib.request for opening and reading URLs
> * urllib.error containing the exceptions raised by urllib.request
> * urllib.parse for parsing URLs
> * urllib.robotparser for parsing robots.txt files
>
> I propose to deprecate all of them. Maybe the deprecation can be
> different for each sub-module?
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/EZ6O7MOPZ4GA75MKTDO7LAELKXUHK2QS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Py

[Python-Dev] Re: It's now time to deprecate the stdlib urllib module

[Damian Shaw]

Pip vendors requests for network calls:
https://github.com/pypa/pip/tree/main/src/pip/_vendor/requests

But still does depend on functions from urllib.parse and urllib.request in
many places:
https://github.com/pypa/pip/blob/main/src/pip/_internal/utils/urls.py

Damian (he/him)

On Sun, Feb 6, 2022 at 9:36 AM Dong-hee Na  wrote:

> I am not an expert about pip,
> but it will be not a problem about installing the pip module once CPython
> removes urllib module from stdlib?
>
> Warm regards,
> Dong-hee
>
> 2022년 2월 6일 (일) 오후 11:13, Victor Stinner 님이 작성:
>
>> Hi,
>>
>> I propose to deprecate the urllib module in Python 3.11. It would emit
>> a DeprecationWarning which warn users, so users should consider better
>> alternatives like urllib3 or httpx: well known modules, better
>> maintained, more secure, support HTTP/2 (httpx), etc.
>>
>> I don't propose to schedule its removal. Let's discuss the removal in
>> 1 or 2 years.
>>
>> --
>>
>> urllib has many abstraction to support a wide range of protocols with
>> "handlers": HTTP, HTTPS, FTP, "local file", proxy, HTTP
>> authentication, HTTP Cookie, etc. A simple HTTP request using Basic
>> Authentication requires 10-20 lines of code, whereas it should be a
>> single line.
>>
>> Users (me included) don't like urllib API which was too complicated
>> for common tasks.
>>
>> --
>>
>> Unhappy users created multiple better alternatives to the stdlib urllib
>> module.
>>
>> In 2008, the "urllib3" module was created to provide an API designed
>> to be as simple as possible for the most common HTTP and HTTPS
>> requests. Example:
>>
>>req = http.request('GET', 'http://httpbin.org/robots.txt').
>>
>> In 2011, the "requests" module based on urllib3 was created.
>>
>> In 2013, the "aiohttp" module based on asyncio was created.
>>
>> In 2015, new "httpx" module was created:
>>
>> req = httpx.get('https://www.example.org/')
>>
>> Not only httpx has a regular "synchronous" API (blocking function
>> calls), but it also has an asynchronous API!
>>
>> Sadly, while HTTP/3 is being developed, it seems like in this list,
>> httpx is the only HTTP client library supporting HTTP/2 currently :-(
>>
>> For HTTP/2, I also found the "httplib2" module.
>>
>> For HTTP/3, I found the "http3" and "aioquic" modules.
>>
>> --
>>
>> Let's come back to urllib:
>>
>> * It's API is too complicated
>> * It doesn't support HTTP/2 nor HTTP/3
>> * It's barely maintained: there are 121 open issues including 3 security
>> issues!
>>
>> The 3 open security issues:
>>
>> * bpo-33661 open 2018;
>> * bpo-36338 open in 2019;
>> * bpo-45795 open in 2021.
>>
>> Usually, it's bad when you refer to an open security issue by its
>> creation year :-(
>>
>> The urllib module has long history of security vulnerabilities. List
>> of *fixed* vulnerabilities:
>>
>> * 2011 (bpo-11662):
>> https://python-security.readthedocs.io/vuln/urllib-redirect.html
>> * 2017 (bpo-30119):
>>
>> https://python-security.readthedocs.io/vuln/urllib-ftp-stream-injection.html
>> * 2017 (bpo-30500):
>>
>> https://python-security.readthedocs.io/vuln/urllib-connects-wrong-host.html
>> * 2019 (bpo-35907):
>> https://python-security.readthedocs.io/vuln/urllib-local-file-scheme.html
>> * 2019 (bpo-38826):
>> https://python-security.readthedocs.io/vuln/urllib-basic-auth-regex.html
>> * 2021 (bpo-42967):
>>
>> https://python-security.readthedocs.io/vuln/urllib-query-string-semicolon-separator.html
>> * 2021 (bpo-43075):
>> https://python-security.readthedocs.io/vuln/urllib-basic-auth-regex2.html
>> * 2021 (bpo-44022):
>> https://python-security.readthedocs.io/vuln/urllib-100-continue-loop.html
>>
>> urllib is a package made of 4 parts:
>>
>> * urllib.request for opening and reading URLs
>> * urllib.error containing the exceptions raised by urllib.request
>> * urllib.parse for parsing URLs
>> * urllib.robotparser for parsing robots.txt files
>>
>> I propose to deprecate all of them. Maybe the deprecation can be
>> different for each sub-module?
>>
>> Victor
>> --
>> Night gathers, and now my watch begins. It shall not end until my death.
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/EZ6O7MOPZ4GA75MKTDO7LAELKXUHK2QS/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/E6GN2THYCNQ2Q3CGMSH7GRCDFOOFDDCQ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@pytho

[Python-Dev] Re: It's now time to deprecate the stdlib urllib module

[Damian Shaw]

That was just one example, here are others in the pip code base that
urllib.request is used for more than the pathname functions, they are all
vendored or tests but would still be disruptive to remove:

https://github.com/pypa/pip/blob/main/tests/lib/local_repos.py
https://github.com/pypa/pip/blob/main/src/pip/_vendor/webencodings/mklabels.py
https://github.com/pypa/pip/blob/main/src/pip/_vendor/requests/compat.py
https://github.com/pypa/pip/blob/main/src/pip/_vendor/distlib/compat.py

In particular the vendored library, and replacement you suggest, "requests"
is very dependent on the proxy functions such as "getproxies" that are
currently in urllib.requests. More than once I've had to go down the rabbit
hole of seeing where those functions get that info for each platform.

Damian (he/him)


On Sun, Feb 6, 2022 at 11:10 AM Victor Stinner  wrote:

> On Sun, Feb 6, 2022 at 3:48 PM Damian Shaw 
> wrote:
> >
> > Pip vendors requests for network calls:
> https://github.com/pypa/pip/tree/main/src/pip/_vendor/requests
> >
> > But still does depend on functions from urllib.parse and urllib.request
> in many places:
> https://github.com/pypa/pip/blob/main/src/pip/_internal/utils/urls.py
>
> Aha, it doesn't use urllib.request to open a HTTP connection, it only
> uses pathname2url() and url2pathname() functions of urllib.request.
> Maybe we can keep these functions. I'm not sure why they don't belong
> to urllib.parse.
>
> If urllib.parse is widely used, maybe we can keep this module.
>
> Victor
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ACA7AU4W6XB35PA6O4IYBPQSQD3HFLFS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: New PEP website is horrible to read on mobile device

[Damian Shaw]

I tried on Pixel 6 Pro with Android 12 with both the latest stable Chrome
and Firefox and didn't see the reported issue.

One thing I would mention though is people who can reproduce it check if
you have any extensions enabled or other tools that can block network
traffic. Sometimes privacy based extensions and tools can have false
positives and block resources required to render sites correctly.

Damian (he/him)

On Tue, Mar 15, 2022 at 6:00 PM David Mertz, Ph.D. 
wrote:

> I get pretty much the same thing as the OP on Chrome 99.0.4844.58; Android
> 11; Pixel 2 XL; Build RP1A.201005.004.A1.
>
> However, it gets more readable if I force Desktop site and zoom a bit.
> These facts are pretty common for a lot of websites, and I never gave it
> much thought.  But yes, the mobile version could be better responsive.
>
> On Tue, Mar 15, 2022 at 5:17 PM Terry Reedy  wrote:
>
>> On 3/15/2022 10:06 AM, Nathan Cook wrote:
>> > Please make https://peps.python.org/  more
>> > responsive to various form factors
>> >
>> > See attached screenshot from Chrome version 99.0.4844.58 on my Pixel
>> > 3aXL running Android 12
>>
>> Are you sure that the site looks any different that it did at the older
>> url, www.python.org/peps/ ?  In any case, when I go there with Chrome on
>> my (old) Nexus 6P, pages are originally wrapped to the screen size.  If
>> I zoom in, a horizontal scrollbar is added (and suppressed) and I have
>> to scroll horizontally with my finger.  On the phone, Google search page
>> results act the same.  So do the wikipedia pages I tested.  (On my
>> desktop monitor, they rewrap on both Chrome and Firefox, although the
>> layout is different in the two browswers.)
>>
>>
>> --
>> Terry Jan Reedy
>>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/SVJRYYCZ2YEAJ7OH6TE3TV4TEX4TBUE4/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> --
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/HDYPGCGIAK6UOIAOTNBEI6HAB7DEPQEM/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/H2Y7QUUWZNZAH6QOBAWZIXT7HGCUGH2Z/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [RELEASE] Python 3.10.4 and 3.9.12 are now available out of schedule

[Damian Shaw]

I guess the docs aren't updated yet and the changes are listed as "Python
Next": https://docs.python.org/3.10/whatsnew/changelog.html#changelog ?

Damian(he/him)

On Thu, Mar 24, 2022 at 8:13 AM Łukasz Langa  wrote:

> Did anybody say cursed releases
> ?
> Well, it turns out that 3.10.3 and 3.9.11 both shipped a regression which
> caused those versions not to build on Red Hat Enterprise Linux 6. While
> this 11-year-old version is now out of maintenance support
> , it’s still
> used in production workloads. Some of those rely on Python 3.9 and/or 3.10.
> In particular, our own manylinux2010
> 
> image used to build widely compatible Linux wheels is based on CentOS 6.
> (Don’t worry, we do have newer manylinux* variants, see PEP 599
>  and PEP 600
>  for details.)
>
> Due to the out-of-schedule release, the respective versions released today
> contain a very limited set of changes. Python 3.9.12 only contains 12 other
> bug fixes on top of 3.9.11. Python 3.10.4 only contains 10 other bug fixes
> on top of 3.10.3.
>
> Get 3.10.4 here: Python Release 3.10.4 | Python.org
> 
> Get 3.9.12 here: Python Release 3.9.12 | Python.org
> 
>
> Hopefully, the third time’s a charm and we’ll return no sooner than May
> with the regularly scheduled bug fix releases of 3.9 and 3.10.
>
> We
> hope you enjoy the new releases
>
> Your friendly release team,
> Łukasz Langa @ambv 
> Pablo Galindo Salgado @pablogsal 
> Ned Deily @nad 
> Steve Dower @steve.dower 
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/SGB4B572RDPZ7SIJ5A6NZAI3Z3GKNIXA/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AHLDJISO2YFGXIZD3ON5F6QDPMVE3DD5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Damian Shaw]

> If urllib is removed, I would very much like to preserve at least the
functionality of urlparse /somewhere/.

Given every alternative library to urllib relies on urllib.parse (and some
rely on urllib.request), as well as popular libraries like pip, in this
hypothetical it would definitely need to be maintained somewhere.

Damian (he/him)

On Mon, Mar 28, 2022 at 1:38 PM lincoln auster [they/them] <
lincolnaus...@gmail.com> wrote:

>
> "Coyot Linden (Glenn Glazer)"  writes:
> > P.S. There are uses for urllib outside of standard web programming. I
> > recently needed to handle strings in query parameter format
> > (`?...&...&...`) and found urllib parse() the easiest way of doing
> > that even though I wasn't taking them in as a webservice. Deleting the
> > library would break this code. :(
>
> +1. If urllib is removed, I would very much like to preserve at least
> the functionality of urlparse /somewhere/.
>
> --
> Lincoln Auster
> they/them
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/C3JZDCWLCMED32Q4MFHMITI2QQGK2ME5/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4FL6DXXIHEC44MQUXTRU4T5E3MIDM4T4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Are "Batteries Included" still a Good Thing? [was: It's now time to deprecate the stdlib urllib module]

[Damian Shaw]

I'm probably overly stressing a well understood point here that
urllib.parse is incredibly widely used and critical to many foundational
libraries in Python. But I just came across this today that:

conda is migrating from urllib3 for parsing to urllib.parse:
https://github.com/conda/conda/pull/11373

And not to mention urllib3 itself somewhat depends on urllib.parse:
https://github.com/urllib3/urllib3/search?q=urllib.parse

Damian (he/him)

On Tue, Mar 29, 2022 at 6:59 PM Alex Waygood  wrote:

> There's also the "Experts index" in the devguide:
> https://devguide.python.org/experts/#experts
>
> Best,
> Alex
>
>  Original message 
> From: Skip Montanaro 
> Date: 29/03/2022 22:36 (GMT+00:00)
> To: "Eric V. Smith" 
> Cc: Python Dev 
> Subject: [Python-Dev] Re: Are "Batteries Included" still a Good Thing?
> [was: It's now time to deprecate the stdlib urllib module]
>
> There's the CODEOWNERS file:
>> https://github.com/python/cpython/blob/main/.github/CODEOWNERS
>
>
> Thanks. Never would have thought there was such a thing. I was looking for
> files with "maintain" in them. Skimming it, it would seem that most of the
> stuff in Lib or Modules isn't really associated with a particular person or
> small group.
>
> Skip
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/CN7SXNP2D4ML4U2OBIUC4DGQOKEYNSKE/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QYCAECVPQEZ5JX7LTLDA3IWB6CUUUOZ5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal to deprecate mailcap

[Damian Shaw]

> I searched grep.app and found no significant usage.

Maybe someone wants to inform mitmproxy?

It's a very popular tool and it comes up using that tool when searching for
"import mailcap" using grep.app:
https://grep.app/search?q=import%20mailcap
https://github.com/mitmproxy/mitmproxy/blob/main/mitmproxy/tools/console/master.py#L2

On Thu, Apr 14, 2022 at 3:06 PM Jelle Zijlstra 
wrote:

>
>
> El jue, 14 abr 2022 a las 11:47, Brett Cannon ()
> escribió:
>
>> A CVE has been opened against mailcap (see
>> https://github.com/python/cpython/issues/68966 for details). I'm not
>> aware of anyone trying to maintain the module and Victor did a search
>> online and didn't find any use of the module in the top 5000 projects on
>> PyPI (see the issue). The module is also under 300 lines of Python code
>> that only  (https://github.com/python/cpython/blob/main/Lib/mailcap.py),
>> so vendoring wouldn't be burdensome.
>>
>> As such, I'm proposing we deprecate mailcap in 3.11 and remove it in
>> 3.13. Any explicit objections?
>>
>
> Agree on deprecating. I searched grep.app and found no significant usage.
>
> Do you know why this module wasn't included in PEP 594? Should we do
> another audit of old modules to deprecate them before they cause problems?
>
>
>> ___
>> Python-Dev mailing list -- python-dev@python.org
>> To unsubscribe send an email to python-dev-le...@python.org
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-dev@python.org/message/EB2BS4DBWSTBIOPQL5QTBSIOBORWSCMJ/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ON7R7LD7OVPOF5XXPXJWE7AKCRAPQ7IN/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MRW7DDFMD4G4D46AIRO3JICFKK5HSN6X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal to deprecate mailcap

[Damian Shaw]

I didn't spot anyone else report this to mitmproxy so I raised an issue to
make them aware: https://github.com/mitmproxy/mitmproxy/issues/5297

On Tue, Apr 26, 2022 at 3:30 AM Victor Stinner  wrote:

> On Tue, Apr 26, 2022 at 5:47 AM Brett Cannon  wrote:
> > After talking about this in the SC today, we agreed to deprecate mailcap
> under the auspices of PEP 594:
> https://github.com/python/peps/commit/701999a91dc5f976c00d5bde1510226ebd9c7822
> .
>
> Good. I proposed https://github.com/python/cpython/pull/91951 to
> implement the deprecation in Python 3.11.
>
> Fixing or documenting the shell injection vulnerability CVE-2015-20107
> is still being discussed at:
> https://github.com/python/cpython/issues/68966
>
> Victor
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/Y4IAWAWYTNKSIAVTXJGV2ZMQTFV5WYTT/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/INWQMSKTJCJGSVIZUDKV5V6FIJVYO3VA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Raw strings ending with a backslash

[Damian Shaw]

That PR seems to make \' and \" not special in general right?

I think this is a more limited proposal, to only change the behavior when \
is at the end of a string, so the only behavior difference would never
receiving the error "SyntaxError: EOL while scanning string literal"

In which case there should be no backwards compatibility issue.

Damian

On Sat, May 28, 2022 at 12:20 PM Serhiy Storchaka 
wrote:

> 28.05.22 12:22, Steven D'Aprano пише:
> > Now that we have a new parser for CPython, can we fix the old gotcha
> > that raw strings cannot end in a backslash?
> >
> > Its an FAQ and has come up again on the bug tracker.
> >
> > https://docs.python.org/3/faq/design.html#id26
> >
> > https://github.com/python/cpython/issues/93314
>
> I do not think that we can allow this, and it is not related to parser.
>
> Few years ago I experimented with such change:
> https://github.com/python/cpython/pull/15217
>
> You can see that it breaks even some stdlib code, and it will definitely
> break many third-party packages and examples. Technically we can do
> this, but the benefit is small in comparison with the cost.
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/WWWRFQK4AG52GD3L6WT6QLRGTY2VQRQ2/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/O3STZ54BRQ3T352PIMNEZWCRBGP6FE2O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Raw strings ending with a backslash

[Damian Shaw]

My understanding was that was part of the question being asked, is it
possible to know what with the new PEG parser?

On Sat, May 28, 2022 at 1:25 PM Serhiy Storchaka 
wrote:

> 28.05.22 14:57, Damian Shaw пише:
> > That PR seems to make \' and \" not special in general right?
> >
> > I think this is a more limited proposal, to only change the behavior
> > when \ is at the end of a string, so the only behavior difference would
> > never receiving the error "SyntaxError: EOL while scanning string
> literal"
> >
> > In which case there should be no backwards compatibility issue.
>
> How do you know that it is at the end of a string?
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/BFGT3H57CFTEOKWA3NXBSPUDE4JF4C2H/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4QGGU7GJ3K4TGZFF2X4DPDZVDEMVJECD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: New contribution to Python built-in libraries.

[Damian Shaw]

I don't understand the purpose of this function, for example if I run:
strangelt("Hello World", 3, "Bye")

I get the output:
HeByeByeo WorByed

What is the common use case for this?

And would it not be simpler and faster to implement it with this 1 liner:
def strangelt(s,j,m):
return s.replace(s[j], m)


On Sun, Aug 28, 2022 at 10:29 PM Kiruthik.M.R 11-B 
wrote:

> Greetings, I'm Kiruthik from India. I've been studying coding for the past
> two years. I know that strings are immutable in Python. But, most of the
> people can't change the elements in the string as default. So, I came up
> with a solution by creating a function called *strangelt. *This function
> has three parameters in it and works faster. It has the ability to get a
> string from the user and make changes to the element of the string using
> index values. I found that it would be a pretty good solution for people's
> problems. I hereby kindly request you to include this function in Python
> built-in libraries from the release of the next version.
>
> Expansion of strangelt: *str*ing+ch*ang*e+*el*ement
> *Syntax:*
> *string_variable=strangelt(string,index,new value to be replaced)*
>
> Thank You,
> Kiruthik.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/DTJRUMDJLL5WLJZKMRKEE7OGAFZIIXFW/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DX6IPXSLG6TM7YWQGEZWLWTUTR2LPXMB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Small lament...

[Damian Shaw]

In case people missed it, this Pip PR:
https://github.com/pypa/pip/pull/11914

On Mon, Apr 3, 2023 at 8:41 PM Barry Warsaw  wrote:

> I heard it on reasonably believable authority that the FLUFL took the year
> off.  Lamentable.
>
> -Barry
>
> > On Apr 1, 2023, at 11:19, Skip Montanaro 
> wrote:
> >
> > Just wanted to throw this out there... I lament the loss of waking up on
> April 1st to see a creative April Fool's Day joke on one or both of these
> lists, often from our FLUFL... Maybe such frivolity still happens, just not
> in the Python ecosystem? I know you can still import "this" or
> "antigravity", but those are now old (both introduced before 2010). When
> was the last time a clever easter egg was introduced or an April Fool's Day
> joke played?
> >
> > ¯\_(ツ)_/¯
> >
> > Skip
> >
> > ___
> > Python-Dev mailing list -- python-dev@python.org
> > To unsubscribe send an email to python-dev-le...@python.org
> > https://mail.python.org/mailman3/lists/python-dev.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/Q62W2Q6R6XMX57WK2CUGEENHMT3C3REF/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/NGP4A6GN76T4NGBILAGYTZO4OQORBZM4/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RPZWBDDGYWYKYCTFFT323N6NMNBGIFBC/
Code of Conduct: http://python.org/psf/codeofconduct/