[issue41314] PEP 563 and annotations __future__ mandatory version

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The PEP was accepted apparently by Guido sometime after the second posting 
21-Nov-2017.  Guido added the annotations entry to __future__ on 1/26/2018 with 
the 'mandatory' version changed to 4.0, which means 'some indefinite undecided 
future version'.  I consider '4.0' to the intentional and definitive until 
Guido or Lucasz say otherwise.

When PEP implementations differ from the PEP, we do not usually go back and 
revise the PEP.  The implementation rules unless clearly buggy.

--
nosy: +gvanrossum, lukasz.langa, terry.reedy
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
title: __future__ doc and PEP 563 conflict -> PEP 563 and annotations 
__future__ mandatory version
versions: +Python 3.8, Python 3.9 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41282] Deprecate and remove distutils

[Terry J. Reedy]

Change by Terry J. Reedy :


--
versions:  -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41330] Inefficient error-handle for CJK encodings

[Ma Lin]

New submission from Ma Lin :

CJK encode/decode functions only have three error-handler fast-paths:
replace
ignore
strict  
See the code: [1][2]

If use other built-in error-handlers, need to get the error-handler object, and 
call it with an Unicode Exception argument. See the code: [3]

But the error-handler object is not cached, it needs to be looked up from a 
dict every time, which is very inefficient.


Another possible optimization is to write fast-path for common error-handlers, 
Python has these built-in error-handlers:

strict
replace
ignore
backslashreplace
xmlcharrefreplace
namereplace
surrogateescape
surrogatepass (only for utf-8/utf-16/utf-32 family)

For example, maybe `xmlcharrefreplace` is heavily used in Web application, it 
can be implemented as a fast-path, so that no need to call the error-handler 
object every time.
Just like the `xmlcharrefreplace` fast-path in `PyUnicode_EncodeCharmap` [4].

[1] encode function:
https://github.com/python/cpython/blob/v3.9.0b4/Modules/cjkcodecs/multibytecodec.c#L192

[2] decode function:
https://github.com/python/cpython/blob/v3.9.0b4/Modules/cjkcodecs/multibytecodec.c#L347

[3] `call_error_callback` function:
https://github.com/python/cpython/blob/v3.9.0b4/Modules/cjkcodecs/multibytecodec.c#L82

[4] `xmlcharrefreplace` fast-path in `PyUnicode_EncodeCharmap`:
https://github.com/python/cpython/blob/v3.9.0b4/Objects/unicodeobject.c#L8662

--
components: Unicode
messages: 373871
nosy: ezio.melotti, malin, vstinner
priority: normal
severity: normal
status: open
title: Inefficient error-handle for CJK encodings
type: performance
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41311] Add a function to get a random sample from an iterable (reservoir sampling)

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> This comment suggest that you have missed the general
> motivation for reservoir sampling.

Please don't get personal.  I've devoted a good deal of time thinking about 
your proposal.  Tim is also giving it an honest look. Please devote some time 
to honestly thinking about what we have to say.

FWIW, this is an area of expertise for me.  I too have been fascinated with 
reservoir sampling for several decades, have done a good deal of reading on the 
topic, and routinely performed statistical sampling as part of my job (where it 
needed to be done in a legally defensible manner).


> The idea of reservoir sampling is that you want to sample from
> an iterator, you only get one chance to iterate over it, and 
> you don't know a priori how many items it will yield.

Several thoughts:

* The need for sampling a generator or one-time stream of data is in the 
"almost never" category.  Presumably, that is why you don't find it in numpy or 
Julia.

* The examples you gave involved dicts or sets.  These aren't one-chance 
examples and we do know the length in advance.

* Whether talking about sets, dicts, generators, or arbitrary iterators, 
"sample(list(it), k)" would still work.  Both ways still have to consume the 
entire input before returning.  So really this is just an optimization, one 
that under some circumstances runs a bit faster, but one that forgoes a number 
of desirable characteristics of the existing tool.  

* IMO, sample_iter() is hard to use correctly.  In most cases, the users would 
be worse off than they are now and it would be challenging to communicate 
clearly under what circumstances they would be marginally better off.

At any rate, my recommendation stands.  This should not be part of standard 
library random module API.  Perhaps it could be a recipe or a see-also link.  
We really don't have to do this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29447] Add/check os.PathLike support for the tempfile module's 'dir' arguments

[Christoph Anton Mitterer]

Change by Christoph Anton Mitterer :


--
nosy: +calestyo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41314] __future__ doc and PEP 563 conflict

[wyz23x2]

Change by wyz23x2 :


--
components: +Library (Lib)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41220] add optional make_key argument to lru_cache

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I've left this open for a week to collect comments.  I concur with Felipe that 
this should be the responsibility of the caller.  And I concur with Jim that 
the use cases are likely too rare to warrant shoehorning in the extra 
functionality.

For my part, I'm concerned that this would be bug bait.  The result of the 
function would need to be uniquely associated with a particular output, 
independent of the contents of the mutable arguments.  I don't think it is good 
design for function calls to be disregarding the part of the input that was 
originally used to compute the output — in a way that notion conflict with the 
core idea of the of the lru_cache giving the same outputs for the same inputs.

Likewise, debugging and exception handing would be complicated by having two 
underlying user functions.  Exceptions could arise from three sources: the 
make_key() function, the cached function, and the lru cache internals.

Further, if the make_key() function turns out not to be unique, the ensuring 
bugs would be hard to find.  In the OP's later example, if we get two distinct 
inputs which happen to have the same timestamp, the error would pass without 
warning and would be hard to reproduce.  Similarly, the make_key() function has 
to assume that the mutable portion of the data hasn't mutated, but a key 
feature of mutable data is that it can mutate.  That too would result in a hard 
to find bug.

Also, while the OP has a specific use case in mind, we can't know in advance 
how others will use make_key().  It could be misused to convert mutable data to 
immutable data, possibly resulting in a cached function that is slower than the 
original (caching mean() for example).  Or the make_key() function could had 
side-effects.  The OP's original "my_list[0]" example showed yet another way 
that bugs could arise. It is problematic that that particular bug might not 
have been caught by a test suite that didn't know to call the function twice 
with a common first argument but with differing subsequent arguments.

Another possible bug trap is that it would make it easier to accidentally cache 
mutable function results without getting a warning.  For example:
  @lru_cache(128, make_key = lambda target, size, uniq_id, data: uniq_id)
  def search(target, size, uniq_id, data):
   i = data.index(target)
   return data[i : i+size]# <== bug because this is mutable if data 
is mutable
   
I think the suggestion was a neat idea (and thought provoking), but I think we 
should decline because 1) the use case is uncommon 2) because it makes the API 
harder to understand 3) because it is bug bait and 4) because the 
responsibility should probably be with the caller.

Thank you for the suggestion.  It was inspired.  Don't be discouraged — many of 
my proposals don't get accepted as well.  Please continue to submit suggestions.



P.S. make_key() shouldn't be called a key-function so that we don't make that 
term ambiguous.  Elsewhere key-functions are all about comparison logic rather 
than hashing.  They can be created by functools.cmp_to_key(), they tend to be 
used only once per data element, and the same function tends to be inoperable 
with all the tools that accept key-functions.  So the term make_key() was 
likely the better terminology.  Perhaps get_unique_id() would have been even 
less ambiguous.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41271] Add support for io_uring to cpython

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I think that the answer is maybe, eventually, and if so, an ioring module that 
any event framework can use and a separate asyncio module for its use with 
asyncio.  I say maybe because the lwn article suggests that additions and 
revisions might continue for a decade or more.  A pypi py_iorin wrapper for 
lib_ioring might need revision with each Linux x.y release for a long time.

The opening question could have been raised on the python-ideas or possibly 
pydev lists.  Since this list is for patches to cpython, creating and 
maintaining a py_ioring package on pypi should be discussed elsewhere.  In the 
meanwhile, this issue could be closed as 'later' ("Issue is to be worked on in 
a later release cycle.").

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41315] Add mathematical functions as wrappers to decimal.Decimal methods

[Raymond Hettinger]

Raymond Hettinger  added the comment:

FWIW, I don't really buy into the use case.  In my experience a nest of 
existing functions that are designed for floats will somewhere being using a 
float constant like 0.5, e, pi, or tau.  So, just feeding in a decimal input 
isn't sufficient to get it to work.  Also, for your idea to work, the function 
couldn't explicitly reference math.exp(x); instead, it must just use exp() so 
that you could substitute, "from decimal import exp" for "from math import 
exp". 

Also, this proposal would be a limited utility because most of the functions in 
the math module don't have an existing equivalent in the decimal methods.  The 
inconvenient fact is that the decimal module wasn't designed to be a drop-in 
substitute for floats — it's principal design objectives were somewhat 
different.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41271] Add support for io_uring to cpython

[Giampaolo Rodola']

Change by Giampaolo Rodola' :


--
nosy: +giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41297] Remove doctest import from heapq

[Raymond Hettinger]

Raymond Hettinger  added the comment:

If not, could modulegraph add a flag to drop imports commonly found in main 
sections:  doctest, unittest, argparse, etc.?   

Asking the standard library to change seems like the tail wagging the dog — it 
only paints over the problem since third-party modules, other standard library 
modules, and user code commonly use main sections as well.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41311] Add a function to get a random sample from an iterable (reservoir sampling)

[Oscar Benjamin]

Oscar Benjamin  added the comment:

> At its heart, this a CPython optimization to take advantage of list() being 
> slower than a handful of islice() calls.

This comment suggest that you have missed the general motivation for reservoir 
sampling. Of course the stdlib can not satisfy all use cases so this can be out 
of scope as a feature. It is not the case though that this is a CPython 
optimisation.

The idea of reservoir sampling is that you want to sample from an iterator, you 
only get one chance to iterate over it, and you don't know a priori how many 
items it will yield. The classic example of that situation is reading from a 
text file but in general it maps neatly onto Python's concept of iterators. The 
motivation for generators/iterators in Python is that there are many situations 
where it is better to avoid building a concrete in-memory data structure and it 
can be possible to avoid doing so with appropriately modified algorithms (such 
as this one). 

The core use case for this feature is not sampling from an in-memory data 
structure but rather sampling from an expensive generator or an iterator over a 
file/database. The premise is that it is undesirable or perhaps impossible to 
build a list out of the items of the iterable. In those contexts the 
comparative properties of sample/choices are to some extent irrelevant because 
those APIs can not be used or should be avoided because of their overhead in 
terms of memory or other resources.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41311] Add a function to get a random sample from an iterable (reservoir sampling)

[Raymond Hettinger]

Raymond Hettinger  added the comment:

More thoughts:

* If sample_iter() were added, people would expect a choices_iter() as well.

* Part of the reason that Set support was being dropped from sample() is that 
it was rarely used and that it was surprising that it was a O(n) operation 
instead of O(k).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41311] Add a function to get a random sample from an iterable (reservoir sampling)

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I've put more thought into the proposal and am going to recommend against it.

At its heart, this a CPython optimization to take advantage of list() being 
slower than a handful of islice() calls.  It also gains a speed benefit by 
dropping the antibias logic because random() is faster than _randbelow().  IMO, 
this doesn't warrant an API extension.  I'm not looking forward to years of 
teaching people that there are two separate APIs for sampling without 
replacement and that the second one is almost never what they should use.

A few years ago, GvR rejected adding a pre-sizing argument to dicts even though 
there were some cases where it gave improved performance.  His rationale was 
that it was challenging for a user to know when they were better off and when 
they weren't.  It added a new complication that easily led to suboptimal 
choices. IMO, this new API puts the users in a similar situation.  There are a 
number of cases where a person is worse off, sometimes much worse off.

This new code runs O(n) instead of O(k).  It eats more entropy. It loses the 
the antibias protections.  The API makes it less explicit that the entire input 
iterable is consumed.  It can only be beneficial is the input is not a 
sequence.  When k gets bigger, the repeated calls to islice() become more 
expensive than a single call to list.   And given the math library functions 
involved, I not even sure that this code can guarantee it gives the same 
results across platforms.

Even if the user makes a correct initial decision about which API to use, the 
decision can become invalidated when the population sizes or sample sizes 
change over time.

Lastly, giving users choices between two substantially similar tools typically 
makes them worse off.  It creates a new burden to learn, remember, and 
distinguish the two.  It's really nice that we currently have just one sample() 
and that it behaves well across a broad range of cases — you generally get a 
good result without having to think about it.  Presumably, that was the wisdom 
behind having one-way-to-do-it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41024] doc: Explicitly mention use of 'enum.Enum' as a valid container for 'choices' argument of 'argparse.ArgumentParser.add_argument'

[Raymond Hettinger]

Raymond Hettinger  added the comment:


New changeset 760552ceb8c5f5ca4f1bf13f47543b42b25e0b83 by Miss Islington (bot) 
in branch '3.9':
bpo-41024: doc: Explicitly mention use of 'enum.Enum' as a valid container for 
'… (GH-20964) (GH-21527)
https://github.com/python/cpython/commit/760552ceb8c5f5ca4f1bf13f47543b42b25e0b83


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41024] doc: Explicitly mention use of 'enum.Enum' as a valid container for 'choices' argument of 'argparse.ArgumentParser.add_argument'

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Thanks for the PR.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41311] Add a function to get a random sample from an iterable (reservoir sampling)

[Raymond Hettinger]

Change by Raymond Hettinger :


--
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41024] doc: Explicitly mention use of 'enum.Enum' as a valid container for 'choices' argument of 'argparse.ArgumentParser.add_argument'

[Raymond Hettinger]

Raymond Hettinger  added the comment:

The backport failed for 3.9.  I'll trigger it again.

Don't see a need to go back to older versions.  This is a minor informational 
note, not a bug.

--
versions:  -Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41024] doc: Explicitly mention use of 'enum.Enum' as a valid container for 'choices' argument of 'argparse.ArgumentParser.add_argument'

[miss-islington]

Change by miss-islington :


--
pull_requests: +20663
status: pending -> open
pull_request: https://github.com/python/cpython/pull/21527

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41329] IDLE not saving .py files

[Terry J. Reedy]

Terry J. Reedy  added the comment:

In calling this a duplicate, and suggesting the fix above, I am assuming that 
a) you are using the new 3.8.4 and b) you have a non-ascii character in the .py 
file or files you tested.  Since several years ago, the is the only case of 
save failing that I know of.  If either is not the case, please reopen and 
report exact os, python version, minimal file that does not save, and any other 
possibly relevant details you can think of.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41329] IDLE not saving .py files

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Add 'import io' to the top of /Lib/idlelib/iomenu.py.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> IDLE: add missing import io in iomenu.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41311] Add a function to get a random sample from an iterable (reservoir sampling)

[Tim Peters]

Tim Peters  added the comment:

Julia's randsubseq() doesn't allow to specify the _size_ of the output desired. 
It picks each input element independently with probability p, and the output 
can be of any size from 0 through the input's size (with mean output length 
p*length(A)). Reservoir sampling is simply irrelevant to that, although they 
almost certainly use a form of skip-generation internally.

The quoted docs don't make much sense: for any given p, O(p*N) = O(N). I'm 
guessing they're trying to say that the mean of the number of times the RNG is 
called is p*N.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41297] Remove doctest import from heapq

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Ronald, can modulegraph be made smarter with respect to sections of code 
guarded by __name__ == '__main__'?That Python idiom is old and pervasive.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41319] IDLE 3.8 can not save and run this file

[Terry J. Reedy]

Terry J. Reedy  added the comment:

宋嘉腾 put 'import io' at the top of idlelib/iomenu.py

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41329] IDLE not saving .py files

[Nick.Lupariello]

New submission from Nick.Lupariello :

IDLE will not save .py files. Neither Ctrl + S nor file -> Save file nor file 
-> Save file as function as intended and the .py file is not updated. 
This happens for both 32 bit and 64 bit distributions on multiple computers 
with 4 GB of ram running on a AMD A6-9220e RADEON R4. Tested installation in 
multiple locations. 
IDLE will not write to file anywhere in the file system including documents, 
desktop, and the installation folder itself.

--
assignee: terry.reedy
components: IDLE
messages: 373854
nosy: nicklupe13, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE not saving .py files
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41311] Add a function to get a random sample from an iterable (reservoir sampling)

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Other implementations aren't directly comparable, but I thought I would check 
to see what others were doing:

* Scikit-learn uses reservoir sampling but only when k / n > 0.99.  Also, it 
requires a follow-on step to shuffle the selections.

* numpy does not use reservoir sampling.

* Julia's randsubseq() does not use reservoir sampling.  The docs guarantee 
that, "Complexity is linear in p*length(A), so this function is efficient even 
if p is small and A is large."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41328] Hudson CI is not available anymore

[Dmytro Litvinov]

New submission from Dmytro Litvinov :

In the documentation (https://docs.python.org/3.8/library/unittest.html) there 
is a mention of Hudson(http://hudson-ci.org/) as continuous integration system 
for tests. According to 
wikipedia(https://en.wikipedia.org/wiki/Hudson_(software)), Hudson "Having been 
replaced by Jenkins, Hudson is no longer maintained[9][10] and was announced as 
obsolete in February 2017.[11]" 

My recommendation for that is to change the mention of "Hudson" to "Jenkins".

I am ready to prepare PR for that change.

--
assignee: docs@python
components: Documentation
messages: 373852
nosy: DmytroLitvinov, docs@python
priority: normal
severity: normal
status: open
title: Hudson CI is not available anymore
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41327] Windows Store "stub" Python executables give confusing behaviour

[Tzu-ping Chung]

Tzu-ping Chung  added the comment:

FWIW, I tweeted about this a while ago (January) and IIRC Steve said there’s 
plan to change that. https://twitter.com/uranusjr/status/1212450480917340160

--
nosy: +uranusjr

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41322] unittest: Generator test methods will always be marked as passed

[Zachary Ware]

Zachary Ware  added the comment:

In the same vein as Serhiy's suggestion, I would like to see unittest raise a 
DeprecationWarning when a test method returns anything other than `None`, and 
eventually switch that to an error (probably TypeError or ValueError).

--
nosy: +zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41327] Windows Store "stub" Python executables give confusing behaviour

[Paul Moore]

Paul Moore  added the comment:

I thought that might be the answer. But does anyone know where I can repost 
this as an issue against the Store distribution? I'm happy to report there if I 
can find out how...

Hopefully Steve can point me in the right direction.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41323] Perform "peephole" optimization directly on control-flow graph.

[Raymond Hettinger]

Raymond Hettinger  added the comment:

+1 This looks like a nice improvement.  I've long wanted the optimizations to 
be moved upstream.  

At the time the peephole logic was first written, operating directly on the 
code object was the only option that kept optimizations separate from the core 
compiler logic. But disassembling and reassembling bytecode was always awkward 
— it is nice to no longer have to do so.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41327] Windows Store "stub" Python executables give confusing behaviour

[Brett Cannon]

Brett Cannon  added the comment:

Closing as "third-party" as those stubs are controlled by Microsoft Windows and 
has nothing to do with us as a project beyond that they install the Windows 
Store copy that Steve uploads (although this is all feedback to Steve who has 
connections on the Windows team).

And I will say I believe there are shims on some Linux distros like Ubuntu for 
things like venv, pip, etc. which are not included in-box which people 
typically expect to be there.

--
nosy: +brett.cannon
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41323] Perform "peephole" optimization directly on control-flow graph.

[Brett Cannon]

Change by Brett Cannon :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41327] Windows Store "stub" Python executables give confusing behaviour

[Paul Moore]

Paul Moore  added the comment:

See, for example, https://github.com/pypa/packaging-problems/issues/379 as an 
illustration of the user (and project maintainer!) confusion this causes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41327] Windows Store "stub" Python executables give confusing behaviour

[Paul Moore]

New submission from Paul Moore :

First of all, I do know that this is an issue with the Windows Store 
distribution, rather than the python.org one. But (a) I don't know where to 
report a bug against the Store implementation except here, and (b) it's 
arguably a case of the Windows implementation "stealing" the Python command, so 
worth flagging against core Python.

The problem is that Windows 10 installs `python` and `python3` executables by 
default, which open the Windows Store offering to install Python. That's not a 
bad thing - it's nice to have Python available with the OS! Although (see 
below) hijacking the `python` and `python3` commands for this purpose has some 
problems.

But those stubs silently do nothing when run with command line arguments, and 
because the python.org distribution doesn't put Python on PATH by default, 
users end up with the stubs available even if they install python.org Python.

We're getting a lot of bug reports from users as a result of this, when they 
follow standard instructions like "execute `python -m pip`". With the stub on 
the path, this silently does nothing, even though the user has installed 
Python. This is a very bad user experience, and not one that projects like pip 
can do anything to alleviate, other than having extremely convoluted "how to 
run pip" commands:

"""
To run pip, type `python -m pip` if you're on Unix, and `py -m pip` on Windows. 
Unless you're using the Windows Store version when you should do `python -m 
pip` on Windows too. Or on Unix when you might need to do `python3 -m pip`. 
Or...
"""

I don't have a good answer to this issue. Maybe the Windows Store stubs could 
detect core Python and do something better if it's installed? Maybe the stubs 
should print an explanatory message if run with command line arguments? Maybe 
Store Python should be available on Windows via some less error-prone mechanism?

I'm pretty sure if a Linux distribution shipped a `python` command that didn't 
run Python the way users expected, there would be complaints. After all, we 
have PEP 394 that explicitly states how we expect Unix systems to work. Maybe 
we need something similar for Windows?

--
assignee: steve.dower
components: Windows
messages: 373845
nosy: paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Windows Store "stub" Python executables give confusing behaviour
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41326] Build failure in blurb-it repo: "Failed building wheel for yarl"

[Mariatta]

Mariatta  added the comment:

Adding Victor Stinner to nosy

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41326] Build failure in blurb-it repo: "Failed building wheel for yarl"

[Mariatta]

New submission from Mariatta :

We're seeing travis CI build failure against the nightly Python build v3.10.0a0.

Build log: https://travis-ci.org/github/python/blurb_it/jobs/707532881

Last few lines of the build log:


In file included from 
/opt/python/3.10-dev/include/python3.10/unicodeobject.h:1026:0,
   from /opt/python/3.10-dev/include/python3.10/Python.h:97,
   from yarl/_quoting.c:4:
  /opt/python/3.10-dev/include/python3.10/cpython/unicodeobject.h:551:42: note: 
declared here
   Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
^
  yarl/_quoting.c: In function ‘__Pyx_decode_c_bytes’:
  yarl/_quoting.c:9996:9: warning: ‘PyUnicode_FromUnicode’ is deprecated 
[-Wdeprecated-declarations]
   return PyUnicode_FromUnicode(NULL, 0);
   ^
  In file included from 
/opt/python/3.10-dev/include/python3.10/unicodeobject.h:1026:0,
   from /opt/python/3.10-dev/include/python3.10/Python.h:97,
   from yarl/_quoting.c:4:
  /opt/python/3.10-dev/include/python3.10/cpython/unicodeobject.h:551:42: note: 
declared here
   Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
^
  error: command '/usr/bin/gcc' failed with exit code 1
  
  ERROR: Failed building wheel for yarl


I'm not familiar with that part of codebase. If anyone has any insight, it 
would be appreciated. Thanks.

--
messages: 373843
nosy: Mariatta
priority: normal
severity: normal
status: open
title: Build failure in blurb-it repo: "Failed building wheel for yarl"
type: compile error
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41324] Add a minimal decimal capsule API

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I probably won't try it out explicitly (it's basically cumbersome for us to 
test with non-release Pythons, because of our dependencies to Numpy and 
Cython), but reviewing the API should be enough anyway.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1823] Possible to set invalid Content-Transfer-Encoding on email.mime.multipart.MIMEMultipart

[Joannah Nanjekye]

Joannah Nanjekye  added the comment:

I agree with @Victor. I removed the easy tag on this easy

--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1823] Possible to set invalid Content-Transfer-Encoding on email.mime.multipart.MIMEMultipart

[Joannah Nanjekye]

Change by Joannah Nanjekye :


--
keywords:  -easy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41317] sock_accept() does not remove server socket reader on cancellation

[Alex Grönholm]

Alex Grönholm  added the comment:

This bug is the same as https://bugs.python.org/issue30064 except for 
sock_accept().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41325] Document addition of `mock.call_args.args` and `mock.call_args.kwargs` in 3.8

[Jordan Speicher]

Change by Jordan Speicher :


--
keywords: +patch
pull_requests: +20662
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21525

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41325] Document addition of `mock.call_args.args` and `mock.call_args.kwargs` in 3.8

[Jordan Speicher]

New submission from Jordan Speicher :

`args` and `kwargs` were added to unittest `mock.call_args` in 
https://bugs.python.org/issue21269 however documentation was not updated to 
state that this was added in python 3.8

--
assignee: docs@python
components: Documentation
messages: 373839
nosy: docs@python, uspike
priority: normal
severity: normal
status: open
title: Document addition of `mock.call_args.args` and `mock.call_args.kwargs` 
in 3.8
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39280] Don't allow datetime parsing to accept non-Ascii digits

[Ram Rachum]

Change by Ram Rachum :


--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41304] [CVE-2020-15801] python 38 embed ignore python38._pth file on windows

[Steve Dower]

Steve Dower  added the comment:


New changeset a16ac4e43c8ed15bf2fca52df3a0a5de26ad2705 by Miss Islington (bot) 
in branch '3.9':
bpo-41304: Update NEWS to include CVE-2020-15801 reference (GH-21521)
https://github.com/python/cpython/commit/a16ac4e43c8ed15bf2fca52df3a0a5de26ad2705


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41304] [CVE-2020-15801] python 38 embed ignore python38._pth file on windows

[Steve Dower]

Steve Dower  added the comment:


New changeset 79ed1a53fa44a1b74e3c24c9d5f698abd9610921 by Miss Islington (bot) 
in branch '3.8':
bpo-41304: Update NEWS to include CVE-2020-15801 reference (GH-21521)
https://github.com/python/cpython/commit/79ed1a53fa44a1b74e3c24c9d5f698abd9610921


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41304] [CVE-2020-15801] python 38 embed ignore python38._pth file on windows

[miss-islington]

Change by miss-islington :


--
pull_requests: +20660
pull_request: https://github.com/python/cpython/pull/21524

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41304] [CVE-2020-15801] python 38 embed ignore python38._pth file on windows

[miss-islington]

Change by miss-islington :


--
pull_requests: +20661
pull_request: https://github.com/python/cpython/pull/21523

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41304] [CVE-2020-15801] python 38 embed ignore python38._pth file on windows

[miss-islington]

Change by miss-islington :


--
pull_requests: +20659
pull_request: https://github.com/python/cpython/pull/21522

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41304] [CVE-2020-15801] python 38 embed ignore python38._pth file on windows

[Steve Dower]

Change by Steve Dower :


--
pull_requests: +20658
pull_request: https://github.com/python/cpython/pull/21521

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41304] [CVE-2020-15801] python 38 embed ignore python38._pth file on windows

[Steve Dower]

Steve Dower  added the comment:

This is now assigned CVE-2020-15801

--
title: python 38 embed ignore python38._pth file on windows -> [CVE-2020-15801] 
python 38 embed ignore python38._pth file on windows

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21625] Make help() beginner helpful when no PAGER or LESS variable

[Zackery Spytz]

Change by Zackery Spytz :


--
keywords: +patch
nosy: +ZackerySpytz
nosy_count: 7.0 -> 8.0
pull_requests: +20657
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/21520

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41024] doc: Explicitly mention use of 'enum.Enum' as a valid container for 'choices' argument of 'argparse.ArgumentParser.add_argument'

[Vincent Férotin]

Vincent Férotin  added the comment:

Unless I am mistaken, merged pull-request was not backported from master to 
other potential branches (3.5 -> 3.9 included). There was a message from GitHub 
'miss-islington' bot 
(https://github.com/python/cpython/pull/20964#issuecomment-646991186) saying, 
if I understand correctly, to manually cherry-pick merged commit to other 
branches. Am I correct?
And if so, is there some automatic process for this cherry-picking to re-run, 
or do someone have to do it manually?

--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41024] doc: Explicitly mention use of 'enum.Enum' as a valid container for 'choices' argument of 'argparse.ArgumentParser.add_argument'

[Vincent Férotin]

Vincent Férotin  added the comment:

Explicitly add Raymond Hettinger to nosy list, as he reviewed the corresponding 
PR.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41315] Add mathematical functions as wrappers to decimal.Decimal methods

[Jean Abou Samra]

Jean Abou Samra  added the comment:

I would argue that given a function,

from math import *

def naive_calc_pi(n=100):
u = sqrt(8)
v = 4
for _ in range(n):
v = 2*u*v/(u + v)
u = sqrt(u*v)
return u

when you realize that floats have limited precision (happened to me several 
times while being taught and teaching Python), you could just replace

from math import *

with

from decimal import *

and use it, instead of the current

from decimal import Decimal as D

def sqrt(x):
return D(x).sqrt()

Of course, you can define these, but as I repeatedly ended up doing this, I 
just thought I'd bring the idea upstream.

Another, perhaps more important argument is readability. We all think in terms 
of functions: log(x), not x.log(). I find it a significant fact that NumPy has 
both numpy.dot(A, B) and numpy.ndarray.dot(self, B), but the former is 
generally preferred (the method's documentation points to the function and the 
first dozen Google search results for "numpy dot product" yield the function). 
It makes expressions resemble what we are used to: compare

(a + b).tan().log() = (a.tan() + b.tan()).sqrt()/(1 - a.tan()*b.tan()).sqrt()
with
sqrt(tan(a + b)) = sqrt(tan(a) + tan(b))/sqrt(1 - tan(a)*tan(b))

> Also, the APIs aren't completely harmonious because the Decimal methods 
> accept an optional context object.

I don't see a problem with the functions also accepting this parameter. 
(np.sin() has many parameters after all.)

Overall, I think this modest addition would bring an improvement in the 
usability of the decimal module. I can make a PR.

--
nosy: +Jean_Abou_Samra

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38656] mimetypes for python 3.7.5 fails to detect matroska video

[David K. Hess]

David K. Hess  added the comment:

@michael-lazar a documentation change seems the path of least resistance given 
the complicated history of this module. +1 from me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41324] Add a minimal decimal capsule API

[Stefan Krah]

Stefan Krah  added the comment:

Adding Daniele Varrazzo, in case this is useful for the PostgreSQL
adapter.

--
nosy: +piro

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41321] Calculate timestamp is wrong in datetime.datetime

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Serhiy, you are right. I wrote below script that could explain the difference.

# ../backups/bpo41321.py
import datetime
import zoneinfo

for tz in zoneinfo.available_timezones():
diff = datetime.datetime(1986, 5, 4, 7, 13, 22, 
tzinfo=zoneinfo.ZoneInfo(tz)).timestamp() - datetime.datetime(1986, 5, 4, 0, 0, 
0, tzinfo=zoneinfo.ZoneInfo(tz)).timestamp()
if diff != 26002:
print(diff, tz)

for tz in zoneinfo.available_timezones():
diff = datetime.datetime(1986, 5, 2, 7, 13, 22, 
tzinfo=zoneinfo.ZoneInfo(tz)).timestamp() - datetime.datetime(1986, 5, 2, 0, 0, 
0, tzinfo=zoneinfo.ZoneInfo(tz)).timestamp()
if diff != 26002:
print("Diff using second day", diff, tz)

$ ./python ../backups/bpo41321.py
22402.0 Asia/Harbin
22402.0 Asia/Shanghai
22402.0 PRC
22402.0 Asia/Chongqing
22402.0 Asia/Chungking

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41324] Add a minimal decimal capsule API

[Stefan Krah]

Stefan Krah  added the comment:

Also as a note for Mark and Raymond: This API is for exact conversions
and avoids the use of the context except for raising ConversionSyntax.

I'll add documentation once Antoine has tried it out for his use case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41311] Add a function to get a random sample from an iterable (reservoir sampling)

[Oscar Benjamin]

Oscar Benjamin  added the comment:

All good points :)

Here's an implementation with those changes and that shuffles but gives the 
option to preserve order. It also handles the case W=1.0 which can happen at 
the first step with probability 1 - (1 - 2**53)**k.

Attempting to preserve order makes the storage requirements expected 
O(k*log(k)) rather than deterministic O(k) but note that the log(k) part just 
refers to the values list growing larger with references to None: only k of the 
items from iterable are stored at any time. This can be simplified by removing 
the option to preserve order which would also make it faster in the 
small-iterable case.

There are a few timings below for choosing from a dict vs converting to a list 
and using sample (I don't have a 3.9 build immediately available to use 
choices). Note that these benchmarks are not the primary motivation for 
sample_iter though which is the case where the underlying iterable is much more 
expensive in memory and/or time and where the length is not known ahead of time.



from math import exp, log, log1p, floor
from random import random, randrange, shuffle as _shuffle
from itertools import islice


def sample_iter(iterable, k=1, shuffle=True):
"""Choose a sample of k items from iterable

shuffle=True (default) gives the items in random order
shuffle=False preserves the original ordering of the items
"""
iterator = iter(iterable)
values = list(islice(iterator, k))

irange = range(len(values))
indices = dict(zip(irange, irange))

kinv = 1 / k
W = 1.0
while True:
W *= random() ** kinv
# random() < 1.0 but random() ** kinv might not be
# W == 1.0 implies "infinite" skips
if W == 1.0:
break
# skip is geometrically distributed with parameter W
skip = floor( log(random())/log1p(-W) )
try:
newval = next(islice(iterator, skip, skip+1))
except StopIteration:
break
# Append new, replace old with dummy, and keep track of order
remove_index = randrange(k)
values[indices[remove_index]] = None
indices[remove_index] = len(values)
values.append(newval)

values = [values[indices[i]] for i in irange]

if shuffle:
_shuffle(values)

return values


Timings for a large dict (1,000,000 items):

In [8]: n = 6   
   

In [9]: d = dict(zip(range(10**n), range(10**n)))   
   

In [10]: %timeit sample_iter(d, 10) 
   
16.1 ms ± 363 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

In [11]: %timeit sample(list(d), 10)
   
26.3 ms ± 1.88 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)


Timings for a small dict (5 items):

In [14]: d2 = dict(zip(range(5), range(5))) 
   

In [15]: %timeit sample_iter(d2, 2) 
   
14.8 µs ± 539 ns per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [16]: %timeit sample(list(d2), 2)
   
6.27 µs ± 457 ns per loop (mean ± std. dev. of 7 runs, 10 loops each)


The crossover point for this benchmark is around 10,000 items with k=2. 
Profiling at 10,000 items with k=2 shows that in either case the time is 
dominated by list/next so the time difference is just about how efficiently we 
can iterate vs build the list. For small dicts it is probably possible to get a 
significant factor speed up by removing the no shuffle option and simplifying 
the routine.


> Although why it keeps taking k'th roots remains a mystery to me ;-)

Thinking of sample_iter_old, before doing a swap the uvals in our reservoir 
look like:

  U0 = {u[1], u[2], ... u[k-1], W0}
  W0 = max(V0)

Here u[1] ... u[k-1] are uniform in (0, W0). We find a new u[n] < W0 which we 
swap in while removing W0 and afterwards we have

  U1 = {u[1], u[2], ... u[k-1], u[k]}
  W1 = max(U1)

Given that U1 is k iid uniform variates in (0, W0) we have that

  W1 = W0 * max(random() for _ in range(k)) = W0 * W'

Here W' has cdf x**k and so by the inverse sampling method we can generate it 
as random()**(1/k). That gives the update rule for sample_iter:

  W *= random() ** (1/k)

--

___
Python tracker 

___

[issue25538] Traceback from __exit__ method is misleading

[Tadhg McDonald-Jensen]

Tadhg McDonald-Jensen  added the comment:

uploaded `with_traceback_fixed+async.py` because I forgot to include async with 
in my first code.

--
Added file: https://bugs.python.org/file49323/with_traceback_fixed+async.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41241] Unnecessary Type casting in 'if condition'

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

Hi Wansoo, welcome to Python!


Changes that are purely cosmetic but do not change the underlying meaning of 
the code are usually refused as they create unwanted code churn. You will find 
some info about this and all the other aspects of contributing to Python in the 
Python Dev Guide: https://devguide.python.org/.


If you want to find an issue on which you could work, please have a look at the 
`easy` `easy (C)` and `newcomer_friendly` issues in the bug tracker. You can 
find them using the search function of https://bugs.python.org/

--
nosy: +remi.lapeyre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41324] Add a minimal decimal capsule API

[Stefan Krah]

Change by Stefan Krah :


--
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22194] access to cdecimal / libmpdec API

[Stefan Krah]

Stefan Krah  added the comment:

Closing in favor of #41324, which adds just the most important
functions.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Add a minimal decimal capsule API

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41322] unittest: Generator test methods will always be marked as passed

[Alexander Hungenberg]

Alexander Hungenberg  added the comment:

I would also strongly vote for raising an error if the test method is a 
generator - not even silently turn it into a list.

It would be straight forward to implement various checks (like the ones you 
mentioned for generators, coroutines, ...) - and they will absolutely help to 
prevent unintended errors, especially by more junior developers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41324] Add a minimal decimal capsule API

[Stefan Krah]

New submission from Stefan Krah :

This adds a minimal decimal capsule API.  As can be seen from the patch,
adding anything to decimal while doing it properly is quite labor and
testing intensive, so the intent is to *keep* the API minimal!

That said, some functions are really necessary:

  1) PyDec_TypeCheck() -- for obvious reasons.

  2) PyDec_Alloc() -- create new decimals.

  3) PyDec_Get()   -- get the mpd_t, enables the use of all 
libmpdec functions.

  4) PyDec_AsUint128Triple()   -- export the decimal as (sign, hi, lo, exp).

  5) PyDec_FromUint128Triple() -- create a decimal from (sign, hi, lo, exp).


4) and 5) have been requested by Antoine for a real world use case.  (hi, lo)
is the coefficient as a __uint128_t split in two uint64_t.


Antoine, could you verify that this is sufficient for the database use case?

--
nosy: +mark.dickinson, pitrou, rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41319] IDLE 3.8 can not save and run this file

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Already fixed.  I have asked the fix go in 3.8.5 on Monday.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41319] IDLE 3.8 can not save and run this file

[Terry J. Reedy]

Change by Terry J. Reedy :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> IDLE: add missing import io in iomenu.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25538] Traceback from __exit__ method is misleading

[Tadhg McDonald-Jensen]

Tadhg McDonald-Jensen  added the comment:

This is doable, the feature that decides which line is shown in the traceback 
is the `lnotab` structure in the relevant code object, the structure is 
described here 
(https://github.com/python/cpython/blob/3.8/Objects/lnotab_notes.txt) and it 
supports negative line offsets so it'd be possible to get the with statement 
closing statements to backup the line numbers to point to the with statement in 
question.

I have attached a python script which actually achieves this. 
`correct_with_cleanup_traceback` when passed a function changes the relevant 
line numbers so the traceback actually point to the with statement when the 
error is thrown in the __exit__, most of the relevant work is just dealing with 
the weird format of lnotab.

This is the error thrown by the test script, it has 2 cascading failures in 
exit so that I could confirm it worked for nested withs and it absolutely 
points to the relevant with statement as I have highlighted on the traceback,

Traceback (most recent call last):
  File "/Users/tadhgmcdonald-jensen/Documents/test.py", line 71, in my_test
>>  with Test(True) as will_fail_first: 
<<<- HERE
  File "/Users/tadhgmcdonald-jensen/Documents/test.py", line 67, in __exit__
raise Exception("error in exit")
Exception: error in exit

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/tadhgmcdonald-jensen/Documents/test.py", line 78, in 
my_test()
  File "/Users/tadhgmcdonald-jensen/Documents/test.py", line 70, in my_test
>>  with Test(True) as fail_during_handling:
<<<- HERE 
  File "/Users/tadhgmcdonald-jensen/Documents/test.py", line 67, in __exit__
raise Exception("error in exit")
Exception: error in exit

--
Added file: https://bugs.python.org/file49322/with_traceback_monkeypatch.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41300] IDLE: add missing import io in iomenu.py

[Terry J. Reedy]

Terry J. Reedy  added the comment:

#41319 should be closed as a duplicate, but site will not currently allow me to 
do so.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41321] Calculate timestamp is wrong in datetime.datetime

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

It depends on timezone. I guess there was a switch to daylight saving time at 
that time and place.

--
nosy: +belopolsky, lemburg, p-ganssle, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41319] IDLE 3.8 can not save and run this file

[wyz23x2]

wyz23x2  added the comment:

Is bpo-41300 related? It talks about non-ASCII in IDLE.

--
nosy: +wyz23x2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41319] IDLE 3.8 can not save and run this file

[wyz23x2]

Change by wyz23x2 :


--
nosy:  -wyz23x2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41241] Unnecessary Type casting in 'if condition'

[wyz23x2]

Change by wyz23x2 :


--
pull_requests: +20655
pull_request: https://github.com/python/cpython/pull/21518

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41324] Add a minimal decimal capsule API

[Stefan Krah]

Change by Stefan Krah :


--
keywords: +patch
pull_requests: +20656
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/21519

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41241] Unnecessary Type casting in 'if condition'

[wyz23x2]

Change by wyz23x2 :


--
pull_requests:  -20654

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41324] Add a minimal decimal capsule API

[Stefan Krah]

Change by Stefan Krah :


--
assignee: skrah
components: Extension Modules
nosy: skrah
priority: normal
severity: normal
stage: needs patch
status: open
title: Add a minimal decimal capsule API
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41241] Unnecessary Type casting in 'if condition'

[wyz23x2]

Change by wyz23x2 :


--
nosy: +wyz23x2
nosy_count: 3.0 -> 4.0
pull_requests: +20654
pull_request: https://github.com/python/cpython/pull/21518

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41322] unittest: Generator test methods will always be marked as passed

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I would raser raise error if the test method returns something suspicious, like 
generator or coroutine. Or maybe if it returns anything except None.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41300] IDLE: add missing import io in iomenu.py

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Problem also reported on #41319

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40941] Merge generator.gi_running and frame executing flag into single frame state

[Mark Shannon]

Mark Shannon  added the comment:


New changeset cb9879b948a19c9434316f8ab6aba9c4601a8173 by Mark Shannon in 
branch 'master':
bpo-40941: Unify implicit and explicit state in the frame and generator objects 
into a single value. (GH-20803)
https://github.com/python/cpython/commit/cb9879b948a19c9434316f8ab6aba9c4601a8173


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41322] unittest: Generator test methods will always be marked as passed

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

This is a duplicate of issue15551 and as per discussion in the thread my 
approach is incoherent with design of unittest to execute generators. I propose 
closing this as duplicate with same resolution as not fixed unless there is a 
change required here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41323] Perform "peephole" optimization directly on control-flow graph.

[Mark Shannon]

Change by Mark Shannon :


--
keywords: +patch
pull_requests: +20653
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/21517

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41323] Perform "peephole" optimization directly on control-flow graph.

[Mark Shannon]

New submission from Mark Shannon :

Currently we perform various bytecode improvements as a pass on the code 
objects after generating the code object.

This requires parsing the bytecode to find instructions, recreating the CFG, 
and rewriting the line number table.

If we perform the optimizations directly on the CFG we can avoid all that 
additional work.

This would save hundreds of lines of code and avoid coupling the optimization 
to the line number table format.

--
assignee: Mark.Shannon
messages: 373811
nosy: Mark.Shannon
priority: normal
severity: normal
stage: needs patch
status: open
title: Perform "peephole" optimization directly on control-flow graph.
type: performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41322] unittest: Generator test methods will always be marked as passed

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

https://github.com/python/cpython/blob/8e836bb21ce73f0794fd769db5883c29680dfe47/Lib/unittest/case.py#L548
 . _callTestMethod just calls the test method and doesn't check for the method 
to be a generator function to be iterated through. In Python 3.8 the call to 
test method was separated out as _callTestMethod. So something like below added 
in the original method should work. I guess there is an existing issue for this.


import unittest
from unittest import TestCase

class BuggyTestCase(TestCase):

def _callTestMethod(self, method):
import inspect

if inspect.isgeneratorfunction(method):
list(method())
else:
method()

def test_generator(self):
self.assertTrue(False)
yield None

if __name__ == "__main__":
unittest.main()


python3.8 test_foo.py
F
==
FAIL: test_generator (__main__.BuggyTestCase)
--
Traceback (most recent call last):
  File "test_foo.py", line 10, in _callTestMethod
list(method())
  File "test_foo.py", line 15, in test_generator
self.assertTrue(False)
AssertionError: False is not true

--
Ran 1 test in 0.000s

FAILED (failures=1)

--
nosy: +ezio.melotti, michael.foord, rbcollins, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41297] Remove doctest import from heapq

[Ronald Oussoren]

Ronald Oussoren  added the comment:

I have no opinion on the proposed change.

The "disruption" alex c talks about is that this imports gets seen by 
modulegraph (and hence pyinstaller and py2app), which will then include doctest 
and all its dependencies in standalone bundles even though doctest isn't 
actually used.

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41321] Calculate timestamp is wrong in datetime.datetime

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

I cannot reproduce the report on 3.8 and master. Both of the instances return 
26002 which is 7 hours, 13 minutes and 22 seconds. Can you please attach a 
script that I can run to reproduce the difference?

python3
Python 3.8.0 (v3.8.0:fa919fdf25, Oct 14 2019, 10:23:27)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> datetime.datetime(1986, 5, 4, 7, 13, 22).timestamp() - 
>>> datetime.datetime(1986, 5, 4, 0, 0, 0).timestamp()
26002.0
>>> datetime.datetime(1986, 5, 2, 7, 13, 22).timestamp() - 
>>> datetime.datetime(1986, 5, 2, 0, 0, 0).timestamp()
26002.0

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41322] unittest: Generator test methods will always be marked as passed

[Alexander Hungenberg]

New submission from Alexander Hungenberg :

The following testcase will always be marked as passed:

from unittest import TestCase

class BuggyTestCase(TestCase):
def test_generator(self):
self.assertTrue(False)
yield None


It happened to us that someone accidentally made the test method a generator 
function. That error was caught very late, because it always appears to have 
executed correctly.

Maybe an additional check can be introduced in the unittest module, to check if 
a test_ method was executed correctly?

--
components: Library (Lib)
messages: 373807
nosy: defreng
priority: normal
severity: normal
status: open
title: unittest: Generator test methods will always be marked as passed
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39584] multiprocessing.shared_memory: MacOS crashes by running attached Python code

[Vinay Sharma]

Vinay Sharma  added the comment:

Hi, I tried replicating this by truncating normal files but that doesn't crash. 
The above mentioned call of ftruncate only crashes for when the file descriptor 
passed points to a shared memory segment.
And only, multiprocessing.shared_memory is currently creating shared_memory 
using _posixshmem.shm_open.

So, it can be fixed in ftruncate implementation or this can also be handled by 
multiprocessing.shared_memory.

Please let me know your thoughts about the same.

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41321] Calculate timestamp is wrong in datetime.datetime

[dh4931]

New submission from dh4931 :

like so
datetime.datetime(1986, 5, 4, 7, 13, 22).timestamp() - datetime.datetime(1986, 
5, 4, 0, 0, 0).timestamp()
the result is 22402.0, the result is wrong.
but on May 2nd
datetime.datetime(1986, 5, 2, 7, 13, 22).timestamp() - datetime.datetime(1986, 
5, 2, 0, 0, 0).timestamp()
the result is 26002.0, the result is true

--
components: Extension Modules
messages: 373805
nosy: dh4931
priority: normal
severity: normal
status: open
title: Calculate timestamp is wrong in datetime.datetime
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41195] Interface to OpenSSL's security level

[miss-islington]

miss-islington  added the comment:


New changeset 8e836bb21ce73f0794fd769db5883c29680dfe47 by matthewhughes934 in 
branch 'master':
bpo-41195: Add getter for Openssl security level (GH-21282)
https://github.com/python/cpython/commit/8e836bb21ce73f0794fd769db5883c29680dfe47


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38119] resource tracker destroys shared memory segments when other processes should still have valid access

[Vinay Sharma]

Change by Vinay Sharma :


--
pull_requests: +20652
pull_request: https://github.com/python/cpython/pull/21516

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41295] CPython 3.8.4 regression on __setattr__ in multiinheritance with metaclasses

[David Caro]

David Caro  added the comment:

> David, which issue number is this?

It's issue41295, pr #21473

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40816] Add missed AsyncContextDecorator to contextlib

[John Belmonte]

John Belmonte  added the comment:

Thank you heckad!

I'm in need of a decorating asynccontextmanager, and glad that an 
implementation is in the works that I can copy from in the meantime.

I hope Yuri reviews the PR before long.

--
nosy: +John Belmonte

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39959] Bug on multiprocessing.shared_memory

[Vinay Sharma]

Vinay Sharma  added the comment:

@rauanargyn , persist flag won't be good idea because it cannot be supported 
for windows easily, since windows uses a reference counting mechanism to keep 
track of shared memory and frees it as soon as all the processes using it are 
done.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39959] Bug on multiprocessing.shared_memory

[Vinay Sharma]

Vinay Sharma  added the comment:

Hi, shared_memory has lot of issues which are mainly being caused due to 
resource tracking. Initially resource tracking was implemented to keep track of 
semaphores only, but for some reason resource tracker also started to keep 
track of shared_memory.
This causes shared memory to be practically useless when used by unrelated 
processes, because it will be unlinked as soon as a process dies, by processes 
which are yet to be spawned.

There is already a PR open to fix this 
https://github.com/python/cpython/pull/15989/files , by applio(a core 
developer), but for some reason it hasn't been merged yet. I will try to fix 
the conflicts and request it to be merged.

Now, this will fix most of the issues in shared memory, but still the current 
implementation of shared memory for linux won't be consistent with windows 
(which isn't at the moment also). You can read more about the same here: 
https://bugs.python.org/issue38119#msg352050

--
nosy: +vinay0410

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41320] async process closing after event loop closed

[Kuang-che Wu]

New submission from Kuang-che Wu :

(following code is attached as well)
import asyncio
import time

workaround = False

async def slow_proc():
  proc = await asyncio.create_subprocess_exec('sleep', '10', 
stdout=asyncio.subprocess.PIPE)
  try:
return await proc.stdout.read()
  except asyncio.CancelledError:
if workaround:
  proc.terminate()
  time.sleep(0.1)  # hope the machine is not too busy

async def func():
  try:
return await asyncio.wait_for(slow_proc(), timeout=0.1)
  except asyncio.TimeoutError:
return 'timeout'

def main():
  while True:
print('test')
asyncio.run(func())

main()

Run above code, it may work without error message (expected behavior).

However, depends on timing, it may show warning/error messages
case 1.
Loop <_UnixSelectorEventLoop running=False closed=True debug=False> that 
handles pid 1257652 is closed

case 2.
Exception ignored in: 
Traceback (most recent call last):
  File "/usr/lib/python3.8/asyncio/base_subprocess.py", line 126, in __del__
self.close()
  File "/usr/lib/python3.8/asyncio/base_subprocess.py", line 104, in close
proto.pipe.close()
  File "/usr/lib/python3.8/asyncio/unix_events.py", line 536, in close
self._close(None)
  File "/usr/lib/python3.8/asyncio/unix_events.py", line 560, in _close
self._loop.call_soon(self._call_connection_lost, exc)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 719, in call_soon
self._check_closed()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 508, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

Although running tasks will be cancelled when asyncio.run is finishing, 
subprocess' exit handler may be invoked after the event loop is closed.

In above code, I provided a workaround. However it just mitigates the problem 
and not really fix the root cause.

This is related to https://bugs.python.org/issue35539

p.s. My test environment is Debian 5.5.17

--
components: asyncio
files: cancel_proc.py
messages: 373799
nosy: asvetlov, kcwu, yselivanov
priority: normal
severity: normal
status: open
title: async process closing after event loop closed
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file49321/cancel_proc.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41316] tarfile: Do not write full path in FNAME field

[Artem Bulgakov]

Artem Bulgakov  added the comment:

Hi. My PR doesn't remove the possibility to add tree into tar file. It only 
fixes header for GZIP compression. Any data after this header is not affected.

You can test it by creating two archives with the same data but one with my 
patch and the second without. All bytes after header are equal.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41316] tarfile: Do not write full path in FNAME field

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +ethan.furman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com