[issue36139] release GIL on mmap dealloc

[Inada Naoki]

Inada Naoki  added the comment:

Your patch is modifying obmalloc.
It uses MAP_PRIVATE|MAP_ANONYMOUS mmap.
Does it really take log time?  How?

#1572968 is different.  It is mmap module.  It can use file-backed mmap.  It is 
I/O.

--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35892] Fix awkwardness of statistics.mode() for multimodal datasets

[Raymond Hettinger]

Change by Raymond Hettinger :


--
keywords: +patch
pull_requests: +12099
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13850] Summary tables for argparse add_argument options

[Stéphane Wirtel]

Stéphane Wirtel  added the comment:

Hi @Raymond,

Yep, I understand about the "look", I have seen the result after the build of 
the doc but the patch was like that and I did not want to change the initial 
patch. 

Would you suggest one layout and I could update the PR with your 
recommendations?

Thanks for your review

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35892] Fix awkwardness of statistics.mode() for multimodal datasets

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Attached a draft PR for discussion purposes.  Let me know what you think (I'm 
not wedded to any part of it).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36144] Dictionary addition.

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> In Python, the plus operator for sequences (strings, lists, 
> tuples) is non-commutative.

For sequences, that is obvious and expected, but not so much with mappings 
where the order of overlapping keys is determined by the left operand and the 
value associated with those keys is determined by the right operand.

Also with sequences the + operator actually means "add to", but with 
dictionaries it means "add/or replace" which is contrary to the normal meaning 
of plus.  I think that was one of Guido's reasons for favoring "|" instead of 
"+" for set-to-set operations.

> We already have a syntax for dict merging: {**d1, **d2}. 
> It works with arbitrary mappings,

This is a good point.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36139] release GIL on mmap dealloc

[Inada Naoki]

Inada Naoki  added the comment:

Sorry, I messed up.  Your PR is changing mmap module, not obmalloc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36141] configure: error: could not find pthreads on your system during cross compilation

[muhzi]

muhzi  added the comment:

Well, it solves the problem with the configure script. 

My pyconfig.h.in contains:

/* Define to 1 if you have the  header file. */
#undef HAVE_PTHREAD_H

/* Define to 1 if you have the `pthread_init' function. */
#undef HAVE_PTHREAD_INIT

/* Define to 1 if you have the `pthread_kill' function. */
#undef HAVE_PTHREAD_KILL

/* Define to 1 if you have the `pthread_sigmask' function. */
#undef HAVE_PTHREAD_SIGMASK

The problem extends after the configuration. It's as if the android headers are 
not visible to configure. All the macros are undefined in the pyconfig file too 
btw.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36139] release GIL on mmap dealloc

[Davide Rizzo]

Davide Rizzo  added the comment:

Yes, this is mmap module. I found this to be slow for posix-shm-backed mmaps. 
Several milliseconds, like 20ms for a 128 MB object. Maybe the same can happen 
with private|anon mmaps? I will follow up with more numbers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36144] Dictionary addition.

[Stefan Behnel]

Stefan Behnel  added the comment:

> We already have a syntax for dict merging: {**d1, **d2}. 

Which doesn't mean that "d1 + d2" isn't much more intuitive than this 
special-character heavy version. It takes me a while to see the dict merge 
under that heap of stars. And that's already the shortest example.


> It works with arbitrary mappings,

The RHS of "d += M" doesn't have to be a dict IMHO, it could be any mapping. 
And even "dict(X) + M" doesn't look all too bad to me, even though there's 
"dict(X, **M)".


> Use of the + operator is a temptation to produce new dictionaries rather than 
> update an existing dict in-place which is usually what you want.

That's why there would be support for "+=". The exact same argument already 
fails for lists, where concatenation is usually much more performance critical 
than for the average little dict. (And remember that most code isn't 
performance critical at all.)


> We already have ChainMap() which presents a single view of multiple mappings 
> with any copying.

Which is a different use case that is unlikely to go away with this proposal.


> makes it less clear that you're working with dicts.

This is a valid argument, although it always depends on the concrete code what 
the most readable way to express its intentions is. Again, this doesn't really 
differ for lists.

Let's wait for the PEP, I'd say.

--
nosy: +scoder

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36141] configure: error: could not find pthreads on your system during cross compilation

[Xavier de Gaye]

Xavier de Gaye  added the comment:

> The problem extends after the configuration. It's as if the android headers 
> are not visible to configure. All the macros are undefined in the pyconfig 
> file too btw.

pyconfig.h is generated by configure from pyconfig.h.in.

Please attach the new output of running configure with 
ac_cv_pthread_is_default=yes (please do not paste it in your post).

BTW what is the NDK version ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36141] configure: error: could not find pthreads on your system during cross compilation

[muhzi]

muhzi  added the comment:

OK! NDK version is 19.1.5304403

Cool, so I uploaded the output of configure (attachments)

Sorry for the confusion about pyconfig.h, what I was trying to say is after 
providing ac_cv_pthread_is_default=yes to configure I get errors in make:

Python/pytime.c:911:9: error: implicit declaration of function 
'pytime_fromtimespec' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
if (pytime_fromtimespec(tp, &ts, raise) < 0) {
^
Python/pytime.c:911:9: note: did you mean 'pytime_fromtimeval'?
Python/pytime.c:336:1: note: 'pytime_fromtimeval' declared here
pytime_fromtimeval(_PyTime_t *tp, struct timeval *tv, int raise)
^
Python/pytime.c:911:9: warning: this function declaration is not a prototype 
[-Wstrict-prototypes]
if (pytime_fromtimespec(tp, &ts, raise) < 0) {
^

--
Added file: https://bugs.python.org/file48176/config.log

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36144] Dictionary addition.

[Mark Dickinson]

Change by Mark Dickinson :


--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30235] Validate shutil supports path-like objects, update docs accordingly

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

One thing to note is the copy_function parameter: if a path-like object is 
given as input should the copy_function receive the same object instead of a 
path string?


>>> copytree(path_like_src, path_like_dst, copy_function=foo)


Because of issue33695 in case of copytree(..., copy_function=...) we currently 
force to always pass path strings if copy_function is specified:
https://github.com/python/cpython/blob/ed1deb0719f0ac1b08a374e30ad26a701d4d51a2/Lib/shutil.py#L450

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36137] SSL verification fails for some sites inside windows docker container

[Christian Heimes]

Christian Heimes  added the comment:

Err, I meant #36011

The enum cert store trick only breaks on a fresh installation. If you have used 
Windows for a bit (e.g. downloaded Firefox with Edge *g*), the root CA store is 
filled with common CA certs. Your certutil trick also works, as it triggers 
cert retrieval and update from Windows Update Server.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31652] make install fails: no module _ctypes

[Thrlwiti]

Change by Thrlwiti :


--
nosy: +THRlWiTi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36141] configure: error: could not find pthreads on your system during cross compilation

[Xavier de Gaye]

Xavier de Gaye  added the comment:

This is a different problem. Please create a new issue and attach pyconfig.h.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29571] test_re is failing when local is set for `en_IN`

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Similar issue reported on debian9.8 stretch with python 3.7.2 and en_IN : 
issue36134

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36139] release GIL on mmap dealloc

[Davide Rizzo]

Davide Rizzo  added the comment:

munmap() of private maps is usually pretty fast but not negligible (2 ms for 
1GB). Shared maps are much slower. For some reason, shared maps explicitly 
backed by POSIX shared memory stand in between but are still pretty slow.

If someone cares about file-backed mmaps I can test those too. I thought this 
is already significant to justify releasing the GIL.


This is on Linux 4.4:


shared anon mmap 1048576 bytes
mmap time 6,393 ns
write time  449,062 ns
munmap time 100,205 ns
--
private anon mmap 1048576 bytes
mmap time 2,168 ns
write time  308,966 ns
munmap time  36,930 ns
--
posix shm + mmap 1048576 bytes
mmap time13,299 ns
write time  369,305 ns
close time1,545 ns
munmap time  26,759 ns
--

shared anon mmap 134217728 bytes
mmap time 4,641 ns
write time   64,508,536 ns
munmap time  13,592,556 ns
--
private anon mmap 134217728 bytes
mmap time 6,116 ns
write time   25,402,084 ns
munmap time 388,976 ns
--
posix shm + mmap 134217728 bytes
mmap time29,034 ns
write time   66,826,645 ns
close time3,707 ns
munmap time   3,475,977 ns
--

shared anon mmap 1073741824 bytes
mmap time11,127 ns
write time  508,227,373 ns
munmap time  94,885,306 ns
--
private anon mmap 1073741824 bytes
mmap time 7,133 ns
write time  199,933,903 ns
munmap time   2,361,036 ns
--
posix shm + mmap 1073741824 bytes
mmap time24,868 ns
write time  527,566,819 ns
close time4,015 ns
munmap time  21,179,674 ns
--

--
Added file: https://bugs.python.org/file48177/unmap.c

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29397] linux/random.h present but cannot be compiled

[Emmanuel Arias]

Change by Emmanuel Arias :


--
nosy: +eamanu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36145] android arm cross compilation fails, h

[muhzi]

New submission from muhzi :

This is a follow up of #36141, I'm trying to build python for android armv7a.
The problem was the configure script fails to find pthread_create in the 
android headers. Now after getting past the configuration, I get a build error:

armv7a-linux-androideabi16-clang -c -mfloat-abi=softfp -mfpu=vfpv3-d16 
-Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 
-Wall-std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter 
-Wno-missing-field-initializers -Wstrict-prototypes 
-Werror=implicit-function-declaration   -I. -I./Include-DPy_BUILD_CORE -o 
Python/pytime.o Python/pytime.c
Python/pytime.c:911:9: error: implicit declaration of function 
'pytime_fromtimespec' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
if (pytime_fromtimespec(tp, &ts, raise) < 0) {
^
Python/pytime.c:911:9: note: did you mean 'pytime_fromtimeval'?
Python/pytime.c:336:1: note: 'pytime_fromtimeval' declared here
pytime_fromtimeval(_PyTime_t *tp, struct timeval *tv, int raise)
^
Python/pytime.c:911:9: warning: this function declaration is not a prototype 
[-Wstrict-prototypes]
if (pytime_fromtimespec(tp, &ts, raise) < 0) {
^

The declaration for pytime_fromtimespec needs the token HAVE_CLOCK_GETTIME to 
be defined to 1, which isn't the case (as per pyconfig.h). I checked the 
android headers and time.h seems to have the necessary function for this token. 

I uploaded pyconfig.h for reference (attached).

--
components: Cross-Build
files: pyconfig.h
messages: 336828
nosy: Alex.Willmer, muhzi, xdegaye
priority: normal
severity: normal
status: open
title: android arm cross compilation fails, h
type: compile error
versions: Python 3.7
Added file: https://bugs.python.org/file48178/pyconfig.h

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36145] android arm cross compilation fails, config issue

[muhzi]

Change by muhzi :


--
title: android arm cross compilation fails, h -> android arm cross compilation 
fails, config issue

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36139] release GIL on mmap dealloc

[Christian Heimes]

Christian Heimes  added the comment:

The change sounds like a good idea and should be backported, too.

IIRC mmap() performance also depends on MMU and TLB speed. In the past I have 
seen paravirtualized systems with poor MMU performance that caused fork() to be 
slow and Redis to hang.

--
nosy: +christian.heimes, davin
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29397] linux/random.h present but cannot be compiled

[Christian Heimes]

Christian Heimes  added the comment:

It's a CentOS 5 system with Kernel 2.6.18. The getrandom() syscall was added in 
Linux Kernel version 3.17. Your OS simply lacks the necessary feature to use 
linux/random.h. Python will automatically fall back to /dev/urandom.

--
nosy: +christian.heimes
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35975] Put back the ability to parse files where async/await aren't keywords

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Would not be simpler to just drop the support of Python versions <3.7 in new 
MyPy versions?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36145] android arm cross compilation fails, config issue

[Xavier de Gaye]

Xavier de Gaye  added the comment:

There is a bug in the implementation of pymonotonic() in Python/pytime.c:

The call to pytime_fromtimespec() is not encapsulated by an #ifdef 
HAVE_CLOCK_GETTIME.

@muhzi
It seems that your pyconfig.h is generated with configure failing to find the 
NDK headers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36145] android arm cross compilation fails, config issue

[Xavier de Gaye]

Change by Xavier de Gaye :


--
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36134] test failure : test_re; recipe for target 'test' failed

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> test_re is failing when local is set for `en_IN`

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31652] make install fails: no module _ctypes

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36145] android arm cross compilation fails, config issue

[muhzi]

muhzi  added the comment:

Yes, pretty much so, which I think was what caused the pthread problem. Why 
would it fail to find the headers?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36126] Reference count leakage in structseq_repr

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 69b4a17f342146d6b7a73975a37678db9916aa75 by Serhiy Storchaka 
(Gao, Xiang) in branch '2.7':
bpo-36126: Fix ref count leakage in structseq_repr. (GH-12035)
https://github.com/python/cpython/commit/69b4a17f342146d6b7a73975a37678db9916aa75


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36126] Reference count leakage in structseq_repr

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
type: behavior -> resource usage

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23794] http package should support HTTP/2

[Ludovic Gasc]

Change by Ludovic Gasc :


--
nosy:  -Ludovic.Gasc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36145] android arm cross compilation fails, config issue

[Paul Ganssle]

Change by Paul Ganssle :


--
nosy: +p-ganssle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36143] Auto-generate Lib/keyword.py

[Joannah Nanjekye]

Joannah Nanjekye  added the comment:

I am working on this. Shouldn't there be a PR by tomorrow from me. Someone else 
can take it on.

--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36143] Auto-generate Lib/keyword.py

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Do not haste with this. After implementing pgen in Python, Lib/keyword.py can 
be generated directly by pgen.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36145] android arm cross compilation fails, config issue

[Xavier de Gaye]

Xavier de Gaye  added the comment:

Obviously because you do not tell clang where to find those headers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36143] Auto-generate Lib/keyword.py

[Joannah Nanjekye]

Joannah Nanjekye  added the comment:

Noted. Releasing this in favor of when pgen is implemented in Python to solve 
this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36143] Auto-generate Lib/keyword.py

[Guido van Rossum]

Guido van Rossum  added the comment:

A similar thing seems to be the list of keywords in Lib/pydoc.py. The recipe 
says

# CAUTION: if you change one of these dictionaries, be sure to adapt the
#  list of needed labels in Doc/tools/extensions/pyspecific.py and
#  regenerate the pydoc_data/topics.py file by running
#  make pydoc-topics
#  in Doc/ and copying the output file into the Lib/ directory.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36146] Refactor setup.py

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +12100
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36146] Refactor setup.py

[STINNER Victor]

New submission from STINNER Victor :

The detect_modules() method of setup.py became longer and longer over
the years. It is now 1128 lines long. It's way too long: it becomes
very hard to track the lifetime of a variable and many variables are
overriden on purpose or not. Shorter functions help to track the
lifetime of variables, ease review and reduce the number of bugs.

--
components: Build
messages: 336841
nosy: vstinner
priority: normal
severity: normal
status: open
title: Refactor setup.py
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36143] Auto-generate Lib/keyword.py

[Guido van Rossum]

Guido van Rossum  added the comment:

(However that doesn't seem to work. Anyway, there's a list of all keywords 
there.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36146] Refactor setup.py

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12101

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36143] Auto-generate Lib/keyword.py

[Guido van Rossum]

Guido van Rossum  added the comment:

Hum, it seems the right solution was to arrange for async and await to appear 
in the list of keywords despite their special status. Nevertheless in theory 
there's something that can be done here to auto-regenerate the list derived 
from the Grammar.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21879] str.format() gives poor diagnostic on placeholder mismatch

[Joannah Nanjekye]

Joannah Nanjekye  added the comment:

@dmzz, You can convert the patch into a PR on github or Open a PR with your own 
implementation.

If working with this same patch, the next step would be to test the patch if it 
is still in good shape now before opening a PR.

--
nosy: +nanjekyejoannah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36143] Auto-generate Lib/keyword.py

[STINNER Victor]

STINNER Victor  added the comment:

> Do not haste with this. After implementing pgen in Python, Lib/keyword.py can 
> be generated directly by pgen.

That's bpo-35808 and PR 11814 which is under review.

Anyway, at the end, it would be nice if Lib/keyword.py would be regenerated by 
"make regen-all". Currently, "regen-all" doesn't regenerate everything. For 
example, configure is not regenerated.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21879] str.format() gives poor diagnostic on placeholder mismatch

[Francisco Couzo]

Francisco Couzo  added the comment:

I'm preparing the PR myself

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34631] Upgrade to OpenSSL 1.1.1b

[Christian Heimes]

Christian Heimes  added the comment:

Hi macOS and Windows devs,

as I explained in 
https://mail.python.org/pipermail/python-dev/2019-February/156470.html we need 
to update Python 3.7 to OpenSSL 1.1.1 eventually. 1.1.0 will reach EOL in 
September.

--
components: +Windows, macOS
nosy: +ned.deily, paul.moore, ronaldoussoren, steve.dower, tim.golden, zach.ware
title: Upgrade to OpenSSL 1.1.1 -> Upgrade to OpenSSL 1.1.1b
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34631] Upgrade to OpenSSL 1.1.1b

[Christian Heimes]

Change by Christian Heimes :


--
keywords: +patch
pull_requests: +12102
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36144] Dictionary addition.

[Josh Rosenberg]

Josh Rosenberg  added the comment:

scoder: dict(X, **M) is broken unless M is known to be string keyed (it used to 
work, but in Python 3, it will raise a TypeError). It's part of the argument 
for the additional unpacking generalizations from PEP 448; {**X, **M} does what 
dict(X, **M) is trying to do, but without abusing the keyword argument passing 
convention.

You also claim "It takes me a while to see the dict merge under that heap of 
stars", but that's at least as much about the newness of PEP 448 (and for many 
Python coders, a complete lack of familiarity with the pre-existing varargs 
unpacking rules for functions) as it is about the punctuation; after all, you 
clearly recognize dict(X, **M) even though it's been wrong in most contexts for 
years.

In any event, I'm a strong -1 on this, for largely the same reasons as Raymond 
and others:

1. It doesn't provide any new functionality, just one more way to do it; += is 
satisfied by .update, + is satisfied (more generally and efficiently) by the 
unpacking generalizations

2. It's needlessly confusing; addition is, for all existing types in the 
standard library I can think of, lossless; the information from both sides of 
the + is preserved in some form, either by addition or concatenation (and in 
the concatenation case, addition is happening, just to the length of the 
resulting sequence, and order is preserved). Addition for dictionaries would 
introduce new rules specific to dicts that do not exist for any other type 
regarding loss of values, non-additive resulting length, etc. Those rules would 
likely be similar to those of dict literals and the update method, but they'd 
need to be made explicit. By contrast, the PEP 448 unpacking generalization 
rules followed the existing rules for dict literals; no special rules occur, it 
just behaves intuitively (if you already knew the rules for dict literals 
without unpacking being involved).

3. Almost any generic, duck-typing based code for which addition makes sense 
will not make sense for dicts simply because it loosens the definition of 
addition too much to be useful, so best case, it still raises TypeError (when 
dicts added to non-dict things), worst case, it silently operates in a way that 
violates the rules of both addition and concatenation rather than raising a 
TypeError that the generic code could use to determine the correct thing to do.

4. The already mentioned conflict with Counter (which already has an addition 
operator, with lossless semantics)

5. (Minor) It means PyDict_Type needs a non-NULL tp_as_number, so now it's 
slightly slower to reject dicts as being non-numeric at the C layer

Problem #2 could be used to argue for allowing | instead of + (which would also 
resolve #4, and parts of #3), since | is already used for unioning with sets, 
and this operation is much closer to a union operation than addition or 
concatenation. Even so, it would still be misleading; at least with sets, there 
is no associated value, so it's still mostly lossless (you lose the input 
lengths, but the unique input data is kept); with dicts, you'd be losing values 
too.

Basically, I think the PEP 448 unpacking syntax should remain as the "one-- and 
preferably only one --obvious way to" combine dictionaries as a one-liner. It's 
more composable, since it allows adding arbitrary additional key/value pairs, 
and more efficient, since it allows combining more than two dicts at once with 
no additional temporaries: dicta + dictb + dictc requires "dictab" to be made 
first, then thrown away after dictab + dictc produces dictabc, while {**dicta, 
**dictb, **dictc} builds dictabc directly.

The only real argument I can see for not sticking to unpacking is that it 
doesn't allow for arbitrary dict-like things to produce new dict-like things 
directly; you'd have to rewrap as myspecialdict({**speciala, **specialb}). But 
I don't think that's a flaw worth fixing if it means major changes to the 
behavior of what I'm guessing is one of the three most commonly used types in 
Python (along with int and tuple, thanks to the integration of dicts into so 
many facets of the implementation).

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36144] Dictionary addition.

[Guido van Rossum]

Guido van Rossum  added the comment:

I changed my mind and am now in favor. Most of the arguments against could also 
be used against list+list. Counter addition is actually a nice special case of 
this -- it produces the same keys but has a more sophisticated way of merging 
values for common keys. Please read the python-ideas thread!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36144] Dictionary addition.

[Josh Rosenberg]

Josh Rosenberg  added the comment:

Also note: That Python ideas thread that xtreak linked ( 
https://mail.python.org/pipermail/python-ideas/2015-February/031748.html ) 
largely rejected the proposal a couple weeks before PEP 448 was approved. At 
the time, the proposal wasn't just about +/+=; that was the initial proposal, 
but operator overloading was heavily criticized for the failure to adhere to 
either addition or concatenation semantics, so alternate constructors and 
top-level functions similar to sorted were proposed as alternatives (e.g. 
merged(dicta, dictb)). The whole thread ended up being about creating an 
approved, built-in way of one-lining: d3 = d1.copy(); d3.update(d2)

A key quote though is that this was needed because there was no other option 
without rolling your own merged function. Andrew Barnert summarized it best:

"I'm +1 on constructor, +0.5 on a function (whether it's called updated or 
merged, whether it's in builtins or collections), +0.5 on both constructor and 
function, -0.5 on a method, and -1 on an operator.

"Unless someone is seriously championing PEP 448 for 3.5, in which case I'm 
-0.5 on anything, because it looks like PEP 448 would already give us one 
obvious way to do it, and none of the alternatives are sufficiently nicer than 
that way to be worth having another."

As it happens, PEP 448 was put in 3.5, and we got the one obvious way to do it.

Side-note: It occurs to me there will be one more "way to do it" in 3.8 
already, thanks to PEP 572:

(d3 := d1.copy()).update(d2)

I think I'll stick with d3 = {**d1, **d2} though. :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6766] Cannot modify dictionaries inside dictionaries using Managers from multiprocessing

[Dusan Gligoric]

Dusan Gligoric <7...@protonmail.com> added the comment:

Hey folks,

This is still an issue with 3.7.2

===

# Python 3.7.2 (default, Jan 10 2019, 23:51:51) 
# [GCC 8.2.1 20181127] on linux

from multiprocessing import  Manager

manager = Manager()
d = manager.dict({})

d["test"] = {"a": 123}
# update fails
d["test"]["a"] = 321
# add fails
d["test"]["b"] = 321

print(d)

--
nosy: +dusan76
versions: +Python 3.7 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36146] Refactor setup.py

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4cbea518a0827d23a41a45b03a8af729c2f16605 by Victor Stinner in 
branch 'master':
bpo-36146: Refactor setup.py (GH-12093)
https://github.com/python/cpython/commit/4cbea518a0827d23a41a45b03a8af729c2f16605


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36018] Add a Normal Distribution class to the statistics module

[Raymond Hettinger]

Change by Raymond Hettinger :


--
pull_requests: +12103

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36146] Refactor setup.py

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12104

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36146] Refactor setup.py

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12105

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36018] Add a Normal Distribution class to the statistics module

[miss-islington]

miss-islington  added the comment:


New changeset ef17fdbc1c274dc84c2f611c40449ab84824607e by Miss Islington (bot) 
(Raymond Hettinger) in branch 'master':
bpo-36018: Add special value tests and make minor tweaks to the docs (GH-12096)
https://github.com/python/cpython/commit/ef17fdbc1c274dc84c2f611c40449ab84824607e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Steve Dower]

Steve Dower  added the comment:

> But think of the poor user who doesn’t have that expertise or ability to hack 
> on an installed Python’s site.py file.

This is actually part of the thinking behind the reportabug tool I started (and 
why when you format it as raw text you get the listing of everything in any 
directory on sys.path - mostly because I haven't added a Markdown rendering of 
that). If the answer is to enhance that and tell users "run `reportabug 
mybrokenmodule` and send me the output", well, that's why I put it on GitHub :) 
https://github.com/zooba/reportabug

I see no reason to hold up adding pth logging to -v, so anyone interested 
please feel free to do a PR.

The only reason I see to hold up PE 10131 (docs update) is because it documents 
the rationale for using arbitrary code execution in a pth file. Since we 
clearly want to get rid of it, I don't think we should in any way rationalize 
it in the docs.

Once these are done, I think we'll have to reevaluate whether .pth files are 
actually a problem in their normal behavior, and whether the benefit outweighs 
the cost. But since we're all agreed that they aren't easy to debug and contain 
features we all want to get rid of, there's not much point using the current 
state to do the cost/benefit analysis. Let's fix the bits we can fix first and 
then see where we stand.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36144] Dictionary addition.

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Current python-ideas thread for the issue : 
https://mail.python.org/pipermail/python-ideas/2019-February/055509.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29571] test_re is failing when local is set for `en_IN`

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +12106
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29571] test_re is failing when local is set for `en_IN`

[STINNER Victor]

STINNER Victor  added the comment:

Ah, I can reproduce the bug on Fedora 29 using "LANG=en_IN ./python -m test -v 
test_re".

The problem is that locale.getlocale() is not reliable: it pretends that the 
locale encoding is ISO8859-1, whereas the real encoding is UTF-8:

$ LANG=en_IN ./python 
Python 3.8.0a2+ (heads/master:4cbea518a0, Feb 28 2019, 18:19:44) 
>>> chr(224).encode('ISO8859-1')
b'\xe0'
>>> import _testcapi
>>> _testcapi.DecodeLocaleEx(b'\xe0', 0, 'strict')
Traceback (most recent call last):
  File "", line 1, in 
RuntimeError: decode error: pos=0, reason=decoding error

>>> import locale

# Wrong encoding
>>> locale.getlocale(locale.LC_CTYPE)
('en_IN', 'ISO8859-1')
>>> locale.setlocale(locale.LC_CTYPE, None)
'en_IN'
>>> locale._parse_localename('en_IN')
('en_IN', 'ISO8859-1')

# Real encoding
>>> locale.getpreferredencoding()
'UTF-8'
>>> locale.nl_langinfo(locale.CODESET)
'UTF-8'


Attached PR 12099 fix the issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Anthony Sottile]

Anthony Sottile  added the comment:

> contain features we all want to get rid of

I don't think even this is unanimous.  Things like registering codecs, 
instrumenting coverage in subprocesses, etc. all seem like legitimate uses of 
the arbitrary code execution feature

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36131] test.test_urllib2net.TimeoutTest ftp related tests fail due to ftp://www.pythontest.net/ being unavailable

[STINNER Victor]

STINNER Victor  added the comment:

I discussed with Ernest on #python-infra. It was an issue with the DNS 
"resolving the host back to 127.0.1.1 (...) so vsftpd is telling users to 
connect back to themselves".

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13096] ctypes: segfault with large POINTER type names

[Charalampos Stratakis]

Change by Charalampos Stratakis :


--
pull_requests: +12107

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13096] ctypes: segfault with large POINTER type names

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

It seems the python2 backport was incomplete as a PyMem_Free is missing, making 
buf leak.

--
nosy: +cstratak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13096] ctypes: segfault with large POINTER type names

[Charalampos Stratakis]

Change by Charalampos Stratakis :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36147] [2.7] Coverity scan: Modules/_ctypes/cfield.c , Variable "result" going out of scope

[Charalampos Stratakis]

New submission from Charalampos Stratakis :

Coverity scan on python2 resulted in this error.

Python-2.7.15/Modules/_ctypes/cfield.c:1297: alloc_fn: Storage is returned from 
allocation function "PyString_FromString".
Python-2.7.15/Objects/stringobject.c:143:5: alloc_fn: Storage is returned from 
allocation function "PyObject_Malloc".
Python-2.7.15/Objects/obmalloc.c:982:5: alloc_fn: Storage is returned from 
allocation function "malloc".
Python-2.7.15/Objects/obmalloc.c:982:5: return_alloc_fn: Directly returning 
storage allocated by "malloc".
Python-2.7.15/Objects/stringobject.c:143:5: var_assign: Assigning: "op" = 
"PyObject_Malloc(37UL + size)".
Python-2.7.15/Objects/stringobject.c:164:5: return_alloc: Returning allocated 
memory "op".
Python-2.7.15/Modules/_ctypes/cfield.c:1297: var_assign: Assigning: "result" = 
storage returned from "PyString_FromString((char *)ptr)".
Python-2.7.15/Modules/_ctypes/cfield.c:1311: leaked_storage: Variable "result" 
going out of scope leaks the storage it points to.
1309|   } else
1310|   /* cannot shorten the result */
1311|-> return PyString_FromStringAndSize(ptr, size);
1312|   }
1313|   

This was fixed on python3 with 
https://github.com/python/cpython/commit/19b52545df898ec911c44e29f75badb902924c0b

Partially backporting this change for this file should fix the issue.

--
components: Extension Modules
messages: 336859
nosy: cstratak, vstinner
priority: normal
severity: normal
status: open
title: [2.7] Coverity scan: Modules/_ctypes/cfield.c , Variable "result" going 
out of scope
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Feb 28, 2019, at 09:40, Anthony Sottile  wrote:
> 
> I don't think even this is unanimous.  Things like registering codecs, 
> instrumenting coverage in subprocesses, etc. all seem like legitimate uses of 
> the arbitrary code execution feature

Except pth files are a terrible interface for that, given all the other 
problems, including weird wall-of-code inducing restrictions on what actually 
gets executed.

I’m in agreement with Steve Dower in principle here.  I would like to see a 
solution that deprecates and eventually removes arbitrary code execution in pth 
files, leaves sys.path extension alone (for now ), and improves the 
discoverability and debuggability of magical pth files.

What I think Anthony is looking for are ways to register “start up functions” 
that get executed automatically when the Python interpreter starts up.  Perhaps 
somewhat analogous to atexit functions?  But if we’re going to officially 
support a feature like that, I think a PEP would be the right vehicle to suss 
out all the gory details, like, should these things be global across all 
invocations of the interpreter, how a user or application would disable that, 
how would bugs in start up functions get discovered, reported, and debugged, 
what if any execution order guarantees should be made, etc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36147] [2.7] Coverity scan: Modules/_ctypes/cfield.c , Variable "result" going out of scope

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

Also the change from PyUnicode_FromStringAndSize to PyBytes_FromStringAndSize 
happened here: https://bugs.python.org/issue8966

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35931] pdb: "debug print(" crashes with SyntaxError

[daniel hahler]

daniel hahler  added the comment:

Re-opening: it currently still crashes with a NameError when using "debug 
doesnotexist", or "debug doesnotexist()".

Will create a new PR for this.

--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36147] [2.7] Coverity scan: Modules/_ctypes/cfield.c , Variable "result" going out of scope

[Charalampos Stratakis]

Change by Charalampos Stratakis :


--
keywords: +patch
pull_requests: +12108
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Anthony Sottile]

Anthony Sottile  added the comment:

> What I think Anthony is looking for are ways to register “start up functions” 
> that get executed automatically when the Python interpreter starts up

yes, this is what I want to still exist :)


my hope is that there's a clear standards-track replacement *before* 
deprecating .pth (which currently satisfies my usecases for startup functions)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35931] pdb: "debug print(" crashes with SyntaxError

[daniel hahler]

Change by daniel hahler :


--
pull_requests: +12109
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36148] smtplib.SMTP.sendmail: mta status codes only accessible by local variables

[sls]

New submission from sls :

MTA status codes (visible via setdebuglevel(1)) are not accessible as the 
`sendmail`-method stores them only locally (code, resp).

I suggest to store the mta status codes, for instance: "250, b'2.0.0 Ok: queued 
as XYZ" etc. in an instance attribute (tuple) to access them on smtp-sessions. 

As an email developer those information are very important to me.

--
components: Library (Lib)
messages: 336864
nosy: sls
priority: normal
severity: normal
status: open
title: smtplib.SMTP.sendmail: mta status codes only accessible by local 
variables
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36149] use of uninitialised memory in cPickle.

[Thomas Wouters]

New submission from Thomas Wouters :

There is a bug in cPickle that makes it use uninitialised memory when reading a 
truncated pickle from a file (an actual C FILE*, not just any file-like 
object). Using MemorySanitizer:

% ./python
Python 2.7.15 (default, redacted, redacted) 
[GCC 4.2.1 Compatible Clang (redacted)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile, cPickle
>>> f = tempfile.TemporaryFile()
>>> f.write('F')
>>> f.seek(0)
>>> cPickle.load(f)
Uninitialized bytes in __interceptor_strlen at offset 1 inside [0x70101b50, 
3)
==23453==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x561a9110d51b in PyString_FromFormatV Objects/stringobject.c:241:22
#1 0x561a912ba454 in PyErr_Format Python/errors.c:567:14
#2 0x561a91303bc8 in PyOS_string_to_double Python/pystrtod.c
#3 0x561a90b4a7d7 in load_float Modules/cPickle.c:3618:9
#4 0x561a90b48ca7 in load Modules/cPickle.c:4779:17
#5 0x561a90b56d9c in cpm_load Modules/cPickle.c:5758:11
#6 0x561a91260b89 in call_function Python/ceval.c:4379:17

The problem is Modules/cPickle:readline_file end-of-file handling logic:

for (; i < (self->buf_size - 1); i++) {
if (feof(self->fp) ||
(self->buf[i] = getc(self->fp)) == '\n') {
self->buf[i + 1] = '\0';
*s = self->buf;
return i + 1;
}
}

When feof(self->pf) becomes true, the code skips over writing to self->buf[i] 
(which hasn't been written to yet), only writes to self->buf[i+1], and returns 
self->buf.

There is an additional problem that the code fails to check for the EOF return 
of getc(), which means it may write EOF-cast-to-a-char to self->buf[i] without 
realising it's meant to be an EOF. (EOF is usually -1, but I don't remember if 
that's a valid cast or undefined behaviour.) Theoretically this could cause 
invalid, truncated pickles to be read succesfully, but with the wrong value for 
the last item. (It could perhaps do even worse things with files that change 
while cPickle is reading them, but that's harder to reason about.)

(Use of uninitialised memory is a potential security issue, although so is 
using pickles, so I'm conflicted about the bug type...)

--
components: Interpreter Core
messages: 336865
nosy: twouters
priority: normal
severity: normal
status: open
title: use of uninitialised memory in cPickle.
type: security
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36148] smtplib.SMTP.sendmail: mta status codes only accessible by local variables

[sls]

Change by sls :


--
keywords: +patch
pull_requests: +12111
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36149] use of uninitialised memory in cPickle.

[Thomas Wouters]

Change by Thomas Wouters :


--
keywords: +patch
pull_requests: +12112
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36148] smtplib.SMTP.sendmail: mta status codes only accessible by local variables

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
components: +email
nosy: +barry, r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36150] Possible assertion failures due to _ctypes.c's PyCData_reduce()

[Zackery Spytz]

New submission from Zackery Spytz :

The PyBytes_FromStringAndSize() and PyObject_GetAttrString() calls in 
PyCData_reduce() are not checked for failure.

--
components: Extension Modules, ctypes
messages: 336866
nosy: ZackerySpytz
priority: normal
severity: normal
status: open
title: Possible assertion failures due to _ctypes.c's PyCData_reduce()
type: crash
versions: Python 2.7, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36150] Possible assertion failures due to _ctypes.c's PyCData_reduce()

[Zackery Spytz]

Change by Zackery Spytz :


--
keywords: +patch
pull_requests: +12113
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36150] Possible assertion failures due to _ctypes.c's PyCData_reduce()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Does not Py_BuildValue() check the argument for "N" for NULL?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36151] Incorrect answer when calculating 11//3

[Aiden Zhou]

New submission from Aiden Zhou :

When calculating 11//3, the last digit of the decimals is wrongly rounded to 5 
as obviously it needs to be rounded to 7. IDLE (Python 3.7 32-bit)

--
assignee: terry.reedy
components: IDLE
files: Screenshot (102)_LI.jpg
messages: 336868
nosy: azihdoeun, terry.reedy
priority: normal
severity: normal
status: open
title: Incorrect answer when calculating 11//3
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48179/Screenshot (102)_LI.jpg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36151] Incorrect answer when calculating 11//3

[Aiden Zhou]

Change by Aiden Zhou :


Added file: https://bugs.python.org/file48180/Screenshot (102)_LI.jpg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36151] Incorrect answer when calculating 11//3

[Aiden Zhou]

Change by Aiden Zhou :


Removed file: https://bugs.python.org/file48180/Screenshot (102)_LI.jpg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36151] Incorrect answer when calculating 11/3

[Zachary Ware]

Zachary Ware  added the comment:

This has nothing to do with IDLE and everything to do with how floating point 
numbers work; have a read through 
https://docs.python.org/3/tutorial/floatingpoint.html for an introduction to 
floating point.  If you need exact decimal math, use the `decimal` module 
instead.

--
assignee: terry.reedy -> 
components:  -IDLE
nosy: +zach.ware
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
title: Incorrect answer when calculating 11//3 -> Incorrect answer when 
calculating 11/3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36150] Possible assertion failures due to _ctypes.c's PyCData_reduce()

[Christian Heimes]

Christian Heimes  added the comment:

Yes, N and O accept NULL and pass the exception up the call stack. It's not a 
bug. 
https://docs.python.org/3/c-api/arg.html?highlight=py_buildvalue#c.Py_BuildValue

--
nosy: +christian.heimes
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36148] smtplib.SMTP.sendmail: mta status codes only accessible by local variables

[R. David Murray]

R. David Murray  added the comment:

Thanks for the PR, but this is a duplicate of #29539, which I think has a 
better API proposal.  Since the original author never actually submitted a PR 
there, perhaps you could pick up his work (after pinging the issue).

--
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed
superseder:  -> Deprecate string concatenation without plus
versions: +Python 3.8 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36148] smtplib.SMTP.sendmail: mta status codes only accessible by local variables

[R. David Murray]

R. David Murray  added the comment:

Sorry, that should be #29539.

--
superseder: Deprecate string concatenation without plus -> [smtplib] collect 
response data for all recipients

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29539] [smtplib] collect response data for all recipients

[R. David Murray]

Change by R. David Murray :


--
nosy: +sls

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36151] Incorrect answer when calculating 11/3

[Terry J. Reedy]

Terry J. Reedy  added the comment:

To add to Zach's answer:

Please do not mark issues for IDLE just because you use IDLE to send your code 
to Python and display the returned string answer. If in any doubt, rerun the 
same code directly with Python either interactively or from the command line.  
(Note: Windows requires "", not '', on command line.)

C:\Users\Terry>python -c "print(11/3)"
3.6665 

As for the result: 11/3 cannot be exactly represented as a binary (or decimal) 
finite-precision floating point number.  In this case, the nearest binary float 
is less that 11/3.  Python prints floats with up to 17 decimal digits.We 
can get more with the decimal module.

>>> decimal.Decimal(11/3)
Decimal('3.666518636930049979127943515777587890625')

We can also check with decimal float literals
>>> 11/3 == 3.6665)
True

This expression is true for all 18 digit literals from
3.66630 to
3.66674

It would have been better to ask about 11/3 on python-list (for instance), 
where other beginners could see the answer, rather than here.  I will repost my 
answer there.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36151] Incorrect answer when calculating 11/3

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Aiden: in the future, please do not post unnecessary screenshots of text. They 
are hostile to the blind and slight-impaired, they make it impossible to copy 
your code and run it ourselves, and they cannot be searched for.

Instead, copy and paste the text demonstrating the code and the results:

py> 11/3
3.6665


Thank you.

Zachary: while the rest of your comments were excellent, please don't tell 
people that Decimal is "exact". Decimal is a floating point format just like 
floats are, and so suffers the same kinds of rounding errors as binary floats. 
In fact, Decimal often suffers from larger rounding errors than binary, which 
is why serious numeric data scientists still use binary float.

The advantages of Decimal over binary float is that you can configure how much 
precision is used, and that any number you can write out exactly in base 10 can 
be represented exactly as a Decimal. But it still has to round off values which 
cannot be represented exactly as Decimals.

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Ivan Pozdeev]

Change by Ivan Pozdeev :


--
pull_requests: +12114

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33944] Deprecate and remove pth files

[Ivan Pozdeev]

Ivan Pozdeev  added the comment:

On second thought, the inability to debug code that runs at startup, before 
user code ever gets control, is a fundamental issue (this problem arises for 
any software that has startup code), so such a facility in stock codebase has a 
merit.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29539] [smtplib] collect response data for all recipients

[sls]

sls  added the comment:

Did you make a PR ? I didn't notice this issue and did a quick fix: 
https://github.com/python/cpython/pull/12104

I was hoping this or similar fixes would be implemented in 3.7.x

Anyways. I'd suggest to return both success and errors for each recipient. 
sendmail returns a dict already. Why not adding each status?

Maybe using something like "mta_result" as variable instead of senderrs, which 
is in my opinion more clear to understand. Also, this would have the advantage 
to parse the return value easily and you're able to access more verbose 
information like mta msg id and so on.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29571] test_re is failing when local is set for `en_IN`

[STINNER Victor]

STINNER Victor  added the comment:

> This seems to have broken test_re on Windows, see 
> https://ci.appveyor.com/project/python/cpython/build/3.7.0a0.1

It seems like the ANSI code page is 1252 ("cp1252").

== CPython 3.7.0a0 (master:d31b28e16a2387d0251df948ef5d1b33d4357652, Mar 5 
2017, 21:47:06) [MSC v.1900 32 bit (Intel)]
==   Windows-2012ServerR2-6.3.9600-SP0 little-endian
==   hash algorithm: siphash24 32bit
==  cwd: C:\projects\cpython\build\test_python_1844
==  encodings: locale=cp1252, FS=utf-8
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, 
dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, 
verbose=0, bytes_warning=2, quiet=0, hash_randomization=1, isolated=0)
Using random seed 5949816

...

FAIL: test_locale_flag (test.test_re.ReTests)
--
Traceback (most recent call last):
  File "C:\projects\cpython\lib\test\test_re.py", line 1422, in test_locale_flag
self.assertTrue(pat.match(bletter))
AssertionError: None is not true

> getpreferredencoding() takes a completely different path on windows
> (returns a codepage) and isn't related to the C locale.

On my Windows 10 with Python 3.8, getpreferredencoding() (and 
getpreferredencoding(False)) returns "cp1252", getlocale(LC_CTYPE)[1] returns 
"1252". Python has an alias "1252" for "cp1252".

On Windows, getpreferredencoding() is implemented as 
_locale._getdefaultlocale()[1]. _getdefaultlocale()[1] is implemented with:

PyOS_snprintf(encoding, sizeof(encoding), "cp%d", GetACP());

At the end, it's the ANSI code page (1252).

--

I don't understand how the change ace5c0fdd9b962e6e886c29dbcea72c53f051dc4 
introduced a regression. And so I don't understand how commit 
21a74312f2d1ddee71fade709af49d078085ec30 (revert) could fix anything.

--

On my PR 12099, two Windows CI run and both succeeded:

* AppVeyor: pythoninfo says "locale.encoding: cp1252"
  https://ci.appveyor.com/project/python/cpython/builds/22726025
* Windows PR Tests on Azure Pipeline: pythoninfo also says "locale.encoding: 
cp1252"

When the change ace5c0fdd9b962e6e886c29dbcea72c53f051dc4 was merged, Python had 
no working Windows CI. Things evolved at lot in the meanwhile.

I also tested manually my PR 12099 on my Windows 10 VM which also uses cp1252: 
test_re pass.

--

re.LOCALE flag of re.compile() for a bytes pattern uses the following function 
of Modules/_sre.c:

LOCAL(int)
char_loc_ignore(SRE_CODE pattern, SRE_CODE ch)
{
return ch == pattern
|| (SRE_CODE) sre_lower_locale(ch) == pattern
|| (SRE_CODE) sre_upper_locale(ch) == pattern;
}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29571] test_re is failing when local is set for `en_IN`

[STINNER Victor]

STINNER Victor  added the comment:


New changeset ab71f8b793f7b42853ccd2a127ae7720adc5bcb4 by Victor Stinner in 
branch 'master':
bpo-29571: Fix test_re.test_locale_flag() (GH-12099)
https://github.com/python/cpython/commit/ab71f8b793f7b42853ccd2a127ae7720adc5bcb4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29571] test_re is failing when local is set for `en_IN`

[miss-islington]

Change by miss-islington :


--
pull_requests: +12115

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35975] Put back the ability to parse files where async/await aren't keywords

[Guido van Rossum]

Guido van Rossum  added the comment:

> Would not be simpler to just drop the support of Python versions <3.7 in new 
> MyPy versions?

Not really -- mypy has a lot of users who run it over (very) large code bases 
that can't easily be upgraded. Basically mypy has to support all Python 
versions that haven't reached their end of life yet. (And it's also important 
that mypy not be constrained to the same Python version as the target code.)

I suppose we could just stick with the existing typed_ast that supports 3.4 
through 3.7, but we actually have a use case for the end_lineno and 
end_col_offset fields that were just added to ast in 3.8, and in general we'd 
like to support any future grammar elements and ast features.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36152] IDLE: Remove close_when_done from colorizer close()

[Cheryl Sabella]

New submission from Cheryl Sabella :

Remove the unused `close_when_done` parameter from `close()` in 
`colorizer.ColorDelegator()`.

* The second parameter to close() is called `close_when_done` and it is 
expected to contain a toplevel widget that has a destroy() method.

* Originally, the editor window had code that would send self.top (if 
colorizing was in process) as the value for this parameter:

doh = colorizing and self.top
self.color.close(doh) # Cancel colorization

* This was changed via this commit 
(https://github.com/python/cpython/commit/8ce8a784bd672ba42975dec752848392ff9a7797)
 in 2007 to instead be:
 
self.color.close(False)
self.color = None

The value of `False` made it so the destroy code in colorizer wouldn't be run 
even though `None` or leaving the parameter off would have been more clear.

In any case, this `close_when_done` hasn't been used since 2007.

--
assignee: cheryl.sabella
components: IDLE
messages: 336880
nosy: cheryl.sabella, terry.reedy
priority: normal
severity: normal
status: open
title: IDLE: Remove close_when_done from colorizer close()
type: enhancement
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36152] IDLE: Remove close_when_done from colorizer close()

[Cheryl Sabella]

Change by Cheryl Sabella :


--
keywords: +patch
pull_requests: +12116
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com