[issue30157] csv.Sniffer.sniff() regex error

[Louie Lu]

Changes by Louie Lu :


--
title: csn.Sniffer.sniff() regex error -> csv.Sniffer.sniff() regex error

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30153] lru_cache should support invalidations

[Raymond Hettinger]

Raymond Hettinger added the comment:

If you're interested, here is a starting point for experimenting with any 
variations you want (invalidate a specific entry, changeable maxsize, 
pickle/unpickle, expiration of entries after a specific time, inspection of the 
internal contents, ability to inject known values, testing whether a specific 
argument tuple is in the cache, logging of cache access, or just about anything 
you could do with a regular dictionary):

class LRU(OrderedDict):

def __init__(self, func, maxsize=128):
self.maxsize = 128
self.func = func

def __call__(self, *args):
if args in self:
value = self[args]
self.move_to_end(args)
return value
value = self.func(*args)
if len(self) >= self.maxsize:
self.popitem(False)
self[args] = value
return value

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29617] Drop Python 3.4 support from asyncio

[INADA Naoki]

Changes by INADA Naoki :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29617] Drop Python 3.4 support from asyncio

[INADA Naoki]

INADA Naoki added the comment:


New changeset 3e2ad8ec61a322370a6fbdfb2209cf74546f5e08 by INADA Naoki in branch 
'master':
bpo-29617: Remove Python 3.3 support from asyncio (GH-232)
https://github.com/python/cpython/commit/3e2ad8ec61a322370a6fbdfb2209cf74546f5e08


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30052] URL Quoting page links to function Bytes instead of defintion

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
stage:  -> patch review
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30157] csn.Sniffer.sniff() regex error

[Jake Davis]

New submission from Jake Davis:

Line 220 of Lib/csv.py has an extra `>` in the first group:

r'(?P>[^\w\n"\'])

--
components: Library (Lib)
messages: 292249
nosy: jcdavis1983
priority: normal
pull_requests: 1389
severity: normal
status: open
title: csn.Sniffer.sniff() regex error
versions: Python 2.7, Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30153] lru_cache should support invalidations

[Raymond Hettinger]

Raymond Hettinger added the comment:

Sorry Jesús, I don't want to add feature creep to the LRU cache.

FWIW, it is very easy to make your own variants from collections.OrderedDict.

--
resolution:  -> rejected
stage: needs patch -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25949] Lazy creation of __dict__ in OrderedDict

[Jesús Cea Avión]

Changes by Jesús Cea Avión :


--
nosy: +jcea

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30132] [Windows] test_distutils leaks a vc140.pdb file

[STINNER Victor]

STINNER Victor added the comment:


New changeset f6448e5d65c349576df6e83b8324b9c208e77615 by Victor Stinner in 
branch 'master':
bpo-30132: distutils test_build_ext() uses temp_cwd() (#1278)
https://github.com/python/cpython/commit/f6448e5d65c349576df6e83b8324b9c208e77615


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30132] [Windows] test_distutils leaks a vc140.pdb file

[STINNER Victor]

Changes by STINNER Victor :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30107] python.core file created when running tests on AMD64 FreeBSD CURRENT Non-Debug 3.x buildbot

[STINNER Victor]

STINNER Victor added the comment:

The commit 2a1aed04b0943636f605543522e16cca1dc23e70 introduced a regression on 
macOS. The PR https://github.com/python/cpython/pull/1279 should fix it.

http://buildbot.python.org/all/builders/x86-64%20El%20Capitan%203.x/builds/87/steps/test/logs/stdio

==
FAIL: test_daemon_threads_shutdown_stderr_deadlock (test.test_io.CMiscIOTest)
--
Traceback (most recent call last):
  File 
"/Users/buildbot/buildarea/3.x.billenstein-elcapitan/build/Lib/test/test_io.py",
 line 3772, in test_daemon_threads_shutdown_stderr_deadlock
self.check_daemon_threads_shutdown_deadlock('stderr')
  File 
"/Users/buildbot/buildarea/3.x.billenstein-elcapitan/build/Lib/test/test_io.py",
 line 3766, in check_daemon_threads_shutdown_deadlock
self.assertFalse(err.strip('.!'))
AssertionError: '2017-04-24 16:57:21.432 defaults[41046:2462851] \nThe 
domain/default pair of (com.apple.CrashReporter, DialogType) does not exist\n' 
is not false

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30107] python.core file created when running tests on AMD64 FreeBSD CURRENT Non-Debug 3.x buildbot

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +1388

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1662581] the re module can perform poorly: O(2**n) versus O(n**2)

[Gregory P. Smith]

Changes by Gregory P. Smith :


--
versions: +Python 3.7 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1662581] the re module can perform poorly: O(2**n) versus O(n**2)

[Gregory P. Smith]

Gregory P. Smith added the comment:

Note that https://pypi.python.org/pypi/re2 exists today as well and offers a re 
module compatible interface.  I haven't tried it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30132] [Windows] test_distutils leaks a vc140.pdb file

[STINNER Victor]

STINNER Victor added the comment:

Ok, I found the guilty: test_build_ext, see linked PR which should fix the 
warning.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30132] [Windows] test_distutils leaks a vc140.pdb file

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +1387

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30124] Fix C aliasing issue in Python/dtoa.c to use strict aliasing on Clang 4.0

[STINNER Victor]

STINNER Victor added the comment:

> As an experiment, and to show what would be needed (at minimum), I have 
> attempted to make Python/dtoa.c completely aliasing-safe here:
> https://github.com/DimitryAndric/cpython/commit/29c3f6f5cd771fce5630f127b9e7054593e3160c

Would it be technically possible to completely get ride of the union? For 
example, it would allow to replace:

   set_dval(rv, get_dval(rv) + sulp(rv, bc));
with:
   rv += sulp(rv, bc);

Is it possible to replace Big0 and Big1 with Big double?

etc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30124] Fix C aliasing issue in Python/dtoa.c to use strict aliasing on Clang 4.0

[Dimitry Andric]

Dimitry Andric added the comment:

There is a "#pragma clang optimize", but it only has the options "on" or "off", 
for specific source locations.  I guess that would defeat the purpose a little 
bit. :(

As an experiment, and to show what would be needed (at minimum), I have 
attempted to make Python/dtoa.c completely aliasing-safe here:

https://github.com/DimitryAndric/cpython/commit/29c3f6f5cd771fce5630f127b9e7054593e3160c

This allowed to remove the -fno-strict-aliasing flag again, and it succeeded 
all tests, even with clang 5.0.0.

It basically replaces the direct union member accesses with getters and 
setters, which do the right thing for clang.  Note that even though those 
getters and setters use memcpy(), this is actually completely optimized away in 
the resulting assembly.  (Old compilers might not fare that well, though.)

In any case, while these are mostly mechanical changes, it is still a lot of 
code churn, and it should really be reviewed by the original maintainer of 
dtoa, David M. Gay.  I have no idea whether he is still active, though...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30131] test_logging leaks a "dangling" thread

[STINNER Victor]

STINNER Victor added the comment:


New changeset 4bcfa3a2363c30d406c6826e291b30c33a649d18 by Victor Stinner in 
branch 'master':
bpo-30131: Cleanup threads in test_logging (#1275)
https://github.com/python/cpython/commit/4bcfa3a2363c30d406c6826e291b30c33a649d18


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30131] test_logging leaks a "dangling" thread

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +1386

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28851] namedtuples field_names sequence preferred

[Cheryl Sabella]

Cheryl Sabella added the comment:

Hope it's OK that I've made a PR for these changes.

--
nosy: +csabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28851] namedtuples field_names sequence preferred

[Cheryl Sabella]

Changes by Cheryl Sabella :


--
pull_requests: +1385

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29581] __init_subclass__ causes TypeError when used with standard library metaclasses (such as ABCMeta)

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
stage:  -> backport needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[STINNER Victor]

STINNER Victor added the comment:

Come on, yet another crash from property_descr_get()??? It's the 3rd time... Do 
we really need this micro-optimization?

Previous bugs and workarounds:

- issue #26811
- issue #24276

Using the FASTCALL calling convention, no temporary tuple is created to pass 
arguments if you use the _PyObject_FastCall() API and if the called function 
supports this calling convention.

Sadly, namedtuple.attr uses operator.itergetter, itemgetter_call() doesn't 
support the FASTCALL, and my tp_fastcall patch was rejected: issue #29259.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30148] Pathological regex behaviour

[Tim Peters]

Tim Peters added the comment:

Yes, that example takes time exponential in the number of blanks to (fail to) 
match - each time you add a blank to `input`, it essentially doubles the time 
required.

It's _possible_ for an implementation to deduce that `(\s+)+` is an insanely 
inefficient way to spell `\s+`, like it's _possible_ for an implementation to 
deduce that 10**10**10 - 10**10**10 is an insanely inefficient way to spell 0.

Python's does not.  To understand what's going on, Friedl's book "Mastering 
Regular Expressions" is an excellent source.

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30148] Pathological regex behaviour

[Jussi Pakkanen]

Jussi Pakkanen added the comment:

This is slow even when ignores is set to a non-empty value. It's not as slow 
but the real slowdown is in the whitespace regex. Here is a minimal sample:

input = '  abc'
re.search(r'(\s+)+d', input)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30095] HTMLCalendar allow custom classes

[Oz Tiram]

Oz Tiram added the comment:

Apparently, I forgot to push the second branch. It is now pushed.

I renamed the class attribute names on the v1 branch. I still need to do
the following:

 - Add docs
 - Fix existing tests and add new tests demonstrating the usage of the new 
attributes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The regression was added by the fix for issue26811. PR 1272 applies the 
alternate patch from issue26811. This doesn't harm the performance.

$ ./python.patched -m perf timeit -q --compare-to ./python.default -s "from 
collections import namedtuple; P = namedtuple('P', 'x y'); p = P(1, 2)" 
--duplicate 1000 "p.x"
Mean +- std dev: [python.default] 128 ns +- 7 ns -> [python.patched] 121 ns +- 
7 ns: 1.05x faster (-5%)

I thought about tests, but I don't know what is the best place for them. Seems 
other environment variables that controls debug output are not tested too.

--
nosy: +haypo
stage:  -> patch review
versions: +Python 3.5, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[Oren Tirosh]

Oren Tirosh added the comment:

In addition to fixing this - perhaps PYTHONDUMPREFS or something similar should 
be added to test automation? It is apparently capable of uncovering some bugs 
that none of the other reference and recnt debugging tools could find.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26811] segfault due to null pointer in tuple

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +1384

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +1383

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5115] Extend subprocess.kill to be able to kill process groups

[Eryk Sun]

Eryk Sun added the comment:

Using a Windows job object should be paired with the creation flag 
CREATE_SUSPENDED. Callers may also need CREATE_BREAKAWAY_FROM_JOB, but setting 
that creation flag shouldn't be integrated into Popen.

The child has to be created suspended to ensure it doesn't spawn another 
process and exit before it's added to the job. Once it's in the job, call 
ResumeThread to start it. 

On Windows Vista and 7, the child may need to break away from Python's current 
job, if allowed. These older versions of Windows don't implement nested jobs, 
so adding the child to a job will fail if it's already in one. The job used by 
py.exe for python.exe isn't a problem in this case since it's configured for 
child processes to automatically break away, but python.exe may have been 
started directly and added to a job that's not configured for silent breakaway.

--
nosy: +eryksun
versions: +Python 3.7 -Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30052] URL Quoting page links to function Bytes instead of defintion

[Cheryl Sabella]

Changes by Cheryl Sabella :


--
pull_requests: +1382

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30039] Resuming a 'yield from' stack is broken if a signal arrives in the middle

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
assignee:  -> yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30156] PYTHONDUMPREFS segfaults on exit

[Oren Tirosh]

New submission from Oren Tirosh:

Reproduce: 

Py_DEBUG build
PYTHONDUMPREFS=1 ./python -c pass
(big dump of reference information)
Segmentation fault 

git-bisected to commit 7822f151b68e40376af657d267ff774439d9adb9

--
components: Interpreter Core
messages: 292232
nosy: orent, serhiy.storchaka
priority: normal
severity: normal
status: open
title: PYTHONDUMPREFS segfaults on exit
type: crash
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30152] Reduce the number of imports for argparse

[Wolfgang Maier]

Changes by Wolfgang Maier :


--
nosy: +wolma

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29822] inspect.isabstract does not work on abstract base classes during __init_subclass__

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
stage: patch review -> backport needed
versions:  -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29822] inspect.isabstract does not work on abstract base classes during __init_subclass__

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset fcfe80ec2592fed8b3941c79056a8737abef7d3b by Serhiy Storchaka 
(Nate) in branch 'master':
bpo-29822: Make inspect.isabstract() work during __init_subclass__. (#678)
https://github.com/python/cpython/commit/fcfe80ec2592fed8b3941c79056a8737abef7d3b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30155] Add ability to get/set tzinfo on datetime instances in C API

[Daniel Lenski]

Changes by Daniel Lenski :


--
nosy: +dlenski

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30155] Add ability to get/set tzinfo on datetime instances in C API

[Anthony Tuininga]

New submission from Anthony Tuininga:

Right now there is no documented way to create a datetime instance with a 
tzinfo instance. The documented macros all hard code the value Py_None for the 
tzinfo parameter. Using the PyObject_Call() method instead of the macro for 
creating a datetime instance is ~5x slower.

In addition, there is no macro or method for getting the tzinfo from an 
existing datetime instance. Perhaps creating DATE_GET_TZINFO and 
TIME_GET_TZINFO would be acceptable?

The enhancement 10381 (http://bugs.python.org/issue10381) would also be needed.

I can provide a GitHub PR if that would be helpful. I first want to make sure 
that such an effort would be appreciated!

--
components: Library (Lib)
messages: 292230
nosy: atuining
priority: normal
severity: normal
status: open
title: Add ability to get/set tzinfo on datetime instances in C API
type: enhancement
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30153] lru_cache should support invalidations

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

This is a duplicate of rejected issue28178.

See also related issue17528, issue18577, issue19859, issue23030, issue26082, 
and issue28112.

--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29804] test_ctypes test_pass_by_value fails on arm64 (aarch64) architecture

[Chi Hsuan Yen]

Changes by Chi Hsuan Yen :


--
nosy: +Chi Hsuan Yen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30052] URL Quoting page links to function Bytes instead of defintion

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +georg.brandl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30153] lru_cache should support invalidations

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Could cache_clear() clear the entire cache or discard just the cached call 
without arguments?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30104] clang 4.0 miscompiles dtoa.c, giving broken float parsing and printing

[STINNER Victor]

STINNER Victor added the comment:

> I also note that -fno-strict-aliasing is being included in 2.7's compile 
> arguments

Ah right, PyObject structures of Python 2 doesn't respect strict aliasing...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30153] lru_cache should support invalidations

[Steven D'Aprano]

Steven D'Aprano added the comment:

Perhaps the existing ``cache_clear`` method could take optional arguments?

def cache_clear(self, *args, **kw):
if not (args or kw):
# clear the entire cache
else:
# clear just the cache entry for *args, **kw

--
components: +Library (Lib)
nosy: +rhettinger, steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28969] lru_cache is not threadsafe

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests:  -855

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26534] subprocess.check_output with shell=True ignores the timeout

[Martin Panter]

Martin Panter added the comment:

Issue 5115 is already open with patch that has an alternative API to the 
low-level “killpg” method. It also has a Windows implementation and tests. I 
suggest to focus this bug on the higher-level “kill_group” option.

--
dependencies: +Extend subprocess.kill to be able to kill process groups

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30124] Fix C aliasing issue in Python/dtoa.c to use strict aliasing on Clang 4.0

[Eric V. Smith]

Eric V. Smith added the comment:

> Unfortunately, as far as I know, clang still does not support
> function-level optimization pragmas.  Maybe it was implemented
> recently, but then you would still have to have a workaround>
> for older versions.

I realize the answer is probably "no", but I'll ask anyway. Do you know if they 
support whole-module optimization pragmas? That would still be better than 
mucking with makefiles to get a per-module flag.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30124] Fix C aliasing issue in Python/dtoa.c to use strict aliasing on Clang 4.0

[Dimitry Andric]

Dimitry Andric added the comment:

Note that gcc has documented accessing union members in this way as an 
"implementation defined" feature:
https://gcc.gnu.org/onlinedocs/gcc/Structures-unions-enumerations-and-bit-fields-implementation.html#Structures-unions-enumerations-and-bit-fields-implementation

They specifically mention an example that is very similar to the dtoa pattern 
as being allowed under *their* implementation, here:
https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#Type-punning

However, whether that means that it is generally allowed by all standards is 
unfortunately still rather vague.  I have seen more than one complaint that the 
standards committees should make this very explicit, instead of weaseling it 
into post-release footnotes.

That said, I'd like to reply to some other posters here.

Eric V. Smith writes:
> At most, I think Mark's idea to use -fno-strict-aliasing only on dtoa.c and 
> nowhere else would be the better approach.

Indeed, this is exactly the solution I chose for FreeBSD.  Just the one file 
that needs it is compiled with -fno-strict-aliasing.  The caveat is that this 
might not work when link-time optimization is in effect.

Serhiy Storchaka writes:
> Could we use Clang specific pragma in dtoa.c rather than a compiler option?

Unfortunately, as far as I know, clang still does not support function-level 
optimization pragmas.  Maybe it was implemented recently, but then you would 
still have to have a workaround for older versions.

STINNER Victor writes:
> If we decide to go for the -fno-strict-aliasing only for dtoa.c, I suggest to 
> use it also for GCC. GCC might decide to also optimize dtoa.c further in the 
> future.

Theoretically they could, but as I pointed out above, they explicitly document 
this as a feature of their union implementation.  I estimate the probability of 
them dropping this in the future to be near zero.

> I don't think that the flag has a major impact on performance if it's 
> restricted to dtoa.c, and it would simplify the build system to only have 
> "per compiler" flags. (Ex: Does ICC also "miscompile" dtoa?)

Disabling strict aliasing for just that file, or even just the one function it 
applies to (by splitting it off to a separate file, for instance) should not 
result in different assembly output, unless the compiler is very old and/or 
dumb.

> FreeBSD uses the following syntax to only add the flag on a specific C file. 
> Does it work with GNU and BSD make? (is it a "portable" syntax?)
> 
>CFLAGS.gdtoa_${src}+=-fno-strict-aliasing

No, this is specifically a feature of BSD's bsd.sys.mk infrastructure.  It's 
most likely not compatible with GNU make.

--
nosy: +dim

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30154] subprocess.run with stderr connected to a pipe won't timeout when killing a never-ending shell commanad

[Martin Panter]

Martin Panter added the comment:

This is similar to the problem described in Issue 26534, which proposes 
“kill_group” and “killpg” APIs as a solution.

(FYI you should put a shebang at the start of the shell script, or call it as 
“sh -c test.sh”, to fix the “Exec format error”.)

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26534] subprocess.check_output with shell=True ignores the timeout

[Martin Panter]

Martin Panter added the comment:

I don’t know enough about process groups and sessions to properly review, but 
some things that stand out:

* Patch is missing documentation and tests
* If the “killpg” just wraps os.killpg, perhaps adding the method is not 
justified
* Are there any race conditions with killing a process group that has already 
exited. When does a process group get freed and potentially reused (so you may 
kill the wrong group)? The “send_signal” method (and others) have a check to 
avoid signalling an unrelated process.
* The method is called killpg, and the doc string mentions SIGKILL, but the 
implementation says SIGTERM
* What happens if you use killpg without starting a new session? If it kills 
the parent process as well, that sounds like a source of subtle bugs that may 
only be detected in unexpected cases (e.g. Ctrl+C or timeout)
* Be aware of Issue 25942. It is not clear what should happen to the child 
process(es) when the timeout happens, or when the “communicate” call is 
interrupted.
* What platforms does this support and what happens if there is no platform 
support?

--
nosy: +martin.panter
stage:  -> patch review
type: behavior -> enhancement
versions: +Python 3.7 -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30154] subprocess.run with stderr connected to a pipe won't timeout when killing a never-ending shell commanad

[Martijn Pieters]

Martijn Pieters added the comment:

Apologies, I copied the wrong sleep 10 demo. The correct demo is:

cat >test.sh< #!/bin/sh
> sleep 10
> EOF
time bin/python -c "import subprocess; subprocess.run(['./test.sh'], 
stderr=subprocess.PIPE, timeout=3)"
Traceback (most recent call last):
  File 
"/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.6/subprocess.py",
 line 405, in run
stdout, stderr = process.communicate(input, timeout=timeout)
  File 
"/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.6/subprocess.py",
 line 836, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
  File 
"/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.6/subprocess.py",
 line 1497, in _communicate
self._check_timeout(endtime, orig_timeout)
  File 
"/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.6/subprocess.py",
 line 864, in _check_timeout
raise TimeoutExpired(self.args, orig_timeout)
subprocess.TimeoutExpired: Command '['./test.sh']' timed out after 3 seconds

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "", line 1, in 
  File 
"/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.6/subprocess.py",
 line 410, in run
stderr=stderr)
subprocess.TimeoutExpired: Command '['./test.sh']' timed out after 3 seconds

real0m10.054s
user0m0.033s
sys 0m0.015s

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30152] Reduce the number of imports for argparse

[Berker Peksag]

Berker Peksag added the comment:

> The patch also makes argparse itself be imported only when the module
> is used as a script, not just imported.

+1. I'd move this into its own PR.

--
nosy: +berker.peksag

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30104] clang 4.0 miscompiles dtoa.c, giving broken float parsing and printing

[Kubilay Kocak]

Kubilay Kocak added the comment:

2.7 branch on koobs-freebsd-current worker doesn't appear to be failing 
currently, but there may be differences in (or a lack of equivalent) coverage 
compared to 3.*.

I also note that -fno-strict-aliasing is being included in 2.7's compile 
arguments:

cc -pthread -c -fno-strict-aliasing -OPT:Olimit=0 -g -O2 -g -O0 -Wall 
-Wstrict-prototypes

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30154] subprocess.run with stderr connected to a pipe won't timeout when killing a never-ending shell commanad

[Martijn Pieters]

New submission from Martijn Pieters:

You can't time out a process tree that includes a never-ending process, *and* 
which redirects stderr:

cat >test.sh< /dev/null  # never-ending
EOF
chmod +x test.sh
python -c "import subprocess; subprocess.run(['./test.sh'], 
stderr=subprocess.PIPE, timeout=3)"

This hangs forever; the timeout kicks in, but then the kill on the child 
process fails and Python forever tries to read stderr, which won't produce 
data. See 
https://github.com/python/cpython/blob/v3.6.1/Lib/subprocess.py#L407-L410. The 
`sh` process is killed, but listed as a zombie process and the `cat` process 
has migrated to parent id 1:

^Z
bg
jobs -lr
[2]- 21906 Running bin/python -c "import subprocess; 
subprocess.run(['./test.sh'], stderr=subprocess.PIPE, timeout=3)" &
pstree 21906
-+= 21906 mjpieters bin/python -c import subprocess; 
subprocess.run(['./test.sh'], stderr=subprocess.PIPE, timeout=3)
 \--- 21907 mjpieters (sh)
ps -j | grep 'cat /dev/random'
mjpieters 24706 1 24704  01 Rs0030:26.54 cat /dev/random
mjpieters 24897 99591 24896  02 R+   s0030:00.00 grep cat 
/dev/random

Killing Python at that point leaves the `cat` process running indefinitely.

Replace the `cat /dev/random > /dev/null` line with `sleep 10`, and the 
`subprocess.run()` call returns after 10+ seconds:

cat >test.sh<", line 1, in 
  File 
"/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.6/subprocess.py",
 line 403, in run
with Popen(*popenargs, **kwargs) as process:
  File 
"/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.6/subprocess.py",
 line 707, in __init__
restore_signals, start_new_session)
  File 
"/Users/mjpieters/Development/Library/buildout.python/parts/opt/lib/python3.6/subprocess.py",
 line 1326, in _execute_child
raise child_exception_type(errno_num, err_msg)
OSError: [Errno 8] Exec format error

real0m12.326s
user0m0.041s
sys 0m0.018s

When you redirect stdin instead, `process.communicate()` does return, but the 
`cat` subprocess runs on indefinitely nonetheless; only the `sh` process was 
killed.

Is this something subprocess.run should handle better (perhaps by adding in a 
second timeout poll and a terminate())? Or should the documentation be updated 
to warn about this behaviour instead (with suitable advice on how to write a 
subprocess that can be killed properly).

--
components: Library (Lib)
messages: 292217
nosy: mjpieters
priority: normal
severity: normal
status: open
title: subprocess.run with stderr connected to a pipe won't timeout when 
killing a never-ending shell commanad
type: behavior
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30153] lru_cache should support invalidations

[Jesús Cea Avión]

New submission from Jesús Cea Avión:

I think that "functools.lru_cache()" should have the ability to "invalidate" a 
(possibly cached) value. Something like:

@functools.lru_cache()
def func(param):
  ...

func.invalidate(PARAM) # discard this cached call, or ignore if not cached

--
messages: 292216
nosy: jcea
priority: normal
severity: normal
stage: needs patch
status: open
title: lru_cache should support invalidations
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28969] lru_cache is not threadsafe

[Jesús Cea Avión]

Changes by Jesús Cea Avión :


--
nosy: +jcea

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30150] raw debug allocators to not return malloc alignment

[Mattia Rizzolo]

Changes by Mattia Rizzolo :


--
nosy: +mapreri

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30052] URL Quoting page links to function Bytes instead of defintion

[Cheryl Sabella]

Cheryl Sabella added the comment:

Since no one has said that this change shouldn't be made, I'll start working on 
this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29751] PyLong_FromString documentation wrong on numbers with leading zero and base=0

[Cheryl Sabella]

Cheryl Sabella added the comment:

Oh, I didn't expect that.  That is so cool!  Thanks Mariatta.  :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30095] HTMLCalendar allow custom classes

[Oz Tiram]

Oz Tiram added the comment:

@Walter,

I agree with you about consistent naming. Personally, I would prefer to name 
all the variables "styles". But I preferred not to break past compatibility 
too. So I guess we are stuck for a while with "classes". I will rename the 
variables. As for documenting that, I would also do that.
Thanks for the feedback.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30095] HTMLCalendar allow custom classes

[Walter Dörwald]

Walter Dörwald added the comment:

The second link is a 404.

For the v1 patch:

The variable names are a bit inconsistent: The first uses "classes" all others 
use "styles". This should be consistent within itself and with the existing 
code, i.e. "classes" should be used.

Also each class attribute should be preceded with a comment, explaining what 
the CSS class is used for.

As these are now made public to be overwritten by subclasses, I wonder wether 
it would make sense to document these class attributes in 
Doc/library/calendar.rst

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30124] Fix C aliasing issue in Python/dtoa.c to use strict aliasing on Clang 4.0

[STINNER Victor]

STINNER Victor added the comment:

> Could we use Clang specific pragma in dtoa.c rather than a compiler option?

If we decide to go for the -fno-strict-aliasing only for dtoa.c, I suggest to 
use it also for GCC. GCC might decide to also optimize dtoa.c further in the 
future. I don't think that the flag has a major impact on performance if it's 
restricted to dtoa.c, and it would simplify the build system to only have "per 
compiler" flags. (Ex: Does ICC also "miscompile" dtoa?)

FreeBSD uses the following syntax to only add the flag on a specific C file. 
Does it work with GNU and BSD make? (is it a "portable" syntax?)

   CFLAGS.gdtoa_${src}+=-fno-strict-aliasing

See https://svnweb.freebsd.org/changeset/base/313706 (linked from 
http://bugs.python.org/issue30104#msg292001).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30152] Reduce the number of imports for argparse

[STINNER Victor]

Changes by STINNER Victor :


--
nosy: +Chi Hsuan Yen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29442] Replace optparse with argparse in setup.py

[STINNER Victor]

STINNER Victor added the comment:

I consider that the issue #30152 is now a dependency of this change.

--
dependencies: +Reduce the number of imports for argparse

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30104] clang 4.0 miscompiles dtoa.c, giving broken float parsing and printing

[STINNER Victor]

STINNER Victor added the comment:

> This appears to require backporting to 3.6 and 3.5 which are also failing:

I'm aware of that and I plan to fix these versions as well (2.7 also, no?), but 
I would like to first decide if we "fix" dtoa.c aliasing, or if we restrict 
-fno-strict-aliasing flag on dtoa.c: see discussion on the issue #30124. It 
seems like restricting the flag to dtoa.c is the favorite option.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30104] clang 4.0 miscompiles dtoa.c, giving broken float parsing and printing

[Kubilay Kocak]

Changes by Kubilay Kocak :


--
stage:  -> backport needed
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30104] clang 4.0 miscompiles dtoa.c, giving broken float parsing and printing

[Kubilay Kocak]

Kubilay Kocak added the comment:

Thank you for investigating Victor.

This appears to require backporting to 3.6 and 3.5 which are also failing:

FAIL: test_repr (test.test_float.ReprTestCase)
--
Traceback (most recent call last):
  File 
"/usr/home/buildbot/python/3.5.koobs-freebsd-current.nondebug/build/Lib/test/test_float.py",
 line 674, in test_repr
self.assertEqual(v, eval(repr(v)))
AssertionError: 5.3110001e+245 != 5.3110002e+245

--
Ran 42 tests in 0.242s

FAILED (failures=1, skipped=1)
7 tests failed again:
test_cmath test_float test_json test_marshal test_math
test_statistics test_strtod

--
versions: +Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Martin Panter]

Martin Panter added the comment:

A while ago I wrote a patch targetting Issue 22359 that may be a starting point 
for “make regen”: . It 
pulled out three recipes into separate “phony” targets: “make graminit 
importlib importlib_external”.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11913] sdist refuses README.rst

[Berker Peksag]

Berker Peksag added the comment:

PR 563 has been merged.

--
assignee: merwok -> 
components:  -Distutils2
nosy: +berker.peksag, dstufft
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed
type: behavior -> enhancement
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29999] repr() of ImportError misses keyword arguments name and path

[Berker Peksag]

Berker Peksag added the comment:

Including 'path' may make the repr less readable in some cases. +1 for 'name', 
though.

--
nosy: +berker.peksag

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30124] Fix C aliasing issue in Python/dtoa.c to use strict aliasing on Clang 4.0

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Could we use Clang specific pragma in dtoa.c rather than a compiler option?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30124] Fix C aliasing issue in Python/dtoa.c to use strict aliasing on Clang 4.0

[Mark Dickinson]

Mark Dickinson added the comment:

@haypo: for your strict-aliasing notes collection, I highly recommend the 
recent paper "Detecting Strict Aliasing Violations" by P. Cuoq et. al.

http://trust-in-soft.com/wp-content/uploads/2017/01/vmcai.pdf

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30144] Import collections ABC from collections.abc rather than collections

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Issue30152 replaces some collections.abc with _collections_abc for performance 
reasons.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29442] Replace optparse with argparse in setup.py

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See also issue30152. It includes Chi's change for argparse (actually it was 
inspired by Chi's change) and much more.

--
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30152] Reduce the number of imports for argparse

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +1381

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30152] Reduce the number of imports for argparse

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Since argparse becomes more used as standard way of parsing command-line 
arguments, the number of imports involved when import argparse becomes more 
important. Proposed patch reduces that number by 10 modules.

Unpatched:

$ ./python -c 'import sys; s = set(sys.modules); import argparse; print(len(s), 
len(sys.modules), len(set(sys.modules) - s)); print(sorted(set(sys.modules) - 
s))'
35 65 30
['_collections', '_functools', '_heapq', '_locale', '_operator', '_sre', 
'_struct', 'argparse', 'collections', 'collections.abc', 'copy', 'copyreg', 
'enum', 'functools', 'gettext', 'heapq', 'itertools', 'keyword', 'locale', 
'operator', 're', 'reprlib', 'sre_compile', 'sre_constants', 'sre_parse', 
'struct', 'textwrap', 'types', 'warnings', 'weakref']

$ ./python -S -c 'import sys; s = set(sys.modules); import argparse; 
print(len(s), len(sys.modules), len(set(sys.modules) - s)); 
print(sorted(set(sys.modules) - s))'
23 61 38
['_collections', '_collections_abc', '_functools', '_heapq', '_locale', 
'_operator', '_sre', '_stat', '_struct', 'argparse', 'collections', 
'collections.abc', 'copy', 'copyreg', 'enum', 'errno', 'functools', 
'genericpath', 'gettext', 'heapq', 'itertools', 'keyword', 'locale', 
'operator', 'os', 'os.path', 'posixpath', 're', 'reprlib', 'sre_compile', 
'sre_constants', 'sre_parse', 'stat', 'struct', 'textwrap', 'types', 
'warnings', 'weakref']

Patched:

$ ./python -c 'import sys; s = set(sys.modules); import argparse; print(len(s), 
len(sys.modules), len(set(sys.modules) - s)); print(sorted(set(sys.modules) - 
s))'
35 55 20
['_collections', '_functools', '_locale', '_operator', '_sre', 'argparse', 
'collections', 'copyreg', 'enum', 'functools', 'itertools', 'keyword', 
'operator', 're', 'reprlib', 'sre_compile', 'sre_constants', 'sre_parse', 
'types', 'weakref']

$ ./python -S -c 'import sys; s = set(sys.modules); import argparse; 
print(len(s), len(sys.modules), len(set(sys.modules) - s)); 
print(sorted(set(sys.modules) - s))'
23 51 28
['_collections', '_collections_abc', '_functools', '_locale', '_operator', 
'_sre', '_stat', 'argparse', 'collections', 'copyreg', 'enum', 'errno', 
'functools', 'genericpath', 'itertools', 'keyword', 'operator', 'os', 
'os.path', 'posixpath', 're', 'reprlib', 'sre_compile', 'sre_constants', 
'sre_parse', 'stat', 'types', 'weakref']

The patch defers importing rarely used modules. For example textwrap and 
gettext are used only for output a help and error messages.

The patch also makes argparse itself be imported only when the module is used 
as a script, not just imported. The patch also replaces importing 
collections.abc with _collections_abc in some other basic modules (like 
pathlib), this could allow to avoid importing the collections package if it is 
not used.

Unavoided imports:

* functools is used in re for decorating _compile_repl with lru_cache.
* collections is used in functools for making CacheInfo a named tuple.
* enum is used in re for creating RegexFlag.
* types is used in enum for decorating some properties with 
DynamicClassAttribute.

--
components: Library (Lib)
messages: 292200
nosy: bethard, haypo, rhettinger, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Reduce the number of imports for argparse
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30141] If you forget to call do_handshake, then everything seems to work but hostname checking is disabled

[Christian Heimes]

Christian Heimes added the comment:

The PR doesn't fix all bugs with the current approach. In the auto-handshake 
case, the struct members peer_cert and handshake_done are not set correctly. 
I'll look into the matter. Perhaps I can set them in the handshake or verify 
callback.

if (self->peer_cert)
X509_free (self->peer_cert);
self->peer_cert = SSL_get_peer_certificate(self->ssl);
self->handshake_done = 1;

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30140] Binary arithmetic does not always call subclasses first

[Mark Dickinson]

Mark Dickinson added the comment:

> could you point me to where this logic is implemented in CPython's source?

Most of the relevant code is in Objects/abstract.c and Objects/typeobject.c.

A BINARY_ADD opcode (for example) ends up calling PyNumber_Add:

https://github.com/python/cpython/blob/v3.6.1/Objects/abstract.c#L913

which in turn calls binary_op1, where you see an explicit check for the nb_add 
slots of the two operands being identical:

https://github.com/python/cpython/blob/v3.6.1/Objects/abstract.c#L769-L770

For a user-defined class, the slots themselves are defined in typeobject.c. 
Here's where nb_add is defined:

https://github.com/python/cpython/blob/v3.6.1/Objects/typeobject.c#L5952

and here's the explicit check for overloading in the SLOT1BIN macro definition:

https://github.com/python/cpython/blob/v3.6.1/Objects/typeobject.c#L5796

There's also an explicit test for the arithmetic operation behaviour in 
Lib/test/test_descr.py. In short, I doubt this was ever a bug: everything 
points to this being a deliberate design decision. I hope someone on 
python-ideas can elaborate on the rationale behind that design decision (and 
also on why that rationale doesn't apply to comparisons).

In contrast, it does seem plausible to me that the *comparison* failure to 
check for an explicit override may have been accidental.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30151] Race condition in use of _PyOS_SigintEvent on windows

[Eryk Sun]

Changes by Eryk Sun :


--
components: +Windows
nosy: +paul.moore, steve.dower, tim.golden, zach.ware
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30151] Race condition in use of _PyOS_SigintEvent on windows

[Nathaniel Smith]

Nathaniel Smith added the comment:

Oh, I should also say that this isn't actually affecting me, I just figured 
that once I was aware of the bug it was worth making a record here. Might be a 
good starter bug for someone trying to get into CPython's internals :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30151] Race condition in use of _PyOS_SigintEvent on windows

[Nathaniel Smith]

New submission from Nathaniel Smith:

As pointed out in this stackoverflow answer: 
http://stackoverflow.com/a/43578450/
and since I seem to be collecting signal-handling bugs these days :-), there's 
a race condition in how the interpreter uses _PyOS_SigintEvent to allow 
control-C to break out of functions like time.sleep on Windows. Suppose we have 
a call to time.sleep(), and the user hits control-C while it's running.

What's supposed to happen is:

- the windows implementation of pysleep in Modules/timemodule.c does 
ResetEvent(hInterruptEvent)
- then it blocks waiting for the interrupt event to be set *or* the timeout to 
expire
- the C-level signal handler runs in a new thread, which sets the "hey a signal 
arrived" flag and then sets the event
- the main thread wakes up because the event is set, and runs 
PyErr_CheckSignals()
- this notices that that the signal has arrived and runs the Python-level 
handler, all is good

But what can happen instead is:

- before doing CALL_FUNCTION, the eval loop checks to see if any signals have 
arrived. They haven't.
- then the C implementation of time.sleep starts executing.
- then a signal arrives; the signal handler sets the flag and sets the event
- then the main thread clears the event again
- then it blocks waiting for the event to be set or the timeout to expire. But 
the C-level signal handler's already done and gone, so we don't realize that 
the flag is set and we should wake up and run the Python-level signal handler.

The solution is that immediately *after* calling 
ResetEvent(_PyOS_SigintEvent()) but *before* sleeping, we should call 
PyErr_CheckSignals(). This catches any signals that arrived before we called 
ResetEvent, and any signals that arrive after that will cause the event to 
become set and wake us up, so that eliminates the race condition.

This same race-y pattern seems to apply to appear in Modules/timemodule.c, 
Modules/_multiprocessing/semaphore.c, and Modules/_winapi.c. _winapi.c also 
handles the event in a weird way that doesn't make sense to me – if the user 
hits control-C it raises an OSError instead of running the signal handler? OTOH 
I *think* Modules/_io/winconsoleio.c already handles the race condition 
correctly, and I don't dare make a guess about whatever Parser/myreadline.c is 
doing.

--
components: Interpreter Core
messages: 292196
nosy: njs
priority: normal
severity: normal
status: open
title: Race condition in use of _PyOS_SigintEvent on windows
versions: Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30144] Import collections ABC from collections.abc rather than collections

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset 2e576f5aec1f8f23f07001e2eb3db9276851a4fc by Serhiy Storchaka in 
branch 'master':
bpo-30144: Import collections ABC from collections.abc rather than collections. 
(#1263)
https://github.com/python/cpython/commit/2e576f5aec1f8f23f07001e2eb3db9276851a4fc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23404] 'make touch' does not work with git clones of the source repository

[Nick Coghlan]

Nick Coghlan added the comment:

As Martin notes, my concern is that I can go for years without having to touch 
the low level files (since syntax changes are rare), so it's important to me to 
make it easy to remember and/or relearn how to regenerate them.

However, suitable make commands (e.g. a "make regen" target, with "make 
regen-X" subtargets) would qualify as "easy to remember", and would also be 
amenable to automated testing in combination with a `make clobber-gen` command.

(Tangent on target naming: I think 'rebuild' would be confusing, while 
'regenerate' is a bit long, especially if there are more specific subtargets. 
'regen' is just a shortened version of the latter)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com