[issue41395] pickle and pickletools cli interface doesn't close input and output file.
Adam added the comment: Hi, First-time contributor here, I've made a patch in follow-up to the discussions that happened in Amir's patch in regards to this. I'd appreciate it if someone would be able to take a look and review it! https://github.com/python/cpython/pull/32257 -- ___ Python tracker <https://bugs.python.org/issue41395> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41395] pickle and pickletools cli interface doesn't close input and output file.
Change by Adam : -- nosy: +achhina nosy_count: 7.0 -> 8.0 pull_requests: +30326 pull_request: https://github.com/python/cpython/pull/32257 ___ Python tracker <https://bugs.python.org/issue41395> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46863] Python 3.10 OpenSSL Configuration Issues
Adam added the comment: Many thanks Christian, that resolved the issue! I really appreciate your efforts here. -- ___ Python tracker <https://bugs.python.org/issue46863> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46863] Python 3.10 OpenSSL Configuration Issues
Adam added the comment: Many thanks Christian, see the attached for the output of the commands on Python 3.9.10 and 3.10.2, along with a diff removing version numbers and memory addresses. I've run the commands on the Ubuntu distribution, we can also run the same for the Centos VM, if helpful. There are a few differences in the outputs but nothing that appears obviously the cause. -- Added file: https://bugs.python.org/file50654/python_details.tar.gz ___ Python tracker <https://bugs.python.org/issue46863> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46863] Python 3.10 OpenSSL Configuration Issues
Adam added the comment: Update, the Pyenv team confirmed that they do not install OpenSSL in linux, its only installed for MacOS, and it should be built using the system OpenSSL within Linux. We're investigating further to attempt to debug the issue. Interestingly the OpenSSL build flags for both Python versions appear to be the same. `Trying link with OPENSSL_LDFLAGS=; OPENSSL_LIBS=-lssl -lcrypto; OPENSSL_INCLUDES=` I've attached the build logs for both the Python 3.9.10 and 3.10.2 build, in case you're able to review. Many thanks. https://github.com/pyenv/pyenv/issues/2257 -- Added file: https://bugs.python.org/file50653/python_builds.tar.gz ___ Python tracker <https://bugs.python.org/issue46863> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38854] Decorator with paren tokens in arguments breaks inspect.getsource
Change by Adam Hopkins : -- nosy: +ahopkins nosy_count: 3.0 -> 4.0 pull_requests: +29736 pull_request: https://github.com/python/cpython/pull/31605 ___ Python tracker <https://bugs.python.org/issue38854> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46873] inspect.getsource with some lambdas in decorators does not get the full source
Adam Hopkins added the comment: Duplicate of https://bugs.python.org/issue38854 Sorry I didn't come across our before submitting. -- resolution: -> duplicate stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue46873> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46873] inspect.getsource with some lambdas in decorators does not get the full source
Change by Adam Hopkins : -- keywords: +patch pull_requests: +29728 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31605 ___ Python tracker <https://bugs.python.org/issue46873> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46873] inspect.getsource with some lambdas in decorators does not get the full source
Adam Hopkins added the comment: Sorry about that. I am doing some more digging to see if I can find the route of it and a proposal for a non-breaking patch. The problem seems to be in BlockFinder.tokeneater. -- type: behavior -> versions: +Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue46873> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46873] inspect.getsource with some lambdas in decorators does not get the full source
New submission from Adam Hopkins : I believe the following produces an unexpected behavior: from inspect import getsource def bar(*funcs): def decorator(func): return func return decorator @bar(lambda x: bool(True), lambda x: False) async def foo(): ... print(getsource(foo)) The output shows only the decorator declaration and none of the function: @bar(lambda x: bool(True), lambda x: False) >From my investigation, it seems like this requires the following conditions to >be true: - lambdas are passed in decorator arguments - there is more than one lambda - at least one of the lambdas has a function call Passing the lambdas as default function arguments seems okay: async def foo(bar=[lambda x: bool(True), lambda x: False]): ... A single lambda seems okay: @bar(lambda x: bool(True)) async def foo(): ... Lambdas with no function calls also seem okay: @bar(lambda x: not x, lambda: True) async def foo(): ... Tested this on: - Python 3.10.2 - Python 3.9.9 - Python 3.8.11 - Python 3.7.12 -- messages: 414149 nosy: ahopkins2 priority: normal severity: normal status: open title: inspect.getsource with some lambdas in decorators does not get the full source versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46873> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46863] Python 3.10 OpenSSL Configuration Issues
Adam added the comment: Yes agreed, it may well be a Pyenv issue. Interestingly we can demonstrate that the global OpenSSL crypto policies is respected with the 3.9.10 version, through adjusting the policy. The ssl error occurs with the default policy setting and is resolved with the legacy policy setting. With 3.10.2 this is no longer the case. I can’t see any obvious changes to the build recipe that would cause this. -- ___ Python tracker <https://bugs.python.org/issue46863> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46863] Python 3.10 OpenSSL Configuration Issues
Adam added the comment: I found the Python build recipes and Pyenv does appear to install OpenSSL from source. The only difference I can see, aside from the Python version, is an update on the OpenSSL versions; openssl-1.1.1l (3.9.10) to openssl-1.1.1k (3.10.2). The OpenSSL release notes do not appear to suggest anything relevant. https://github.com/pyenv/pyenv/blob/master/plugins/python-build/share/python-build/3.10.2 https://github.com/pyenv/pyenv/blob/master/plugins/python-build/share/python-build/3.9.10 https://github.com/pyenv/pyenv/blob/master/plugins/python-build/bin/python-build https://www.openssl.org/news/openssl-1.1.1-notes.html -- ___ Python tracker <https://bugs.python.org/issue46863> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46863] Python 3.10 OpenSSL Configuration Issues
Adam added the comment: Thanks for the quick reply. On both Ubuntu and Centos, I’m installing Python using Pyenv, testing with 3.9.10 and 3.10.2. Pyenv provides a verbose install flag, I can rebuild the Python versions and review the build commands, if helpful? I’m testing with clean Linux distributions and I believe there is only one OpenSSL installed and available. I don’t know if it’s possible to gain more details from the Python ssl module to confirm? I did confirm the OpenSSL versions aligns using ssl.OPENSSL_VERSION. Command: pyenv install 3.10.2 --verbose https://github.com/pyenv/pyenv -- ___ Python tracker <https://bugs.python.org/issue46863> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46863] Python 3.10 OpenSSL Configuration Issues
New submission from Adam Pinckard : Python 3.10 does not appear to respecting the OpenSSL configuration within linux. Testing completed using Pyenv on both Ubuntu 20.04.4 and Centos-8. Note PEP 644 which requires OpenSSL >= 1.1.1 is released in Python 3.10. We operate behind a corporate proxy / firewall which causes an SSL error where the Diffie-Hellman key size is too small. In previous Python versions this is resolved by updating the OpenSSL configuration, e.g. downgrading the linux crypto policies `sudo update-crypto-policies --set LEGACY`. The issue is reproducible in both Ubuntu 20.04.4 and Centos-8. In both linux distributions the SSL error is resolvable in earlier Python version, using the OpenSSL configurations, but the configuration is not respected with Python 3.10.2. See the details below on the kernel versions, linux distributions, and Openssl versions, many thanks in advance. 1. Python 3.10.2 Error: (py_3_10_2) ➜ py_3_10_2 pip install --upgrade pip WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: DH_KEY_TOO_SMALL] dh key too small (_ssl.c:997)'))': /simple/pip/ 2. Ubuntu details uname -a Linux Horatio 5.13.0-30-generic #33~20.04.1-Ubuntu SMP Mon Feb 7 14:25:10 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description:Ubuntu 20.04.4 LTS Release:20.04 Codename: focal openssl version -a OpenSSL 1.1.1f 31 Mar 2020 built on: Wed Nov 24 13:20:48 2021 UTC platform: debian-amd64 options: bn(64,64) rc4(16x,int) des(int) blowfish(ptr) compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-dnfdFp/openssl-1.1.1f=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_TLS_SECURITY_LEVEL=2 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2 OPENSSLDIR: "/usr/lib/ssl" ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-1.1" Seeding source: os-specific 2. Centos-8 details uname -a Linux localhost.localdomain 5.4.181-1.el8.elrepo.x86_64 #1 SMP Tue Feb 22 10:00:15 EST 2022 x86_64 x86_64 x86_64 GNU/Linux cat /etc/centos-release CentOS Stream release 8 openssl version -a OpenSSL 1.1.1k FIPS 25 Mar 2021 built on: Thu Dec 2 16:40:48 2021 UTC platform: linux-x86_64 options: bn(64,64) md2(char) rc4(16x,int) des(int) idea(int) blowfish(ptr) compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -O3 -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wa,--noexecstack -Wa,--generate-missing-build-notes=yes -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DZLIB -DNDEBUG -DPURIFY -DDEVRANDOM="\"/dev/urandom\"" -DSYSTEM_CIPHERS_FILE="/etc/crypto-policies/back-ends/openssl.config" OPENSSLDIR: "/etc/pki/tls" ENGINESDIR: "/usr/lib64/engines-1.1" Seeding source: os-specific engines: rdrand dynamic -- assignee: christian.heimes components: SSL messages: 414072 nosy: adam, christian.heimes priority: normal severity: normal status: open title: Python 3.10 OpenSSL Configuration Issues type: behavior versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue46863> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46467] Rounding 5, 50, 500 behaves differently depending on preceding value
New submission from Adam Ulrich : round(250,-2) returns 200 round(350,-2) returns 400 round(450,-2) returns 400 round(550,-2) returns 600 round(5,-1) returns 0 round(15,-1) returns 20 round(500,-3) returns 0 round(1500,-3) returns 2000 expected: round of 5 to consistently rounds up. -- components: Interpreter Core messages: 411222 nosy: adam.ulrich priority: normal severity: normal status: open title: Rounding 5,50,500 behaves differently depending on preceding value type: behavior versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue46467> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45865] Old syntax in unittest
Adam Johnson added the comment: Okay, I updated the PR to only remove inheritance from object. Should I reopen the ticket? (Not sure of the etiquette.) Perhaps I could later submit a second patch for use of `super()`, and so on? -- ___ Python tracker <https://bugs.python.org/issue45865> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25834] getpass falls back when sys.stdin is changed
Adam Bartoš added the comment: Sorry, I don't. But my use case is not relevant any more since my package was a workround for problems with entering Unicode interactively on Windows, and these problems were resolved in Python since then. -- ___ Python tracker <https://bugs.python.org/issue25834> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23882] unittest discovery doesn't detect namespace packages when given no parameters
Adam Johnson added the comment: I just reported https://bugs.python.org/issue45864 , and closed as duplicate of this. -- nosy: +adamchainz ___ Python tracker <https://bugs.python.org/issue23882> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45864] unittest does not discover tests in PEP420 packages
Change by Adam Johnson : -- stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue45864> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45864] unittest does not discover tests in PEP420 packages
Adam Johnson added the comment: It's exactly that ticket. I missed that when searching for duplicates - I only searched for "pep420" and not "namespace packages". Mea culpa. -- resolution: -> duplicate ___ Python tracker <https://bugs.python.org/issue45864> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45865] Old syntax in unittest
Change by Adam Johnson : -- keywords: +patch pull_requests: +27934 stage: -> patch review pull_request: https://github.com/python/cpython/pull/29698 ___ Python tracker <https://bugs.python.org/issue45865> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45865] Old syntax in unittest
New submission from Adam Johnson : I often browse the unittest code in order to write extensions. It still uses some Python 2-isms like classes inheriting from object, it would be nice to clean that up. -- components: Tests messages: 406757 nosy: adamchainz priority: normal severity: normal status: open title: Old syntax in unittest type: enhancement ___ Python tracker <https://bugs.python.org/issue45865> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45864] unittest does not discover tests in PEP420 packages
New submission from Adam Johnson : unittest's test discovery does not descend into directories without `__init__.py`. This avoids discovering test modules that are otherwise valid and importable, after PEP 420. I've seen this more than once where there were valid looking test files not being discovered, and they bit rot. The tests had been run individually when created but never again. (I created [flake8-no-pep420](https://pypi.org/project/flake8-no-pep420/) to avoid this problem on my projects.) For example, take this directory structure: ``` $ tree . └── tests └── test_thing.py 1 directory, 1 file $ cat tests/test_thing.py 1/0 ``` It's valid to import the naughty file, which crashes: ``` $ python -c 'import tests.test_thing' Traceback (most recent call last): File "", line 1, in File "/.../tests/test_thing.py", line 1, in 1/0 ZeroDivisionError: division by zero ``` But unittest does not discover it: ``` $ python -m unittest -- Ran 0 tests in 0.000s OK ``` But, after creating an empty `__init__.py`, the tests doth fail: ``` $ touch tests/__init__.py $ python -m unittest E == ERROR: tests.test_thing (unittest.loader._FailedTest) -- ImportError: Failed to import test module: tests.test_thing Traceback (most recent call last): File "/.../unittest/loader.py", line 436, in _find_test_path module = self._get_module_from_name(name) File "/.../unittest/loader.py", line 377, in _get_module_from_name __import__(name) File "/.../tests/test_thing.py", line 1, in 1/0 ZeroDivisionError: division by zero -- Ran 1 test in 0.000s FAILED (errors=1) ``` -- components: Tests messages: 406756 nosy: adamchainz priority: normal severity: normal status: open title: unittest does not discover tests in PEP420 packages type: behavior ___ Python tracker <https://bugs.python.org/issue45864> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45639] Support modern image formats in MIME types
Change by Adam Konrad : -- keywords: +patch pull_requests: +27523 stage: -> patch review pull_request: https://github.com/python/cpython/pull/29259 ___ Python tracker <https://bugs.python.org/issue45639> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45639] Support modern image formats in MIME types
New submission from Adam Konrad : Modern image types webp and avif are not recognized by the mimetypes module. Problem: Many tools are written in Python and running on macOS. Good example is the AWS CLI. Running commands like "s3 sync" will save files with .webp and .avif extensions with incorrect "binary/octet-stream" Content-Type to S3. This creates additional problems with serving these resources over HTTP. The webp and avif image types are supported by most browsers: https://caniuse.com/#feat=webp https://caniuse.com/#feat=avif While webp is fully supported and largely used, it is not officially registered with IANA. Avif is currently less popular, it is fully registered with IANA. https://www.iana.org/assignments/media-types/media-types.xhtml Please consider the attached GitHub PR as a fix to these MIME Content-Type issues. -- components: Library (Lib) messages: 405145 nosy: adamkonrad priority: normal severity: normal status: open title: Support modern image formats in MIME types type: enhancement versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue45639> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45337] Create venv with pip fails when target dir is under userappdata using Microsoft Store python
Change by Adam Yoblick : -- type: -> behavior ___ Python tracker <https://bugs.python.org/issue45337> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45337] Create venv with pip fails when target dir is under userappdata using Microsoft Store python
New submission from Adam Yoblick : Repro steps: 1. Install Python 3.9 from the Microsoft Store 2. Try to create a virtual environment under the userappdata folder, using a command line similar to the following: "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2032.0_x64__qbz5n2kfra8p0\python3.9.exe" -m venv "C:\Users\advolker\AppData\Local\Microsoft\CookiecutterTools\env" 3. Observe the following error: Error: Command '['C:\\Users\\advolker\\AppData\\Local\\Microsoft\\CookiecutterTools\\env\\Scripts\\python.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 106. Note that creating a venv without pip DOES work: "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2032.0_x64__qbz5n2kfra8p0\python3.9.exe" -m venv "C:\Users\advolker\AppData\Local\Microsoft\CookiecutterTools\env" --without-pip BUT the venv is NOT at the specified location. This is because the Windows Store app creates a redirect when creating the venv, and that redirect is only visible from within the python executable. This means that python doesn't respect the redirect when trying to install pip into the newly created venv. -- components: Windows messages: 402983 nosy: AdamYoblick, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Create venv with pip fails when target dir is under userappdata using Microsoft Store python versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue45337> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45235] argparse does not preserve namespace with subparser defaults
Change by Adam Schwalm : -- keywords: +patch pull_requests: +26832 stage: -> patch review pull_request: https://github.com/python/cpython/pull/28420 ___ Python tracker <https://bugs.python.org/issue45235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45235] argparse does not preserve namespace with subparser defaults
New submission from Adam Schwalm :
The following snippet demonstrates the problem. If a subparser flag has a
default set, argparse will override the existing value in the provided
'namespace' if the flag does not appear (e.g., if the default is used):
import argparse
parser = argparse.ArgumentParser()
sub = parser.add_subparsers()
example_subparser = sub.add_parser("example")
example_subparser.add_argument("--flag", default=10)
print(parser.parse_args(["example"], argparse.Namespace(flag=20)))
This should return 'Namespace(flag=20)' because 'flag' already exists in the
namespace, but instead it returns 'Namespace(flag=10)'. This intended behavior
is described and demonstrated in the second example here:
https://docs.python.org/3/library/argparse.html#default
Lib's behavior is correct for the non-subparser cause.
--
components: Library (Lib)
messages: 402060
nosy: ALSchwalm
priority: normal
severity: normal
status: open
title: argparse does not preserve namespace with subparser defaults
type: behavior
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python
3.9
___
Python tracker
<https://bugs.python.org/issue45235>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28474] WinError(): Python int too large to convert to C long
Change by Adam Meily : -- keywords: +patch pull_requests: +26407 stage: -> patch review pull_request: https://github.com/python/cpython/pull/27959 ___ Python tracker <https://bugs.python.org/issue28474> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27484] Some Examples in Format String Syntax are incorrect or poorly worded
Change by Adam Meily : -- nosy: +meilyadam nosy_count: 5.0 -> 6.0 pull_requests: +26405 pull_request: https://github.com/python/cpython/pull/27959 ___ Python tracker <https://bugs.python.org/issue27484> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28474] WinError(): Python int too large to convert to C long
Adam Meily added the comment: I can potentially take a stab at writing up a PR for this. I've also seen this affecting other locations that eventually call FormatMessage, including: - ctypes.format_error() - this original issue - os.strerror() - OSError(winerror=X) I will most likely look into fixing all three. -- nosy: +meilyadam ___ Python tracker <https://bugs.python.org/issue28474> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44253] tkinter searches for tk.tcl in wrong directory
Adam Stewart added the comment: Thanks, that does help. Spack uses both `--with-tcltk-includes` and `--with-tcltk-libs`, and actually RPATHs the libraries in place. According to otool, that is all working fine: $ otool -L /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/python-3.8.10-fkj5vkn3tpottyv6yqoj5ucz2emstpvo/lib/python3.8/lib-dynload/_tkinter.cpython-38-darwin.so /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/python-3.8.10-fkj5vkn3tpottyv6yqoj5ucz2emstpvo/lib/python3.8/lib-dynload/_tkinter.cpython-38-darwin.so: /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tcl-8.6.11-n7nea33urrk25rkoqpsc2tdcgai5u4z2/lib/libtcl8.6.dylib (compatibility version 8.6.0, current version 8.6.11) /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tk-8.6.11-ydmhrbboheucxsuhrnyoxqaihgna5dfe/lib/libtk8.6.dylib (compatibility version 8.6.0, current version 8.6.11) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1) So like you initially thought, the problem isn't that tkinter/_tkinter can't find tcl, it's that tcl can't find tk. I'll talk more with the tcl developers and see how tcl is trying to find tk. Thanks for all of your help! -- ___ Python tracker <https://bugs.python.org/issue44253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44253] tkinter searches for tk.tcl in wrong directory
Adam Stewart added the comment: And... now it's not working again. Can you clarify exactly how tkinter finds tk/tcl? Does it rely on TCL_LIBRARY or TK_LIBRARY env vars? TCLLIBPATH? If I use all of these env vars, tkinter finds tcl/tk, but commands like: $ python -m tkinter $ python -c 'import tkinter; tkinter._test()' open a window and immediately minimize it. If I try to maximize the window it immediately closes, so something is definitely wrong with my installation. -- ___ Python tracker <https://bugs.python.org/issue44253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44253] tkinter searches for tk.tcl in wrong directory
Adam Stewart added the comment: I think I FINALLY figured out the problem. We were setting `TCLLIBPATH` to `/lib/tk8.6` when it should be `/lib`. With this change, tkinter seems to work for me. Thanks for all of your help! -- ___ Python tracker <https://bugs.python.org/issue44253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44253] tkinter searches for tk.tcl in wrong directory
Adam Stewart added the comment: Thanks, in that case it sounds like the problem is that Spack installs tcl and tk to separate directories, but since tk depends on tcl and not the other way around, tcl has no way of knowing where tk is installed. I'll see if I can convince the other Spack devs to combine tcl and tk into a single package. -- ___ Python tracker <https://bugs.python.org/issue44253> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44253] tkinter searches for tk.tcl in wrong directory
New submission from Adam Stewart : I'm trying to install Python with tkinter support using the Spack package manager. Spack adds the following flags to configure during install: ``` '--with-tcltk-libs=-L/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tcl-8.6.11-n7nea33urrk25rkoqpsc2tdcgai5u4z2/lib -ltcl8.6 -L/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tk-8.6.11-ydmhrbboheucxsuhrnyoxqaihgna5dfe/lib -ltk8.6' ``` It also sets the following environment variables: ``` TCLLIBPATH='/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tcl-8.6.11-n7nea33urrk25rkoqpsc2tdcgai5u4z2/lib/tcl8.6 /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tk-8.6.11-ydmhrbboheucxsuhrnyoxqaihgna5dfe/lib/tcl8.6 /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tk-8.6.11-ydmhrbboheucxsuhrnyoxqaihgna5dfe/lib64/tcl8.6'; export TCLLIBPATH TCL_LIBRARY=/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tcl-8.6.11-n7nea33urrk25rkoqpsc2tdcgai5u4z2/lib; export TCL_LIBRARY ``` The install seems to correctly pick up tk/tcl and builds correctly. However, when I try to use tkinter, I see the following run-time error: ``` $ python Python 3.8.10 (default, May 27 2021, 13:28:01) [Clang 12.0.0 (clang-1200.0.32.29)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import tkinter >>> tkinter._test() Traceback (most recent call last): File "", line 1, in File "/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/python-3.8.10-fkj5vkn3tpottyv6yqoj5ucz2emstpvo/lib/python3.8/tkinter/__init__.py", line 4557, in _test root = Tk() File "/Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/python-3.8.10-fkj5vkn3tpottyv6yqoj5ucz2emstpvo/lib/python3.8/tkinter/__init__.py", line 2270, in __init__ self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use) _tkinter.TclError: Can't find a usable tk.tcl in the following directories: /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tk-8.6.11-ydmhrbboheucxsuhrnyoxqaihgna5dfe/lib /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tcl-8.6.11-n7nea33urrk25rkoqpsc2tdcgai5u4z2/lib/tcl8.6/tk8.6 /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tcl-8.6.11-n7nea33urrk25rkoqpsc2tdcgai5u4z2/lib/tcl8.6/tk8.6/Resources/Scripts /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tk-8.6.11-ydmhrbboheucxsuhrnyoxqaihgna5dfe/lib/tcl8.6/tk8.6 /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tk-8.6.11-ydmhrbboheucxsuhrnyoxqaihgna5dfe/lib/tcl8.6/tk8.6/Resources/Scripts /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tk-8.6.11-ydmhrbboheucxsuhrnyoxqaihgna5dfe/lib64/tcl8.6/tk8.6 /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tk-8.6.11-ydmhrbboheucxsuhrnyoxqaihgna5dfe/lib64/tcl8.6/tk8.6/Resources/Scripts /Users/Adam/spack/opt/spack/darwin-ca talina-x86_64/apple-clang-12.0.0/tcl-8.6.11-n7nea33urrk25rkoqpsc2tdcgai5u4z2/lib/tk8.6 /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tcl-8.6.11-n7nea33urrk25rkoqpsc2tdcgai5u4z2/lib/tk8.6/Resources/Scripts /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tcl-8.6.11-n7nea33urrk25rkoqpsc2tdcgai5u4z2/tk8.6 /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/tcl-8.6.11-n7nea33urrk25rkoqpsc2tdcgai5u4z2/tk8.6/Resources/Scripts /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/python-3.8.10-fkj5vkn3tpottyv6yqoj5ucz2emstpvo/lib/tk8.6 /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/python-3.8.10-fkj5vkn3tpottyv6yqoj5ucz2emstpvo/lib/tk8.6/Resources/Scripts /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/python-3.8.10-fkj5vkn3tpottyv6yqoj5ucz2emstpvo/lib/tk8.6 /Users/Adam/spack/opt/spack/darwin-catalina-x86_64/apple-clang-12.0.0/lib/tk8.6 /Users/Adam/spack/opt/spack/dar win-catalina-x86_64/apple-clang-12.0.0/python-3.8.10-fkj5vkn3tpottyv6yqoj5ucz2emstpvo/library This probably means that tk wasn't installed properly. ``` It seems that tkinter searches for tk.tcl in `/lib`, but tk.tcl is actually installed in `/lib/tk8.6`. I asked the tk developers, but it looks like `/lib/tk8.6` is indeed the correct installation location: https://core.tcl-lang.org/tk/tktview/447bd3e4abe17452d19a80e6840dcc8a2603fcbc Is there a way to tell tkinter where to find tk.tcl? If not, can we modify the default search path to search in `/lib/tk8.6`? Related to https://github.com/spack/spack/issues/23780 -- components: Tkinter messages: 394584 nosy: ajstewart priority: normal severity: normal status: open title: tkinter searches for tk.tcl in wrong directory type: cr
[issue37658] In some cases asyncio.wait_for can lead to socket leak.
Adam Liddell added the comment: Wrapping every resource allocating call like that is what we were trying to avoid, since it makes wait_for go from a simple one-line helper to something you have to be very careful with. Conceptually, a user should expect that wait_for should behave the exact same as awaiting the underlying awaitable, just with auto-cancellation. The problem with the current wait_for is that there is a gap where the underlying task may have completed but a cancellation arrives. In this case, we need to raise the cancellation to be a good asyncio citizen, but the underlying task has no opportunity to act on the cancellation (to free the resource) since it is already complete and cannot be re-entered. So the resource returned by the completed task gets stuck in limbo, since we can't return it and we can't assume a generic 'close' behaviour. See my comment in the PR for a suggestion about an alternative structure for wait_for, which may avoid this gap and hence prevent the leak (but I have not tested it!) -- ___ Python tracker <https://bugs.python.org/issue37658> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37658] In some cases asyncio.wait_for can lead to socket leak.
Adam Liddell added the comment: Some discussion leading up to that change is here https://github.com/MagicStack/asyncpg/pull/548 and in the issues it links. -- ___ Python tracker <https://bugs.python.org/issue37658> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition
Change by Adam Liddell : -- nosy: +aaliddell ___ Python tracker <https://bugs.python.org/issue42130> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43486] Python 3.9 installer not updating ARP table
Adam added the comment: The 64 installer doesn't even show up in the ARP table, only Python Launcher. -- ___ Python tracker <https://bugs.python.org/issue43486> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43486] Python 3.9 installer not updating ARP table
New submission from Adam : 1. Install 3.9.0 using the following command line options: python-3.9.0.exe /quiet InstallAllUsers=1 2. Install 3.9.2 using the following command line options: python-3.9.2.exe /quiet InstallAllUsers=1 3. Observe that 3.9.2 successfully installed, however the ARP table does not reflect the latest version (see first screenshot in the attachment) it still shows 3.9.0 as installed. 4. Uninstall 3.9.2 using the following command line options: python-3.9.2.exe /uninstall /silent 5. Observe that Python 3.9.0 is still listed as installed in the ARP table. Looking in the registry, all Python installed products are removed except for Python Launcher. Maybe it is by design to leave Python Launcher on the system, maybe not, but I think keeping the ARP table tidy would reduce confusion for users. See second screenshot in the attachment. -- components: Installation files: 1.jpg messages: 388615 nosy: codaamok priority: normal severity: normal status: open title: Python 3.9 installer not updating ARP table type: behavior versions: Python 3.9 Added file: https://bugs.python.org/file49873/1.jpg ___ Python tracker <https://bugs.python.org/issue43486> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42967] [security] urllib.parse.parse_qsl(): Web cache poisoning - `; ` as a query args separator
Adam Goldschmidt added the comment: > The difference is that semicolon is defined in a previous specification. I understand, but this will limit us in the future if the spec changes - though I don't have strong feelings regarding this one. > Dear all, now that Adam has signed the CLA, I have closed my PR in favor of > Adam's because I think 2 open PRs might split everyone's attention. Instead, > I'll focus on reviewing Adam's PR. Sorry for any inconvenience caused. ❤ -- ___ Python tracker <https://bugs.python.org/issue42967> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42967] [security] urllib.parse.parse_qsl(): Web cache poisoning - `; ` as a query args separator
Adam Goldschmidt added the comment: > That doesn’t feel necessary to me. I suspect most links use &, some use ;, > nothing else is valid at the moment and I don’t expect a new separator to > suddenly appear. IMO the boolean parameter to also recognize ; was better. That's reasonable. However, I think that we are making this change in order to treat the semicolon as a "custom" separator. In that case, why not let the developer decide on a different custom separator for their own use cases? What's the difference between a semicolon and something else? -- ___ Python tracker <https://bugs.python.org/issue42967> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42967] [security] urllib.parse.parse_qsl(): Web cache poisoning - `; ` as a query args separator
Adam Goldschmidt added the comment: > I _didn't_ change the default - it will allow both '&' and ';' still. Eric > showed a link above that still uses semicolon. So I feel that it's strange to > break backwards compatibility in a patch update. Maybe we can make just '&' > the default in Python 3.10, while backporting the ability to specify > separators to older versions so it's up to users? I like this implementation. I definitely think we should not break backwards compatibility and only change the default in Python 3.10. -- ___ Python tracker <https://bugs.python.org/issue42967> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42967] [security] urllib.parse.parse_qsl(): Web cache poisoning - `; ` as a query args separator
Adam Goldschmidt added the comment: I haven't noticed, I'm sorry. I don't mind closing mine, just thought it could be a nice first contribution. Our PRs are different though - I feel like if we are to implement this, we should let the developer choose the separator and not limit to just `&` and `;` - but that discussion probably belongs in the PR. -- ___ Python tracker <https://bugs.python.org/issue42967> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42967] [security] urllib.parse.parse_qsl(): Web cache poisoning - `; ` as a query args separator
Change by Adam Goldschmidt : -- pull_requests: +23120 pull_request: https://github.com/python/cpython/pull/24297 ___ Python tracker <https://bugs.python.org/issue42967> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42967] Web cache poisoning - `;` as a query args separator
New submission from Adam Goldschmidt : The urlparse module treats semicolon as a separator (https://github.com/python/cpython/blob/master/Lib/urllib/parse.py#L739) - whereas most proxies today only take ampersands as separators. Link to a blog post explaining this vulnerability: https://snyk.io/blog/cache-poisoning-in-popular-open-source-packages/ When the attacker can separate query parameters using a semicolon (;), they can cause a difference in the interpretation of the request between the proxy (running with default configuration) and the server. This can result in malicious requests being cached as completely safe ones, as the proxy would usually not see the semicolon as a separator, and therefore would not include it in a cache key of an unkeyed parameter - such as `utm_*` parameters, which are usually unkeyed. Let’s take the following example of a malicious request: ``` GET /?link=http://google.com_content=1;link='>alert(1) HTTP/1.1 Host: somesite.com Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,imag e/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9 Accept-Encoding: gzip, deflate Accept-Language: en-US,en;q=0.9 Connection: close ``` urlparse sees 3 parameters here: `link`, `utm_content` and then `link` again. On the other hand, the proxy considers this full string: `1;link='>alert(1)` as the value of `utm_content`, which is why the cache key would only contain `somesite.com/?link=http://google.com`. A possible solution could be to allow developers to specify a separator, like werkzeug does: https://github.com/pallets/werkzeug/blob/6784c44673d25c91613c6bf2e614c84465ad135b/src/werkzeug/urls.py#L833 -- components: C API messages: 385266 nosy: AdamGold priority: normal severity: normal status: open title: Web cache poisoning - `;` as a query args separator type: security versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue42967> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18838] The order of interactive prompt and traceback on Windows
Adam Bartoš added the comment: The order is fine on Python 3.8, Windows 10. -- ___ Python tracker <https://bugs.python.org/issue18838> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue18838] The order of interactive prompt and traceback on Windows
Adam Bartoš added the comment:
So far I could reproduce the issue on Python 3.7, Windows Vista 64bit. I'll try
with newer versions.
The output I got:
>>> from subprocess import *
>>> Popen("py -i foo.py", stdin=PIPE, stdout=PIPE, stderr=PIPE).communicate()
(b'', b'>>> Traceback (most recent call last):\r\n File "foo.py", line 2, in
\r\n1/0\r\nZeroDivisionError: division by zero\r\n\r\n')
--
status: pending -> open
___
Python tracker
<https://bugs.python.org/issue18838>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42184] pdb exits unexpectedly when calling args
New submission from Adam Merchant : When an objects __repr__ or __str__ methods return None a TypeError is raised. However if this object is passed to a function and `args` is called from within pdb, pdb will immediately exit. Attached to this is bug_example.py which contains a simple example of how to reproduce this. Depending on circumstances this can make debugging difficult. exact python version that this happened with: Python 3.6.11 -- files: bug_example.py messages: 379838 nosy: xgenadam priority: normal severity: normal status: open title: pdb exits unexpectedly when calling args type: behavior versions: Python 3.6 Added file: https://bugs.python.org/file49546/bug_example.py ___ Python tracker <https://bugs.python.org/issue42184> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41283] The parameter name for imghdr.what in the documentation is wrong
New submission from Adam Eltawla : I noticed the parameter name for imghdr.what in the documentation is wrong Link: https://docs.python.org/3.8/library/imghdr.html?highlight=imghdr function imghdr.what(filename, h=None) In reality: def what(file, h=None): It is 'file' not 'filename'. -- assignee: docs@python components: Documentation messages: 373551 nosy: aeltawela, docs@python priority: normal severity: normal status: open title: The parameter name for imghdr.what in the documentation is wrong type: enhancement versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue41283> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41008] multiprocessing.Connection.poll raises BrokenPipeError on Windows
New submission from David Adam : On Windows 10 (1909, build 18363.900) in 3.7.7 and 3.9.0b3, poll() on a multiprocessing.Connection object can produce an exception: -- import multiprocessing def run(output_socket): for i in range(10): output_socket.send(i) output_socket.close() def main(): recv, send = multiprocessing.Pipe(duplex=False) process = multiprocessing.Process(target=run, args=(send,)) process.start() send.close() while True: if not process._closed: if recv.poll(): try: print(recv.recv()) except EOFError: process.join() break if __name__ == "__main__": main() -- On Linux/macOS this prints 0-9 and exits successfully, but on Windows produces a backtrace as follows: File "mptest.py", line 17, in main if recv.poll(): File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.179.0_x64__qbz5n2kfra8p0\lib\multiprocessing\connection.py", line 262, in poll return self._poll(timeout) File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.179.0_x64__qbz5n2kfra8p0\lib\multiprocessing\connection.py", line 333, in _poll _winapi.PeekNamedPipe(self._handle)[0] != 0): BrokenPipeError: [WinError 109] The pipe has been ended -- messages: 371748 nosy: zanchey priority: normal severity: normal status: open title: multiprocessing.Connection.poll raises BrokenPipeError on Windows type: behavior versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue41008> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40847] New parser considers empty line following a backslash to be a syntax error, old parser didn't
Adam Williamson added the comment: I'm not the best person to ask what I'd "consider" to be a bug or not, to be honest. I'm just a Fedora packaging guy trying to make our packages build with Python 3.9 :) If this is still an important question, I'd suggest asking the folks from the Black issue and PR I linked to, that's the "real world" case if any. -- ___ Python tracker <https://bugs.python.org/issue40847> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40911] Unexpected behaviour for += assignment to list inside tuple
Adam Cmiel added the comment: Got it, I didn't realize that the last step of augmented assignment is (in this case) assigning the result of __iadd__ back to the tuple. Thanks for the explanations! -- ___ Python tracker <https://bugs.python.org/issue40911> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40911] Unexpected behaviour for += assignment to list inside tuple
New submission from Adam Cmiel : Python version: Python 3.8.3 (default, May 15 2020, 00:00:00) [GCC 10.1.1 20200507 (Red Hat 10.1.1-1)] on linux Description: When assigning to a tuple index using +=, if the element at that index is a list, the list is extended and a TypeError is raised. a = ([],) try: a[0] += [1] except TypeError: assert a != ([1],) # assertion fails else: assert a == ([1],) The expected behaviour is that only one of those things would happen (probably the list being extended with no error, given that a[0].extend([1]) works fine). -- components: Interpreter Core messages: 370990 nosy: Adam Cmiel priority: normal severity: normal status: open title: Unexpected behaviour for += assignment to list inside tuple type: behavior versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue40911> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40848] compile() can compile a bare starred expression with `PyCF_ONLY_AST` flag with the old parser, but not the new one
Adam Williamson added the comment: Realized I forgot to give it, so in case it's important, the context here is the black test suite: https://github.com/psf/black/issues/1441 that test suite has a file full of expressions that it expects to be able to parse this way (it uses `ast.parse()`, which in turn calls `compile()` with this flag). A bare (*starred) line is part of that file: https://github.com/psf/black/blob/master/tests/data/expression.py#L149 and has been for as long as black has existed. Presumably if this isn't going to be fixed we'll need to adapt this black test file to test a starred expression in a 'valid' way, somehow. -- ___ Python tracker <https://bugs.python.org/issue40848> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40848] compile() can compile a bare starred expression with `PyCF_ONLY_AST` flag with the old parser, but not the new one
New submission from Adam Williamson :
Not 100% sure this would be considered a bug, but it seems at least worth
filing to check. This is a behaviour difference between the new parser and the
old one. It's very easy to reproduce:
sh-5.0# PYTHONOLDPARSER=1 python3
Python 3.9.0b1 (default, May 29 2020, 00:00:00)
[GCC 10.1.1 20200507 (Red Hat 10.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from _ast import *
>>> compile("(*starred)", "", "exec", flags=PyCF_ONLY_AST)
>>>
sh-5.0# python3
Python 3.9.0b1 (default, May 29 2020, 00:00:00)
[GCC 10.1.1 20200507 (Red Hat 10.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from _ast import *
>>> compile("(*starred)", "", "exec", flags=PyCF_ONLY_AST)
Traceback (most recent call last):
File "", line 1, in
File "", line 1
(*starred)
^
SyntaxError: invalid syntax
That is, you can compile() the expression "(*starred)" with PyCF_ONLY_AST flag
set with the old parser, but not with the new one. Without PyCF_ONLY_AST you
get a SyntaxError with both parsers, though a with the old parser, the error
message is "can't use starred expression here", not "invalid syntax".
--
components: Interpreter Core
messages: 370620
nosy: adamwill
priority: normal
severity: normal
status: open
title: compile() can compile a bare starred expression with `PyCF_ONLY_AST`
flag with the old parser, but not the new one
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue40848>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40847] New parser considers empty line following a backslash to be a syntax error, old parser didn't
New submission from Adam Williamson :
While debugging issues with the black test suite in Python 3.9, I found one
which black upstream says is a Cpython issue, so I'm filing it here.
Reproduction is very easy. Just use this four-line tester:
print("hello, world")
\
print("hello, world 2")
with that saved as `test.py`, check the results:
sh-5.0# PYTHONOLDPARSER=1 python3 test.py
hello, world
hello, world 2
sh-5.0# python3 test.py
File "/builddir/build/BUILD/black-19.10b0/test.py", line 3
^
SyntaxError: invalid syntax
The reason black has this test (well, a similar test - in black's test, the
file *starts* with the backslash then the empty line, but the result is the
same) is covered in https://github.com/psf/black/issues/922 and
https://github.com/psf/black/pull/948 .
--
components: Interpreter Core
messages: 370618
nosy: adamwill
priority: normal
severity: normal
status: open
title: New parser considers empty line following a backslash to be a syntax
error, old parser didn't
type: behavior
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue40847>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11395] print(s) fails on Windows with long strings
Adam Bartoš added the comment:
I've been hit by this issue recently. On my configuration, print("a" * 10215)
fails with an infinite loop of OSErrors (WinError 8). This even cannot by
interrupted with Ctrl-C nor the exception can be catched.
- print("a" * 10214) is fine
- print("a" * 10215) is fine when preceeded by print("b" * 2701), but not when
preceeded by print("b" * 2700)
- the problem (or at least with these numbers) occurs only when the code is
saved in a script, and this is run by double-clicking the file (i.e. run by
Windows ShellExecute I guess), not by "py test.py" or interactively.
My configuration is Python 3.7.3 64 bit on Windows Vista 64 bit. I wonder if
anyone is able to reproduce this on their configuration.
--
nosy: +Drekin
___
Python tracker
<https://bugs.python.org/issue11395>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect
Change by Adam Meily : -- pull_requests: +17546 pull_request: https://github.com/python/cpython/pull/18159 ___ Python tracker <https://bugs.python.org/issue39439> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect
Change by Adam Meily : -- pull_requests: +17545 pull_request: https://github.com/python/cpython/pull/18158 ___ Python tracker <https://bugs.python.org/issue39439> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect
Change by Adam Meily : -- pull_requests: +17544 pull_request: https://github.com/python/cpython/pull/18157 ___ Python tracker <https://bugs.python.org/issue39439> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38092] environment variables not passed correctly using new virtualenv launching in windows and python3.7+
Change by Adam Meily : -- pull_requests: +17543 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/18157 ___ Python tracker <https://bugs.python.org/issue38092> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect
Adam Meily added the comment: OK, that makes sense. For 3.7, I can create a PR for that corrects the order of arguments passed into _winapi.CreateProcess For 3.8 / master, the problem appears to be that the check in popen_spawn_win32.py to set the subprocess env is failing because sys.executable != spawn.get_executable() -- spawn.get_executable() is returning sys._base_executable. So, can you confirm that the fix is to just change spawn.get_executable() to return sys.executable, like it was prior to the PR mentioned in the other ticket? -- ___ Python tracker <https://bugs.python.org/issue39439> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39439] Windows Multiprocessing in Virtualenv: sys.prefix is incorrect
New submission from Adam Meily : I upgraded from Python 3.7.1 to 3.7.6 and began noticing a behavior that was breaking my code. My code detects if it's running in a virtualenv. This check worked in 3.7.1 but is broken in 3.7.6. >From the documentation, sys.prefix and sys.exec_prefix should point to the >virtualenv when one is active. However, I'm seeing that both of these >constants are pointing to the system installation directory and not my >virtualenv when I am in a multiprocessing child. Here is an example output of >a test application running in 3.7.6 (I've attached the test script to this >ticket): Parent process = sys.prefix: C:\Users\user\project\venv sys.exec_prefix: C:\Users\user\project\venv sys.base_prefix: C:\Program Files\Python37 sys.base_exec_prefix: C:\Program Files\Python37 = Subprocess = sys.prefix: C:\Program Files\Python37 sys.exec_prefix: C:\Program Files\Python37 sys.base_prefix: C:\Program Files\Python37 sys.base_exec_prefix: C:\Program Files\Python37 = I would expect that sys.prefix and sys.exec_prefix to be identical in the parent and child process. I verified that this behavior is present in 3.7.5, 3.7.6, and 3.8.1. I am on a Windows 10 x64 system. Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)] on win32 -- components: Windows files: multiproc_venv_prefix.py messages: 360581 nosy: meilyadam, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Windows Multiprocessing in Virtualenv: sys.prefix is incorrect versions: Python 3.7, Python 3.8 Added file: https://bugs.python.org/file48862/multiproc_venv_prefix.py ___ Python tracker <https://bugs.python.org/issue39439> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39201] Threading.timer leaks memory in 3.8.0/3.8.1
Adam added the comment: I filed a bug for this a few weeks ago, and then found another ticket about the same issue before: https://bugs.python.org/issue37788 My ticket: https://bugs.python.org/issue39074 The memory leak was from a change introduced about 6 months ago: https://github.com/python/cpython/commit/468e5fec8a2f534f1685d59da3ca4fad425c38dd -- nosy: +krypticus ___ Python tracker <https://bugs.python.org/issue39201> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37788] fix for bpo-36402 (threading._shutdown() race condition) causes reference leak
Adam added the comment: I ran into this bug as well, and opened an issue for it (before I saw this issue): https://bugs.python.org/issue39074 Was there a conclusion on the best way to fix this? It seems like the previous __del__ implementation would correct the resource leakage by removing the _tstate_lock from _shutdown_locks. -- nosy: +krypticus ___ Python tracker <https://bugs.python.org/issue37788> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39074] Threading memory leak in _shutdown_locks for non-daemon threads
Adam added the comment: Looks like this issue might be a duplicate of https://bugs.python.org/issue37788 -- ___ Python tracker <https://bugs.python.org/issue39074> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39074] Threading memory leak in _shutdown_locks for non-daemon threads
Change by Adam : -- type: resource usage -> security ___ Python tracker <https://bugs.python.org/issue39074> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39074] Threading memory leak in _shutdown_locks for non-daemon threads
New submission from Adam :
When running 3.7, we noticed a memory leak in threading._shutdown_locks when
non-deamon threads are started but "join()" or "is_alive()" is never called.
Here's a test to illustrate the growth:
=
import threading
import time
import tracemalloc
def test_leaking_locks():
tracemalloc.start(10)
snap1 = tracemalloc.take_snapshot()
def print_things():
print('.', end='')
for x in range(500):
t = threading.Thread(target=print_things)
t.start()
time.sleep(5)
print('')
gc.collect()
snap2 = tracemalloc.take_snapshot()
filters = []
for stat in
snap2.filter_traces(filters).compare_to(snap1.filter_traces(filters),
'traceback')[:10]:
print("New Bytes: {}\tTotal Bytes {}\tNew blocks: {}\tTotal blocks: {}:
".format(stat.size_diff, stat.size, stat.count_diff ,stat.count))
for line in stat.traceback.format():
print(line)
=
=
Output in v3.6.8:
New Bytes: 840 Total Bytes 840 New blocks: 1 Total blocks: 1:
File "/usr/local/lib/python3.6/threading.py", line 884
self._bootstrap_inner()
New Bytes: 608 Total Bytes 608 New blocks: 4 Total blocks: 4:
File "/usr/local/lib/python3.6/tracemalloc.py", line 387
self.traces = _Traces(traces)
File "/usr/local/lib/python3.6/tracemalloc.py", line 524
return Snapshot(traces, traceback_limit)
File "/gems/tests/integration/endpoint_connection_test.py", line 856
snap1 = tracemalloc.take_snapshot()
File "/usr/local/lib/python3.6/site-packages/_pytest/python.py", line 198
testfunction(**testargs)
File "/usr/local/lib/python3.6/site-packages/pluggy/callers.py", line 187
res = hook_impl.function(*args)
File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 87
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/usr/local/lib/python3.6/site-packages/pluggy/manager.py", line 93
return self._inner_hookexec(hook, methods, kwargs)
File "/usr/local/lib/python3.6/site-packages/pluggy/hooks.py", line 286
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/usr/local/lib/python3.6/site-packages/_pytest/python.py", line 1459
self.ihook.pytest_pyfunc_call(pyfuncitem=self)
File "/usr/local/lib/python3.6/site-packages/_pytest/runner.py", line 111
item.runtest()
==
Output in v3.7.4:
New Bytes: 36000Total Bytes 36000 New blocks: 1000Total
blocks: 1000:
File "/usr/local/lib/python3.7/threading.py", line 890
self._bootstrap_inner()
File "/usr/local/lib/python3.7/threading.py", line 914
self._set_tstate_lock()
File "/usr/local/lib/python3.7/threading.py", line 904
self._tstate_lock = _set_sentinel()
New Bytes: 32768Total Bytes 32768 New blocks: 1 Total blocks:
1:
File "/usr/local/lib/python3.7/threading.py", line 890
self._bootstrap_inner()
File "/usr/local/lib/python3.7/threading.py", line 914
self._set_tstate_lock()
File "/usr/local/lib/python3.7/threading.py", line 909
_shutdown_locks.add(self._tstate_lock)
=
It looks like this commit didn't take into account the tstate_lock cleanup that
happens in the C code, and it's not removing the _tstate_lock of completed
threads from the _shutdown_locks once the thread finishes, unless the code
manually calls "join()" or "is_alive()" on the thread:
https://github.com/python/cpython/commit/468e5fec8a2f534f1685d59da3ca4fad425c38dd
Let me know if I can provide more clarity on this!
--
messages: 358551
nosy: krypticus
priority: normal
severity: normal
status: open
title: Threading memory leak in _shutdown_locks for non-daemon threads
type: resource usage
versions: Python 3.7, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue39074>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38839] Some unused functions in test suite
New submission from Adam Johnson : Whilst developing a new unused function check for flake8 ( https://github.com/PyCQA/pyflakes/pull/485 ) I ran it against the CPython source code and found some uncalled functions. -- messages: 356919 nosy: adamchainz priority: normal pull_requests: 16745 severity: normal status: open title: Some unused functions in test suite type: enhancement ___ Python tracker <https://bugs.python.org/issue38839> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37951] Disallow fork in a subinterpreter broke subprocesses in mod_wsgi daemon mode
Adam Williamson added the comment: It's this function: https://github.com/freeipa/freeipa/blob/master/ipalib/install/kinit.py#L66 The function `run` is imported from `ipapython.ipautil`, it's defined here: https://github.com/freeipa/freeipa/blob/master/ipapython/ipautil.py#L391 all of this is being run inside a WSGI. -- ___ Python tracker <https://bugs.python.org/issue37951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37951] Disallow fork in a subinterpreter broke subprocesses in mod_wsgi daemon mode
Adam Williamson added the comment: Well, now our (Fedora QA's) automated testing of FreeIPA is showing what looks like a problem with preexec_fn (rather than fork) being disallowed: https://bugzilla.redhat.com/show_bug.cgi?id=1759290 Login to the FreeIPA webUI is failing, and at the time it fails we see this error message on the server end: [Mon Oct 07 09:22:19.521604 2019] [wsgi:error] [pid 32989:tid 139746234119936] [remote 10.0.2.102:56054] ipa: DEBUG: args=['/usr/bin/kinit', 'admin', '-c', '/run/ipa/ccaches/kinit_32989', '-E'] [Mon Oct 07 09:22:19.521996 2019] [wsgi:error] [pid 32989:tid 139746234119936] [remote 10.0.2.102:56054] ipa: DEBUG: Process execution failed [Mon Oct 07 09:22:19.522189 2019] [wsgi:error] [pid 32989:tid 139746234119936] [remote 10.0.2.102:56054] ipa: INFO: 401 Unauthorized: preexec_fn not supported within subinterpreters -- nosy: +adamwill status: pending -> open ___ Python tracker <https://bugs.python.org/issue37951> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue26510] [argparse] Add required argument to add_subparsers
Change by Adam Stewart : -- pull_requests: +16178 pull_request: https://github.com/python/cpython/pull/16588 ___ Python tracker <https://bugs.python.org/issue26510> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1583] Patch for signal.set_wakeup_fd
Adam Olsen added the comment: signalmodule.c has a hack to limit it to the main thread. Otherwise there's all sorts of platform-specific behaviour. -- ___ Python tracker <https://bugs.python.org/issue1583> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1583] Patch for signal.set_wakeup_fd
Adam Olsen added the comment: signal-safe is different from thread-safe (despite conceptual similarities), but regardless it's been a long time since I last delved into this so I'm quite rusty. I could be doing it all wrong. -- ___ Python tracker <https://bugs.python.org/issue1583> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1583] Patch for signal.set_wakeup_fd
Adam Olsen added the comment: Converting to/from sig_atomic_t could have a compile time check on currently supported platforms and isn't buggy for them. For platforms with a different size you could do a runtime check, only allowing a fd in the range of 0-254 (with 255 reserved); that could sometimes fail, yes, but at least it's explicit, easily understood failure. Just using int would fail in undefined ways down the road, likely writing to a random fd instead (corrupting whatever it was doing), with no way to trace it back. Unpacking the int would mean having one sig_atomic_t for 'invalid', using that instead of INVALID_FD, plus an array of sig_atomic_t for the fd itself. Every time you want to change the fd you first set the 'invalid' flag, then the individual bytes, then clear 'invalid'. -- ___ Python tracker <https://bugs.python.org/issue1583> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1583] Patch for signal.set_wakeup_fd
Adam Olsen added the comment: Disagree; if you're writing signal-handling code you should be very careful to do it properly, even if that's only proper for your current platform. If you can't do it properly you should find an alternative that doesn't involve signals. The fact that sig_atomic_t is only 1 byte on VxWorks strongly implies using int WILL fail in strange ways on that platform. I can see three options: 1) use pycore_atomic.h, implementing it for VxWorks if you haven't already. This also implies sig_atomic_t could have been int but wasn't for some reason, such as performance. 2) disable wakeup_fd entirely. It's obscure, GNOME being the biggest user I can think of. 3) unpack the int into an array of sig_atomic_t. Only the main thread writes to it so this method is ugly but viable. -- ___ Python tracker <https://bugs.python.org/issue1583> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1583] Patch for signal.set_wakeup_fd
Adam Olsen added the comment: The fd field may be written from the main thread simultaneous with the signal handler activating and reading it out. Back in 2007 the only POSIX-compliant type allowed for that was sig_atomic_t, anything else was undefined. Looks like pycore_atomic.h should have alternatives now but I'm not at all familiar with it. -- ___ Python tracker <https://bugs.python.org/issue1583> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35192] pathlib mkdir throws FileExistsError when not supposed to
Adam Dunlap added the comment: Thank your for your response. I'm running python 3.5.2. The linked issue is indeed a duplicate of this one. To reproduce, you can run two instances of the attached script at the same time, i.e. python3 pymkdir.py & python3 pymkdir.py. It is a race condition so won't happen every time, but with the number of directories created it should happen pretty frequently (remove the directories between tests with rm -rf 1) I'm running Ubuntu 16.04.5 LTS. I don't know if you know anything about Ubuntu's release process, but it would be nice if the linked fix could be backported to work on that platform. -- Added file: https://bugs.python.org/file47917/pymkir.py ___ Python tracker <https://bugs.python.org/issue35192> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue35192] pathlib mkdir throws FileExistsError when not supposed to
New submission from Adam Dunlap : I have 2 processes (one C, one python) that are started at the same time and both need a certain directory tree to exist to continue, so both processes create the directory tree while ignoring errors about it already existing. In the python process, I'm using pathlib's mkdir function (https://docs.python.org/3/library/pathlib.html#pathlib.Path.mkdir) with parents=True, exist_ok=True, but this is sometimes throwing a FileExistsError. I don't expect this because the documentation says "If exist_ok is true, FileExistsError exceptions will be ignored (same behavior as the POSIX mkdir -p command), but only if the last path component is not an existing non-directory file." The last component is never a non-directory file. I believe what is happening is that mkdir realizes that the parent doesn't exist, so it recursively tries to make the parent directory. However, when it recurses, it uses the default exists_ok value of False. Before the recursive call can make the parent directory, the other process makes the directory. This causes the inner call to throw a FileExistsError. I believe the correct thing to do is to always pass True for exists_ok in the recursive call. -- components: Library (Lib) messages: 329484 nosy: Adam Dunlap priority: normal severity: normal status: open title: pathlib mkdir throws FileExistsError when not supposed to type: behavior versions: Python 3.5 ___ Python tracker <https://bugs.python.org/issue35192> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1154351] add get_current_dir_name() to os module
Change by Marc Adam Anderson : -- nosy: -marcadam ___ Python tracker <https://bugs.python.org/issue1154351> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27165] Skip callables when displaying exception fields in cgitb
Change by Adam Bielański : -- pull_requests: +9085 ___ Python tracker <https://bugs.python.org/issue27165> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29883] asyncio: Windows Proactor Event Loop UDP Support
Adam Meily added the comment: I've rebased onto upstream master and I fixed the CI build. -- ___ Python tracker <https://bugs.python.org/issue29883> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33440] Possible lazy import opportunities in `pathlib`
Change by Adam Forsyth <a...@adamforsyth.net>: -- keywords: +patch pull_requests: +6505 stage: -> patch review ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33440> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32627] Header dependent _uuid build failure on Fedora 27
Adam <a...@netbsd.org> added the comment: Some systems might have both uuid.h and uuid/uuid.h. For example, NetBSD provides /usr/include/uuid.h, and one might also install libuuid from a package, and the latter has uuid/uuid.h. To fix this, do not include both files, when both have been detected. Here is a patch: --- Modules/_uuidmodule.c.orig +++ Modules/_uuidmodule.c @@ -3,8 +3,7 @@ #include "Python.h" #ifdef HAVE_UUID_UUID_H #include -#endif -#ifdef HAVE_UUID_H +#elif defined(HAVE_UUID_H) #include #endif -- nosy: +a...@netbsd.org ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32627> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33420] __origin__ invariant broken
Adam Paszke <adam.pas...@gmail.com> added the comment: Of course, I'm not expecting this to be 100% reliable, and so I'm ok with the answer that the comment is now outdated. I'd like to avoid adding extra dependencies for so simple things, so I guess I'll just special case that in my code for now. It would be great if the typing module had some tools for introspection built in! -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33420> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33420] __origin__ invariant broken
New submission from Adam Paszke <adam.pas...@gmail.com>: Hi everyone, I have a module that needs to inspect type annotations on a few functions. One of the types I need to special case on is typing.Tuple, and I used code like this to detect it: if getattr(annotation, '__origin__', None) == typing.Tuple: ... else: ... This was based on the comment from the typing module (Lib/typing.py:609) that specified this particular invariant on the __origin__ attribute: > __origin__ keeps a reference to a type that was subscripted, e.g., Union[T, int].__origin__ == Union; Everything worked just fine until I checked it on the alpha release of Python 3.7 in my CI. Turns out, that in that release we have typing.Tuple[str, int].__origin__ == tuple and not (which is the case in e.g. 3.6) typing.Tuple[str, int].__origin__ == typing.Tuple I know this is not a documented attribute, so it can change, but I wanted to highlight that it's either a regression, or the comment will need to be updated, so people won't try to depend on that. -- components: Library (Lib) messages: 316127 nosy: apaszke priority: normal severity: normal status: open title: __origin__ invariant broken type: behavior versions: Python 3.7, Python 3.8 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33420> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33291] issue in the default setup.py which comes with the Python 3.6.5 installer
Change by Adam Klinger <adamjklin...@gmail.com>: -- title: Cannot Install Stegano Package - Python 3.6.5 Base -> issue in the default setup.py which comes with the Python 3.6.5 installer ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33291> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33291] Cannot Install Stegano Package - Python 3.6.5 Base
New submission from Adam Klinger <adamjklin...@gmail.com>: There seems to be an issue in the default setup.py which comes with the Python 3.6.5 installer. Upon trying to install an additional package through pip the below is observed: C:\Users\adamj\Desktop>pip install stegano Collecting stegano Downloading https://files.pythonhosted.org/packages/77/76/07a61c042ac1a1cb3445689b4f140800b16d0e883a46049e221d7a92de66/Stegano-0.8.4.tar.gz (243kB) 100% || 245kB 3.3MB/s Complete output from command python setup.py egg_info: Traceback (most recent call last): File "", line 1, in File "C:\Users\adamj\AppData\Local\Temp\pip-install-sh3rd6wj\stegano\setup.py", line 26, in readme = f.read() File "c:\python36\lib\encodings\cp1252.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0] UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 1569: character maps to -- components: Unicode messages: 315373 nosy: Adam Klinger, ezio.melotti, vstinner priority: normal severity: normal status: open title: Cannot Install Stegano Package - Python 3.6.5 Base versions: Python 3.6 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33291> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31087] asyncio.create_subprocess_* should honor `encoding`
Adam <adam.g...@wp.pl> added the comment: After reading the docs more carefully, it's now plain to me that text encoding is not supported yet, so actually it's not a bug :) However the docs should be improved (and then an assertion could be added too) to prevent people from falling into this trap. Only the `universal_newlines` parameter is explicitly mentioned, while others (including `encoding` and `errors``) are passed to `subprocess.Popen`, which falsely suggests that they should work fine. Moreover, the `std*` properties of the subprocess have a `_transport._pipe.encoding` set to the encoding passed to `asyncio.create_subprocess_*`, but apparently it's not used at all. IMHO this is too messy. Alternatively this option could be implemented, which would require a new kind of StreamReader and StreamWriter. -- title: asyncio.create_subprocess_* do not honor `encoding` -> asyncio.create_subprocess_* should honor `encoding` type: behavior -> enhancement versions: +Python 3.8 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31087> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31087] asyncio.create_subprocess_* do not honor `encoding`
Adam <adam.g...@wp.pl> added the comment: It occurs both on Python 3.6 and 3.7 RC, so maybe it should be fixed in the 3.7 release. -- nosy: +adampl type: -> behavior versions: +Python 3.7 Added file: https://bugs.python.org/file47531/asyncio_encoding_test.py ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31087> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32988] datetime.datetime.strftime('%s') always uses local timezone, even with aware datetimes
Adam Williamson <awill...@redhat.com> added the comment: Yeah, I've added a comment there. I agree we can keep subsequent discussion in that issue. Closing this as a dupe. I actually have the same thought as you, but I suspect making something that "worked" before start throwing an error might be a hard sell for some. Perhaps at least some kind of warning? -- resolution: -> duplicate stage: -> resolved status: open -> closed ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32988> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12750] add cross-platform support for %s strftime-format code
Adam Williamson <awill...@redhat.com> added the comment: On the "attractive nuisance" angle: I just ran right into this problem, and reported https://bugs.python.org/issue32988 . As I suggested there, if Python doesn't try to fix this, I'd suggest it should at least *explicitly document* that using %s is unsupported and dangerous in more than one way (might not work on all platforms, does not do what it should for 'aware' datetimes on platforms where it *does* work). I think explicitly telling people NOT to use it would be better than just not mentioning it. At least for me, when I saw real code using it and that the docs just didn't mention it, my initial thought was "I guess it must be OK, and the docs just missed it out for some reason". If I'd gone to the docs and seen an explicit note that it's not supported and doesn't work right, that would've been much clearer and I wouldn't have had to figure that out for myself :) For Python 2, btw, the arrow library might be a suitable alternative to suggest: you can do something like this, assuming you have an aware datetime object called 'awaredate' you want to get the timestamp for: import arrow ts = arrow.get(awaredate).timestamp and it does the right thing. -- nosy: +adamwill ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue12750> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32988] datetime.datetime.strftime('%s') always uses local timezone, even with aware datetimes
Adam Williamson <awill...@redhat.com> added the comment:
I'd suggest that if that is the case, it would be better for the docs to
*specifically mention* that `%s` is not supported and should not be used,
rather than simply not mentioning it.
When it's used in real code (note someone in the SO issue mentions "I have been
going crazy trying to figure out why i see strftime("%s") a lot, yet it's not
in the docs") and just *not mentioned* in the docs, this tends to give the
impression that it's something usable that was perhaps just forgotten from the
docs, or something. The situation would be much clearer if the docs said "DO
NOT USE THIS, IT'S DANGEROUS AND DOESN'T DO WHAT YOU THINK" in big letters.
(And suggested using .timestamp() on Python 3.3+, and possibly arrow's
.timestamp on 2.7?)
--
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32988>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32988] datetime.datetime.strftime('%s') always uses local timezone, even with aware datetimes
Adam Williamson <awill...@redhat.com> added the comment: Paul: right. This is on Linux - specifically Fedora Linux, but I don't think it matters. glibc strftime and strptime depend on an underlying struct called 'tm'. 'man strftime' says: %s The number of seconds since the Epoch, 1970-01-01 00:00:00 + (UTC). (TZ) (Calculated from mktime(tm).) And 'man mktime' says: The mktime() function converts a broken-down time structure, expressed as local time, to calendar time representation. ... On success, mktime() returns the calendar time (seconds since the Epoch), expressed as a value of type time_t." I am finding it hard to determine whether various C standards require the tm struct and mktime and strftime and so on to handle timezones, but I'm sort of inclining to the answer that "no they don't". Basically I suspect what's going on in this case is that the timezone information gets lost somewhere in the chain down from Python to system strftime to system mktime, and Python doesn't make any adjustment to the actual date / time values before calling system strftime to try and account for this. I think Python must do *something* more than purely converting to a tm and calling system strftime, though, as %Z does work, which it wouldn't if Python was purely converting to a non-timezone-aware tm struct and calling system strftime, I don't think... -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32988> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32988] datetime.datetime.strftime('%s') always uses local timezone, even with aware datetimes
New submission from Adam Williamson <awill...@redhat.com>:
Test script:
import pytz
import datetime
utc = pytz.timezone('UTC')
print(datetime.datetime(2017, 1, 1, tzinfo=utc).strftime('%s'))
Try running it with various system timezones:
[adamw@xps13k pagure (more-timezone-fun %)]$ TZ='UTC' python /tmp/test2.py
1483228800
[adamw@xps13k pagure (more-timezone-fun %)]$ TZ='America/Winnipeg' python
/tmp/test2.py
1483250400
[adamw@xps13k pagure (more-timezone-fun %)]$ TZ='America/Vancouver' python
/tmp/test2.py
1483257600
That's Python 2.7.14; same results with Python 3.6.4.
This does not seem correct. The correct Unix time for an aware datetime object
should be a constant: for 2017-01-01 00:00 UTC it *is* 1483228800 . No matter
what the system's local timezone, that should be the output of strftime('%s'),
surely. What it seems to be doing instead is just outputting the Unix time for
2017-01-01 00:00 in the system timezone.
I *do* note that strftime('%s') is completely undocumented in Python; neither
https://docs.python.org/2/library/datetime.html#strftime-and-strptime-behavior
nor
https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior
mentions it. However, it does exist, and is used in the real world; I found
this usage of it, and the bug, in a real project, Pagure.
--
components: Library (Lib)
messages: 313169
nosy: adamwill
priority: normal
severity: normal
status: open
title: datetime.datetime.strftime('%s') always uses local timezone, even with
aware datetimes
versions: Python 2.7, Python 3.6
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32988>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31959] Directory at `TemporaryDirectory().name` does not exist
Adam Dangoor <adamdang...@gmail.com> added the comment: Thank you for clearing this up for me. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31959> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31959] Directory at `TemporaryDirectory().name` does not exist
Adam Dangoor <adamdang...@gmail.com> added the comment: > The unexpected behavior occurs on CPython 3.5.3 and CPython 3.6.X but not on > pypy. This suggests that it is something to do with garbage collection. Upon further thought, maybe this is by design, but I still was surprised. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31959> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
