[issue47260] os.closerange() can be no-op in a seccomp sandbox
Gregory P. Smith added the comment: Good catch. -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue47260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47260] os.closerange() can be no-op in a seccomp sandbox
Gregory P. Smith added the comment: New changeset 1c8b3b5d66a629258f1db16939b996264a8b9c37 by Alexey Izbyshev in branch 'main': bpo-47260: Fix os.closerange() potentially being a no-op in a seccomp sandbox (GH-32418) https://github.com/python/cpython/commit/1c8b3b5d66a629258f1db16939b996264a8b9c37 -- ___ Python tracker <https://bugs.python.org/issue47260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34975] start_tls() difficult when using asyncio.start_server()
Gregory P. Smith added the comment: https://discuss.python.org/t/need-reconsideration-of-bpo-34975-add-start-tls-method-to-streams-api/14720 would like to see this reconsidered. reopening. -- nosy: +gregory.p.smith resolution: wont fix -> stage: resolved -> status: closed -> open versions: +Python 3.11 -Python 3.8 ___ Python tracker <https://bugs.python.org/issue34975> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47245] potential undefined behavior with subprocess using vfork() on Linux?
Gregory P. Smith added the comment: Thanks! I agree with you that this is probably not an actual problem on Linux. _I did look at the various glibc architecture vfork.s implementations: Cute tricks used on some where they need to avoid a stack modifying traditional return from vfork()._ As for glibc specifics, I'm mostly thinking of the calls we do in the child. According to the "Standard Description (POSIX.1)" calls to anything other than `_exit()` or `exec*()` are not allowed. But the longer "Linux Description" in that vfork(2) man page does not say that. Which implies merely by omission that calls to other things are okay so long as you understand everything they do to the process heap/stack/state. (I wish it were *explicit* about that) Some of the calls we do from our child_exec() code... many are likely "just" syscall shims and thus fine - but that is technically up to libc. A few others are Py functions that go elsewhere in CPython and while they may be fine for practical reasons today with dangerous bits on conditional branches that technically should not be possible to hit given the state by the time we're at this point in _posixsubprocess, pose a future risk - anyone touching implementations of those is likely unaware of vfork'ed child limitations that must be met. For example if one of the potential code paths that trigger an indirect Py_FatalError() is hit... that fatal exit code is definitely not post-vfork-child safe. The pre-exec child dying via that could screw up the vfork parent process's state. -- title: potential undefined behavior with subprocess using vfork() on Linux -> potential undefined behavior with subprocess using vfork() on Linux? ___ Python tracker <https://bugs.python.org/issue47245> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47245] potential undefined behavior with subprocess using vfork() on Linux
Gregory P. Smith added the comment: Our current assumptions around the use of vfork() are very much glibc specific. Another useful reference for reasoning, comments, and history is https://github.com/golang/go/blob/master/src/syscall/exec_linux.go#L146 `forkAndExecInChild1` -- ___ Python tracker <https://bugs.python.org/issue47245> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47245] potential undefined behavior with subprocess using vfork() on Linux
Change by Gregory P. Smith : -- keywords: +3.10regression ___ Python tracker <https://bugs.python.org/issue47245> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47245] potential undefined behavior with subprocess using vfork() on Linux
Gregory P. Smith added the comment: Immediate action item: Add a way for people to disable vfork at runtime by setting a flag in the subprocess module, just in case. This can be backported to 3.10 - It'd provide an escape hatch for anyone without a need to rebuild Python to disable use of vfork() without resorting to LD_PRELOAD hacks. This is not an urgent issue unless actual practical problems are being observed in the field. -- ___ Python tracker <https://bugs.python.org/issue47245> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47245] potential undefined behavior with subprocess using vfork() on Linux
Gregory P. Smith added the comment: Can you provide a reproducable way to demonstrate evidence of a problem in CPython's use of the Linux libc vfork() implementation? A test case that causes a CPython parent or child process on Linux when built with HAVE_VFORK failing to function properly would help prioritize this because in practice nobody has reported problems in 3.10. (we've deployed the subprocess vfork code into thousands production Python programs at work in the past year w/o issues being reported - though we have a constrained environment with use on only a couple of libc versions and limited set of kernels on a few very common architectures) General thinking (possible dated and incorrect - against what https://man7.org/linux/man-pages/man2/vfork.2.html wording claims with its "or calls any other function" text): Pushing additional data onto the stack in the child process _should_ not a problem. That by definition lands in previously unused pre-allocated stack space. If that page faults, that could map a new page into the process state shared by both the paused parent and running child. But this page mapping should be fine - the child exec that resumes the parent means the parent is the only one who sees it. When the parent process resumes, sure, that data will be in that memory on the unallocated portion of stack, but critically the *stack pointer* in the parent process (a register) never changes. As far as I understand things, registers are not shared between vfork()ed processes. So the parent only sees some temporary data having been written to the unused region of the stack by the since-replaced by exec() child process. No big deal. **Untrue wishful thinking**: If a new stack were needed on a given platform for use in the vfork()ed child, I'd like it to be the job of libc to take care of that for us. glibc sources do no such thing (every vfork supporting architecture has a vfork.S code that appears to make the syscall and jump back to the caller without stack manipulation). -- assignee: -> gregory.p.smith nosy: +gregory.p.smith stage: -> test needed title: Subprocess with vfork() is broken -> potential undefined behavior with subprocess using vfork() on Linux type: -> behavior ___ Python tracker <https://bugs.python.org/issue47245> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46576] test_peg_generator is extremely slow
Gregory P. Smith added the comment: New changeset 612e422c6ea9df60d3382ed1491d8254c283e93f by Jeremy Kloth in branch 'main': bpo-46576: Speed up test_peg_generator by using a static library for shared sources (GH-32338) https://github.com/python/cpython/commit/612e422c6ea9df60d3382ed1491d8254c283e93f -- ___ Python tracker <https://bugs.python.org/issue46576> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47235] Clarify that `assret_called_once_with` sample code is intended typo
Change by Gregory P. Smith : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed versions: -Python 3.8 ___ Python tracker <https://bugs.python.org/issue47235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47235] Clarify that `assret_called_once_with` sample code is intended typo
Gregory P. Smith added the comment: New changeset ac1fb07b6ecb6b93446484f52894914e5199de63 by Gregory P. Smith in branch 'main': bpo-47235: Note where a typo is intentional in code. (GH-32348) https://github.com/python/cpython/commit/ac1fb07b6ecb6b93446484f52894914e5199de63 -- ___ Python tracker <https://bugs.python.org/issue47235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47235] Clarify that `assret_called_once_with` sample code is intended typo
Change by Gregory P. Smith : -- keywords: +patch pull_requests: +30401 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/32348 ___ Python tracker <https://bugs.python.org/issue47235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47235] Clarify that `assret_called_once_with` sample code is intended typo
Gregory P. Smith added the comment: LOL... we didn't anticipate this happening when adding the commonly found typo detection feature. -- assignee: docs@python -> gregory.p.smith nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue47235> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47027] subprocess.run(), subprocess.Popen() should accept file descriptor as cwd parameter
Gregory P. Smith added the comment: this mostly requires plumbing to accept an int as the cwd and plumb that through to the between fork and exec code to call `fchdir(cwd_fd)` on the `int` instead of chdir(cwd) on the `char*`. the Modules/_posixsubprocess.c internals are a bit of a mess today with a bazillion parameters passed to internal functions which makes this a pain... I want to refactor things to use a struct and not need that, but adding this feature is doable regardless. -- components: +Extension Modules -Library (Lib) ___ Python tracker <https://bugs.python.org/issue47027> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47139] pthread_sigmask needs SIG_BLOCK behaviour explaination
Gregory P. Smith added the comment: The "trick" wouldn't be too useful though as this API can't block and the signal flag needs to be processed on the main thread. So I guess documentation it is. The way I think of this is that the signal.pthread_sigmask API is pretty low level. After that API is called, no more signals will _reach the process_, but the interpreter may process some that happened beforehand. So installing a handler is indeed appropriate. Also, it is entirely possible for C extension modules or code embedding Python to call pthread_sigmask in its own threads outside of the Python runtimes knowledge. So we us tracking what signals are blocked on our own may not be accurate. We could instead change the eval loop signal processing code to call `pthread_sigmask(SIG_UNBLOCK, NULL /* set */, );` to retrieve the processes current mask to decide if the flagged signals should be processed by Python or not. BUT... I can imagine complex race cases where that'd surprise people who are chaining multiple signal handlers such as one from outside of Python that saved the Python handler and calls it afterwards. Their C/process-level handler would be called, would chain to Python's "record that signal X happened and set the bit" handler, but Python wouldn't then be guaranteed to call the Python handler code if the sigmask changed before the eval loop did its pending signal check. So I'm still inclined to keep this simple and stick with just documenting best practices for now. An implementation of masking from the python eval handler may need to be something conditionally controllable for differing application situations if added. -- ___ Python tracker <https://bugs.python.org/issue47139> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46607] Add DeprecationWarning to configparser's LegacyInterpolation
Gregory P. Smith added the comment: New changeset 75280944e5ca957eec7f814b9d0608fc84fc5811 by Hugo van Kemenade in branch 'main': bpo-46607: Add DeprecationWarning for LegacyInterpolation, deprecated in docs since 3.2 (GH-30927) https://github.com/python/cpython/commit/75280944e5ca957eec7f814b9d0608fc84fc5811 -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue46607> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47153] __doc__ should generally be writable
Change by Gregory P. Smith : -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue47153> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47139] pthread_sigmask needs SIG_BLOCK behaviour explaination
Gregory P. Smith added the comment: The irony... Documenting the caveat at least seems useful. Your workaround sounds reasonable. I don't love the idea of implementing our own mask blocked/unblocked state check, though it probably wouldn't be very complicated. Might be interesting. Another trick that'd "work" so long as people don't have multiple SIG_BLOCK calls (not something to depend on) is to force a check of our interpreter signal flags right before signal.pthread_sigmask returns to the Python caller. -- nosy: +gregory.p.smith stage: -> needs patch type: -> behavior versions: +Python 3.11 ___ Python tracker <https://bugs.python.org/issue47139> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47164] [C API] Add private "CAST" macros to clean up casts in C code
Change by Gregory P. Smith : -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue47164> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47185] code.replace(co_code=new_code) no longer catch exceptions on Python 3.11
Change by Gregory P. Smith : -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue47185> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47165] [C API] Test that the Python C API is compatible with C++
Gregory P. Smith added the comment: If we can conditionally test new things based on C++XX version, accumulating modern issue regression tests seems useful. Otherwise 11 at minimum. As for why some things trigger this and others don't, my wild _guess_ would be whether the statements appear within an extern "C" block. Though that's not really what that is for so it isn't clear to me. -- components: +Tests nosy: +gregory.p.smith stage: -> test needed ___ Python tracker <https://bugs.python.org/issue47165> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46850] [C API] Move _PyEval_EvalFrameDefault() to the internal C API
Gregory P. Smith added the comment: per https://mail.python.org/archives/list/python-...@python.org/message/GFOMU7LP63JUVFMWNJNZJLUMZDRPTUYJ/ lets roll some of those PRs back and reshape where we move these APIs as despite the PEP they were not as private as other privates. (deferred blocker meaning: block 3.11beta1 - though Pablo could decide to have the rollbacks block 3.11alpha7) -- nosy: +gregory.p.smith, pablogsal priority: normal -> deferred blocker ___ Python tracker <https://bugs.python.org/issue46850> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47133] enhance unittest to show test name and docstring on one line
Gregory P. Smith added the comment: A more fundamental issue behind this: unittest's plain text verbose output is not intended to be parsable and consumable by machines. It is 22 years old and was intended for use in 80 column wide terminals as a progress report while watching it run as much it is a final result. Consider that the legacy use case or format if you wish. Allowing an option for output to one line per test with description could be more likely to require horizontal scrolling or reading past a line wrap in CI setups where text is scrolling regions embedded in scrolling regions. Yuck. BUT not necessarily bad. Just different. The ideas expressed in this thread of making a more clear text format with the important PASS/FAIL/ERROR/SKIP status up front and a very clear pastable test module.class.test_name seem wise. I expect most Python projects with nicer test output formatting desires have simply adopted pytest and never looked back. --- Other python unittest derivative concepts to get clear output of results: We add a reliably machine parsable XML report of test outcomes in absl-py https://github.com/abseil/abseil-py/tree/main/absl/testing. It's IMNSHO a rather annoying implementation - we wish unittest made plugging in additional output formats easier. We use that generated XML format for results from unittest systems in all of our languages at work, it is picked up and parsed by our internal CI system and used to display nicely formatted results when available. I assume pytest also provides machine parsable reporting capabilities but I haven't looked. -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue47133> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47194] Upgrade to zlib v1.2.12 in CPython binary releases
New submission from Gregory P. Smith : zlib v1.2.11 as used in Windows binary releases contains a security issue that, while fixed in its git repo years ago, never wound up in a release or a CVE until just now. Folllow the https://www.openwall.com/lists/oss-security/2022/03/24/1 thread and the and recently assigned CVE-2018-25032. I believe we only ship our own zlib on Windows so this issue is tagged as such. The above oss-security thread is where an idea of severity will come out. -- components: Extension Modules, Windows messages: 416510 nosy: gregory.p.smith, lukasz.langa, ned.deily, pablogsal, paul.moore, steve.dower, tim.golden, zach.ware priority: release blocker severity: normal stage: needs patch status: open title: Upgrade to zlib v1.2.12 in CPython binary releases type: security versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue47194> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47193] Use zlib-ng rather than zlib in binary releases
New submission from Gregory P. Smith : zlib-ng is an optimized zlib library with better performance on most architectures (with contributions from the likes of Google, Cloudflare, and Intel). It is API compatible with zlib. https://github.com/zlib-ng/zlib-ng I believe the only platform we don't use the OS's own zlib on is Windows so I'm tagging this issue Windows. -- components: Windows messages: 416508 nosy: gregory.p.smith, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal stage: needs patch status: open title: Use zlib-ng rather than zlib in binary releases type: performance versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue47193> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47151] subprocess fails when used as init, vfork() returns EINVAL if PID=1
Change by Gregory P. Smith : -- assignee: -> gregory.p.smith resolution: -> fixed stage: patch review -> commit review status: open -> closed ___ Python tracker <https://bugs.python.org/issue47151> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47151] subprocess fails when used as init, vfork() returns EINVAL if PID=1
Gregory P. Smith added the comment: New changeset 4a08c4c469d36f99d3a5e0f17ad82ab35dcf2835 by Gregory P. Smith in branch 'main': bpo-47151: Fallback to fork when vfork fails in subprocess. (GH-32186) https://github.com/python/cpython/commit/4a08c4c469d36f99d3a5e0f17ad82ab35dcf2835 -- ___ Python tracker <https://bugs.python.org/issue47151> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47151] subprocess fails when used as init, vfork() returns EINVAL if PID=1
Gregory P. Smith added the comment: Thanks. I had wondered if this was really a pid=1 restriction or not, but I could definitely imagine kernel scenarios where vfork is simply forbidden regardless. There are a variety of policy mechanisms in kernels, mainline Linux or not, that _could_ do that kind of thing. As much as I'd like to expose that the fallback happened, emitting to stderr isn't friendly and using warnings from this code is complicated so I'm inclined to keep the silent fallback on failure simple as is until someone can demonstrate of it causing a practical problem. Outside of unusual configurations, if this were ever happening when it is not expected, people observing low subprocess performance could strace and witness the vfork syscall failure. I'll merge once our CI is happy. -- ___ Python tracker <https://bugs.python.org/issue47151> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47175] Allow applications to tune the condition that triggers a GIL release and implementation choice in hashlib
New submission from Gregory P. Smith : ## Background All `hashlib` computations and `binascii.crc32` and `zlib.crc32` release the GIL around their computational core. But they use a hard coded length check to determine when to do so, or always do it. That already accomplishes the larger good of releasing the GIL on big computations. But _probably_ just serves to slow down smaller ones when a GIL release adds more overhead than a context switch to another thread could meaningfully provide in terms of forward progress. ## Desire 1 Determine if a threshold should exist at all (should we just always release the GIL for these?) and if so, allow it to be tuned on a per algorithm basis. This comes at the same time as other enhancements like bpo-47102 and its windows and macos cousins could shift us towards using OS kernel APIs for a subset of algorithms where available - which may effectively "always release" the GIL on OS APIs that are virtual IO call based such as bpo-47102's. ## Desire 2 When multiple implementations of an algorithm may be available, allow the user to configure data length thresholds for when each one is triggered. Without meaningfully slowing most things down by adding such logic. Different implementations have different setup and finalization time which can make them more useful for large data rather than tiny. --- I'm marking this low priority as it veers towards over-optimization. :) Creating benchmarks and a thing to live in Tools/ that people could run on their target platform to provide a tuning suggestion of what thresholds work best for their needs. Related inspiring work: OSes often benchmark several algorithm implementations up front to pick a "best" to use for a given platform (ex: see what the Linux kernel does for hashes and raid algorithms). This extends that idea and acknowledges latency as important, not exclusively thru-put. -- messages: 416401 nosy: gregory.p.smith priority: low severity: normal stage: needs patch status: open title: Allow applications to tune the condition that triggers a GIL release and implementation choice in hashlib type: enhancement ___ Python tracker <https://bugs.python.org/issue47175> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47151] subprocess fails when used as init, vfork() returns EINVAL if PID=1
Gregory P. Smith added the comment: Any possibility that you can test the attached PR as pid 1? -- ___ Python tracker <https://bugs.python.org/issue47151> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47151] subprocess fails when used as init, vfork() returns EINVAL if PID=1
Change by Gregory P. Smith : -- keywords: +patch pull_requests: +30263 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/32186 ___ Python tracker <https://bugs.python.org/issue47151> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44733] Feature request: maxtasksperchild for ProcessPoolExecutor
Gregory P. Smith added the comment: yep, branch off of a recent main. -- ___ Python tracker <https://bugs.python.org/issue44733> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33178] Add support for BigEndianUnion and LittleEndianUnion in ctypes
Gregory P. Smith added the comment: Thanks for the contribution! (our attention spans can be scattered and random, apologies for the delay) -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue33178> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33178] Add support for BigEndianUnion and LittleEndianUnion in ctypes
Gregory P. Smith added the comment: New changeset dc2d8404a3ab6288ce112c71da8c65c34cd3087e by Dave Goncalves in branch 'main': bpo-33178: Add BigEndianUnion, LittleEndianUnion classes to ctypes (GH-25480) https://github.com/python/cpython/commit/dc2d8404a3ab6288ce112c71da8c65c34cd3087e -- ___ Python tracker <https://bugs.python.org/issue33178> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47151] subprocess fails when used as init, vfork() returns EINVAL if PID=1
Change by Gregory P. Smith : -- components: +Extension Modules -Library (Lib) stage: -> needs patch title: vfork() returns EINVAL if PID=1 -> subprocess fails when used as init, vfork() returns EINVAL if PID=1 type: -> behavior ___ Python tracker <https://bugs.python.org/issue47151> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44493] Missing terminated NUL in the length of sockaddr_un
Change by Gregory P. Smith : -- resolution: -> fixed ___ Python tracker <https://bugs.python.org/issue44493> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44493] Missing terminated NUL in the length of sockaddr_un
Gregory P. Smith added the comment: New changeset dae09c2b819c2683ad870733451c050b59c3eb93 by Miss Islington (bot) in branch '3.9': [3.9] bpo-44493: Add missing terminated NUL in sockaddr_un's length (GH-26866) (GH-32140) (GH-32156) https://github.com/python/cpython/commit/dae09c2b819c2683ad870733451c050b59c3eb93 -- ___ Python tracker <https://bugs.python.org/issue44493> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44493] Missing terminated NUL in the length of sockaddr_un
Gregory P. Smith added the comment: New changeset f6b3a07b7df60dc04d0260169ffef6e9796a2124 by ty in branch 'main': bpo-44493: Add missing terminated NUL in sockaddr_un's length (GH-26866) https://github.com/python/cpython/commit/f6b3a07b7df60dc04d0260169ffef6e9796a2124 -- ___ Python tracker <https://bugs.python.org/issue44493> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47124] explore hashlib use of the Apple CryptoKit macOS
Gregory P. Smith added the comment: I only pointed to that API after a brief search without looking at details (Swift? oops!). If there is one available from C that'd also make sense to consider. The only things I expect, relevant to hashlib, that would be accelerated by OS native APIs most platforms are SHA2, maybe SHA1, and sometimes HMAC using those. I'm in no position to judge if there is value in using them, I'm just assuming there might be. The irony is that builds without OpenSSL are rare, so unless the OS native APIs provide tangible benefits it may not matter. (ex: the Linux APIs may allow for an efficient zero-copy variant of the new `hashlib.file_digest()` function) -- ___ Python tracker <https://bugs.python.org/issue47124> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47125] Explore hashlib use of the Windows Crypto API NG
New submission from Gregory P. Smith : https://docs.microsoft.com/en-us/windows/win32/seccng/creating-a-hash-with-cng See if these are worthwhile vs using OpenSSL for the hashlib algorithms it supports. OS APIs can in theory take better advantage of HW acceleration for performance. Verify this on supported architectures. Regardless it would allow some things to be fast if someone wanted to use a build without OpenSSL. I'm not a Windows or Microsoft user. I'm filing this for completeness, someone else would need to do the work. similar macOS issue: https://bugs.python.org/issue47124 similar Linux issue: https://bugs.python.org/issue47102 -- components: Extension Modules, Windows messages: 416033 nosy: gregory.p.smith, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: Explore hashlib use of the Windows Crypto API NG type: performance ___ Python tracker <https://bugs.python.org/issue47125> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47124] explore hashlib use of the Apple CryptoKit macOS
New submission from Gregory P. Smith : https://developer.apple.com/documentation/cryptokit/ in macOS 10.15+ This is a common place for platform specific hardware acceleration to be exposed to the user (especially on SoCs which often have non-standard hardware - Like Apples... which is presumably why they create this). What they offer is limited, but when present and running on a recent enough macOS, using their and SHA2 and HMAC(SHA2) implementations as well as Insecure.SHA1 is probably better than OpenSSL's. **Verify this.** It'd also allow those to be fast in a non-openssl build (as if anyone does those). I know little about mac building and packaging and how to have something target an older OS and use a 10.15+ API. So if this winds up only being used from aarch64 macOS builds (10.15+ by definition IIRC?) that could also work. I leave this issue for a macOS Apple API friendly person to take on. This issue is cousin to the Linux one: https://bugs.python.org/issue47102 -- components: Extension Modules, macOS messages: 416032 nosy: gregory.p.smith, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: explore hashlib use of the Apple CryptoKit macOS type: enhancement ___ Python tracker <https://bugs.python.org/issue47124> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44493] Missing terminated NUL in the length of sockaddr_un
Change by Gregory P. Smith : -- assignee: -> gregory.p.smith ___ Python tracker <https://bugs.python.org/issue44493> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44493] Missing terminated NUL in the length of sockaddr_un
Gregory P. Smith added the comment: It looks like the length would be short by one in Python before this change, meaning binding or connecting to a non-anonymous named AF_UNIX socket potentially loses the last character of the path name intended to be bound? Depending on if the OS uses the length or trusts a null termination? That should be an observable behavior change. It also suggests that fixing this could break code that has been working around this bug forever by adding an extra character when binding or connecting to a non-anonymous AF_UNIX socket? Is that true? In what circumstances is this practically observed? -- ___ Python tracker <https://bugs.python.org/issue44493> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44493] Missing terminated NUL in the length of sockaddr_un
Gregory P. Smith added the comment: Bug filing tip for ty/zonyitoo: Describe the problem in the text when filing the bug. Include its consequences and how it is observed in practice. A bug filed just saying "look at this code, this is wrong" does not communicate a need for anyone to spend time on an issue or help others find an existing issue describing misbehavior they may be seeing. -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue44493> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39298] add BLAKE3 to hashlib
Gregory P. Smith added the comment: You missed the key "And certainly more efficient in terms of watt-secs/byte" part. -- ___ Python tracker <https://bugs.python.org/issue39298> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28080] Allow reading member names with bogus encodings in zipfile
Gregory P. Smith added the comment: Thanks Serhiy! -- resolution: -> fixed stage: patch review -> commit review status: open -> closed ___ Python tracker <https://bugs.python.org/issue28080> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39298] add BLAKE3 to hashlib
Gregory P. Smith added the comment: Rust based anything comes with a baseline level of Rust code overhead. https://stackoverflow.com/questions/29008127/why-are-rust-executables-so-huge That seems expected. -- ___ Python tracker <https://bugs.python.org/issue39298> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47102] explore hashlib use of the Linux Kernel CryptoAPI
Gregory P. Smith added the comment: Neat. I've never used the API, just filing a breadcrumb suggesting we see if it makes sense. Being I/O based, that even takes care of GIL releasing from Python. :) -- ___ Python tracker <https://bugs.python.org/issue47102> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39298] add BLAKE3 to hashlib
Gregory P. Smith added the comment: Performance wise... The SHA series have hardware acceleration on modern CPUs and SoCs. External libraries such as OpenSSL are in a position to provide implementations that make use of that. Same with the Linux Kernel CryptoAPI (https://bugs.python.org/issue47102). Hardware accelerated SHAs are likely faster than blake3 single core. And certainly more efficient in terms of watt-secs/byte. -- ___ Python tracker <https://bugs.python.org/issue39298> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39298] add BLAKE3 to hashlib
Gregory P. Smith added the comment: To anyone else who comes along with motivation: I'm fine with blake3 being in hashlib, but I don't want us to guarantee it by carrying the implementation of the algorithm in the CPython codebase itself unless it gains wide industry standard-like adoption status. We should feel free to link to both the Rust blake3 and C blake3-py packages from the hashlib docs regardless. -- ___ Python tracker <https://bugs.python.org/issue39298> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47102] explore hashlib use of the Linux Kernel CryptoAPI
New submission from Gregory P. Smith : Linux kernels provide a CryptoAPI. This is a common place for platform specific hardware accelerated hash algorithms to be exposed to the user (especially on SoCs which often have non-standard hardware). https://www.kernel.org/doc/html/v4.10/crypto/userspace-if.html https://www.kernel.org/doc/html/v5.17/crypto/userspace-if.html https://www.chronox.de/libkcapi.html hashlib currently uses OpenSSL when possible for performance. We could also look at querying the kernel API. How to decide between the two implementations when both are present is something TBD. This would probably be best done via a configure time check for libkcapi? -- messages: 415896 nosy: christian.heimes, gregory.p.smith priority: normal severity: normal status: open title: explore hashlib use of the Linux Kernel CryptoAPI type: enhancement ___ Python tracker <https://bugs.python.org/issue47102> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39298] add BLAKE3 to hashlib
Gregory P. Smith added the comment: Because I don't think blake3 or blake2 _(though we've shipped it already so there's a challenge in making changes https://bugs.python.org/issue47095)_ are important enough to be _guaranteed_ present in all builds (our release binaries would include them). Depending on an external library for those to exist makes sense. I do not want CPython to get into the business of maintaining a complicated build process in-tree for third party architecture specific optimized code for non-core functionality purposes. That is best handled outside of the project & on CI and binary release hosts. I'm okay with blake3 in hashlib if we can avoid gaining another /impl/ tree that is a copy of large third party code and our own build system for it. Q: What benefits does having blake3 builtin vs getting it from PyPI bring? Q: Should we instead provide a way for third party provided hashes to be registered in `hashlib` similar to `codecs.register()`? I view the NIST standard hashes as important enough to attempt to guarantee as present (all the SHAs and MD5) as built-in. Others should really demonstrate practical application popularity to gain included battery status rather than just using PyPI. -- ___ Python tracker <https://bugs.python.org/issue39298> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37901] 21 tests fail when run on an IPv6-only host
Gregory P. Smith added the comment: Here's how I created an IPv6-only host: https://gist.github.com/gpshead/f4f394593674e5f7a58e9424b4dba989 -- ___ Python tracker <https://bugs.python.org/issue37901> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47095] Deprecate blake2's tree hashing feature
Gregory P. Smith added the comment: In the short term we should prefer libb2 linkage when available. As for deprecation, it'd be useful to research how often the options going away are used in code in PyPI packages and in Github repos to understand the deprecation impact. The PyPI landscape for blake2 modules is not great because we've had it in hashlib for a while. One of those, or a new one, would need to be created by someone who needs the non openssl features. ultimate goal: simplify what's in Modules/_blake2/impl/ if it cannot be removed. Use an external library for the implementation when possible (and in all our binary releases. Those are better maintained to take advantage of performance or hw features over time. -- ___ Python tracker <https://bugs.python.org/issue47095> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39298] add BLAKE3 to hashlib
Gregory P. Smith added the comment: correction: our md5/sha1/sha2/sha3 code is not gone yet, but they are simple C implementations used as a fallback when the provider of optimal versions are unavailable (openssl for those). That keeps the copies of code in our tree simple and most people use the optimal library version. -- ___ Python tracker <https://bugs.python.org/issue39298> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47090] Make zlib required on all platforms (simplifies code)
Gregory P. Smith added the comment: Bringing this up on discord, others point out that the windows build requires zlib for convenience when we transitioned from having a vendored copy in our repo and that smaller "embedded" use cases may not like this if they don't already need the dep. But there have been no complaints about it on the Windows side. `binascii.crc32` was one thing that motivated me to look into "just require zlib" to get out of carrying our own suboptimal crc32 code. along those lines, we should recommend people choose https://github.com/zlib-ng/zlib-ng rather than zlib.net for better performance. looking over my PR, it can make for some awkward code with zlib right next to others that we treat as optionals. good bad or indifferent? i'm leaning towards indifferent and still enjoying fewer lines of code. -- ___ Python tracker <https://bugs.python.org/issue47090> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39298] add BLAKE3 to hashlib
Gregory P. Smith added the comment: hashlib creator and other maintainer here: I do not think it was a good idea for us to add blake2 to hashlib the way we did. So blake3 should not be presumed as a given, at least not done in the same manner. Background: While OpenSSL gained _some_ blake2 support in 2016, around when we were adding it to hashlib (not a coincidence), we made a mistake: We offered an overly complex API. OpenSSL's own support for blake2 is a subset, not sufficient to be used as a replacement for the API we exposed so we are stuck with our vendored copy with no easy way off. https://github.com/openssl/openssl/issues/980 OpenSSL is not going to gain native blake3 support. https://github.com/openssl/openssl/issues/11613 Given that I don't want to see us gain new vendored copies of significant but non-critical third party hash code in our tree (Modules/_blake3/impl/ in PR 31686) for anything but a known fixed term need (ex: the sha2 libtomcrypt code is gone from our tree as was clearly going to happen from the start), the only way I think we should include blake3 support is if there is already a plan for that code to leave our tree in the future with a high probability of success. A `configure.ac` check for an installed blake3 library to build and link against would be appropriate. Along with updating relevant CI systems and Windows and macOS release build systems to have that available. That'd significantly shrink the PR to reasonable size. This means blake3 support should be considered optional as anyone doing their own CPython build may not have it. This is primarily a documentation issue: list it as such and provide one official documented API to detect its availability. Our binary releases will include it as will most OS distro packagers. It also means implementation details, performance and platform tuning are **not our problem** but those of the OS distro or external library provider. Regarding setup.py, what Christian says is true, that is on its way out. Do not add new non-trivial statements to it as that just creates more work for those working to untangle the mess. Getting rid of the /impl/ build in favor of an autoconf detected library gets rid of that mess. I'll file a separate issue to track moving blake2 in the same direction so we can lose it's /impl/. -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue39298> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47090] Make zlib required on all platforms (simplifies code)
Change by Gregory P. Smith : -- keywords: +patch pull_requests: +30133 pull_request: https://github.com/python/cpython/pull/32043 ___ Python tracker <https://bugs.python.org/issue47090> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47090] Make zlib required on all platforms (simplifies code)
New submission from Gregory P. Smith : We have a pile of conditionals and extra code in CPython to deal with building on systems that do not have zlib. The zlib C library has been around forever at this point and should be present on every system in the world. zlib is already required on Windows to build CPython. Proposal: simplify our code by removing the conditionals around zlib being optional. I'm attaching a draft PR to show what this would look like. -- assignee: gregory.p.smith components: Build, Extension Modules, Library (Lib) messages: 415750 nosy: gregory.p.smith priority: normal severity: normal stage: patch review status: open title: Make zlib required on all platforms (simplifies code) versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue47090> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28080] Allow reading member names with bogus encodings in zipfile
Gregory P. Smith added the comment: Your PR looks good to me. I agree with not making it easy to _write_ zipfiles with non-standard encoding used for names. There is a possibility that someone wants that ability when writing zip files (not yet clear) in https://bugs.python.org/issue40172. I don't like it, but if the need exists, that feature can be addressed on that issue via relevant options. -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue28080> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40172] ZipInfo corrupts file names in some old zip archives
Gregory P. Smith added the comment: Examining Lib/zipfile.py code, the existing code makes sense. Python's zipfile module produces modern zipfiles when writing by setting the utf-8 flag and storing the filename as utf-8 when it is not ASCII. This is desirable for use with all normal zip implementations in the past 10-15 years. When decoding a zipfile, if the utf-8 flag is not set, we assume cp437 per the pkware zip appnotes.txt "spec". So our reading is correct as well, even for very old files. This is being strict in what we produce an lenient in what we accept. caveats? yes: If someone does need to produce zipfiles for use with ancient software that does not support utf-8, that also does not identify the unknown utf-8 flag as an error condition, it will interpret the name in a corrupt manner for non-ascii names. Similarly, even if written with cp437 names (as PR 19335 would do), in old zip system implementations where the implementation blindly uses the users locale encoding instead of cp437, it will always see corrupt data in that scenario. (aka mojibake?) These are not what I'd expect to be normal use cases. Do you have a common practical example of a need for this? (The PR on issue28080 provides a way to _read_ legacy zip files that used a codec other than cp437 if you know what it was.) --- https://www.loc.gov/preservation/digital/formats/fdd/fdd000354.shtml may also be of interest regarding the zip format. -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue40172> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47027] subprocess.run(), subprocess.Popen() should accept file descriptor as cwd parameter
Gregory P. Smith added the comment: Basically you want it to call fchdir() instead of chdir() when passed a fd (integer) instead of a string/Path-like. That makes sense and should be a reasonably straight forward set of changes to _posixsubprocess.c. (A way to convert a fd into a Path-like object would _not_ work as that'd reintroduce the TOCTOU on the directory - that'd be a pathlib feature request anyways, not a subprocess one) -- stage: -> needs patch versions: +Python 3.11 -Python 3.10, Python 3.9 ___ Python tracker <https://bugs.python.org/issue47027> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38256] binascii.crc32 is not 64-bit clean when USE_ZLIB_CRC32
Gregory P. Smith added the comment: 3.8 and older are in security fix only mode. 3.9+ have been fixed. realistically this was a rare issue with multiple workarounds. The 3.9 and 3.10 fixes were strictly bugfix only. The 3.11 fix included some performance improvements. -- resolution: -> fixed stage: patch review -> commit review status: open -> closed ___ Python tracker <https://bugs.python.org/issue38256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38256] binascii.crc32 is not 64-bit clean when USE_ZLIB_CRC32
Gregory P. Smith added the comment: New changeset 58a7e130375776b192a99b013bc563205a639edc by Gregory P. Smith in branch '3.9': bpo-38256: Fix binascii.crc32 large input. (GH-32000) (GH-32013) (GH-32015) https://github.com/python/cpython/commit/58a7e130375776b192a99b013bc563205a639edc -- ___ Python tracker <https://bugs.python.org/issue38256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38256] binascii.crc32 is not 64-bit clean when USE_ZLIB_CRC32
Change by Gregory P. Smith : -- pull_requests: +30103 pull_request: https://github.com/python/cpython/pull/32015 ___ Python tracker <https://bugs.python.org/issue38256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38256] binascii.crc32 is not 64-bit clean when USE_ZLIB_CRC32
Gregory P. Smith added the comment: New changeset 4c989e19c84ec224655bbbde9422e16d4a838a80 by Gregory P. Smith in branch '3.10': [3.10] bpo-38256: Fix binascii.crc32 large input. (GH-32000) (GH-32013) https://github.com/python/cpython/commit/4c989e19c84ec224655bbbde9422e16d4a838a80 -- ___ Python tracker <https://bugs.python.org/issue38256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47064] thread QOS attribute on macOS
Gregory P. Smith added the comment: Definitely not an OS specific concept. Big.little configs are all over the place on all OSes and architectures these days. With a variety of ways OSes are experimenting with scheduling for them (nevermind that Android and iOS have been doing this for a decade). from scheduling policies such as users, cgroup membership, container membership, group membership, nice level, to QoS attributes as you mention here... If a given OS has specific APIs we should expose them via the os or _thread modules as appropriate. But unless multiple OSes congeal around common concepts, there isn't much we can do for abstraction. Perhaps a generic way to supply a generic OS specific settings instance to threading.Thread for implementation defined configuration purposes before Thread.start() is called. -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue47064> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47065] test_curses fails if terminal defaults to bright white text (15)
Gregory P. Smith added the comment: can we reliably query for what the default is and add that to the expectations? or just skip the test if the defaults don't match our expectations (less ideal)? if those are a challenge to do reliably - adding 15, 0 to the expected set may be the most practical. -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue47065> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47054] "SyntaxError: non-default argument follows default argument" confuses
Change by Gregory P. Smith : -- assignee: -> gregory.p.smith ___ Python tracker <https://bugs.python.org/issue47054> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47054] "SyntaxError: non-default argument follows default argument" confuses
Change by Gregory P. Smith : -- keywords: +patch nosy: +gregory.p.smith nosy_count: 4.0 -> 5.0 pull_requests: +30102 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/32014 ___ Python tracker <https://bugs.python.org/issue47054> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue25489] sys.exit() caught in async event loop exception handler
Change by Gregory P. Smith : -- title: sys.exit() caught in exception handler -> sys.exit() caught in async event loop exception handler ___ Python tracker <https://bugs.python.org/issue25489> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38256] binascii.crc32 is not 64-bit clean when USE_ZLIB_CRC32
Change by Gregory P. Smith : -- pull_requests: +30101 pull_request: https://github.com/python/cpython/pull/32013 ___ Python tracker <https://bugs.python.org/issue38256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38256] binascii.crc32 is not 64-bit clean when USE_ZLIB_CRC32
Gregory P. Smith added the comment: New changeset 9d1c4d69dbc800ac344565119337fcf490cdc800 by Gregory P. Smith in branch 'main': bpo-38256: Fix binascii.crc32() when inputs are 4+GiB (GH-32000) https://github.com/python/cpython/commit/9d1c4d69dbc800ac344565119337fcf490cdc800 -- ___ Python tracker <https://bugs.python.org/issue38256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47040] Fix confusing versionchanged note in crc32 and adler32
Gregory P. Smith added the comment: New changeset 6d290d5862375799e997f1192ef56abca4e9182e by Ma Lin in branch '3.10': [3.10] bpo-47040: improve document of checksum functions (GH-31955) (GH-32002) https://github.com/python/cpython/commit/6d290d5862375799e997f1192ef56abca4e9182e -- ___ Python tracker <https://bugs.python.org/issue47040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39640] fall back os.fdatasync() to fsync() on POSIX systems without fdatasync() support
Gregory P. Smith added the comment:
The os module provides a pretty low level simple shim over platform APIs. It is
better for logic like this to live in a higher level application library rather
than make big assumptions on the part of the user.
```
try:
os.fdatasync(fd)
except Exception as err:
logging.debug("fdatasync(fd) failed %s, falling back to fsync(fd)", err)
os.fsync(fd)
```
--
nosy: +gregory.p.smith
resolution: -> rejected
stage: patch review -> resolved
status: open -> closed
type: -> enhancement
___
Python tracker
<https://bugs.python.org/issue39640>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27198] Adding an assertClose() method to unittest.TestCase
Gregory P. Smith added the comment: I agree with the decision, assertAlmostEqual is where the feature belongs. >From a practical point of view I suspect a lot of people who want this in the >wider world today use just `assert math.isclose(...)` despite the less useful >error message. -- nosy: +gregory.p.smith status: pending -> open versions: +Python 3.11 -Python 3.6 ___ Python tracker <https://bugs.python.org/issue27198> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38256] binascii.crc32 is not 64-bit clean when USE_ZLIB_CRC32
Change by Gregory P. Smith : -- assignee: -> gregory.p.smith ___ Python tracker <https://bugs.python.org/issue38256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47040] Fix confusing versionchanged note in crc32 and adler32
Gregory P. Smith added the comment: If you want to backport the documentation updates, feel free to make PRs for that. -- assignee: docs@python -> gregory.p.smith resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue47040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47040] Fix confusing versionchanged note in crc32 and adler32
Gregory P. Smith added the comment: FYI - https://bugs.python.org/issue38256 covers the 32-bit bug. -- ___ Python tracker <https://bugs.python.org/issue47040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38256] binascii.crc32 is not 64-bit clean when USE_ZLIB_CRC32
Change by Gregory P. Smith : -- keywords: +patch pull_requests: +30089 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/32000 ___ Python tracker <https://bugs.python.org/issue38256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38256] binascii.crc32 is not 64-bit clean when USE_ZLIB_CRC32
Gregory P. Smith added the comment: it depends on the build. USE_ZLIB_CRC32 causes it due to zlib's 32-bitness as noted my marko. $ ./python Python 3.11.0a6+ (heads/main-dirty:b3f2d4c8ba, Mar 19 2022, 15:32:04) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import binascii, zlib >>> bigdata=memoryview(bytearray((1<<32) + 100)) >>> >>> print(binascii.crc32(bigdata)) 2575877834 >>> crc = binascii.crc32(bigdata[:1000]) >>> crc = binascii.crc32(bigdata[1000:], crc) >>> print(crc) 2838121701 >>> >>> print(zlib.crc32(bigdata)) 2838121701 >>> crc = zlib.crc32(bigdata[:1000]) >>> crc = zlib.crc32(bigdata[1000:], crc) >>> print(crc) 2838121701 >>> -- resolution: works for me -> stage: resolved -> needs patch status: closed -> open title: binascii.crc32 is not 64-bit clean -> binascii.crc32 is not 64-bit clean when USE_ZLIB_CRC32 versions: +Python 3.10, Python 3.11, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue38256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38256] binascii.crc32 is not 64-bit clean
Gregory P. Smith added the comment: ``` $ python3.8 Python 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import binascii, zlib >>> bigdata=memoryview(bytearray((1<<32) + 100)) >>> >>> print(binascii.crc32(bigdata)) 2838121701 >>> crc = binascii.crc32(bigdata[:1000]) >>> crc = binascii.crc32(bigdata[1000:], crc) >>> print(crc) 2838121701 >>> >>> print(zlib.crc32(bigdata)) 2838121701 >>> crc = zlib.crc32(bigdata[:1000]) >>> crc = zlib.crc32(bigdata[1000:], crc) >>> print(crc) 2838121701 ``` -- nosy: +gregory.p.smith resolution: -> works for me stage: -> resolved status: open -> closed versions: -Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue38256> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47040] Fix confusing versionchanged note in crc32 and adler32
Change by Gregory P. Smith : -- title: Remove invalid versionchanged in doc -> Fix confusing versionchanged note in crc32 and adler32 ___ Python tracker <https://bugs.python.org/issue47040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47040] Remove invalid versionchanged in doc
Gregory P. Smith added the comment: New changeset b3f2d4c8bab52573605c96c809a1e2162eee9d7e by Ma Lin in branch 'main': bpo-47040: improve document of checksum functions (gh-31955) https://github.com/python/cpython/commit/b3f2d4c8bab52573605c96c809a1e2162eee9d7e -- ___ Python tracker <https://bugs.python.org/issue47040> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45033] Calls to PyErr_PrintEx in destructors cause calling async functions to incorrectly return None
Change by Gregory P. Smith : -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue45033> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46948] [CVE-2022-26488] Escalation of privilege via Windows Installer
Gregory P. Smith added the comment: Is there anything on our end we can do to prevent this kind of issue in the future? Am I wrong to see this as just fixing our package to avoid a design flaw in Windows OS level package management? Certainly other packages in the world must run into similar problems. -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue46948> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue38738] Fix formatting of True and False
Gregory P. Smith added the comment: New changeset 46a116c1c9f6b60a3d35ab9a419f8eee5de2542e by Géry Ogam in branch 'main': bpo-38738: Fix formatting of True and False in the threading documentation (GH-31678) https://github.com/python/cpython/commit/46a116c1c9f6b60a3d35ab9a419f8eee5de2542e -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue38738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46896] add support for watching writes to selected dictionaries
Gregory P. Smith added the comment: Per interpreter seems best. If someone using this feature writes a buggy implementation of a callback that doesn't chain reliably, that is a bug in their code and all of the fallout from that is "just" a bug to be fixed in said code. Think of it like a C signal handler, the OS doesn't chain for you - it is the signal handler installer's responsibility to chain to the previous one. Not an unusual pattern for callback hooks in C. We already have such global C callback hooks in SetProfile and SetTrace. https://docs.python.org/3/c-api/init.html#profiling-and-tracing Those don't even provide for chaining. We could do the same here and not let a hook be set more than once instead of providing a Get API to allow for manual chaining. That seems less useful. -- ___ Python tracker <https://bugs.python.org/issue46896> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46896] add support for watching writes to selected dictionaries
Change by Gregory P. Smith : -- nosy: +Mark.Shannon ___ Python tracker <https://bugs.python.org/issue46896> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46896] add support for watching writes to selected dictionaries
Gregory P. Smith added the comment: At first quick glance, this makes sense and the API looks reasonable. Question: what happens on interpreter shutdown? Shutdown obviously finalized and clears out most all dicts. I guess the C callback simply gets called for each of these? That makes sense. Just wondering if there are any ramifications of that. The callback is in C so it shouldn't have issues with this. A pyperformance suite run on an interpreter modified to support this but having no callbacks registered would be useful. (basically judging if there is measurable overhead added by the watched bit check - I doubt it'll be noticeable in most code) -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue46896> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45373] ./configure --enable-optimizations should enable LTO
Gregory P. Smith added the comment: FWIW I agree that we should try adding LTO to --enable-optimizations now. -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue45373> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46784] Duplicated symbols when linking embedded Python with libexpat
Gregory P. Smith added the comment: err "release managers" same thing right? ;) -- ___ Python tracker <https://bugs.python.org/issue46784> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46784] Duplicated symbols when linking embedded Python with libexpat
Gregory P. Smith added the comment: PRs for 3.7 and 3.8 remain up to release blockers. -- components: +Build resolution: -> fixed stage: patch review -> commit review status: open -> closed ___ Python tracker <https://bugs.python.org/issue46784> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46784] Duplicated symbols when linking embedded Python with libexpat
Gregory P. Smith added the comment: New changeset e05e3d20d309603010f2c1194e612f894ad8a985 by Gregory P. Smith in branch '3.10': [3.10] bpo-46784: Add newly exported expat symbols to the namespace. (GH-31397) (GH-31420) https://github.com/python/cpython/commit/e05e3d20d309603010f2c1194e612f894ad8a985 -- ___ Python tracker <https://bugs.python.org/issue46784> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46200] Discourage logging f-strings due to security considerations
Gregory P. Smith added the comment: A new system of logging APIs has been on several of our (core dev and otherwise) minds ever since f-strings were introduced. For this specific issue, agreed that documentation is key. The old logging APIs cannot change. And practically preventing passing non-literal constant values in as the first parameter within their implementation while there are theoretical ways to _maybe_ do that, I expect none would be maintainable or performant. For https://www.python.org/dev/peps/pep-0675/#logging-format-string-injection, my gut feeling is you want logging functions annotated as is a duality using overload. ``` @overload def info(str) -> None: ... @overload def info(msg: LiteralString, *args: object) -> None: ``` As that prevents the untrusted template in msg so you could use a lone f-str. This larger conversation is better turned into a python-dev or discuss.python.org thread than done in our poor bug tracker as anything more than the doc update and musing around the annotations turns into an evolved logging system design discussion. (a nice thread internally at work just restarted about this and linked here... people want f-logging!) -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue46200> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46784] Duplicated symbols when linking embedded Python with libexpat
Change by Gregory P. Smith : -- pull_requests: +29557 pull_request: https://github.com/python/cpython/pull/31420 ___ Python tracker <https://bugs.python.org/issue46784> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46793] expose expat XML billion laughs attack mitigation APIs
New submission from Gregory P. Smith : Quoting from https://github.com/python/cpython/pull/31397#issuecomment-1044796561 """ XML_SetBillionLaughsAttackProtectionActivationThreshold XML_SetBillionLaughsAttackProtectionMaximumAmplification I still hope that someone can make those two^^ accessible (with additional glue code) to the user on pyexpat level in CPython. """ - Sebastian Pipping @hartwork -- messages: 413513 nosy: gregory.p.smith priority: normal severity: normal stage: needs patch status: open title: expose expat XML billion laughs attack mitigation APIs type: enhancement versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue46793> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46784] Duplicated symbols when linking embedded Python with libexpat
Change by Gregory P. Smith : -- nosy: +gregory.p.smith ___ Python tracker <https://bugs.python.org/issue46784> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20342] Endianness not detected correctly due to AC_RUN_IFELSE macros
Gregory P. Smith added the comment: I'll leave it open, nothing about this should be ppc64 vs ppc64el specific. Our cross compilation story has historically been not great. Building for a target of one endianness from a build host of other endianness... sounds like exactly one of many kinds of thing that we might have a mess with until someone can confirm. (I'm not setup with a big endian toolchain to build for a qemu target right now so I can't easily confirm) -- ___ Python tracker <https://bugs.python.org/issue20342> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46648] `test.test_urllib2.MiscTests.test_issue16464` flaky due to external connection
Change by Gregory P. Smith : -- assignee: -> sobolevn ___ Python tracker <https://bugs.python.org/issue46648> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
