[issue46888] SharedMemory.close() destroys memory

[Eryk Sun]

Eryk Sun  added the comment:

> The persistent mode sounds just like Python shared memory also works
> on Linux (where I can have these /dev/shm/* files even after the 
> Python process ends) but I think on Windows, Python is not using 
> the persistent mode and thus the shared memory goes away, in contrast
> to how it works on Linux.

Unix has the notion that everything is a file as a central organizing concept. 
As such, Linux opts to implement POSIX shm_open() [1] with a tmpfs [2] 
filesystem that's mounted on /dev/shm. tmpfs filesystems exist in virtual 
memory. They're not persistent in the sense of being created on a physical disk 
that provides persistent storage.

Windows has the notion that everything is an object as a central organizing 
concept. Internally, the system has a "\" root object directory, which contains 
other object directories and object symbolic links (unrelated to filesystem 
symlinks). Each object directory, including the root directory, contains named 
kernel objects. 

Named device objects are normally created in r"\Device", such as 
r"\Device\HarddiskVolume2", and global symbolic links to devices are created in 
r"\GLOBAL??", such as r"\GLOBAL??\C:" -> r"\Device\HarddiskVolume2" for the 
"C:" drive. The root registry key object is r"\REGISTRY", which contains 
dynamic key objects such as r"\REGISTRY\MACHINE" (referenced via the 
pseudohandle HKEY_LOCAL_MACHINE) and "\REGISTRY\USER" (referenced via the 
pseudohandle HKEY_USERS), which in turn contain other keys such as 
"\REGISTRY\MACHINE\SOFTWARE" on which registry hives are mounted.

For naming global kernel objects and session 0 objects, the Windows API 
internally uses the directory r"\BaseNamedObjects". For interactive Windows 
sessions, it uses r"\Sessions\\BaseNamedObjects" and, for app 
containers, subdirectories of r"\Sessions\\AppContainerNamedObjects". To explicitly name an object in the global 
directory, use the path r"Global\". The "Global" prefix is 
implemented as an object symbolic link to r"\BaseNamedObjects". Of course, 
that's an internal implementation detail; the API just refers to the 
r"Global\\" prefix.

Naming a kernel object in r"\BaseNamedObjects" is nearly equivalent to creating 
a file in a tmpfs filesystem in Linux, with one major difference. Objects are 
reference counted, with a handle reference count and a pointer reference count. 
Opening a handle increments both counters, but kernel code can use just a 
pointer reference instead of opening a handle. By default, objects are 
temporary. As such, when their pointer reference count is decremented to 0, 
they get unlinked from the object namespace, if they're named objects, and 
deallocated.

It's possible to create a permanent object in the object namespace, either 
initially via the OBJ_PERMANENT attribute [3], or later on via 
NtMakePermanentObject(handle). However, making an object permanent is 
considered a super privilege in Windows, so privileged in fact that the Windows 
API doesn't support this capability, at least not as far as I know. By default, 
SeCreatePermanentPrivilege is only granted to the SYSTEM account. Also, 
creating a named section object (i.e. file mapping object) in a 
session-restricted directory such as r"\BaseNamedObjects" requires 
SeCreateGlobalPrivilege, which is only granted by default to administrators and 
system service accounts. 

Unix and Linux are less concerned about creating 'permanent' files and global 
shared memory in virtual filesystems such as tmpfs. The lifetime while the 
system is running is left up to the creator, which has to manually remove the 
file, such as via shm_unlink() [4]. Stale files could accumulate in "/dev/shm" 
when processes crash, which is why a separate resource tracker is required, 
implementing what the Windows kernel provides by default.

---
[1] https://pubs.opengroup.org/onlinepubs/9699919799/functions/shm_open.html
[2] https://www.kernel.org/doc/Documentation/filesystems/tmpfs.txt
[3] 
https://docs.microsoft.com/en-us/windows/win32/api/ntdef/ns-ntdef-_object_attributes
[4] https://pubs.opengroup.org/onlinepubs/9699919799/functions/shm_unlink.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46896] add support for watching writes to selected dictionaries

[Gregory P. Smith]

Gregory P. Smith  added the comment:

At first quick glance, this makes sense and the API looks reasonable.

Question: what happens on interpreter shutdown?

Shutdown obviously finalized and clears out most all dicts.  I guess the C 
callback simply gets called for each of these?  That makes sense.  Just 
wondering if there are any ramifications of that.  The callback is in C so it 
shouldn't have issues with this.

A pyperformance suite run on an interpreter modified to support this but having 
no callbacks registered would be useful.  (basically judging if there is 
measurable overhead added by the watched bit check - I doubt it'll be 
noticeable in most code)

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43853] [sqlite3] Improve sqlite3_value_text() error handling

[Erlend E. Aasland]

Change by Erlend E. Aasland :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46195] Annotated and Optional get_type_hints buggy interaction

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:

This is now fixed in 3.11, but we'll leave 3.10 and 3.9 alone. Thanks for your 
bug report!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46195] Annotated and Optional get_type_hints buggy interaction

[Jelle Zijlstra]

Jelle Zijlstra  added the comment:


New changeset 20a1c8ee4bcb1c421b7cca1f3f5d6ad7ce30a9c9 by Nikita Sobolev in 
branch 'main':
bpo-46195: Do not add `Optional` in `get_type_hints` (GH-30304)
https://github.com/python/cpython/commit/20a1c8ee4bcb1c421b7cca1f3f5d6ad7ce30a9c9


--
nosy: +Jelle Zijlstra

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46848] Use optimized string search function in mmap.find()

[Dennis Sweeney]

Dennis Sweeney  added the comment:

Thanks for the report!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> performance

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46848] Use optimized string search function in mmap.find()

[Dennis Sweeney]

Dennis Sweeney  added the comment:


New changeset 6ddb09f35b922a3bbb59e408a3ca7636a6938468 by Dennis Sweeney in 
branch 'main':
bpo-46848: Use stringlib/fastsearch in mmap (GH-31625)
https://github.com/python/cpython/commit/6ddb09f35b922a3bbb59e408a3ca7636a6938468


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43853] [sqlite3] Improve sqlite3_value_text() error handling

[miss-islington]

miss-islington  added the comment:


New changeset 3ea2a8f425d26e81d914c54d477e9d56eb27ac98 by Erlend Egeberg 
Aasland in branch '3.9':
[3.9] bpo-43853: Expand test suite for SQLite UDF's (GH-27642) (GH-31030) 
(GH-31586)
https://github.com/python/cpython/commit/3ea2a8f425d26e81d914c54d477e9d56eb27ac98


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46888] SharedMemory.close() destroys memory

[Ronny Rentner]

Ronny Rentner  added the comment:

Thanks for your quick response.

My bigger scope is real time audio and video processing where I use multiple 
processes that need to be synchronized. I use shared memory for that.

As a small spin off, I've hacked together a dict that is using shared memory as 
a storage.

It works like this: It uses one shared memory space for streaming updates. This 
is efficient because only changes are transferred. Once the streaming shared 
memory buffer is full or if any single update to the dict is larger than the 
streaming buffer, it creates a full dump of the whole dict in a new shared 
memory that is just as big as needed. Any user of the dict would then consume 
the full dump.

On Linux that works great. Any user of the dict can create a full dump in a new 
shared memory and all other users of the same dict can consume it.

On Windows, the issue is if the creator process of the full dump goes away, the 
shared memory goes away. This is in contrast to the Python docs, unfortunately.

I don't fully understand the underlying implementations, but I've been looking 
at https://docs.microsoft.com/en-us/dotnet/standard/io/memory-mapped-files and 
I understand there are 2 main modes.

The persistent mode sounds just like Python shared memory also works on Linux 
(where I can have these /dev/shm/* files even after the Python process ends) 
but I think on Windows, Python is not using the persistent mode and thus the 
shared memory goes away, in contrast to how it works on Linux.

PS: You can find the code for this shared dict here 
https://github.com/ronny-rentner/UltraDict - Please note, it's an eary lack and 
not well tested.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6143] IDLE - clear and restart the shell window

[Terry J. Reedy]

Terry J. Reedy  added the comment:

https://stackoverflow.com/questions/71290382/how-do-i-adjust-python-idle-shell-settings-so-that-the-next-line-of-code-i-type
is from someone who, for whatever reason, likes having blank space in the shell 
below the input prompt.  It is not clear whether deleting lines to make that 
happen would be satisfactory, or if all lines must be accessible.

--
versions: +Python 3.11 -Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46891] Crash in ModuleType subclass with __slots__

[Dennis Sweeney]

Change by Dennis Sweeney :


--
nosy: +Mark.Shannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46891] Crash in ModuleType subclass with __slots__

[Dennis Sweeney]

Dennis Sweeney  added the comment:

Bisected to here:

c3f52b4d707a78eb342372a2be00f3eb846a05b9 is the first bad commit
commit c3f52b4d707a78eb342372a2be00f3eb846a05b9
Author: Mark Shannon 
Date:   Wed Jun 23 10:00:43 2021 +0100

bpo-44486: Make sure that modules always have a dictionary. (GH-26847)

* Make sure that modules always have a dictionary.

--
nosy: +Dennis Sweeney

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46841] Inline bytecode caches

[Brandt Bucher]

Change by Brandt Bucher :


--
pull_requests: +29761
pull_request: https://github.com/python/cpython/pull/31640

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46863] Python 3.10 OpenSSL Configuration Issues

[Adam]

Adam  added the comment:

Many thanks Christian, see the attached for the output of the commands on 
Python 3.9.10 and 3.10.2, along with a diff removing version numbers and memory 
addresses. 

I've run the commands on the Ubuntu distribution, we can also run the same for 
the Centos VM, if helpful.

There are a few differences in the outputs but nothing that appears obviously 
the cause.

--
Added file: https://bugs.python.org/file50654/python_details.tar.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46896] add support for watching writes to selected dictionaries

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46899] use of uninitialized value with msan from subprocess_fork_exec

[Yilei Yang]

New submission from Yilei Yang :

The uid & gid variable from 
https://github.com/python/cpython/blob/9833bb91e4d5c2606421d9ec2085f5c2dfb6f72c/Modules/_posixsubprocess.c#L737-L738
 can be passed to the `do_fork_exec` call below uninitialized and cause msan to 
report use-of-uninitialized-value errors.

Those variables are guarded by call_setgid/call_setuid so they aren't really 
used uninitialized in practice. It would just be great if we can make it msan 
clean.

Ideally, the long list of do_fork_exec arguments could also be rewritten in a 
struct.

--
components: Library (Lib)
messages: 414320
nosy: gregory.p.smith, yilei
priority: normal
severity: normal
status: open
title: use of uninitialized value with msan from subprocess_fork_exec
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15373] copy.copy() does not properly copy os.environment

[Eryk Sun]

Eryk Sun  added the comment:

In bpo-28824, I suggested preserving the case of environment variables in 
Windows by using a case-insensitive subclass of str in the encodekey() 
function. This is self-contained by the use of the encodekey() and decodekey() 
functions in the mapping methods such as __iter__(). The reason for this is 
mostly about aesthetics, but also about faithfully displaying the actual 
environment variable names. Opinions vary, but to me "WindowsSdkVerBinPath" is 
both easier on my eyes and easier to read than "WINDOWSSDKVERBINPATH". The 
eyesore factor gets amplified when it's a wall of all upper-cased names 
'screaming' at me.

A copy via dict(os.environ) would use regular str keys and lose the 
case-insensitive, case-preserving property. It would be more useful if 
os.environ.copy() were implemented to return a copy that keeps the 
case-insensitive property for keys but disables updating the process 
environment. This could be implemented by making putenv and unsetenv parameters 
in the constructor. If self.putenv or self.unsetenv is None, then __setitem__() 
or __delitem__() woud have no effect on the process environment. The copy() 
method would return a new _Environ instance for self._data.copy(), one which of 
course disables updating the process environment.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15373] copy.copy() does not properly copy os.environment

[Max Katsev]

Max Katsev  added the comment:

Note that deepcopy doesn't work either, even though it looks like it does at 
the first glance (which is arguably worse since it's harder to notice):

Python 3.8.6 (default, Jun  4 2021, 05:16:01)
>>> import copy, os, subprocess
>>> env_copy = copy.deepcopy(os.environ)
>>> env_copy["TEST"] = "oh no"
>>> os.environ["TEST"]
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/local/fbcode/platform009/lib/python3.8/os.py", line 675, in 
__getitem__
raise KeyError(key) from None
KeyError: 'TEST'
>>> subprocess.run("echo $TEST", shell=True, 
>>> capture_output=True).stdout.decode()
'oh no\n'

--
nosy: +mkatsev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46845] dict: Use smaller entry for Unicode-key only dict.

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46845] dict: Use smaller entry for Unicode-key only dict.

[Inada Naoki]

Inada Naoki  added the comment:


New changeset 9833bb91e4d5c2606421d9ec2085f5c2dfb6f72c by Inada Naoki in branch 
'main':
bpo-46845: Reduce dict size when all keys are Unicode (GH-31564)
https://github.com/python/cpython/commit/9833bb91e4d5c2606421d9ec2085f5c2dfb6f72c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45373] ./configure --enable-optimizations should enable LTO

[Inada Naoki]

Inada Naoki  added the comment:

Can we use --lto=thin when availabe?
And can we not use --lto when building profiling python?

--
nosy: +methane

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

> Draft PR for Cython: https://github.com/cython/cython/pull/4671

Notes on how Cython access PyFrameObject fields: 
https://bugs.python.org/issue40421#msg414314

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40421] [C API] Add getter functions for PyFrameObject and maybe move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

I searched for "\<_f" regex in Cython (0.29.x) branch. I found the following 
code getting or setting PyFrameObject fields.

== Set f_back ==

__Pyx_Coroutine_SendEx() at Utility/Coroutine.c:721:

f->f_back = PyThreadState_GetFrame(tstate);

__Pyx_Coroutine_ResetFrameBackpointer() at Utility/Coroutine.c:775:

Py_CLEAR(f->f_back);


== Set f_lineno ==

__Pyx_PyFrame_SetLineNumber() at Utility/ModuleSetupCode.c:543:

#define __Pyx_PyFrame_SetLineNumber(frame, lineno)  (frame)->f_lineno = 
(lineno)


__Pyx_PyFrame_SetLineNumber() is called by 3 functions:

* __Pyx_AddTraceback()
* __Pyx_call_line_trace_func()
* __Pyx_TraceSetupAndCall()

__Pyx_AddTraceback() pseudo-code:

static void __Pyx_AddTraceback(const char *funcname, int c_line,
   int py_line, const char *filename)
{
py_frame = PyFrame_New(..., py_code, ...);
__Pyx_PyFrame_SetLineNumber(py_frame, py_line);
...
}



== f_localsplus ==

If the CYTHON_FAST_PYCALL macro is defined, sizeof(PyFrameObject) is used to 
get the f_localsplus member.

__Pyx_PyFrame_GetLocalsplus() at Utility/ObjectHandling.c:1996:
---
// Initialised by module init code.
static size_t __pyx_pyframe_localsplus_offset = 0;

#include "frameobject.h"

#define __Pxy_PyFrame_Initialize_Offsets()  \
  ((void)__Pyx_BUILD_ASSERT_EXPR(sizeof(PyFrameObject) == 
offsetof(PyFrameObject, f_localsplus) + Py_MEMBER_SIZE(PyFrameObject, 
f_localsplus)), \
   (void)(__pyx_pyframe_localsplus_offset = ((size_t)PyFrame_Type.tp_basicsize) 
- Py_MEMBER_SIZE(PyFrameObject, f_localsplus)))
#define __Pyx_PyFrame_GetLocalsplus(frame)  \
  (assert(__pyx_pyframe_localsplus_offset), (PyObject **)(((char *)(frame)) + 
__pyx_pyframe_localsplus_offset))
---

== Get f_trace ==

__Pyx_TraceLine() at Utility/Profile.c:238:

if (__Pyx_IsTracing(tstate, 0, 0) && tstate->c_tracefunc && 
$frame_cname->f_trace)


== Set f_frame ==

__Pyx_TraceSetupAndCall() at Utility/Profile.c:299:

(*frame)->f_tstate = tstate;

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45373] ./configure --enable-optimizations should enable LTO

[Ned Deily]

Ned Deily  added the comment:

Sorry, this slipped off my radar and I haven't gone back and checked older 
versions of macOS. But it certainly is true that at least with the current 
versions of the Apple Developer Tools (either the Command Line Tools or Xcode) 
for macOS 11 (Big Sur) and macOS 12 (Monterey), things just work.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46876] Walrus operator not in help

[Alex Waygood]

Change by Alex Waygood :


--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46876] Walrus operator not in help

[Alex Waygood]

Change by Alex Waygood :


--
status: pending -> open
superseder:  -> Assignment expression symbol (walrus) not in built-in help()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45373] ./configure --enable-optimizations should enable LTO

[Gregory P. Smith]

Gregory P. Smith  added the comment:

FWIW I agree that we should try adding LTO to --enable-optimizations now.

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46876] Walrus operator not in help

[Alex Waygood]

Alex Waygood  added the comment:

Thanks for the report! I think this is a duplicate of Issue37082.

--
nosy: +AlexWaygood
resolution:  -> duplicate
stage: needs patch -> resolved
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46896] add support for watching writes to selected dictionaries

[Carl Meyer]

Change by Carl Meyer :


--
title: add support for watching writes to selecting dictionaries -> add support 
for watching writes to selected dictionaries

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

Draft PR for gevent: https://github.com/gevent/gevent/pull/1872

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46895] Allow extensions to set a callback to be invoked when a type is modified

[Matt Page]

Change by Matt Page :


--
versions: +Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46898] Add API to allow extensions to set callback function on creation and destruction of PyCodeObject

[Matt Page]

New submission from Matt Page :

CPython extensions providing optimized execution of Python bytecode (e.g. the 
Cinder JIT) may need to hook into the lifecycle of code objects to determine 
what to optimize or to free resources allocated for code objects that no longer 
exist. We propose adding an API to allow extensions to set callbacks that will 
be invoked whenever code objects are created or destroyed.

Proposed API:

```
typedef enum {
  PYCODE_LCEVT_CREATED,
  PYCODE_LCEVT_DESTROYED
} PyCode_LifecycleEvent;

// A callback to be called when a code object is created or about to be 
destroyed.
typedef void(*PyCode_LifecycleCallback)(
  PyCode_LifecycleEvent event,
  PyCodeObject* code);

void PyCode_SetLifecycleCallback(PyCode_LifecycleCallback callback);
PyCode_LifecycleCallback PyCode_GetLifecycleCallback();

```

--
components: C API
messages: 414309
nosy: carljm, dino.viehland, itamaro, mpage
priority: normal
severity: normal
status: open
title: Add API to allow extensions to set callback function on creation and 
destruction of PyCodeObject
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46897] Add API to allow extensions to set callback function on creation, modification, and destruction of PyFunctionObject

[Matt Page]

New submission from Matt Page :

CPython extensions providing optimized execution of Python bytecode (e.g. the 
Cinder JIT) may need to hook into the lifecycle of function objects to 
determine what to optimize, invalidate previously-optimized functions, or free 
resources allocated for functions that no longer exist. For example, when 
inlining a function, the Cinder JIT will use the bytecode of the inlined 
function that was known at compile-time. If the bytecode for the inlined 
function changes at runtime (i.e. if __code__ was reassigned) the JIT needs to 
invalidate any code into which the function was inlined. We propose adding an 
API to allow extensions to set callbacks that will be invoked whenever 
functions are created, modified, or destroyed.

Proposed API:

```
typedef enum {
  PYFUNC_LCEVT_CREATED,
  PYFUNC_LCEVT_MODIFIED,
  PYFUNC_LCEVT_DESTROYED
} PyFunction_LifecycleEvent;

typedef enum {
  PYFUNC_ATTR_CODE,
  PYFUNC_ATTR_GLOBALS,
  PYFUNC_ATTR_DEFAULTS,
  PYFUNC_ATTR_KWDEFAULTS,
  PYFUNC_ATTR_CLOSURE,
  PYFUNC_ATTR_NOT_APPLICABLE,
} PyFunction_AttrId;

// A callback to be called in response to events in a function's lifecycle.
//
// The callback is invoked after a function is created and before the function 
// is modified or destroyed.
//
// On modification the third argument indicates which attribute was modified
// and the fourth argument is the new value.
// Otherwise the third argument is PYFUNC_ATTR_NOT_APPLICABLE and the fourth
// argument is NULL.
typedef void(*PyFunction_LifecycleCallback)(
  PyFunction_LifecycleEvent event, 
  PyFunctionObject* func,
  PyFunction_AttrId attr,
  PyObject* new_value);

void PyFunction_SetLifecycleCallback(PyFunction_LifecycleCallback callback);
PyFunction_LifecycleCallback PyFunction_GetLifecycleCallback();
```

--
components: C API
messages: 414308
nosy: carljm, dino.viehland, itamaro, mpage
priority: normal
severity: normal
status: open
title: Add API to allow extensions to set callback function on creation, 
modification, and destruction of PyFunctionObject
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46896] add support for watching writes to selecting dictionaries

[Carl Meyer]

New submission from Carl Meyer :

CPython extensions providing optimized execution of Python bytecode (e.g. the 
Cinder JIT), or even CPython itself (e.g. the faster-cpython project) may wish 
to inline-cache access to frequently-read and rarely-changed namespaces, e.g. 
module globals. Rather than requiring a dict version guard on every cached 
read, the best-performing way to do this is is to mark the dictionary as 
“watched” and set a callback on writes to watched dictionaries. This optimizes 
the cached-read fast-path at a small cost to the (relatively infrequent and 
usually less perf sensitive) write path.

We have an implementation of this in Cinder ( 
https://docs.google.com/document/d/1l8I-FDE1xrIShm9eSNJqsGmY_VanMDX5-aK_gujhYBI/edit#heading=h.n2fcxgq6ypwl
 ), used already by the Cinder JIT and its specializing interpreter. We would 
like to make the Cinder JIT available as a third-party extension to CPython ( 
https://docs.google.com/document/d/1l8I-FDE1xrIShm9eSNJqsGmY_VanMDX5-aK_gujhYBI/
 ), and so we are interested in adding dict watchers to core CPython.

The intention in this issue is not to add any specific optimization or cache 
(yet); just the ability to mark a dictionary as “watched” and set a write 
callback.

The callback will be global, not per-dictionary (no extra function pointer 
stored in every dict). CPython will track only one global callback; it is a 
well-behaved client’s responsibility to check if a callback is already set when 
setting a new one, and daisy-chain to the previous callback if so. Given that 
multiple clients may mark dictionaries as watched, a dict watcher callback may 
receive events for dictionaries that were marked as watched by other clients, 
and should handle this gracefully.

There is no provision in the API for “un-watching” a watched dictionary; such 
an API could not be used safely in the face of potentially multiple 
dict-watching clients.

The Cinder implementation marks dictionaries as watched using the least bit of 
the dictionary version (so version increments by 2); this also avoids any 
additional memory usage for marking a dict as watched.

Initial proposed API, comments welcome:

// Mark given dictionary as "watched" (global callback will be called if it is 
modified)
void PyDict_Watch(PyObject* dict);

// Check if given dictionary is already watched
int PyDict_IsWatched(PyObject* dict);

typedef enum {
  PYDICT_EVENT_CLEARED,
  PYDICT_EVENT_DEALLOCED,
  PYDICT_EVENT_MODIFIED
} PyDict_WatchEvent;

// Callback to be invoked when a watched dict is cleared, dealloced, or 
modified.
// In clear/dealloc case, key and new_value will be NULL. Otherwise, new_value 
will be the
// new value for key, NULL if key is being deleted.
typedef void(*PyDict_WatchCallback)(PyDict_WatchEvent event, PyObject* dict, 
PyObject* key, PyObject* new_value);

// Set new global watch callback; supply NULL to clear callback
void PyDict_SetWatchCallback(PyDict_WatchCallback callback);

// Get existing global watch callback
PyDict_WatchCallback PyDict_GetWatchCallback();

The callback will be called immediately before the modification to the dict 
takes effect, thus the callback will also have access to the prior state of the 
dict.

--
components: C API
messages: 414307
nosy: carljm, dino.viehland, itamaro
priority: normal
severity: normal
status: open
title: add support for watching writes to selecting dictionaries
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46894] make install DESTDIR= uses /lib/python3.10/lib-dynload out of DESTDIR

[Дилян Палаузов]

Дилян Палаузов  added the comment:

Once I create /lib/python3.10/lib-dynload/, the next error is:

changing mode of /lib/python3.10/lib-dynload/__pycache__ to 755
running install_scripts
copying build/scripts-3.10/pydoc3.10 -> /bin
error: could not delete '/bin/pydoc3.10': Permission denied
make: *** [Makefile:1761: sharedinstall] Error 1

'/bin/pydoc3.10 is not under DESTDIR

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46895] Allow extensions to set a callback to be invoked when a type is modified

[Matt Page]

Change by Matt Page :


--
title: Type-Modified Callbacks -> Allow extensions to set a callback to be 
invoked when a type is modified

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46891] Crash in ModuleType subclass with __slots__

[Ronald Oussoren]

Ronald Oussoren  added the comment:

The debug build seems to indicate that slot memory is not initiated properly, a 
stack trace (note the argument to visit_validate at frame #7):


  * frame #0: 0x00019b60d9b8 libsystem_kernel.dylib`__pthread_kill + 8
frame #1: 0x00019b640eb0 libsystem_pthread.dylib`pthread_kill + 288
frame #2: 0x00019b57e314 libsystem_c.dylib`abort + 164
frame #3: 0x00010030c4f4 python.exe`fatal_error_exit(status=-1) at 
pylifecycle.c:2624:9
frame #4: 0x00010030c3cc python.exe`fatal_error(fd=2, header=1, 
prefix="_PyObject_AssertFailed", msg="_PyObject_AssertFailed", status=-1) at 
pylifecycle.c:2805:5
frame #5: 0x00010030be20 
python.exe`_Py_FatalErrorFunc(func="_PyObject_AssertFailed", 
msg="_PyObject_AssertFailed") at pylifecycle.c:2821:5
frame #6: 0x000100169fe0 
python.exe`_PyObject_AssertFailed(obj=0x000100d63690, 
expr=0x, msg="PyObject_GC_Track() object is not valid", 
file="../Modules/gcmodule.c", line=2187, function="visit_validate") at 
object.c:2293:5
frame #7: 0x000100351184 
python.exe`visit_validate(op=0xcdcdcdcdcdcdcdcd, parent_raw=0x000100d63690) 
at gcmodule.c:2186:9
frame #8: 0x00010019845c 
python.exe`traverse_slots(type=0x000101813230, self=0x000100d63690, 
visit=(python.exe`visit_validate at gcmodule.c:2183), arg=0x000100d63690) 
at typeobject.c:1207:27
frame #9: 0x000100197fe8 
python.exe`subtype_traverse(self=0x000100d63690, 
visit=(python.exe`visit_validate at gcmodule.c:2183), arg=0x000100d63690) 
at typeobject.c:1228:23
frame #10: 0x000100350fdc 
python.exe`PyObject_GC_Track(op_raw=0x000100d63690) at gcmodule.c:2212:11

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46895] Type-Modified Callbacks

[Matt Page]

New submission from Matt Page :

CPython extensions providing optimized execution of Python bytecode (e.g. the 
Cinder JIT), or even CPython itself (e.g. the faster-cpython project) may wish 
to cache access to lookups in the class hierarchy (e.g. when resolving the 
target of a method call). Extensions that perform these optimizations need to 
know when to invalidate the cached values. CPython already has a mechanism to 
invalidate its internal state (e.g. the global method cache) when a type is 
modified: _PyType_Modified. We propose adding an API to allow extensions to set 
a callback that will be invoked by _PyType_Modified whenever a type, or any 
ancestor of the type in the class hierarchy, changes.

Proposed API:

```
// A callback to be invoked with the modified type and optionally the name of
// the attribute that was modified.
typedef void(*PyType_ModifiedCallback)(PyTypeObject* type, PyObject* attr);

// Set or get the callback. The callback may be cleared by supplying a NULL 
callback.
void PyType_SetModifiedCallback(PyType_ModifiedCallback callback);
PyType_ModifiedCallback PyType_GetModifiedCallback();
```

--
components: C API, Interpreter Core
messages: 414305
nosy: carljm, dino.viehland, itamaro, mpage
priority: normal
severity: normal
status: open
title: Type-Modified Callbacks
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46894] make install DESTDIR= uses /lib/python3.10/lib-dynload out of DESTDIR

[Дилян Палаузов]

New submission from Дилян Палаузов :

With most recent 3.10 (v3.10.2-169-g98e2ee60) I call

./configure --prefix=/  && make && make DESTDIR=/home/d/A install

It fails with:

running build
running build_ext
INFO: Can't locate Tcl/Tk libs and/or headers

Python build finished successfully!
The necessary bits to build these optional modules were not found:
_gdbm _tkinter  nis
readline   
To find the necessary bits, look in setup.py in detect_modules() for the 
module's name.


The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc  pwd   time   

running build_scripts
copying and adjusting /git/cpython/Tools/scripts/pydoc3 -> build/scripts-3.10
copying and adjusting /git/cpython/Tools/scripts/idle3 -> build/scripts-3.10
copying and adjusting /git/cpython/Tools/scripts/2to3 -> build/scripts-3.10
changing mode of build/scripts-3.10/pydoc3 from 644 to 755
changing mode of build/scripts-3.10/idle3 from 644 to 755
changing mode of build/scripts-3.10/2to3 from 644 to 755
renaming build/scripts-3.10/pydoc3 to build/scripts-3.10/pydoc3.10
renaming build/scripts-3.10/idle3 to build/scripts-3.10/idle3.10
renaming build/scripts-3.10/2to3 to build/scripts-3.10/2to3-3.10
running install_lib
creating /lib/python3.10/lib-dynload
error: could not create '/lib/python3.10/lib-dynload': Permission denied
make: *** [Makefile:1761: sharedinstall] Error 1

--
components: Build
messages: 414303
nosy: dilyan.palauzov
priority: normal
severity: normal
status: open
title: make install DESTDIR=  uses /lib/python3.10/lib-dynload out of DESTDIR
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46893] Allow extensions to set the vectorcall field on instances of PyFunctionObject

[Itamar Ostricher]

New submission from Itamar Ostricher :

CPython extensions providing optimized execution of Python bytecode (like 
[Cinder 
JIT](https://docs.google.com/document/d/1l8I-FDE1xrIShm9eSNJqsGmY_VanMDX5-aK_gujhYBI/edit#heading=h.ujldakarfxhh)
 and [Pyjion](https://github.com/tonybaloney/Pyjion))
can benefit from being able to modify the vectorcall field on instances of 
PyFunctionObject to allow calling the optimized path (e.g. JIT-compiled) 
directly.

We propose adding an API to allow extensions to override this field:

```
void PyFunction_SetVectorcall(PyFunctionObject *func, vectorcallfunc 
vectorcall);
```

--
components: Interpreter Core
messages: 414302
nosy: carljm, dino.viehland, itamaro
priority: normal
severity: normal
status: open
title: Allow extensions to set the vectorcall field on instances of 
PyFunctionObject
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46892] Async Call-Stack Reconstruction

[Matt Page]

New submission from Matt Page :

Profiling tools that use the call-stack (i.e. all of them) paint an incomplete 
picture of what’s really going on in async-heavy codebases. They can only show 
the stack of the currently executing task; they miss the chain of awaitables 
that are transitively waiting on the current task. To remedy this, we have 
added support in Cinder to expose the async call-stack. This consists of the 
call stack for the currently executing task, followed by the chain of 
awaitables that are transitively reachable from the currently executing task. 
See below for a clarifying example.

```
async def f1():
  return await f2()

async def f2():
  return await asyncio.ensure_future(f3())

async def f3():
  return await f4()

async def f4():
  await asyncio.sleep(10)
  return 42
```

When retrieved from f4, the two different stacks (top-of-stack last) are:
sync - [f3, f4]
async - [f1, f2, f3, f4] 


We’d like to merge our implementation into CPython so that other heavy users of 
asyncio can benefit. This will consist of a few parts:

1. A new limited C-API to set and retrieve the “awaiter” of an awaitable object.
2. Additions to PyAsyncMethods to store function pointers for setting and 
retrieving the awaiter on instances.
3. An API in managed code to retrieve the async call stack as a list of fully 
qualified names (i.e. :.).

--
components: Interpreter Core, asyncio
messages: 414301
nosy: asvetlov, dino.viehland, itamaro, mpage, yselivanov
priority: normal
severity: normal
status: open
title: Async Call-Stack Reconstruction
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46891] Crash in ModuleType subclass with __slots__

[Ronald Oussoren]

Ronald Oussoren  added the comment:

The crash itself happens during garbage collection:

% python3.11 -Xdev t.py 
  
(master)pyobjc-8
Fatal Python error: Segmentation fault

Current thread 0x000104ce0580 (most recent call first):
  Garbage-collecting
  


And when using a debug build and -Xdev:

../Modules/gcmodule.c:2187: visit_validate: Assertion failed: 
PyObject_GC_Track() object is not valid
Enable tracemalloc to get the memory block allocation traceback

object address  : 0x101a5f690
object refcount : 1
object type : 0x15503f230
object type name: MyModule
object repr : 

Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized

Current thread 0x0001019c0580 (most recent call first):
  File "/Users/ronald/Projects/Forks/cpython/build/t.py", line 12 in 
zsh: abort  ./python.exe -Xdev t.py

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46566] Support -3.11-arm64 in py.exe launcher

[Steve Dower]

Steve Dower  added the comment:

I'm working on this now.

--
assignee:  -> steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6634] [doc] sys.exit() called from threads other than the main one: undocumented behaviour

[Vidhya]

Vidhya  added the comment:

Thanks for your comments :). The PR for the same is:
https://github.com/python/cpython/pull/31639

--
message_count: 21.0 -> 22.0
pull_requests: +29760
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/31639

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46870] Improper Input Validation in urlparse

[Ned Deily]

Change by Ned Deily :


--
nosy: +orsenthil

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46876] Walrus operator not in help

[Ned Deily]

Change by Ned Deily :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python
stage:  -> needs patch
versions: +Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

Draft PR for Cython: https://github.com/cython/cython/pull/4671

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46891] Crash in ModuleType subclass with __slots__

[Ronald Oussoren]

Ronald Oussoren  added the comment:

This is with python 3.11 alpha 5 using the installer for macOS.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46712] Share global string identifiers in deepfreeze

[Eric Snow]

Eric Snow  added the comment:


New changeset 21099fc064c61d59c936a2f6a0db3e07cd5c8de5 by Eric Snow in branch 
'main':
bpo-46712: Let generate_global_objects.py Run on Earlier Python Versions 
(gh-31637)
https://github.com/python/cpython/commit/21099fc064c61d59c936a2f6a0db3e07cd5c8de5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46891] Crash in ModuleType subclass with __slots__

[Ronald Oussoren]

New submission from Ronald Oussoren :

The following crashes the interpreter in Python 3.11, and works fine in older 
versions:

# --- script.py 
import types

class MyModule (types.ModuleType):
__slots__ = (
"_MyModule__a",
"_MyModule__b",
)

def __init__(self, name):
super().__init__(name)

m = MyModule("name")
# -- end of file

The key in this is the ``__slots__`` definition: The script does not crash 
without ``__slots__``, or with a slots tuple with 1 item.

This is a reproducer based on code in PyObjC.

--
components: Interpreter Core
keywords: 3.11regression
messages: 414294
nosy: ronaldoussoren
priority: normal
severity: normal
stage: needs patch
status: open
title: Crash in ModuleType subclass with __slots__
type: crash
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42982] Update suggested number of iterations for pbkdf2_hmac()

[Ned Deily]

Ned Deily  added the comment:


New changeset 7dbb2f8eaf07c105f4d2bb0fe61763463e68372d by Miss Islington (bot) 
in branch '3.10':
bpo-42982: update pbkdf2 example & add another link (GH-30966) (#30968)
https://github.com/python/cpython/commit/7dbb2f8eaf07c105f4d2bb0fe61763463e68372d


--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

Draft PR for greenlet: https://github.com/python-greenlet/greenlet/pull/294

I made these changes close to the Python 3.11 alpha 6 release to be able to 
test "#if PY_VERSION_HEX < 0x30B00A6" to have code compatible with Python 3.11 
alpha 5 and older.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[Guido van Rossum]

Change by Guido van Rossum :


--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46890] venv does not create "python" link in python 3.11

[Ronald Oussoren]

Ronald Oussoren  added the comment:

The change to _base_executable is the real problem. Venv creates the correct 
directory structure if I add a sitecustomize.py to the python path that sets 
_base_executable to the correct value.

# sitecustomize
import sys
sys._base_executable = sys.executable
# EOF

Is sys._base_executable updated after running getpath.py? On first glance 
getpath.py does update config['base_executable'] and _PyConfig_FromDict reads 
that value back, but that's based on a quick scan through the code. I haven't 
tried debugging this yet.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46890] venv does not create "python" link in python 3.11

[Ned Deily]

Ned Deily  added the comment:

As Ronald notes, the issue isn't in venv, it's that the value of 
sys._base_executable has changed between 3.10 and 3.11 for macOS builds.

$ /usr/local/bin/python3.10
Python 3.10.2 (v3.10.2:a58ebcc701, Jan 13 2022, 14:50:16) [Clang 13.0.0 
(clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys._base_executable
'/usr/local/bin/python3.10'
>>> ^D
nad@vana:~$ /usr/local/bin/python3.11
Python 3.11.0a5 (v3.11.0a5:c4e4b91557, Feb  3 2022, 14:54:01) [Clang 13.0.0 
(clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys._base_executable
'/Library/Frameworks/Python.framework/Versions/3.11.0a5_11/Resources/Python.app/Contents/MacOS/Python'
>>>

The 3.11 value is incorrect for the reasons Ronald noted.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10515] csv sniffer does not recognize quotes at the end of line

[Irit Katriel]

Change by Irit Katriel :


--
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed
superseder:  -> csv.Sniffer.sniff() regex error

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25415] [io doc] Reword "there is no public constructor"

[Terry J. Reedy]

Terry J. Reedy  added the comment:

"IOBase" is a public name and IOBase has an __init__ method.  It definitely has 
a public constructor.  However, like other abstract base classes, it is not 
meant to be directly instantiated by users other than writers of other classes. 
 Ditto for the 3 other io ABCs.  Because this is generic to all ABCs, it need 
not be repeated for each.

The proposed replacement is the opposite of the truth.  Rather than accepting 
no arguments, IOBase, like the ABCs in numbers and collections.abc, has a 
generic signature and accepts any arguments (which I believe are ignored).

>>> io.IOBase.__init__.__text_signature__
'($self, /, *args, **kwargs)'
>>> io.IOBase(3, a=5)


A class, such as `object`, accepting no arguments other that self raises.  (At 
one time, object had the generic signature above.)

>>> object(3)
Traceback (most recent call last):
  File "", line 1, in 
object(3)
TypeError: object() takes no arguments

I think that the fix should be to delete the original incorrect statement and 
not replace it with another incorrect statement.

Most abstract base classes are in collections.abc and numbers.  They are not 
specifically labelled 'abstract base class' in their docstrings because they 
are in modules consisting more or less entirely of ABCs.  Since io is mostly a 
module of concrete classes, I think it appropriate to specifically label them, 
as they are now.  Once so labeled, nothing need be added that is true of all 
ABCs.

--
title: [doc] "there is no public constructor" should be reworded in io module 
documentation -> [io doc] Reword "there is no public constructor"

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46890] venv does not create "python" link in python 3.11

[Steve Dower]

Steve Dower  added the comment:

The _base_executable change might be, though it should still be preferring the 
environment variables here, but I don't think I touched anything that would 
affect which symlinks are created by venv.

--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15002] urllib2 does not download 4 MB file completely using ftp

[Irit Katriel]

Irit Katriel  added the comment:

2.7 backport is no longer relevant.

--
nosy: +iritkatriel
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20970] [doc] contradictory documentation for prog option of argparse

[Irit Katriel]

Irit Katriel  added the comment:

@lilbludot  - is this your GitHub account? https://github.com/lilbludot

(I am asking in order to credit you as coauthor on the GitHub version of your 
patch).

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46890] venv does not create "python" link in python 3.11

[Eric Snow]

Eric Snow  added the comment:

This may be related to the getpath.py work Steve did.

--
nosy: +eric.snow, steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6634] [doc] sys.exit() called from threads other than the main one: undocumented behaviour

[Irit Katriel]

Irit Katriel  added the comment:

Vidhya, I think the sentence you are suggesting to add would overlap with one 
which is already there ("Since exit() ultimately “only” raises an exception, it 
will only exit the process when called from the main thread, and the exception 
is not intercepted.")


It seems to me that what could be improved is the first paragraph, which starts 
with: "Exit from Python.".   Maybe instead it could say something like "Raise a 
SystemExit exception, which has the effect of ..." and then say what it is 
(i.e., that it exits python if you are in the main thread, unless the exception 
is caught... ).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46712] Share global string identifiers in deepfreeze

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +29759
pull_request: https://github.com/python/cpython/pull/31637

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[STINNER Victor]

STINNER Victor  added the comment:

I plan to update Cython, greenlet and gevent for this change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46890] venv does not create "python" link in python 3.11

[Ronald Oussoren]

Ronald Oussoren  added the comment:

The root cause likely is the calculation of sys._base_executable. In 3.10 this 
is {sys.prefix}/bin/python3.10, while in 3.11 this is 
{sys.prefix}/Resources/Python.app/Contents/MacOS/Python.

The venv library uses the incorrect value and therefore creates an incorrect 
virtual environment.

--
components: +Interpreter Core

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46890] venv does not create "python" link in python 3.11

[Ronald Oussoren]

Ronald Oussoren  added the comment:

This is be related to how the virtual environment is populated. In 3.10 the 
"python3.10" name in the environment is a symlink to sys.executable. In 3.10 
"Python" (not the capital) is a symlink to the binary in Python.app and 
"python3.11" is a symlink to "Python".  Given that the filesystem is case 
preserving there is no "python" name.

The behaviour in 3.11 is clearly a bug: the virtual environment is no longer 
using the launcher binary in "{sys.prefix]/bin" but directly uses the "real" 
binary (in a framework build), and because of that scripts cannot use system 
APIs that expect to run in an application bundle. 

Listing "env/bin" in Python 3.10:
-rw-rw-r--  1 ronald  staff  9033 Mar  1 18:11 Activate.ps1
-rw-rw-r--  1 ronald  staff  2018 Mar  1 18:11 activate
-rw-rw-r--  1 ronald  staff   944 Mar  1 18:11 activate.csh
-rw-rw-r--  1 ronald  staff  2086 Mar  1 18:11 activate.fish
-rwxr-xr-x  1 ronald  staff   269 Mar  1 18:11 pip
-rwxr-xr-x  1 ronald  staff   269 Mar  1 18:11 pip3
-rwxr-xr-x  1 ronald  staff   269 Mar  1 18:11 pip3.10
lrwxr-xr-x  1 ronald  staff10 Mar  1 18:11 python -> python3.10
lrwxr-xr-x  1 ronald  staff10 Mar  1 18:11 python3 -> python3.10
lrwxr-xr-x  1 ronald  staff65 Mar  1 18:11 python3.10 -> 
/Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10

In python3.11:
total 72
-rw-rw-r--  1 ronald  staff  9033 Mar  1 17:23 Activate.ps1
lrwxr-xr-x  1 ronald  staff93 Mar  1 17:23 Python -> 
/Library/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python
-rw-rw-r--  1 ronald  staff  2018 Mar  1 17:23 activate
-rw-rw-r--  1 ronald  staff   944 Mar  1 17:23 activate.csh
-rw-rw-r--  1 ronald  staff  2086 Mar  1 17:23 activate.fish
-rwxr-xr-x  1 ronald  staff   265 Mar  1 17:23 pip
-rwxr-xr-x  1 ronald  staff   265 Mar  1 17:23 pip3
-rwxr-xr-x  1 ronald  staff   265 Mar  1 17:23 pip3.11
lrwxr-xr-x  1 ronald  staff 6 Mar  1 17:23 python3 -> Python
lrwxr-xr-x  1 ronald  staff 6 Mar  1 17:23 python3.11 -> Python

--
components: +macOS
nosy: +ned.deily
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[Brandt Bucher]

Brandt Bucher  added the comment:

I'm also very uncomfortable with the lack of review on these PRs. The most 
recent one (https://github.com/python/cpython/pull/31583) was open for less 
than 30 minutes before merging, from 6:57 to 7:22 am in my local time zone.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46836] [C API] Move PyFrameObject to the internal C API

[Brandt Bucher]

Brandt Bucher  added the comment:

Victor, can we please revert these changes? They broke Greenlet, a required 
dependency for three of our performance benchmarks:

https://github.com/python-greenlet/greenlet/issues/288#issuecomment-1055632607

I've already spent considerable effort contributing workarounds for other 3.11 
breakages in Greenlet and coaxing them to put out a compatible release:

https://github.com/python-greenlet/greenlet/pull/280

These changes, however, just seem like needless breakage that are now also 
impacting our performance work.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46890] venv does not create "python" link in python 3.11

[Zachary Ware]

Change by Zachary Ware :


--
title: vent does not create "python" link in python 3.11 -> venv does not 
create "python" link in python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46890] vent does not create "python" link in python 3.11

[Ronald Oussoren]

New submission from Ronald Oussoren :

In python3.10 and earlier "python3 -m venv something" creates 
"something/bin/python" as a symlink to the interpreter itself.

In python3.11 (a5) the same command no longer creates "something/bin/python", 
but only the ".../python3" symlink. 

With this change the "python" command no longer refers to the interpreter in an 
activated virtualenv, but to a different binary on $PATH (if any).

I tested using the Python 3.11a5 installer for macOS on python.org.

--
components: Library (Lib)
keywords: 3.11regression
messages: 414278
nosy: ronaldoussoren
priority: normal
severity: normal
status: open
title: vent does not create "python" link in python 3.11
type: behavior
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46841] Inline bytecode caches

[Mark Shannon]

Mark Shannon  added the comment:


New changeset 3b0f1c5a710eff289dc44bec972dbaea353cc54f by Mark Shannon in 
branch 'main':
bpo-46841: Use inline cache for `BINARY_SUBSCR`. (GH-31618)
https://github.com/python/cpython/commit/3b0f1c5a710eff289dc44bec972dbaea353cc54f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28516] contextlib.ExitStack.__enter__ has trivial but undocumented behavior

[Vidhya]

Vidhya  added the comment:

@Jelle: Thanks. The PR is submitted at
https://github.com/python/cpython/pull/31636

--
keywords: +patch
message_count: 9.0 -> 10.0
pull_requests: +29758
pull_request: https://github.com/python/cpython/pull/31636

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46856] datetime.max conversion

[Joris Geysens]

Joris Geysens  added the comment:

I looked at this a bit more in detail.
What happens it the following, starting in the datetime fromtimestamp fragment :

converter = _time.gmtime if utc else _time.localtime
y, m, d, hh, mm, ss, weekday, jday, dst = converter(t)

That will call the system gmtime_r(), which indeed returns Sat Jan  1 00:00:00 
1

I have not looked at the cpp implementation for that method and I am not sure 
if this is something that can be improved.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset e91b0a7139d4a4cbd2351ccb5cd021a100cf42d2 by Kumar Aditya in 
branch 'main':
bpo-46541: remove usage of _Py_IDENTIFIER from _ssl module (GH-31599)
https://github.com/python/cpython/commit/e91b0a7139d4a4cbd2351ccb5cd021a100cf42d2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17505] [doc] email.header.Header.__unicode__ does not decode header

[Vidhya]

Vidhya  added the comment:

The latest versions 3.9, 3.10 and 3.11 are updated in the issue. So I thought 
like it still applies.

@irit: Any suggestions on what needs to be done for current revisions?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46889] enable sqlite math functions on macos

[lyinch]

Change by lyinch :


--
keywords: +patch
pull_requests: +29757
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31635

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46889] enable sqlite math functions on macos

[lyinch]

New submission from lyinch :

Similar to these PRs: https://github.com/python/cpython/pull/24053/files and 
https://github.com/python/cpython/pull/25892/files which enable the built-in 
sqlite3 math functions for Windows, I want to have support for them on macos. 
As per the sqlite3 documentation: https://sqlite.org/releaselog/3_35_0.html 
this requires compilation with -DSQLITE_ENABLE_MATH_FUNCTIONS . On master, the 
flag is missing for macos : 
https://github.com/python/cpython/blob/main/Mac/BuildScript/build-installer.py#L364

--
components: Library (Lib), macOS
messages: 414272
nosy: lyinch, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: enable sqlite math functions on macos
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29756
pull_request: https://github.com/python/cpython/pull/31634

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46887] ./Programs/_freeze_module fails with MSAN: Uninitialized value was created by an allocation of 'stat.i'

[STINNER Victor]

STINNER Victor  added the comment:

> The Python code calls OBJ_txt2obj("1.3.6.1.5.5.7.3.1", 0) in C: the OpenSSL 
> function.

I reported this issue to OpenSSL: 
https://github.com/openssl/openssl/issues/17784

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46871] Lambda can't be pickled with "spawn" and only "fork"

[Kyle Smith]

Change by Kyle Smith :


--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46871] Lambda can't be pickled with "spawn" and only "fork"

[Kyle Smith]

Kyle Smith  added the comment:

I think that's fair, thanks for the conversation at least. I understand python 
mp a little bit more now...

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46887] ./Programs/_freeze_module fails with MSAN: Uninitialized value was created by an allocation of 'stat.i'

[STINNER Victor]

STINNER Victor  added the comment:

> The Python code calls OBJ_txt2obj("1.3.6.1.5.5.7.3.1", 0) in C: the OpenSSL 
> function.

This error is unrelated to Python, but comes from OpenSSL.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46887] ./Programs/_freeze_module fails with MSAN: Uninitialized value was created by an allocation of 'stat.i'

[STINNER Victor]

STINNER Victor  added the comment:

> It looks like a bug in clang MSAN:
> https://github.com/llvm/llvm-project/issues/54131

I wrote GH-31633 to work around the false alarm on stat() and fstat().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46887] ./Programs/_freeze_module fails with MSAN: Uninitialized value was created by an allocation of 'stat.i'

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +29755
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31633

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9204bb72a2da5885facc747e63d2bd2d654606fe by Victor Stinner in 
branch 'main':
bpo-46633: Skip tests on ASAN and/or MSAN builds (GH-31632)
https://github.com/python/cpython/commit/9204bb72a2da5885facc747e63d2bd2d654606fe


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46887] ./Programs/_freeze_module fails with MSAN: Uninitialized value was created by an allocation of 'stat.i'

[STINNER Victor]

STINNER Victor  added the comment:

Hum, later "import ssl" in setup.py also fails. Simplified code:
---
import _ssl
print(_ssl.txt2obj('1.3.6.1.5.5.7.3.1', name=False)) # server OID
---

Error:
---
$ ./python x.py 
Uninitialized bytes in MemcmpInterceptorCommon at offset 0 inside 
[0x70100032, 8)
==135651==WARNING: MemorySanitizer: use-of-uninitialized-value
#0 0x49467c in memcmp (/home/vstinner/python/main/python+0x49467c)
#1 0x7f5546825adc in OBJ_bsearch_ex_ (/lib64/libcrypto.so.1.1+0x14eadc)
#2 0x7f5546826ff1 in OBJ_obj2nid (/lib64/libcrypto.so.1.1+0x14fff1)
#3 0x7f554675b434  (/lib64/libcrypto.so.1.1+0x84434)
#4 0x7f554675b6af in d2i_ASN1_OBJECT (/lib64/libcrypto.so.1.1+0x846af)
#5 0x7f5546827702 in OBJ_txt2obj (/lib64/libcrypto.so.1.1+0x150702)
#6 0x7f5546a78c31 in _ssl_txt2obj_impl 
/home/vstinner/python/main/Modules/_ssl.c:5276:11
#7 0x7f5546a78c31 in _ssl_txt2obj 
/home/vstinner/python/main/Modules/clinic/_ssl.c.h:1216:20
(...)

  Uninitialized value was created by a heap allocation
#0 0x466642 in __interceptor_malloc 
(/home/vstinner/python/main/python+0x466642)
#1 0x7f55468276b3 in OBJ_txt2obj (/lib64/libcrypto.so.1.1+0x1506b3)

SUMMARY: MemorySanitizer: use-of-uninitialized-value 
(/home/vstinner/python/main/python+0x49467c) in memcmp
Exiting
---

The Python code calls OBJ_txt2obj("1.3.6.1.5.5.7.3.1", 0) in C: the OpenSSL 
function.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset 0cc63641859b2f60ea65bb7c0b6d1cfcec1e2f1a by Dong-hee Na in branch 
'main':
bpo-46541: Remove usage of _Py_IDENTIFIER from multibytecodec (GH-31475)
https://github.com/python/cpython/commit/0cc63641859b2f60ea65bb7c0b6d1cfcec1e2f1a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46887] ./Programs/_freeze_module fails with MSAN: Uninitialized value was created by an allocation of 'stat.i'

[STINNER Victor]

STINNER Victor  added the comment:

It looks like a bug in clang MSAN:
https://github.com/llvm/llvm-project/issues/54131

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46311] Clean up PyLong_FromLong and PyLong_FromLongLong

[Mark Shannon]

Mark Shannon  added the comment:


New changeset c60e6b6ad7aaf9c72035ff9fb1575e2710aeb4b4 by Mark Dickinson in 
branch 'main':
bpo-46311: Clean up PyLong_FromLong and PyLong_FromLongLong (GH-30496)
https://github.com/python/cpython/commit/c60e6b6ad7aaf9c72035ff9fb1575e2710aeb4b4


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

STINNER Victor  added the comment:

"""
No longer skipped:

* test_capi
* test_ctypes
* test_faulthandler
* test_interpreters
"""

I built Python manually with:

  ./configure --with-pydebug CC=clang LD=clang --with-address-sanitizer 

These tests no longer with ASAN_OPTIONS used on our CI:

ASAN_OPTIONS='detect_leaks=0:allocator_may_return_null=1:handle_segv=0' 
./python -m test (...)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46841] Inline bytecode caches

[Mark Shannon]

Mark Shannon  added the comment:


New changeset 7820a5897e7762df23bff1cbe749652130654a08 by Brandt Bucher in 
branch 'main':
bpo-46841: Use inline caching for `COMPARE_OP` (GH-31622)
https://github.com/python/cpython/commit/7820a5897e7762df23bff1cbe749652130654a08


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6778] False positives given through bisect module (binary search)

[Mark Dickinson]

Change by Mark Dickinson :


--
components: +Extension Modules

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6778] False positives given through bisect module (binary search)

[Mark Dickinson]

Change by Mark Dickinson :


--
components:  -Distutils, Documentation, Extension Modules, Installation, 
Parser, email
nosy:  -barry, docs@python, dstufft, eric.araujo, lys.nikolaou, pablogsal
type: security -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46888] SharedMemory.close() destroys memory

[Eryk Sun]

Eryk Sun  added the comment:

> Yes, named memory mappings only exist on Windows until the last 
> reference is closed, so this is a difference due to the underlying OS. 

That's true for the Windows API, so it's true for all practical purposes. In 
the underlying NT API, creating a permanent kernel object is possible by 
setting OBJ_PERMANENT in the initial object attributes [1], or subsequently via 
the undocumented system call NtMakePermanentObject(handle). Creating a 
permanent object requires SeCreatePermanentPrivilege, which normally is granted 
to just the SYSTEM account. An administrator can grant this privilege to any 
user, group, or well-known group, but creating permanent objects should 
generally be limited to drivers and system services. An object can be reverted 
back to a temporary object via NtMakeTemporaryObject(handle).

A named section object (i.e. file mapping) can also be created as a global 
name, i.e. r"Global\{object name}", which is accessible to all sessions. This 
requires SeCreateGlobalPrivilege, which by default is granted to system service 
accounts and administrators. This is separate from whether the section is 
temporary or permanent, but a permanent section object is more likely to be 
needed in the global namespace.

---

[1] 
https://docs.microsoft.com/en-us/windows/win32/api/ntdef/nf-ntdef-initializeobjectattributes
[2] 
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf-wdm-zwmaketemporaryobject

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46888] SharedMemory.close() destroys memory

[Steve Dower]

Steve Dower  added the comment:

Yes, named memory mappings only exist on Windows until the last 
reference is closed, so this is a difference due to the underlying OS. 
The implementation of unlink() recognises this (the entire body is under 
a _USE_POSIX check), but the docs do not reflect it.

The only workaround I can think of would be to create a real file, then 
open it with mmap and give it a tagname, then pass that tagname as the 
name argument of the shared memory object when creating it. It's a bit 
of a pain, but I don't think there's any option on our side given the 
way the API has been designed.

It's not possible for N SharedMemory instances to independently agree 
which one will ignore the close() call and do it in unlink() instead, 
and in any case that still wouldn't "unlink" the name until all other 
references had also been closed.

Can you tell us a bit more about what you're trying to achieve? Perhaps 
knowing how this is actually being relied upon will inspire some ideas 
for how to make it work better.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46888] SharedMemory.close() destroys memory

[Ronny Rentner]

New submission from Ronny Rentner :

According to 
https://docs.python.org/3/library/multiprocessing.shared_memory.html#multiprocessing.shared_memory.SharedMemory.close
 if I call close() on a shared memory, it shall not be destroyed.

Unfortunately this is only true for Linux but not for Windows.

I've tested this in a Windows VM on VirtualBox like this:

```
Python 3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 14:12:15) [MSC v.1929 64 bit 
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import multiprocessing.shared_memory
>>> creator = multiprocessing.shared_memory.SharedMemory(create=True, 
>>> name='mymemory', size=1)
>>> creator.buf[0] = 1
>>> creator.buf[0]
1
>>> # According to  close() is supposed to not destroy 'mymemory' but it does 
>>> destroy it.
>>> creator.close()
>>>
>>> user = multiprocessing.shared_memory.SharedMemory(name='mymemory')
Traceback (most recent call last):
  File "", line 1, in 
  File 
"C:\Users\User\AppData\Local\Programs\Python\Python310\lib\multiprocessing\shared_memory.py",
 line 161, in __init__
h_map = _winapi.OpenFileMapping(
FileNotFoundError: [WinError 2] The system cannot find the file specified: 
'mymemory'
>>> # Shared memory was destroyed by close()
```

--
components: Windows
messages: 414258
nosy: paul.moore, ronny-rentner, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: SharedMemory.close() destroys memory
type: behavior
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46871] Lambda can't be pickled with "spawn" and only "fork"

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Oops, replying by email reopens the ticket. Back to pending you go!

--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46886] ARM64 Windows Non-Debug 3.x: random build error: failure during conversion to COFF: file invalid or corrupt

[Niyas Sait]

Niyas Sait  added the comment:

I cannot reproduce this on local builds on the same machine and the failures 
are also intermittent on CI.

I am going to upgrade OS to Windows 11 and install Visual studio 2022. Hope 
that would resolve the issue.

I will update and keep an eye on the build to see if it would make any 
difference.

--
nosy: +nsait-linaro

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46871] Lambda can't be pickled with "spawn" and only "fork"

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> The "spawn" method requires pickling the data and callable passed to 
> the child proces, and that's not supported for lambda's.

Ah, today I learned something. Kyle, looks like you were right about it 
being due to the lambdas.

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18534] [doc] State clearly that open() 'file' param is "name" attr of the result

[Stanley]

Stanley  added the comment:

Would it be fine to modify FileIO.name like this:

"... given in the constructor. Depending on the type of *name* that was passed 
into the constructor, this may not necessarily e a string."

--
nosy: +slateny

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com