[issue43459] Race conditions when the same source file used to build mutliple extensions

[Michał Górny]

New submission from Michał Górny :

There is a race condition in distutils' build_ext implementation.  When the 
same source file is used to build multiple extensions, distutils attempts to 
build it multiple times using the same output file, in parallel.  This means 
that the link editor can grab the file while another compiler instance is 
overwriting it.  The results vary from compile errors to cryptic dyld failures 
when attempting to load the module.

I've created a trivial reproducer that I've attached in a patch form.  For 
convenience, it's also available on my GitHub: 
https://github.com/mgorny/distutils-build_ext-race

The reproducer consists of two extension modules sharing the same file.  The 
race.sh script attempts to build the extension and then import it.  The process 
is repeated until something fails, e.g.:

+ python3.10 setup.py build_ext -i -j4
running build_ext
building 'bar' extension
creating build
building 'foo' extension
creating build/temp.linux-x86_64-3.10
creating build/temp.linux-x86_64-3.10
x86_64-pc-linux-gnu-gcc-10.2.0 -pthread -fPIC -I/usr/include/python3.10 -c 
bar.c -o build/temp.linux-x86_64-3.10/bar.o
x86_64-pc-linux-gnu-gcc-10.2.0 -pthread -fPIC -I/usr/include/python3.10 -c 
foo.c -o build/temp.linux-x86_64-3.10/foo.o
x86_64-pc-linux-gnu-gcc-10.2.0 -pthread -fPIC -I/usr/include/python3.10 -c 
shared.c -o build/temp.linux-x86_64-3.10/shared.o
x86_64-pc-linux-gnu-gcc-10.2.0 -pthread -fPIC -I/usr/include/python3.10 -c 
shared.c -o build/temp.linux-x86_64-3.10/shared.o
x86_64-pc-linux-gnu-gcc-10.2.0 -pthread -shared -Wl,-O1 -Wl,--as-needed 
-Wl,--hash-style=gnu build/temp.linux-x86_64-3.10/foo.o 
build/temp.linux-x86_64-3.10/shared.o -L/usr/lib64 -o 
/home/mgorny/git/distutils-build_ext-race/foo.cpython-310-x86_64-linux-gnu.so
x86_64-pc-linux-gnu-gcc-10.2.0 -pthread -shared -Wl,-O1 -Wl,--as-needed 
-Wl,--hash-style=gnu build/temp.linux-x86_64-3.10/bar.o 
build/temp.linux-x86_64-3.10/shared.o -L/usr/lib64 -o 
/home/mgorny/git/distutils-build_ext-race/bar.cpython-310-x86_64-linux-gnu.so
+ python3.10 -c 'import foo; import bar'
Traceback (most recent call last):
  File "", line 1, in 
ImportError: 
/home/mgorny/git/distutils-build_ext-race/foo.cpython-310-x86_64-linux-gnu.so: 
undefined symbol: call_shared
+ echo 'Reproduced at iteration 256'
Reproduced at iteration 256
+ break

--
components: Distutils
files: 0001-A-reproducer-for-distutils-build_ext-race-condition.patch
keywords: patch
messages: 388410
nosy: dstufft, eric.araujo, mgorny
priority: normal
severity: normal
status: open
title: Race conditions when the same source file used to build mutliple 
extensions
type: compile error
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9
Added file: 
https://bugs.python.org/file49863/0001-A-reproducer-for-distutils-build_ext-race-condition.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39511] [subinterpreters] Per-interpreter singletons (None, True, False, etc.)

[junyixie]

junyixie  added the comment:

> Which API should be used in C extensions to be "subinterpreter-safe"? ?> 
> Currently, Py_None is a singleton shared by multiple interpreters. > > > 
> Should suddenly all C extensions use a new Py_GetNone() function which > 
> returns the per-interpreter singleton? If yes, that's basically what my > PR 
> 18301 does:

>   #define Py_None Py_GetNone()

after read you [WIP] bpo-39511: Add Py_GetNone() and Py_GetNoneRef() functions 
#18301.

Actually, interp->none shared _Py_NoneStruct variable.

when two interperter modify interp->none refcount,will modify _Py_NoneStruct 
variable.

> the CPU cacheline of common singletons like None, True and False can quickly 
> become a performance bottleneck.

even if add Py_INCREF(none);. 
In the scenario of parallel interpreter, will also have thread safety issues.
> PyStatus
> _Py_InitSingletons(PyThreadState *tstate)
> {
> PyObject *none = &_Py_NoneStruct;
> Py_INCREF(none);
> tstate->interp->none = none;
> return _PyStatus_OK();
> }

--
nosy: +JunyiXie -Mark.Shannon, corona10, eric.snow, jeremy.kloth, jkloth, 
larry, maciej.szulik, nanjekyejoannah, ncoghlan, phsilva, rhettinger, 
shihai1991, steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27820] Possible bug in smtplib when initial_response_ok=False

[Pandu E POLUAN]

Pandu E POLUAN  added the comment:

Hi Senthil,

You're right, it does need a guard. According to my knowledge there is no AUTH 
mechanism that will send more than 3 challenges; they should fail afterwards 
with 535 or similar. Servers that don't do that should be considered 
buggy/broken.

So I've pushed a commit to the GH PR that limits the challenge to 5 times, 
after which it will raise SMTPException. This will protect users of 
smtplib.SMTP from being trapped by a buggy/broken server.


Rgds,
--

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43458] Tutorial should mention about variable scope in try/except/finally

[Marek M]

New submission from Marek M :

It can be helpful to mention that variables defined in try block are visible in 
except/finally block as well. I did not find this info in Python tutorial and 
for me (having C++ background) this is quite unexpected feature.

--
assignee: docs@python
components: Documentation
messages: 388407
nosy: deekox, docs@python
priority: normal
severity: normal
status: open
title: Tutorial should mention about variable scope in try/except/finally
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43457] Include simple file loading and saving functions in JSON standard library.

[Inada Naoki]

Inada Naoki  added the comment:

I created a new vote for naming.
https://discuss.python.org/t/vote-new-function-for-reading-json-from-path/7603

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43044] Python 2.7 documentation links to 404 pages when the library was moved or renamed

[Борис Верховский]

Борис Верховский  added the comment:

This was fixed using redirects in nginx 

https://github.com/python/psf-salt/pull/201

--
resolution: out of date -> fixed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43457] Include simple file loading and saving functions in JSON standard library.

[Inada Naoki]

Inada Naoki  added the comment:

This idea is discussed several times. Last discussion thread is:
https://mail.python.org/archives/list/python-id...@python.org/thread/YHO575YY4FQC3GBDF4SKOWIEAUSY3OQX/#YHO575YY4FQC3GBDF4SKOWIEAUSY3OQX

--
nosy: +methane

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43457] Include simple file loading and saving functions in JSON standard library.

[nervecenter]

New submission from nervecenter :

Python has a "batteries included" approach to standard library construction. To 
that end, commonly used procedures are often included as functions; adding 
sugar to the language is often exchanged for adding sugar to libraries.

One of these common procedures in small-scale scripting tasks is loading a JSON 
file as simple data structures and saving simple data structures as a JSON 
file. This is normally handled using context managers, json.load(), and 
json.dump(). This is a bit cluttered and, I'd argue, not quite as Pythonic as 
the philosophy demands. I have a small file containing this code:

import json

def load_file(filename, *args, **kwargs):
with open(filename, "r") as fp:
data = json.load(fp, *args, **kwargs)
return data

def save_file(data, filename, *args, **kwargs):
with open(filename, "w") as fp:
json.dump(data, fp, *args, **kwargs)

I'd say, toss these two functions into the json module. Those two functions 
contain the clutter. For all other users, loading and saving JSON files become 
one-line function calls. This is convenient and batteries-included.

--
components: Library (Lib)
messages: 388403
nosy: nervecenter
priority: normal
severity: normal
status: open
title: Include simple file loading and saving functions in JSON standard 
library.
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43435] Py_BuildValue("y#".... returns incomplete result

[David Wood]

Change by David Wood :


Removed file: https://bugs.python.org/file49862/crypt.tar.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43435] Py_BuildValue("y#".... returns incomplete result

[David Wood]

David Wood  added the comment:

Attached are the basic files.  As you can see in the example test9.py, I am 
generating a random string, encrypting it, decrypting it, and comparing the 
decrypted result with the original value.

This is intended to run on linux and requires the mcrypt library (libmcrypt-dev 
on debian based distro).  I'm at a loss and any help is greatly appreciated.

--
Added file: https://bugs.python.org/file49861/crypt.tar.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43435] Py_BuildValue("y#".... returns incomplete result

[David Wood]

Change by David Wood :


Added file: https://bugs.python.org/file49862/crypt.tar.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43456] Remove _xxsubinterpreters from sys.stdlib_module_names

[Brett Cannon]

New submission from Brett Cannon :

I noticed that _xxsubinterpreters is in sys.stdlib_module_names but none of the 
other `_xx` modules are included (nor is 'test'). Since _xxsubinterpreters is 
only meant for testing (ATM) I think it should probably be left out.

--
components: Library (Lib)
messages: 388401
nosy: brett.cannon, vstinner
priority: normal
severity: normal
status: open
title: Remove _xxsubinterpreters from sys.stdlib_module_names
type: behavior
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43439] [security] Add audit events on GC functions giving access to all Python objects

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +23578
pull_request: https://github.com/python/cpython/pull/24811

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43439] [security] Add audit events on GC functions giving access to all Python objects

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +23577
pull_request: https://github.com/python/cpython/pull/24810

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43439] [security] Add audit events on GC functions giving access to all Python objects

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset b4f9089d4aa787c5b74134c98e5f0f11d9e63095 by Pablo Galindo in 
branch 'master':
bpo-43439: Add audit hooks for gc functions (GH-24794)
https://github.com/python/cpython/commit/b4f9089d4aa787c5b74134c98e5f0f11d9e63095


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42988] [security] Information disclosure via pydoc -p: /getfile?key=path allows to read arbitrary file on the filesystem

[Miro Hrončok]

Miro Hrončok  added the comment:

Todd Cullum from Red Hat Security team:

"I don't have an account on Python's tracker, would you mind forwarding to 
upstream on my behalf that this is not only locally exploitable, but it can be 
exploited by actors on the adjacent network as well because 
https://github.com/python/cpython/commit/6a396c9807b1674a24e240731f18e20de97117a5
 was introduced in Python 3.7.0 alpha 1. I just used the -n option and got to 
read some of my own files using my cell phone on the WiFi. It does require the 
port to be unblocked by firewall though."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43446] Wrong character in footnote

[Berker Peksag]

Change by Berker Peksag :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43446] Wrong character in footnote

[Berker Peksag]

Berker Peksag  added the comment:


New changeset da602560a4816c88dcf4d75b3e4eea56c8d3bbc2 by Miss Islington (bot) 
in branch '3.9':
bpo-43446: Fix markup in sqlite3 footnote (GH-24806)
https://github.com/python/cpython/commit/da602560a4816c88dcf4d75b3e4eea56c8d3bbc2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43446] Wrong character in footnote

[Berker Peksag]

Berker Peksag  added the comment:


New changeset e89380765df8f0f02c90ad417e164d1597bd0b05 by Miss Islington (bot) 
in branch '3.8':
bpo-43446: Fix markup in sqlite3 footnote (GH-24806)
https://github.com/python/cpython/commit/e89380765df8f0f02c90ad417e164d1597bd0b05


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43446] Wrong character in footnote

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +23575
pull_request: https://github.com/python/cpython/pull/24808

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43446] Wrong character in footnote

[miss-islington]

Change by miss-islington :


--
pull_requests: +23576
pull_request: https://github.com/python/cpython/pull/24809

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43446] Wrong character in footnote

[Berker Peksag]

Berker Peksag  added the comment:


New changeset 62a03cd490f81c0fb01eaceb31aa8a4c7800ed0e by Kamil Turek in branch 
'master':
bpo-43446: Fix markup in sqlite3 footnote (GH-24806)
https://github.com/python/cpython/commit/62a03cd490f81c0fb01eaceb31aa8a4c7800ed0e


--
nosy: +berker.peksag

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43443] Should shelve support dict union?

[Dominik Vilsmeier]

Dominik Vilsmeier  added the comment:

It's true, having `__ior__` but not `__or__` would probably be weird. In the 
end it's just "nice to have", but I'm not even sure that this applies. Calling 
`db.update(...)` is still more explicit than `db |= ...`.  The docs mention that

> This eases the transition from dictionary based scripts to those requiring 
> persistent storage.

For my use cases, however, I always knew right from the beginning that I want 
object persistence between different runs of a script (e.g. for data analysis, 
caching the expensive results), so it was always clear that I'm working with a 
Shelf object and not a dict (i.e. no expectations on the availability of `|=`).

Primarily, this issue was meant to point out the mismatch of 
docs/implementation and not to get `|=` implemented for `Shelf`. In the end, I 
think updating the docs is all that is needed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43443] Should shelve support dict union?

[Brandt Bucher]

Brandt Bucher  added the comment:

+1 for the MutableMapping comment.

We purposely omitted shelve when determining what classes should grow the new 
operators. Guido's thoughts:

> I definitely think we should leave Shelf alone, it's a toy class from a 
> different era.

(https://bugs.python.org/msg364196)

I personally have no experience with shelve, so I'd rather defer to the 
judgement of others here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43455] pathlib mistakenly assumes os.getcwd() is a resolved path in Windows

[Tzu-ping Chung]

Change by Tzu-ping Chung :


--
keywords: +patch
nosy: +uranusjr
nosy_count: 5.0 -> 6.0
pull_requests: +23574
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/17716

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43455] pathlib mistakenly assumes os.getcwd() is a resolved path in Windows

[Eryk Sun]

New submission from Eryk Sun :

pathlib._WindowsFlavour.resolve() mistakenly assume that os.getcwd() returns a 
resolved path in Windows:

s = str(path)
if not s:
return os.getcwd()

I don't think this is a practical problem since `str(path)` should never be an 
empty string. But if there is a concern that the result is an empty string, the 
code should use `s = str(path) or '.'`, and resolve "." like any other relative 
path.

In POSIX the result of getcwd() "shall contain no components that are dot or 
dot-dot, or are symbolic links". In Windows, os.getcwd() calls WinAPI 
GetCurrentDirectoryW(), which returns a fully-qualified path that may contain 
symbolic components that would be resolved in a final path. This includes 
filesystem symlinks and bind mounts (junctions), as well as mapped and 
substitute drives (i.e. drives that resolve to a filesystem directory instead 
of a volume device).

--
components: Library (Lib), Windows
messages: 388393
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: pathlib mistakenly assumes os.getcwd() is a resolved path in Windows
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43440] Enable rtree support in SQLite

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

I've opened bpo-43454 for R*Tree callbacks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43454] [sqlite3] Add support for R*Tree callbacks

[Erlend Egeberg Aasland]

New submission from Erlend Egeberg Aasland :

Ref. bpo-43440

Now that both Windows and macOS builds compile SQLite with R*Tree support, we 
should consider adding support for R*Tree callbacks.

SQLite has two API's:
- sqlite3_rtree_query_callback() for SQLite 3.8.5 and newer.
- sqlite3_rtree_geometry_callback() for SQLite pre 3.8.5.

I suggest using the new API only, because it is more flexible, and it is also 
the one recommended by SQLite.

See https://sqlite.org/rtree.html


Python API:
sqlite3.Connection.create_rtree_query_function()

Too long function name?

As for the callback spec, I'm not sure what's the most pythonic approach?
callback(coords, *params, **query_info):
  coords   # array of coordinates of node or entry to check
  *params  # parameters passed to the SQL function
  **query_info # the rest of the relevant sqlite3_rtree_query_info members
  return (visibility, score)

--
components: Library (Lib)
messages: 388391
nosy: berker.peksag, erlendaasland
priority: normal
severity: normal
status: open
title: [sqlite3] Add support for R*Tree callbacks
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42914] pprint numbers with underscore

[Felipe]

Felipe  added the comment:

All yours! I'm tied up so won't be able to submit the PR

On Thu, 25 Feb 2021 at 10:12, Stéphane Blondon 
wrote:

>
> Stéphane Blondon  added the comment:
>
> I add the same idea but later than you, so I'm interested by such feature.
>
> Felipe: do you want to add a pull request to this issue (with Serhiy
> Storchaka implementation because it's the simplest one)?
>
> If not, I plan to write it.
> I will write it too if there is no reply in one month.
>
> --
> nosy: +sblondon
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43443] Should shelve support dict union?

[Raymond Hettinger]

Raymond Hettinger  added the comment:

One other thought.  While __ior__() could be used as a short-cut for update(), 
the related __or__() method is more problematic because it would need to create 
a new underlying DB.  So maybe this is a can worms that we should not open. It 
would be weird to have __ior__() but not __or__().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43443] Should shelve support dict union?

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> Maybe say that they implement the MutableMapping interface?

Yes, that's a good idea (also add link to the MutableMapping docs).

> Right, that seems like a good idea. But `__ior__` could
> be implemented nevertheless?

Off-hand, I don't see why not.

Adding Brandt to the nosy list to see what he thinks.

--
nosy: +brandtbucher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43453] docs: runtime_checkable example refers to changed behavior in 3.10

[Henry Schreiner]

New submission from Henry Schreiner :

The documentation here: 
https://docs.python.org/3/library/typing.html#typing.runtime_checkable refers 
to "For example, builtins.complex implements __float__(), therefore it passes 
an issubclass() check against SupportsFloat. However, the complex.__float__ 
method exists only to raise a TypeError with a more informative message.".

However, that's not true in Python 3.10 anymore, those methods were thankfully 
removed. See https://docs.python.org/3.10/whatsnew/3.10.html#removed or 
https://bugs.python.org/issue41974 for the removal.

This documentation should either say "before Python 3.10, ...", or pick some 
other example that still is valid. Happy to make the change if I know what 
direction this should go.

--
assignee: docs@python
components: Documentation
messages: 388387
nosy: Henry Schreiner, docs@python
priority: normal
severity: normal
status: open
title: docs: runtime_checkable example refers to changed behavior in 3.10
type: behavior
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43311] bpo-43311: PyInterpreterState_New use thread-specific data tstate before key create .

[STINNER Victor]

STINNER Victor  added the comment:

> PyInterpreterState_New call and use PyThreadState *tstate = 
> _PyThreadState_GET();

It is safe to call _PyThreadState_GET() before _PyGILState_Init().

_PyThreadState_GET() calls 
_Py_atomic_load_relaxed(&_PyRuntime.gilstate.tstate_current), it doesn't use 
autoTSSkey. Or did I miss something?

Are you building Python with --with-experimental-isolated-subinterpreters?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43440] Enable rtree support in SQLite

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

Anytime :)

I'll create an issue for rtree callbacks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43446] Wrong character in footnote

[Kamil Turek]

Change by Kamil Turek :


--
keywords: +patch
pull_requests: +23573
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24806

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43440] Enable rtree support in SQLite

[Steve Dower]

Steve Dower  added the comment:

Perfect, thanks!

Callbacks are a bigger ask, but I don't have any fundamental opposition to 
them. Probably worth opening a new issue, as it'll affect all platforms (I 
assume).

And yeah, the setup.py in the CPython repo doesn't impact any of the Windows 
build at all. It needs to go into PCbuild somewhere as well.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43440] Enable rtree support in SQLite

[Steve Dower]

Steve Dower  added the comment:


New changeset 31818e98d3b845d815e9caf2a3d330341bdc1b33 by Erlend Egeberg 
Aasland in branch 'master':
bpo-43440 : Enable SQLite R*Tree support for windows builds (GH-24797)
https://github.com/python/cpython/commit/31818e98d3b845d815e9caf2a3d330341bdc1b33


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43446] Wrong character in footnote

[Kamil Turek]

Kamil Turek  added the comment:

That's right. Actually, the docs code contains two dashes but I think second of 
them is skipped in build process.

Might be good to wrap it as an inline markup.

--
nosy: +kamilturek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43449] multiprocessing.Pool - crash in subprocess causes deadlock in parent

[Jamie Kirkpatrick]

Jamie Kirkpatrick  added the comment:

More reading around this issue and I stumbled on an existing issue which this 
is a dup of so it can be closed.

https://bugs.python.org/issue22393

--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42658] os.path.normcase() is inconsistent with Windows file system

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7856] Add Big5-ETen codec: Python big5 codec cannot decode \xf9\xd8 bytes (U+7881 expected)

[STINNER Victor]

Change by STINNER Victor :


--
title: cannot decode from or encode to big5 \xf9\xd8 -> Add Big5-ETen codec: 
Python big5 codec cannot decode \xf9\xd8 bytes (U+7881 expected)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7856] cannot decode from or encode to big5 \xf9\xd8

[STINNER Victor]

STINNER Victor  added the comment:

> It looks like the F9D6–F9FE characters all come from the Big5-ETen extension

One option would be to add a new big5eten encoding to Python. Someone has to 
implement the code.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43445] Add frozen modules to sys.stdlib_module_names

[STINNER Victor]

STINNER Victor  added the comment:

> The list of available stdlib modules could change after compiling Python.

It's documented:
"[ sys.stdlib_module_names] is the same on all platforms. Modules which are not 
available on some platforms and modules disabled at Python build are also 
listed."
https://docs.python.org/dev/library/sys.html#sys.stdlib_module_names

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43445] Add frozen modules to sys.stdlib_module_names

[STINNER Victor]

STINNER Victor  added the comment:

Neil Schemenauer: "Not sure the proper place to discuss this but I wonder if 
putting this stdlib module names list in the executable is the best idea."

For the rationale behind sys.stdlib_module_names, please see the bpo-42955 
where alternative were discussed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43452] Microoptimize PyType_Lookup for cache hits

[Dino Viehland]

Change by Dino Viehland :


--
keywords: +patch
pull_requests: +23572
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24804

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43452] Microoptimize PyType_Lookup for cache hits

[Dino Viehland]

New submission from Dino Viehland :

The common case going through _PyType_Lookup is to have a cache hit.  There's 
some small tweaks which can make this a little cheaper:

1) the name field identity is used for a cache hit, and is kept alive by the 
cache.  So there's no need to read the hash code of the name - instead the 
address can be used as the hash.

2) There's no need to check if the name is cachable on the lookup either, it 
probably is, and if it is, it'll be in the cache.

3) If we clear the version tag when invalidating a type then we don't actually 
need to check for a valid version tag bit.

--
components: Interpreter Core
messages: 388377
nosy: dino.viehland
priority: normal
severity: normal
status: open
title: Microoptimize PyType_Lookup for cache hits
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43450] List amnesia

[Christian Heimes]

Christian Heimes  added the comment:

Florian's answer is correct. Thanks!

--
nosy: +christian.heimes
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43450] List amnesia

[Florian Bruhin]

Florian Bruhin  added the comment:

This is not a bug. itertools.product returns an iterator: 
https://docs.python.org/3/glossary.html#term-iterator

Quoting from there:

> [...] every iterator is also iterable and may be used in most places where 
> other iterables are accepted. One notable exception is code which attempts 
> multiple iteration passes. A container object (such as a list) produces a 
> fresh new iterator each time you pass it to the iter() function or use it in 
> a for loop. Attempting this with an iterator will just return the same 
> exhausted iterator object used in the previous iteration pass, making it 
> appear like an empty container.

--
nosy: +The Compiler

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43451] pydoc terminal suboptimal rendering of complex annotations

[David Wilson]

Change by David Wilson :


--
keywords: +patch
pull_requests: +23570
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24802

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43445] Add frozen modules to sys.stdlib_module_names

[Neil Schemenauer]

Neil Schemenauer  added the comment:

Not sure the proper place to discuss this but I wonder if putting this stdlib 
module names list in the executable is the best idea.  The list of available 
stdlib modules could change after compiling Python.  I understand you don't 
want to dynamically crawl the library path the build the list.  That's too 
slow.  However, is there a really strong reason to embed it in the Python 
executable?

Did you consider generating a .py module, containing the list.  E.g. 
"_stdlib_modules.py" inside the lib folder.  Then, you can have site.py or some 
similar startup logic import that module and assign it to 
sys.stdlib_module_names.

--
nosy: +nascheme

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43451] pydoc terminal suboptimal rendering of complex annotations

[David Wilson]

New submission from David Wilson :

When viewing docs for classes that use annotations, pydoc's rendering of 
argument lists is regularly truncated at the terminal edge (if using `less 
-S`), or wrapped in a manner where quickly scanning the output is next to 
impossible.

My 'classic' example is 'pydoc aiohttp.client.ClientSession', where the 
__init__ argument list wraps to 24 lines.

The pull request attached to this issue works around the problem by formatting 
arguments one-per-line if the sum of the arguments would exceed a hard-coded 
width of 150 characters. It is more of an RFC than a suggested fix. It produces 
acceptable formatting, but I'm not sure where the correct fix should be made -- 
in the inspect module, or somehow in the pydoc module, or even what the correct 
fix shuld be.

I will include a before/after screenshot in the pull request

I will attach before/after screenshot

--
components: Library (Lib)
messages: 388373
nosy: dw
priority: normal
severity: normal
status: open
title: pydoc terminal suboptimal rendering of complex annotations
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43450] List amnesia

[Márcio Mocellin]

New submission from Márcio Mocellin :

In Python 3.6.8 (default, Apr 16 2020, 01:36:27) [GCC 8.3.1 20191121 (Red Hat 
8.3.1-5)] on linux, when I materialize the list, it is shown and is deleted. 
Shouldn't the list persist in memory? Is this a bug or is it really?

```python
>>> vet_neg
['EH01', 'EH02', 'EH03']
Categories (3, object): ['EH01', 'EH02', 'EH03']
>>> cenarios
[0, 1]
>>> vet_neg_cenarios = itertools.product(vet_neg, cenarios, cenarios)
>>> vet_neg_cenarios

>>> list(vet_neg_cenarios)
[('EH01', 0, 0), ('EH01', 0, 1), ('EH01', 1, 0), ('EH01', 1, 1), ('EH02', 0, 
0), ('EH02', 0, 1), ('EH02', 1, 0), ('EH02', 1, 1), ('EH03', 0, 0), ('EH03', 0, 
1), ('EH03', 1, 0), ('EH03', 1, 1)]
>>> list(vet_neg_cenarios)
[]
>>> 
```

--
messages: 388372
nosy: marciomocellin
priority: normal
severity: normal
status: open
title: List amnesia
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15097] Improving wording on the thread-safeness of import

[Irit Katriel]

Change by Irit Katriel :


--
stage:  -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43449] multiprocessing.Pool - crash in subprocess causes deadlock in parent

[Jamie Kirkpatrick]

New submission from Jamie Kirkpatrick :

When using multiprocessing.Pool.apply[_async] a crash in the subprocess that is 
assigned the work item results in a deadlock in the parent process.

The parent process remains blissfully unaware of the crash in the subprocess 
and waits for a result forever. The parent process treats this as normal since 
the thread running _maintain_pool handles dead processes and repopulates the 
pool with a replacement subprocess.

See the test-case attached. Its not clear how this case should be handled but 
it can be very hard to trace issues in an application where this condition 
arises since all information about the crashing subprocess is lost (even with 
debug logging for the multiprocessing module enabled).

--
components: Library (Lib)
files: test.py
messages: 388371
nosy: jkp
priority: normal
severity: normal
status: open
title: multiprocessing.Pool - crash in subprocess causes deadlock in parent
type: behavior
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49860/test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12956] builds fail when installing to --prefix with space in path name

[Irit Katriel]

Change by Irit Katriel :


--
resolution:  -> wont fix
stage: needs patch -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43448] exec() ignores scope.

[Eryk Sun]

Eryk Sun  added the comment:

exec(obj, globals(), locals()) is the same as exec(obj). Also, locals in a 
function scope are optimized, so the locals() dict is a snapshot. Modifying the 
snapshot dict doesn't modify the optimized local variables. At most I think 
this issue is a duplicate of bpo-24800, to clarify the implicit local scope and 
behavior.

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43435] Py_BuildValue("y#".... returns incomplete result

[Christian Heimes]

Christian Heimes  added the comment:

A sleep(1) call affects exactly one aspect of the program: the state of the 
PRNG rand(). You re-initialize the process globale RNG in every function call 
with srand((unsigned) time()). time() has a granularity of one second.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42967] [CVE-2021-23336] urllib.parse.parse_qsl(): Web cache poisoning - `; ` as a query args separator

[Riccardo Schirone]

Riccardo Schirone  added the comment:

This CVE was reported against Python, however it does not seem to be Python's 
fault for supporting the `;` separator, which was a valid separator for older 
standards.

@AdamGold for this issue to become a real security problem, it seems that the 
proxy has to be configured to ignore certain parameters in the query. For NGINX 
and Varnish proxies mentioned in the article it seems that by default they use 
the entire request path, host included, and other things as cache key. For 
NGINX in particular I could find some snippets online to manipulate the query 
arguments and split them in arguments, so to remove the "utm_*" arguments, 
however this does not seem a standard(or at least default) behaviour, nor 
something easily supported.

I think that if that is the case and a user has to go out of his way to 
configure the (wrong) splitting of arguments in the proxy, it is not fair to 
blame python for accepting `;` as separator and assigning a CVE against it may 
cause confusion.

For distributions this is problematic as they have 2 choices:
1) "fix" python but with the risk of breaking user's programs/scripts relying 
on the previous API
2) keep older version/unpatched python so that user's programs still work, but 
with a python version "vulnerable" to this CVE.

None of these options is really ideal, especially if the problem is somewhere 
else.

@AdamGold Could you elaborate a bit more on how common it is and how much 
configuration is required for proxies to make `;` a problem in python?

--
nosy: +rschiron

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43435] Py_BuildValue("y#".... returns incomplete result

[Eric V. Smith]

Eric V. Smith  added the comment:

David:

If you could give us an example showing the inputs, the actual outputs, and how 
they differ from what you expect, that would be helpful. Otherwise it's a lot 
of guessing on our part.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43448] exec() ignores scope.

[Keepun]

New submission from Keepun :

exec() ignores scope.

Code:
--
class ExecTest:
def public(self):
h=None
exec("h='It is public'")
print(h)
self._private()
def _private(self):
h=None
exec("h='It is private'", globals(), locals())
print(h)

h = None
exec("h='It is global'")
print(h)

e=ExecTest()
e.public()

Result
--
It is global
None
None

--
Python 3.7.10 (default, Feb 26 2021, 13:06:18) [MSC v.1916 64 bit (AMD64)]
and
Python 3.8.5 (default, Jan 27 2021, 15:41:15) [GCC 9.3.0]

--
components: Interpreter Core
messages: 388366
nosy: Keepun
priority: normal
severity: normal
status: open
title: exec() ignores scope.
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7856] cannot decode from or encode to big5 \xf9\xd8

[Max Bolingbroke]

Max Bolingbroke  added the comment:

As of Python 3.7.9 this also affects \xf9\xd6 which should be \u7881 in 
Unicode. This character is the second character of 宏碁 which is the name of the 
Taiwanese electronics manufacturer Acer.

You can work around the issue using big5hkscs just like with the original 
\xf9\xd8 problem.

It looks like the F9D6–F9FE characters all come from the Big5-ETen extension 
(https://en.wikipedia.org/wiki/Big5#ETEN_extensions, 
https://moztw.org/docs/big5/table/eten.txt) which is so popular that it is a 
defacto standard. Big5-2003 (mentioned in a comment below) seems to be an 
extension of Big5-ETen. For what it's worth, whatwg includes these mappings in 
their own big5 reference tables: https://encoding.spec.whatwg.org/big5.html. 

Unfortunately Big5 is still in common use in Taiwan. It's pretty funny that 
Python fails to decode Big5 documents containing the name of one of Taiwan's 
largest multinationals :-)

--
nosy: +batterseapower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43397] Incorrect conversion path case with german character

[Eryk Sun]

Eryk Sun  added the comment:

ntpath.normcase() needs a platform-dependent implementation that calls 
LCMapStringEx() in Windows, in order to properly agree with case-insensitive 
Windows filesystems. See bpo-42658.

--
nosy: +eryksun
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> os.path.normcase() is inconsistent with Windows file system

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42658] os.path.normcase() is inconsistent with Windows file system

[Eryk Sun]

Change by Eryk Sun :


--
components: +Library (Lib), Unicode
nosy: +ezio.melotti, vstinner
versions: +Python 3.10, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43438] [doc] sys.addaudithook() documentation should be more explicit on its limitations

[JIanqiu Tao]

Change by JIanqiu Tao :


--
nosy: +zkonge

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43439] [security] Add audit events on GC functions giving access to all Python objects

[JIanqiu Tao]

Change by JIanqiu Tao :


--
nosy: +zkonge

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43229] freeze searches libpython3.9.so in /usr/lib instead /usr/lib/x86_64-linux-gnu

[Christian Bachmaier]

Christian Bachmaier  added the comment:

As for some time now, first I have to manually fix 
https://bugs.python.org/issue40350 first, i.e., use the recommended workaround 
not None check for spec.loader in /usr/lib/python3.9/modulefinder.py.


Setting LIBDIR to /usr/lib/x86_64-linux-gnu in 
/usr/lib/python3.9/_sysconfigdata__x86_64-linux-gnu.py
setting seams to fixe the issue linking issue of libpython3.9.so .


However, then the missing PyInit_lazr problem (as indicated in my first 
posting) stays:

...ml__sax__handler.o M_xml__sax__saxutils.o M_xml__sax__xmlreader.o M_xmlrpc.o 
M_xmlrpc__client.o M_zipfile.o M_zipimport.o 
/usr/lib/x86_64-linux-gnu/libpython3.9.so -lexpat   
-L/usr/lib -lz-lexpat  -lcrypt -lpthread -ldl  -lutil 
-lm -lm  -o helloworld
/usr/bin/ld: config.o:(.data.rel+0x278): undefined reference to `PyInit_lazr'
collect2: error: ld returned 1 exit status
make: *** [Makefile:900: helloworld] Error 1

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43444] [sqlite3] Move MODULE_NAME def from setup.py to module.h

[Erlend Egeberg Aasland]

Change by Erlend Egeberg Aasland :


--
keywords: +patch
pull_requests: +23569
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24801

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43375] memory leak in threading ?

[Mark Dickinson]

Mark Dickinson  added the comment:

Thanks; I missed that one. I'll update the state of this one to mark it as a 
duplicate of #37788.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37788] fix for bpo-36402 (threading._shutdown() race condition) causes reference leak

[Mark Dickinson]

Change by Mark Dickinson :


--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43375] memory leak in threading ?

[Mark Dickinson]

Change by Mark Dickinson :


--
resolution:  -> duplicate
superseder:  -> fix for bpo-36402 (threading._shutdown() race condition) causes 
reference leak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43312] Interface to select preferred "user" or "home" sysconfig scheme for an environment

[Matthias Klose]

Matthias Klose  added the comment:

The Debian/Ubuntu packages have a local patch for distutils/setuptools 
introducing an --install-layout option.  Maybe have the same for pip?

The intention with renaming/moving site-packages to 
/usr/lib/python3/dist-packages is to avoid users damaging their system 
installation, like

  sudo python3 setup.py install
  sudo pip install

overriding packages installed with the distro package manager. Just making this 
schema known, and making it the default would again allow pip acting on 
packages managed by the distro package manager.

Otoh, there are use cases, where you want to use the Python provided by the 
Linux distro and run into issues with the custom dist-packages.

 - Creating a venv using the system Python probably should use
   the versioned site-packages.

 - Building on top of a docker image for e.g. a Python App, you
   probably want to install into the dist-packages provided by
   the system Python.

So the problem to solve is

 - let a "sudo pip install" fail by default on the real system
 - let the same install succeed in a docker environment, or any
   other "image".
 - behave transparently on venv and virtualenv installations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43312] Interface to select preferred "user" or "home" sysconfig scheme for an environment

[Matthias Klose]

Matthias Klose  added the comment:

see also https://github.com/pypa/pip/issues/9617

--
nosy: +doko

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43229] freeze searches libpython3.9.so in /usr/lib instead /usr/lib/x86_64-linux-gnu

[Matthias Klose]

Matthias Klose  added the comment:

Please could you edit
/usr/lib/python3.9/_sysconfigdata__x86_64-linux-gnu.py
setting LIBDIR to /usr/lib/x86_64-linux-gnu
and see if that fixes the freeze issue?

that could also be fixed by configuring with --libdir=/usr/lib/x86_64-linux-gnu

but that also moves lib-dynload into the new libdir.

# Detailed destination directories
BINLIBDEST= $(LIBDIR)/python$(VERSION)
LIBDEST=$(SCRIPTDIR)/python$(VERSION)
DESTSHARED= $(BINLIBDEST)/lib-dynload

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43075] ReDoS in urllib.request

[STINNER Victor]

STINNER Victor  added the comment:

I see that you attached a redos_python.py benchmark (which looks like a 
benchmark that I wrote recently ;-)) but you didn't give results. Can you 
please show that your fix is effective to avoid catastrophic performances?

Is this issue related to the CVE-2020-8492? Is it the same issue or is it 
different?
https://python-security.readthedocs.io/vuln/urllib-basic-auth-regex.html

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43375] memory leak in threading ?

[Martin Panter]

Martin Panter  added the comment:

Sounds the same as Issue 37788, which is still open.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43311] bpo-43311: PyInterpreterState_New use thread-specific data tstate before key create .

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3329] API for setting the memory allocator used by Python

[miss-islington]

miss-islington  added the comment:


New changeset ea46c7bc503d1103d60c0ec7023bb52c5defa11d by Miss Islington (bot) 
in branch '3.9':
bpo-3329: Fix typo in PyObjectArenaAllocator doc (GH-24795)
https://github.com/python/cpython/commit/ea46c7bc503d1103d60c0ec7023bb52c5defa11d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43287] Use PEP 590 vectorcall to speed up calls to filter()

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-43447: "Generate vectorcall code to parse arguments using Argument 
Clinic".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43447] Generate vectorcall code to parse arguments using Argument Clinic

[STINNER Victor]

New submission from STINNER Victor :

To optimize the creation of objects, a lot of "tp_new" methods are defined 
twice: once in the legacy way (tp_new slot), once with the new VECTORCALL 
calling convention (tp_vectorcall slot). My concern is that the VECTORCALL 
implementation copy/paste most of the code just to parse arguments, whereas the 
specific code is just a few lines.

Example with the float type constructor:
--
static PyObject *
float_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
{
PyObject *return_value = NULL;
PyObject *x = NULL;

if ((type == _Type) &&
!_PyArg_NoKeywords("float", kwargs)) {
goto exit;
}
if (!_PyArg_CheckPositional("float", PyTuple_GET_SIZE(args), 0, 1)) {
goto exit;
}
if (PyTuple_GET_SIZE(args) < 1) {
goto skip_optional;
}
x = PyTuple_GET_ITEM(args, 0);
skip_optional:
return_value = float_new_impl(type, x);

exit:
return return_value;
}

/*[clinic input]
@classmethod
float.__new__ as float_new
x: object(c_default="NULL") = 0
/

Convert a string or number to a floating point number, if possible.
[clinic start generated code]*/

static PyObject *
float_new_impl(PyTypeObject *type, PyObject *x)
/*[clinic end generated code: output=ccf1e8dc460ba6ba input=f43661b7de03e9d8]*/
{
if (type != _Type) {
if (x == NULL) {
x = _PyLong_GetZero();
}
return float_subtype_new(type, x); /* Wimp out */
}

if (x == NULL) {
return PyFloat_FromDouble(0.0);
}
/* If it's a string, but not a string subclass, use
   PyFloat_FromString. */
if (PyUnicode_CheckExact(x))
return PyFloat_FromString(x);
return PyNumber_Float(x);
}

static PyObject *
float_vectorcall(PyObject *type, PyObject * const*args,
 size_t nargsf, PyObject *kwnames)
{
if (!_PyArg_NoKwnames("float", kwnames)) {
return NULL;
}

Py_ssize_t nargs = PyVectorcall_NARGS(nargsf);
if (!_PyArg_CheckPositional("float", nargs, 0, 1)) {
return NULL;
}

PyObject *x = nargs >= 1 ? args[0] : NULL;
return float_new_impl((PyTypeObject *)type, x);
}
--

Here the float_new() function (tp_new slot) is implemented with Argument 
Clinic: float_new() C code is generated from the [clinic input] DSL: good!

My concern is that float_vectorcall() code is hand written, it's boring to 
write and boring to maintain.

Would it be possible to add a new [clinic input] DSL for vectorcall? I expect 
something like that:
--
static PyObject *
float_vectorcall(PyObject *type, PyObject * const*args,
 size_t nargsf, PyObject *kwnames)
{
if (!_PyArg_NoKwnames("float", kwnames)) {
return NULL;
}

Py_ssize_t nargs = PyVectorcall_NARGS(nargsf);
if (!_PyArg_CheckPositional("float", nargs, 0, 1)) {
return NULL;
}

PyObject *x = nargs >= 1 ? args[0] : NULL;
return float_vectorcall_impl(type, x);
}

static PyObject *
float_vectorcall_impl(PyObject *type, PyObject *x)
{
return float_new_impl((PyTypeObject *)type, x);
}
--
 
where float_vectorcall() C code would be generated, and float_vectorcall_impl() 
would be the only part written manually. float_vectorcall_impl() gets a clean 
API and its body is way simpler to write and to maintain!

--
components: C API
messages: 388354
nosy: corona10, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Generate vectorcall code to parse arguments using Argument Clinic
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43446] Wrong character in footnote

[Sergio Livi]

New submission from Sergio Livi :

Hello,

There seems to be a display error in the sqlite documentation:
https://docs.python.org/3.9/library/sqlite3.html#f1

The footnote says "To get loadable extension support, you must pass 
–enable-loadable-sqlite-extensions to configure."
When actually the configure argument is --enable-etc. The double dash was 
substituted with a long dash (–), which breaks copy/paste for people. Here's an 
example: https://github.com/pyenv/pyenv/issues/1702.

--
assignee: docs@python
components: Documentation
messages: 388353
nosy: docs@python, serl2
priority: normal
severity: normal
status: open
title: Wrong character in footnote
type: enhancement
versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3329] API for setting the memory allocator used by Python

[miss-islington]

miss-islington  added the comment:


New changeset 5ca02c4799c07dba5192f87f9f2378dc24097166 by Miss Islington (bot) 
in branch '3.8':
bpo-3329: Fix typo in PyObjectArenaAllocator doc (GH-24795)
https://github.com/python/cpython/commit/5ca02c4799c07dba5192f87f9f2378dc24097166


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3329] API for setting the memory allocator used by Python

[miss-islington]

Change by miss-islington :


--
pull_requests: +23568
pull_request: https://github.com/python/cpython/pull/24800

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3329] API for setting the memory allocator used by Python

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 17.0 -> 18.0
pull_requests: +23567
pull_request: https://github.com/python/cpython/pull/24799

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3329] API for setting the memory allocator used by Python

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 0d6bd1ca7c683137d52041194f3a2b02219f225a by Victor Stinner in 
branch 'master':
bpo-3329: Fix typo in PyObjectArenaAllocator doc (GH-24795)
https://github.com/python/cpython/commit/0d6bd1ca7c683137d52041194f3a2b02219f225a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43397] Incorrect conversion path case with german character

[Сергей М]

Сергей М  added the comment:

I've found the useful function

https://docs.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-charlowerw

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43198] Operations on sets more than hundred times less efficient with python3.9 than with previous versions

[Eric Martin]

Eric Martin  added the comment:

Thank you for letting us know, I am actually not surprised the issue would 
bugger you and you would change your mind... Thinking further about it and 
experimenting further, I thought it might not be such an edge case and there 
could be a significant cost in some applications. Many thanks Raymond for 
everything you give to the community!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43440] Enable rtree support in SQLite

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

> does compile options set in setup.py propagate to the Windows build, or do we 
> need to set it both places?

Er, forget it. setup.py is of course only used to build the sqlite3 module, not 
sqlite3.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43372] ctypes: test_frozentable fails when make regen-frozen

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-43445 "Add frozen modules to sys.stdlib_module_names".

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13559] Use sendfile where possible in httplib

[Christian Heimes]

Christian Heimes  added the comment:

sendfile() only works for plain HTTP. For technical reasons it does not work 
for HTTPS (*). These days majority of services use HTTPS. Therefore the 
usefulness of sendfile() patch is minimal.

(*) It is possible to use sendfile() for TLS connections, but the feature 
requires a Kernel module that provides kTLS offloading feature, 
https://www.kernel.org/doc/html/latest/networking/tls-offload.html . In user 
space it requires OpenSSL 3.0.0 with kTLS support. 3.0.0 is currently under 
development.

--
nosy: +christian.heimes
versions: +Python 3.10 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43445] Add frozen modules to sys.stdlib_module_names

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +23566
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24798

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43445] Add frozen modules to sys.stdlib_module_names

[STINNER Victor]

New submission from STINNER Victor :

The sys.stdlib_module_names documentation says: "All module kinds are listed: 
pure Python, built-in, frozen and extension modules. Test modules are excluded."
https://docs.python.org/dev/library/sys.html#sys.stdlib_module_names

But I just noticed that frozen modules are not listed!

Attached PR fix this issue.

--
components: Library (Lib)
messages: 388345
nosy: vstinner
priority: normal
severity: normal
status: open
title: Add frozen modules to sys.stdlib_module_names
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43444] [sqlite3] Move MODULE_NAME def from setup.py to module.h

[Erlend Egeberg Aasland]

New submission from Erlend Egeberg Aasland :

Berker, can we please move the MODULE_NAME define from setup.py to 
Modules/_sqlite/module.h? I'm tired of all the undeclared identifier warnings. 
No other module defines their MODULE_NAME in setup.py.

--
components: Library (Lib)
messages: 388344
nosy: berker.peksag, erlendaasland
priority: normal
severity: normal
status: open
title: [sqlite3] Move MODULE_NAME def from setup.py to module.h
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43440] Enable rtree support in SQLite

[Erlend Egeberg Aasland]

Change by Erlend Egeberg Aasland :


--
keywords: +patch
pull_requests: +23565
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24797

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43440] Enable rtree support in SQLite

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

We should also consider adding support for R*Tree query callbacks using 
sqlite3_rtree_query_callback() for SQLite >= 3.8.5, and 
sqlite3_rtree_geometry_callback() for older versions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43443] Should shelve support dict union?

[Dominik Vilsmeier]

Dominik Vilsmeier  added the comment:

Right, that seems like a good idea. But `__ior__` could be implemented 
nevertheless?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43440] Enable rtree support in SQLite

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

> does compile options set in setup.py propagate to the Windows build, or do we 
> need to set it both places?

Seems like it doesn't; correct me if I'm wrong. PR coming up.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43441] mutilcorevm: global variable next_version_tag cause method cache bug

[junyixie]

Change by junyixie :


--
type:  -> crash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43440] Enable rtree support in SQLite

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

Actually, the macOS build already builds with R*Tree support enabled, but it is 
missing from PCbuild/sqlite3.vcxproj. I'm not very familiar with the Windows 
build; does compile options set in setup.py propagate to the Windows build, or 
do we need to set it both places?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40720] accessing mmap of file that is overwritten causes bus error

[Ronald Oussoren]

Ronald Oussoren  added the comment:

What happens here is that the file is truncated, which (more or less) truncates 
the memory mapping. Accessing a memory mapping beyond the length of the file 
results in a SIGBUS signal.

I'm not sure if there is much Python can do about this other than shrinking the 
window for crashes like this by aggressively checking if the file size has 
changed (but even then a crash will happen if another proces truncates the file 
between the time the check is done and the memory is actually accessed).

---

Variant of the script that explicitly truncates the file:

def main():
with tempfile.TemporaryDirectory() as tmp:
tmp_path = pathlib.Path(tmp)
path = tmp_path / "eg"

path.write_bytes(b"Hello, World!")

with path.open("r+b") as rf:
mm = mmap.mmap(rf.fileno(), 0, mmap.MAP_SHARED, mmap.PROT_READ)
rf.truncate(0)
bytes(mm)

if __name__ == "__main__":
main()

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43443] Should shelve support dict union?

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The comment is outdated. Shelf objects also do not support methods copy and 
fromkey. Creating a new Shelve object without specifying a new underlying 
database object does not make much sense.

Maybe say that they implement the MutableMapping interface?

>>> sorted(set(dir(dict)) - set(dir(shelve.Shelf)))
['__ior__', '__or__', '__ror__', 'copy', 'fromkeys']
>>> sorted(set(dir(collections.abc.MutableMapping)) - set(dir(shelve.Shelf)))
[]

--
nosy: +rhettinger, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com