[issue25481] PermissionError in subprocess.check_output() when an inaccessible directory on the path

[Eryk Sun]

Eryk Sun  added the comment:

> So, two interesting questions: does this in fact match the behavior of 
> os._execvpe, and does it match the behavior of the shell? 

I think it's fine. child_exec() tries all paths. It saves the first error 
that's not ENOENT or ENOTDIR. The saved error gets reported back, else ENOENT 
or ENOTDIR. This is the same as os._execvpe:

for dir in path_list:
fullname = path.join(dir, file)
try:
exec_func(fullname, *argrest)
except (FileNotFoundError, NotADirectoryError) as e:
last_exc = e
except OSError as e:
last_exc = e
if saved_exc is None:
saved_exc = e
if saved_exc is not None:
raise saved_exc
raise last_exc

Perhaps the rule for PATH search errors should be documented for os.execvp[e] 
and subprocess.

I think it matches the shell, except, AFAIK, the shell doesn't distinguish 
ENOENT and ENOTDIR errors. For example, where "/noexec" is a directory that has 
no execute access:

$ /bin/sh -c spam
/bin/sh: 1: spam: not found

$ PATH=/noexec:$PATH /bin/sh -c spam
/bin/sh: 1: spam: Permission denied

--
assignee:  -> docs@python
components: +Documentation, Library (Lib)
nosy: +docs@python
versions: +Python 3.10, Python 3.8, Python 3.9 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43478] Disallow Mock spec arguments from being Mocks

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2528] Change os.access to check ACLs under Windows

[Eryk Sun]

Eryk Sun  added the comment:

With increasing use of os.access() in shutil and tempfile, it would be nice to 
have a real implementation of os.access() for Windows. 

Instead of manually evaluating the security of the file/directory, as 
issue2528.2.patch attempts to do, I'd rather just open the file with the 
desired access (e.g. GENERIC_READ, GENERIC_WRITE, GENERIC_EXECUTE). An 
open-based check supports checking for sharing violations, filesystem policy 
(e.g. FILE_READ_ATTRIBUTES granted by the parent directory), non-filesystem 
devices, and access policy implemented by filter drivers in the device stack. 

The code to open the file/directory can be factored out and generalized from 
the stat() implementation. The common open function can implement the flags 
AT_SYMLINK_NOFOLLOW and AT_EACCESS (without which it should temporarily revert 
to the process access token). Also, when a directory is opened with 
GENERIC_WRITE access, it can re-try the open with FILE_DELETE_CHILD access, 
which POSIX includes in write access for a directory.

An S_OK flag could also be supported to ignore a sharing violation in Windows. 
[MS-FSA] section 2.1.5.1.2 (Open of an Existing File) specifies that access 
sharing is checked after the readonly attribute and file security access check. 
So if an open fails with a sharing violation, the caller knows that access was 
otherwise granted.

--
versions: +Python 3.10, Python 3.8, Python 3.9 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43423] Subprocess IndexError possible in _communicate

[miss-islington]

miss-islington  added the comment:


New changeset ad83fde75463dad2df878ff264f52436eb48bc6b by Miss Islington (bot) 
in branch '3.9':
bpo-43423 Fix IndexError in subprocess _communicate function (GH-24777)
https://github.com/python/cpython/commit/ad83fde75463dad2df878ff264f52436eb48bc6b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43478] Disallow Mock spec arguments from being Mocks

[Matthew Suozzo]

New submission from Matthew Suozzo :

An unfortunately common pattern over large codebases of Python tests is for 
spec'd Mock instances to be provided with Mock objects as their specs. This 
gives the false sense that a spec constraint is being applied when, in fact, 
nothing will be disallowed.

The two most frequently observed occurrences of this anti-pattern are as 
follows:

* Re-patching an existing autospec.

  def setUp(self):
mock.patch.object(mod, 'Klass', autospec=True).start()
self.addCleanup(mock.patch.stopall)

  @mock.patch.object(mod, 'Klass', autospec=True)  # :(
  def testFoo(self, mock_klass):
# mod.Klass has no effective spec.

* Deriving an autospec Mock from an already-mocked object

  def setUp(self):
mock.patch.object(mod, 'Klass').start()
...
mock_klass = mock.create_autospec(mod.Klass)  # :(
# mock_klass has no effective spec

This is fairly easy to detect using _is_instance_mock at patch time however it 
can break existing tests.

I have a patch ready and it seems like this error case is not frequent enough 
that it would be disruptive to address.

Another option would be add it as a warning for a version e.g. 3.10 and then 
potentially make it a breaking change in 3.11. However considering this is a 
test-only change with a fairly clear path to fix it, that might be overly 
cautious.

--
components: Tests
messages: 388532
nosy: msuozzo
priority: normal
severity: normal
status: open
title: Disallow Mock spec arguments from being Mocks
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43423] Subprocess IndexError possible in _communicate

[miss-islington]

miss-islington  added the comment:


New changeset 1a5001c606b55226c03fa1046aa8f5e1db2fa67d by Miss Islington (bot) 
in branch '3.8':
bpo-43423 Fix IndexError in subprocess _communicate function (GH-24777)
https://github.com/python/cpython/commit/1a5001c606b55226c03fa1046aa8f5e1db2fa67d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43423] Subprocess IndexError possible in _communicate

[miss-islington]

Change by miss-islington :


--
pull_requests: +23596
pull_request: https://github.com/python/cpython/pull/24831

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43423] Subprocess IndexError possible in _communicate

[miss-islington]

Change by miss-islington :


--
pull_requests: +23595
pull_request: https://github.com/python/cpython/pull/24830

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43423] Subprocess IndexError possible in _communicate

[miss-islington]

Change by miss-islington :


--
pull_requests: +23594
pull_request: https://github.com/python/cpython/pull/24823

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43477] from x import * behavior inconsistent between module types.

[Thomas]

New submission from Thomas :

I'm looking for clarification as to how `from x import *` should operate when 
importing file/directory-based modules versus when importing a sub-module from 
within a directory-based module.

While looking into a somewhat related issue with pylint, I noticed that `from x 
import *` appears to behave inconsistently when called from within a 
directory-based module on a sub-module. Whereas normally `from x import *` 
intentionally does not cause `x` to be added to the current namespace, when 
called within a directory-based module to import from a sub-module (so, `from 
.y import *` in an `__init__.py`, for example), the sub-module (let's say, `y`) 
*does* end up getting added to the importing namespace. From what I can tell, 
this should not be happening. If this oddity has been documented somewhere, I 
may have just missed it, so please let me know if it has been.

This inconsistency is actually setting off pylint (and confusing its AST 
handling code) when you use the full path to reference any member of the 
`asyncio.subprocess` submodule (for example, `asyncio.subprocess.Process`) 
because, according to `asyncio`'s `__init__.py` file, no explicit import of the 
`subprocess` sub-module ever occurs, and yet you can draw the entire path all 
the way to it, and its members. I've attached a generic example of the 
different behaviors (tested with Python 3.9) using simple modules, including a 
demonstration of the sub-module import.

Thomas

--
components: Interpreter Core
files: example.txz
messages: 388530
nosy: kaorihinata
priority: normal
severity: normal
status: open
title: from x import * behavior inconsistent between module types.
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file49871/example.txz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43471] Fails to import bz2 on Ubuntu

[Eric V. Smith]

Change by Eric V. Smith :


--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43267] [sqlite3] Redundant type checks in pysqlite_statement_bind_parameter()

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

$ python -m pyperf compare_to -G  main.json patched.json
Faster (1):
- bind: 63.6 us +- 1.2 us -> 61.8 us +- 0.9 us: 1.03x faster

$ git diff --shortstat master
 1 file changed, 41 insertions(+), 74 deletions(-)

--
keywords: +patch
Added file: https://bugs.python.org/file49870/patch.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43439] [security] Add audit events on GC functions giving access to all Python objects

[Saiyang Gou]

Saiyang Gou  added the comment:

There is a minor issue here. For gc.get_referrers and gc.get_referents, 
probably the format code for PySys_Audit should be "(O)" instead of "O". 
Typically the tuple `args` passed to the hook functions are fixed-length as 
described in the audit events table. Here `objs` itself is a tuple (containing 
variable-length arguments) and directly passed to the audit hook with being 
wrapped by another layer of tuple. If the hook function is to receive a 
variable-length tuple, probably the documentation should be updated to mention 
this.

--
nosy: +gousaiyang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43132] Incorrect handling of PyObject_RichCompareBool() in the _zoneinfo module

[Zackery Spytz]

Change by Zackery Spytz :


--
pull_requests: +23593
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/24829

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43356] PyErr_SetInterrupt should have an equivalent that takes a signal number

[Antoine Pitrou]

Change by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43356] PyErr_SetInterrupt should have an equivalent that takes a signal number

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset ba251c2ae6654bfc8abd9d886b219698ad34ac3c by Antoine Pitrou in 
branch 'master':
bpo-43356: Allow passing a signal number to interrupt_main() (GH-24755)
https://github.com/python/cpython/commit/ba251c2ae6654bfc8abd9d886b219698ad34ac3c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40255] Fixing Copy on Writes from reference counting

[Eric Snow]

Eric Snow  added the comment:

Note that my PR does make the builtin types immortal (at least those 
initialized in _PyTypes_Init()).  However, I did not make the singletons or 
other candidate objects immortal in the PR.  That can be done separately.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40255] Fixing Copy on Writes from reference counting

[Eric Snow]

Eric Snow  added the comment:

FYI, I've just put up a similar PR [1] to Eddie's, with a focus on object 
immortality rather than immutability.  However, if Py_IMMORTAL_CONST_REFCOUNTS 
is defined then it should have the behavior Eddie is after.

[1] https://github.com/python/cpython/pull/24828

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40255] Fixing Copy on Writes from reference counting

[Eric Snow]

Change by Eric Snow :


--
nosy: +eric.snow
nosy_count: 15.0 -> 16.0
pull_requests: +23592
pull_request: https://github.com/python/cpython/pull/24828

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43476] Enabling access to showsyntaxerror for IDLE's shell

[Andre Roberge]

New submission from Andre Roberge :

As a result of https://bugs.python.org/issue43008, IDLE now supports custom 
exception hook for almost all cases except for code entered in the interactive 
shell that result in SyntaxError. It would be useful for some applications if a 
way to replace the error handling currently done by IDLE for this particular 
case was made available to user code.

I consider this to be in the "would be nice to have" category, but in no means 
a high priority item.

--
assignee: terry.reedy
components: IDLE
messages: 388524
nosy: aroberge, terry.reedy
priority: normal
severity: normal
status: open
title: Enabling access to showsyntaxerror for IDLE's shell
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43353] Document that logging.getLevelName() can return a numeric value.

[Mariusz Felisiak]

Mariusz Felisiak  added the comment:

I've prepared PRs with backports.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43353] Document that logging.getLevelName() can return a numeric value.

[Mariusz Felisiak]

Change by Mariusz Felisiak :


--
pull_requests: +23591
pull_request: https://github.com/python/cpython/pull/24826

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43353] Document that logging.getLevelName() can return a numeric value.

[Mariusz Felisiak]

Change by Mariusz Felisiak :


--
pull_requests: +23590
pull_request: https://github.com/python/cpython/pull/24825

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43475] Worst-case behaviour of hash collision with float NaN

[Mark Dickinson]

Mark Dickinson  added the comment:

> I also wonder if there's security implication for servers that process 
> user-submitted input

Yes, the "malicious actor" scenario is another one to consider. But unlike the 
string hashing attack, I'm not seeing a realistic way for the nan hash 
collisions to be used in attacks, and I'm content not to worry about that until 
someone gives an actual proof of concept. Many of Python's hash functions are 
fairly predictable (by design!) and there are already lots of other ways to 
deliberately construct lots of hash collisions with non-string non-float values.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43423] Subprocess IndexError possible in _communicate

[Gregory P. Smith]

Gregory P. Smith  added the comment:

The bug is fixed, Thanks Chris!  There was a refactoring noted as being nice in 
my comments on the primary main branch PR that would be nice to have.  But 
isn't critical.  If you want to make a PR for that, just reuse this bpo-43423 
issue number on the PR and assign it to me.

The backports to 3.9 an 3.8 are approved and should automerge after CI finishes.

--
resolution:  -> fixed
stage: patch review -> commit review
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43423] Subprocess IndexError possible in _communicate

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 7.0 -> 8.0
pull_requests: +23589
pull_request: https://github.com/python/cpython/pull/24824

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43423] Subprocess IndexError possible in _communicate

[Gregory P. Smith]

Gregory P. Smith  added the comment:


New changeset b4fc44bb2d209182390b4f9fdf074a46b0165a2f by Chris Griffith in 
branch 'master':
bpo-43423 Fix IndexError in subprocess _communicate function (GH-24777)
https://github.com/python/cpython/commit/b4fc44bb2d209182390b4f9fdf074a46b0165a2f


--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43475] Worst-case behaviour of hash collision with float NaN

[Cong Ma]

Cong Ma  added the comment:

Sorry, please ignore my rambling about "float() returning aliased object" -- in 
that case the problem with hashing doesn't arise.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43470] Installation of Python 3.6.13 fails on MacOS Big Sur 11.2.3

[Ned Deily]

Ned Deily  added the comment:

[Updated response] I'm sorry you ran into this but, unfortunately, Python 3.6 
is in its security-fix-only phase of its life cycles and, as such, we do not 
provide support on them for new OS versions.  Please see the detailed response 
to Issue43393 for more information. While the particular error you ran into is 
easily worked around, the resulting Python will still not execute correctly on 
macOS 11 Big Sur as there are a number of other run-time issues that would need 
to be fixed. At the moment, only Python 3.9.2 is fully supported on Big Sur.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43438] [doc] sys.addaudithook() documentation should be more explicit on its limitations

[Saiyang Gou]

Saiyang Gou  added the comment:

> Please also keep in mind that sys.addaudithook() does **not** add a global 
> hook. The function adds a per-interpreter hook.

Yes, I'm aware of this. And this should be better documented. When I was 
playing around with audit hooks and reading the source code, I see hooks 
written in Python (added via sys.addaudithook) are per-interpreter (stored in a 
Python list object as part of per-interpreter state) while hooks written in C 
(added via PySys_AddAuditHook) are global (stored in a C linked list). C hooks 
are more robust. The Python hooks are not inherited in a child process created 
via multiprocessing when the start method is "spawn" or "forkserver" (not 
"fork") since they are per-interpreter. I'm not sure whether we should audit 
the creation of new processes/threads via multiprocessing/threading modules 
(and also creation of sub-interpreters when PEP 554 gets accepted).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43470] Installation of Python 3.6.13 fails on MacOS Big Sur 11.2.3

[Ned Deily]

Change by Ned Deily :


--
Removed message: https://bugs.python.org/msg388515

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43470] Installation of Python 3.6.13 fails on MacOS Big Sur 11.2.3

[Ned Deily]

Change by Ned Deily :


--
superseder: Installation of Python 3.6.13 fails on MacOS Big Sur 11.2.3 -> 
Older Python builds are missing a required file on Big Sur

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43469] Python 3.6 fails to run on MacOS (Big Sur 11.2.3)

[Ned Deily]

Ned Deily  added the comment:

I'm sorry you ran into this but, unfortunately, the binaries from Python.org 
macOS installers prior to 3.7 do not work on macOS 11 Big Sur. Since Python 3.6 
and 3.7 are in the security-fix-only phase of the life cycles, we do not 
provide support on them for new OS versions and providing new binary 
installers.  Please see the detailed response to Issue43470 for more 
information.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Older Python builds are missing a required file on Big Sur

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43470] Installation of Python 3.6.13 fails on MacOS Big Sur 11.2.3

[Ned Deily]

Ned Deily  added the comment:

I'm sorry you ran into this but, unfortunately, the binaries from Python.org 
macOS installers prior to 3.7 do not work on macOS 11 Big Sur. Since Python 3.6 
and 3.7 are in the security-fix-only phase of the life cycles, we do not 
provide support on them for new OS versions and providing new binary 
installers.  Please see the detailed response to Issue43470 for more 
information.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Installation of Python 3.6.13 fails on MacOS Big Sur 11.2.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43475] Worst-case behaviour of hash collision with float NaN

[Cong Ma]

Cong Ma  added the comment:

Thank you @mark.dickinson for the detailed analysis.

In addition to your hypothetical usage examples, I am also trying to understand 
the implications for user code.

If judging by the issues people open on GitHub like this: 
https://github.com/pandas-dev/pandas/issues/28363 yes apparently people do run 
into the "NaN as key" problem every now and then, I guess. (I'm not familiar 
enough with the poster's real problem in the Pandas setting to comment about 
that one, but it seems to suggest people do run into "real world" problems that 
shares some common features with this one). There's also StackOverflow threads 
like this (https://stackoverflow.com/a/17062985) where people discuss hashing a 
data table that explicitly use NaN for missing values. The reason seems to be 
that "[e]mpirical evidence suggests that hashing is an effective strategy for 
dimensionality reduction and practical nonparametric estimation." 
(https://arxiv.org/pdf/0902.2206.pdf).

I cannot imagine whether the proposed change would make life easier for people 
who really want NaN keys to begin with. However I think it might reduce the 
exposure to worst-case performances in dict (and maybe set/frozenset), while 
not hurting Python's own self-consistency.

Maybe there's one thing to consider about future compatibility though... 
because the proposed fix depends on the current behaviour that floats (and by 
extension, built-in float-like objects such as Decimal and complex) are not 
cached, unlike small ints and interned Unicode objects. So when you explicitly 
construct a NaN by calling float(), you always get a new Python object. Is this 
guaranteed now on different platforms with different compilers? I'm trying to 
parse the magic in Include/pymath.h about the definition of macro `Py_NAN`, 
where it seems to me that for certain configs it could be a `static const 
union` translation-unit-level constant. Does this affect the code that actually 
builds a Python object from an underlying NaN? (I apologize if this is a bad 
question). But if it doesn't and we're guaranteed to really have Python 
NaN-objects that don't alias if explicitly constructed, is this something 
unlikely to change in the future?

I also wonder if there's security implication for servers that process 
user-submitted input, maybe running a float() constructor on some string list, 
checking exceptions but silently succeeding with "nan": arguably this is not 
going to be as common an concern as the string-key collision DoS now foiled by 
hash randomization, but I'm not entirely sure.

On "theoretical interest".. yes theoretical interests can also be "real world" 
if one teaches CS theory in real world using Python, see 
https://bugs.python.org/issue43198#msg386849

So yes, I admit we're in an obscure corner of Python here. It's a tricky, 
maybe-theoretical, but seemingly annoying but easy-to-fix issue..

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43475] Worst-case behaviour of hash collision with float NaN

[Mark Dickinson]

Mark Dickinson  added the comment:

On third thoughts, of course it *would* help, because the Counter is keeping 
references to the realised NaN values. I think I'll go away now and come back 
when my brain is working.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43475] Worst-case behaviour of hash collision with float NaN

[Mark Dickinson]

Mark Dickinson  added the comment:

Hmm. On second thoughts, the proposed solution wouldn't actually *help* with 
the situation I gave: the elements (lazily) realised from the NumPy array are 
highly likely to all end up with the same address in RAM. :-(

>>> x = np.full(10, np.nan)
>>> for v in x:
... print(id(v))
... del v
... 
4601757008
4601757008
4601757008
4601757008
4601757008
4601757008
4601757008
4601757008
4601757008
4601757008

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43475] Worst-case behaviour of hash collision with float NaN

[Mark Dickinson]

Mark Dickinson  added the comment:

Sigh. When I'm undisputed ruler of the multiverse, I'm going to make "NaN == 
NaN" return True, IEEE 754 be damned. NaN != NaN is fine(ish) at the level of 
numerics; the problems start when the consequences of that choice leak into the 
other parts of the language that care about equality. NaNs just shouldn't be 
considered "special enough to break the rules" (where the rules here are that 
the equivalence relation being used as the basis of equality for a general 
container should actually *be* an equivalence relation - reflexive, symmetric, 
and transitive).

Anyway, more constructively ...

I agree with the analysis, and the proposed solution seems sound: if we're 
going to change this, then using the object hash seems like a workable 
solution. The question is whether we actually do need to change this.

I'm not too worried about backwards compatibility here: if we change this, 
we're bound to break *someone's* code somewhere, but it's hard to imagine that 
there's *that* much code out there that makes useful use of the property that 
hash(nan) == 0. The most likely source of breakage I can think of is in test 
code that checks that 3rd-party float-like things (NumPy's float64, or gmpy2 
floats, or ...) behave like Python's floats.

@Cong Ma: do you have any examples of cases where this has proved, or is likely 
to prove, a problem in real-world code, or is this purely theoretical at this 
point?

I'm finding it hard to imagine cases where a developer *intentionally* has a 
dictionary with several NaN values as keys. (Even a single NaN as a key seems a 
stretch; general floats as dictionary keys are rare in real-world code that 
I've encountered.) But it's somewhat easier to imagine situations where one 
could run into this accidentally. Here's one such:

>>> import collections, numpy as np
>>> x = np.full(10, np.nan)
>>> c = collections.Counter(x)

That took around 4 minutes of non-ctrl-C-interruptible time on my laptop. (I 
was actually expecting it not to "work" as a bad example: I was expecting that 
NaNs realised from a NumPy array would all be the same NaN object, but that's 
not what happens.) For comparison, this appears instant:

>>> x = np.random.rand(10)
>>> c = collections.Counter(x)

And it's not a stretch to imagine having a NumPy array representing a masked 
binary 1024x1024-pixel image (float values of NaN, 0.0 and 1.0) and using 
Counter to find out how many 0s and 1s there were.

On the other hand, we've lived with the current behaviour for some decades now 
without it apparently ever being a real issue.

On balance, I'm +1 for fixing this. It seems like a failure mode that would be 
rarely encountered, but it's quite unpleasant when it *is* encountered.

--
nosy: +rhettinger, tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17519] unittest should not try to run abstract classes

[Nathaniel Manista]

Nathaniel Manista  added the comment:

In the years since this was considered and declined, I wonder if the facts have 
changed sufficiently to make it now worth doing?

I often find myself writing TestCases for interfaces, and those define test_* 
methods that call the interface under test, but of course my TestCase needs to 
be abstract because I'm only testing an interface and not a concrete 
implementation of that interface. It's also the case when I'm writing this kind 
of test that I wish to use a type-checker, and if I can have my abstract 
TestCase inherit from unittest.TestCase, that will satisfy my type-checker's 
questions about why I believe my TestCase has all kinds of assert* methods 
defined that it doesn't otherwise see.

I currently have the impression that if this is cheap enough to do, it may be 
worth doing just for the ergonomics alone? It mightn't make anything impossible 
become possible to do, but I forecast that it would make something difficult to 
do much more straightforward to do.

(I remain a fan of the all-powerful load_tests protocol, but... often it's nice 
to escape all the responsibility that comes with use of it.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43312] Interface to select preferred "user" or "home" sysconfig scheme for an environment

[Piotr Dobrogost]

Change by Piotr Dobrogost :


--
nosy: +piotr.dobrogost

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43475] Worst-case behaviour of hash collision with float NaN

[Mark Dickinson]

Change by Mark Dickinson :


--
nosy: +mark.dickinson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43181] Python macros don’t shield arguments

[STINNER Victor]

STINNER Victor  added the comment:

> Yet... the first argument is still unshielded, passed to a macro that expects 
> one single macro argument.

Are you talking about the current code in the master branch or the 3.9 branch? 
If you are talking about the _PyObject_CAST(ob) call, would you mind to 
elaborate how it is an issue? The code in master LGTM.

> That’s not a regression, it wasn’t shielded in 3.8 either, but why not just 
> parenthesise each macro argument that denotes an expression (as opposed to 
> e.g. name)?

In the master branch, Include/ and its subdirectories contains 158 .h files for 
a total of 20K lines of C code. It's not easy to check all files. If you 
spotted bugs, would you mind to give examples, or a way to automatically detect 
bugs?

As I wrote previously, I dislike macros. If someone is changed, I would prefer 
to convert the function into a static inline function which doesn't have macros 
pitfalls.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43475] Worst-case behaviour of hash collision with float NaN

[Cong Ma]

New submission from Cong Ma :

Summary: CPython hash all NaN values to 0. This guarantees worst-case behaviour 
for dict if numerous existing keys are NaN. I think by hashing NaN using the 
generic object (or "pointer") hash instead, the worst-case situation can be 
alleviated without changing the semantics of either dict or float. However, 
this also requires changes to how complex and Decimal objects hash, and 
moreover incompatible change to sys.hash_info. I would like to hear how Python 
developers think about this matter.



Currently CPython uses the hard-coded macro constant 0 (_PyHASH_NAN, defined in 
Include/pyhash.h) for the hash value of all floating point NaNs. The value is 
part of the sys.hashinfo API and is re-used by complex and Decimal in computing 
its hash in accordance with Python builtin-type documentation. [0]

(The doc [0] specifically says that "[a]ll hashable nans have the same hash 
value.")

This is normally not a great concern, except for the worst case performance. 
The problem is that, since they hash to the same value and they're guaranteed 
to compare unequal to any compatible numeric value -- not even to themselves, 
this means they're guaranteed to collide.

For this reason I'd like to question whether it is a good idea to have all 
hashable NaNs with the same hash value.

There has been some discussions about this over the Web for some time (see 
[1]). In [1] the demo Python script times the insertion of k distinct NaN keys 
(as different objects) into a freshly created dict. Since the keys are distinct 
and are guaranteed to collide with each other (if any), the run time of a 
single lookup/insertion is roughly linear to the existing number of NaN keys. 
I've recreated the same script using with a more modern Python (attached).

I'd suggest a fix for this worst-case behaviour: instead of returning the hash 
value 0 for all NaNs, use the generic object (pointer) hash for these objects. 
As a PoC (also in the attached script), it roughly means

```
class myfloat(float):
def __hash__(self):
if self != self:  # i.e., math.isnan(self)
return object.__hash__(self)
return super().__hash__(self)
```

This will
- keep the current behaviour of dict intact;
- keep the invariant `a == b implies hash(a) == hash(b)` intact, where 
applicable;
- uphold all the other rules for Python numeric objects listed in [0];
- make hash collisions no more likely than with object() instances (dict lookup 
time is amortized constant w.r.t. existing number of NaN keys).

However, it will
- not keep the current rule "All hashable nans have the same hash value";
- not be compatible with the current sys.hash_info API (requires the removal of 
the "nan" attribute from there and documenting the change);
- require congruent modifications in complex and Decimal too.

Additionally, I don't think this will affect module-level NaN "constants" such 
as math.nan and how they behave. The "NaN constant" has never been a problem to 
begin with. It's only the *distinct* NaN objects that may cause the worst-case 
behaviour.



Just for the record I'd also like to clear some outdated info or misconception 
about NaN keys in Python dicts. It's not true that NaN keys, once inserted, 
cannot be retrieved (e.g., as claimed in [1][2]). In Python, they can be, if 
you keep the original key *object* around by keeping a reference to it (or 
obtaining a new one from the dict by iterating over it). This, I think, is 
because Python dict compares for object identity before rich-comparing for 
equality in `lookdict()` in Objects/dictobject.c, so this works for `d = 
dict()`:

```
f = float("nan")
d[f] = "value"
v = d[f]
```

but this fails with `KeyError`, as it should:

```
d[float("nan")] = "value"
v = d[float("nan")]
```

In this regard the NaN float object behaves exactly like the object() instance 
as keys -- except for the collisions. That's why I think at least for floats 
the "object" hash is likely to work. The solution using PRNG [1] (implemented 
with the Go language) is not necessary for CPython because the live objects are 
already distinct.



Links:

[0] https://docs.python.org/3/library/stdtypes.html#hashing-of-numeric-types
[1] https://research.swtch.com/randhash
[2] 
https://readafterwrite.wordpress.com/2017/03/23/how-to-hash-floating-point-numbers/

--
components: Interpreter Core, Library (Lib)
files: nan_key.py
messages: 388508
nosy: congma
priority: normal
severity: normal
status: open
title: Worst-case behaviour of hash collision with float NaN
type: performance
Added file: https://bugs.python.org/file49869/nan_key.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43454] [sqlite3] Add support for R*Tree callbacks

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

Unfortunately, there's no way to detect R*Tree support in sqlite3.h. We could 
run a script that dumps the compile options, and generate a config.h file from 
that:

$ for L in $(sqlite3 ":memory:" "pragma compile_options"); do echo #define 
SQLITE_$L >> config.h; done
$ cat config.h
#define SQLITE_COMPILER=clang-12.0.0
#define SQLITE_ENABLE_DBSTAT_VTAB
#define SQLITE_ENABLE_FTS4
#define SQLITE_ENABLE_FTS5
#define SQLITE_ENABLE_GEOPOLY
#define SQLITE_ENABLE_JSON1
#define SQLITE_ENABLE_RTREE
#define SQLITE_ENABLE_STMTVTAB
#define SQLITE_THREADSAFE=1


For Windows and macOS builds, we've already got the SQLite library compile 
options, so we can just reuse them in CFLAGS when we're building the sqlite3 
module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43452] Microoptimize PyType_Lookup for cache hits

[STINNER Victor]

STINNER Victor  added the comment:

> Geometric mean: 1.04x faster

Hey, it's good to see my new pyperf feature being used :-D IMO it helps to more 
easily compare a large set of benchmarks.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43462] canvas.bbox returns None on 'hidden' items while coords doesn't

[Vincent]

Vincent  added the comment:

... calculate the coordinates using the canvas.coords function

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43462] canvas.bbox returns None on 'hidden' items while coords doesn't

[Vincent]

Vincent  added the comment:

Thank you for your comments. Yes, I would doubt that, too. You would expect to 
get a bounding box regardless what state the canvas item are in. The only way 
to fix this (and I'm open to suggestions), is to create a (custom) function 
that calculate every object belonging to a specific tag and add/subtract the 
coordinates - this is cumbersome.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43435] Py_BuildValue("y#".... returns incomplete result

[David Wood]

Change by David Wood :


Removed file: https://bugs.python.org/file49861/crypt.tar.gz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37193] Memory leak while running TCP/UDPServer with socketserver.ThreadingMixIn

[STINNER Victor]

STINNER Victor  added the comment:

Thank you for fixing the regression Jason R. Coombs ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43462] canvas.bbox returns None on 'hidden' items while coords doesn't

[E. Paine]

E. Paine  added the comment:

This can be easily reproduced in Wish (8.6.11):

% pack [canvas .c]
% .c create rectangle 10 10 90 90
1
% .c bbox 1
9 9 91 91
% .c create rectangle 20 20 80 80 -state hidden
2
% .c bbox 2
%

I doubt this is a bug because the docs 
(https://www.tcl.tk/man/tcl8.6/TkCmd/canvas.htm#M36) say:

> if the matching items have empty bounding boxes (i.e. they have nothing to 
> display) then an empty string is returned

--
nosy: +epaine

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43466] ssl/hashlib: Add configure option to set or auto-detect rpath to OpenSSL libs

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> AFAIK this won't play will with static linking.

It should not be any problem as long as we expose the SSL symbols in the 
dynamic table of the extension (this happens by default). The only nuisance 
would be that users will still need those shared objects around so they can be 
dlopened by the extension.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43474] http.server.BaseHTTPRequestHandler end_header() fails

[grumblor]

grumblor  added the comment:

perhaps simply returning the method if _headers_buffer isn't defined is better 
since there is probably nothing to flush.

def end_headers(self):
if not hasattr(self, '_headers_buffer'):
return 1 
"""Send the blank line ending the MIME headers."""
if self.request_version != 'HTTP/0.9':
self._headers_buffer.append(b"\r\n")
self.flush_headers()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43468] functools.cached_property locking is plain wrong.

[Antti Haapala]

Antti Haapala  added the comment:

Django was going to replace their cached_property by the standard library one 
https://code.djangoproject.com/ticket/30949

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43474] http.server.BaseHTTPRequestHandler end_header() fails

[grumblor]

New submission from grumblor :

Python Version 3.8
http.server version 0.6
This is current install in new xubuntu 20.04 LTS, no idea if this is fixed in 
other version but appears to be present on github 
https://github.com/python/cpython/blob/3.9/Lib/http/server.py
at line 525

http.server.BaseHTTPRequestHandler end_headers() can reference _header_buffer 
array before it is assigned.

Should this be updated to something like the following? This fixes the problem 
of end_headers() failing for me:


def end_headers(self):
if not hasattr(self, '_headers_buffer'):
self._headers_buffer = []
"""Send the blank line ending the MIME headers."""
if self.request_version != 'HTTP/0.9':
self._headers_buffer.append(b"\r\n")
self.flush_headers()


This is my first issue, apologies for any mistakes I might have made.

--
components: Library (Lib)
messages: 388498
nosy: grumblor
priority: normal
severity: normal
status: open
title: http.server.BaseHTTPRequestHandler  end_header() fails
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue5857] Return namedtuples from tokenize token generator

[Sergei Lebedev]

Sergei Lebedev  added the comment:

> I strongly prefer that there not be inner named tuples. 

Hi Raymond, do you still strongly prefer (row, col) to remain unnamed? If so, 
could you comment on what makes you prefer that apart from (row, col) being 
more common than (col, row)? 

Are there any performance implications/runtime costs associated with making it 
(row, col) a namedtuple?

--
nosy: +superbobry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43441] [Subinterpreters]: global variable next_version_tag cause method cache bug

[junyixie]

junyixie  added the comment:

under building Python with --with-experimental-isolated-subinterpreters

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43441] [Subinterpreters]: global variable next_version_tag cause method cache bug

[junyixie]

junyixie  added the comment:

This is a simple fix.
https://github.com/python/cpython/pull/24822/commits/e61ce1dd28a48534ee497aaacb4439193bedfd42

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43441] [Subinterpreters]: global variable next_version_tag cause method cache bug

[junyixie]

Change by junyixie :


--
keywords: +patch
pull_requests: +23588
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24822

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43441] [Subinterpreters]: global variable next_version_tag cause method cache bug

[junyixie]

Change by junyixie :


--
components: +Subinterpreters -Interpreter Core
title: mutilcorevm: global variable next_version_tag cause method cache bug -> 
[Subinterpreters]: global variable next_version_tag cause method cache bug

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43473] Junks in difflib

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43334] venv does not install libpython

[Hasan]

Hasan  added the comment:

Can you please provide more information about this behavior?

--
nosy: +AliyevH

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40521] [subinterpreters] Make free lists and unicode caches per-interpreter

[junyixie]

junyixie  added the comment:

https://github.com/python/cpython/pull/24821/commits/9d7681dbd273b5025fd9b19d1be0a1f978a0b12e

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40521] [subinterpreters] Make free lists and unicode caches per-interpreter

[junyixie]

junyixie  added the comment:

Should Make dtoa bigint free list per-interpreter.

static Bigint *bigint_freelist[Kmax+1]; -> _is { Bigint 
*bigint_freelist[Kmax+1]; }

--
message_count: 43.0 -> 44.0
nosy: +JunyiXie
nosy_count: 5.0 -> 6.0
pull_requests: +23587
pull_request: https://github.com/python/cpython/pull/24821

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43473] Junks in difflib

[Hubert Bonnisseur-De-La-Bathe]

New submission from Hubert Bonnisseur-De-La-Bathe :

Reading first at the documentation of difflib, I thought that the use of junks 
would have produced the result 

s = SequenceMatcher(lambda x : x == " ", "abcd efgh", "abcdefgh")
s.get_matching_blocks()
>>> [Match(a=0, b=0, size=8)]

At a second lecture, it is clear that such evaluation will return in fact two 
matches of length 4.

Would it be nicer to have get_matching_block return the length 8 match ? 

Don't know if it's in the spirit of the lib, I'm just asking.

--
assignee: docs@python
components: Documentation
messages: 388491
nosy: docs@python, hubertbdlb
priority: normal
severity: normal
status: open
title: Junks in difflib
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43438] [doc] sys.addaudithook() documentation should be more explicit on its limitations

[Christian Heimes]

Christian Heimes  added the comment:

Python's dynamic nature makes it hard to implement and reason about audit hooks 
written in Python. sys.addaudithook() is really only design for testing, 
debugging, and playing around with auditing. You absolutely have to write a 
custom interpreter if you want to take auditing serious.

Please also keep in mind that sys.addaudithook() does **not** add a global 
hook. The function adds a per-interpreter hook. It just looks global to most 
people because a process typically has just one interpreter. I have filed 
bpo-43472 to track the issue.

$ cat auditsub.py 
import sys
import _xxsubinterpreters

def hook(*args):
print(args)

sys.addaudithook(hook)

import os
os.system('echo main interpreter')

sub = _xxsubinterpreters.create()
_xxsubinterpreters.run_string(sub, "import os; os.system('echo you got 
pwned')", None)

$ ./python auditsub.py 
('os.system', (b'echo main interpreter',))
main interpreter
you got pwned

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43472] [security][subinterpreters] Add auditing hooks to subinterpreter module

[Christian Heimes]

New submission from Christian Heimes :

The subinterpreters module does not emit any audit events yet. It's possible to 
create a subinterpreter and run arbitrary code through run_string().

We should also improve documentation of sys.addaudithook() and explain what 
'current interpreter' actually means. I guess most users don't realize the 
consequences for subinterpreters.

$ ./python auditsub.py
('os.system', (b'echo main interpreter',))
main interpreter
you got pwned
[heimes@seneca cpython]$ cat au
auditsub.py autom4te.cache/ 
[heimes@seneca cpython]$ cat auditsub.py 
import sys
import _xxsubinterpreters

def hook(*args):
print(args)

sys.addaudithook(hook)

import os
os.system('echo main interpreter')

sub = _xxsubinterpreters.create()
_xxsubinterpreters.run_string(sub, "import os; os.system('echo you got 
pwned')", None)

$ ./python auditsub.py 
('os.system', (b'echo main interpreter',))
main interpreter
you got pwned

--
components: Interpreter Core, Subinterpreters
messages: 388489
nosy: christian.heimes, eric.snow, steve.dower
priority: normal
severity: normal
status: open
title: [security][subinterpreters] Add auditing hooks to subinterpreter module
type: security
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42991] support for splitting multichannel audio fragments in audioop module

[Ramón Fraterman]

Ramón Fraterman  added the comment:

Could someone please have a look at my PR?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43470] Installation of Python 3.6.13 fails on MacOS Big Sur 11.2.3

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
components: +macOS
nosy: +ned.deily, ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43466] ssl/hashlib: Add configure option to set or auto-detect rpath to OpenSSL libs

[Christian Heimes]

Christian Heimes  added the comment:

It's very much the same for OpenSSL 3.0.0: libssl.so and libcrypto.so.

$ ldd build/lib.linux-x86_64-3.10/_ssl.cpython-310-x86_64-linux-gnu.so 
linux-vdso.so.1 (0x7a3cc000)
libssl.so.3 => 
/home/heimes/dev/python/multissl/openssl/3.0.0-alpha12/lib/libssl.so.3 
(0x7f1ab0b66000)
libcrypto.so.3 => 
/home/heimes/dev/python/multissl/openssl/3.0.0-alpha12/lib/libcrypto.so.3 
(0x7f1ab06b1000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x7f1ab065f000)
libc.so.6 => /lib64/libc.so.6 (0x7f1ab0494000)
libdl.so.2 => /lib64/libdl.so.2 (0x7f1ab048d000)
/lib64/ld-linux-x86-64.so.2 (0x7f1ab0c55000)


The external engines and OSSL providers are external plugins. They are very 
much akin to Python's extension modules. OpenSSL loads them with dlopen(), 
dlsym()s an init function and finally calls the init function. It uses either 
RTLD_NOW or RTLD_NOW | RTLD_GLOBAL dlopen() flags.

The engines and OSSL providers depend on libcrypto.so. AFAIK this won't play 
will with static linking.

$ ldd ../multissl/openssl/3.0.0-alpha12/lib/engines-3/afalg.so 
linux-vdso.so.1 (0x7fffa417d000)
libcrypto.so.3 => 
/home/heimes/dev/python/multissl/openssl/3.0.0-alpha12/lib/libcrypto.so.3 
(0x7fbcb3c75000)
libdl.so.2 => /lib64/libdl.so.2 (0x7fbcb3c3e000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x7fbcb3c1c000)
libc.so.6 => /lib64/libc.so.6 (0x7fbcb3a51000)
/lib64/ld-linux-x86-64.so.2 (0x7fbcb4133000)

$ ldd ../multissl/openssl/3.0.0-alpha12/lib/ossl-modules/legacy.so 
linux-vdso.so.1 (0x7ffd3ccc)
libcrypto.so.3 => 
/home/heimes/dev/python/multissl/openssl/3.0.0-alpha12/lib/libcrypto.so.3 
(0x7f5524f36000)
libdl.so.2 => /lib64/libdl.so.2 (0x7f5524eff000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x7f5524edd000)
libc.so.6 => /lib64/libc.so.6 (0x7f5524d12000)
/lib64/ld-linux-x86-64.so.2 (0x7f5525419000)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42967] [CVE-2021-23336] urllib.parse.parse_qsl(): Web cache poisoning - `; ` as a query args separator

[Petr Viktorin]

Petr Viktorin  added the comment:

There's another part of the new implementation that looks a bit fishy: the 
`separator` argument now allows multi-character strings, so you can parse 
'a=1b=2' with separator=''.
Was this intentional?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43452] Microoptimize PyType_Lookup for cache hits

[Inada Naoki]

Inada Naoki  added the comment:

$ ./python -m pyperf compare_to -G --min-speed=3 master.json pytype.json
Slower (1):
- unpack_sequence: 62.2 ns +- 0.6 ns -> 66.1 ns +- 0.9 ns: 1.06x slower

Faster (29):
- nbody: 182 ms +- 1 ms -> 152 ms +- 2 ms: 1.19x faster
- regex_effbot: 4.00 ms +- 0.05 ms -> 3.45 ms +- 0.08 ms: 1.16x faster
- spectral_norm: 178 ms +- 1 ms -> 158 ms +- 2 ms: 1.13x faster
- crypto_pyaes: 140 ms +- 1 ms -> 125 ms +- 1 ms: 1.12x faster
- dulwich_log: 88.0 ms +- 11.8 ms -> 80.1 ms +- 0.5 ms: 1.10x faster
- scimark_fft: 522 ms +- 4 ms -> 477 ms +- 14 ms: 1.09x faster
- mako: 18.2 ms +- 0.1 ms -> 16.7 ms +- 0.1 ms: 1.09x faster
- raytrace: 542 ms +- 4 ms -> 497 ms +- 3 ms: 1.09x faster
- regex_dna: 251 ms +- 1 ms -> 230 ms +- 1 ms: 1.09x faster
- regex_v8: 29.6 ms +- 0.2 ms -> 27.2 ms +- 0.1 ms: 1.09x faster
- pidigits: 227 ms +- 0 ms -> 210 ms +- 0 ms: 1.08x faster
- fannkuch: 564 ms +- 7 ms -> 525 ms +- 4 ms: 1.08x faster
- telco: 7.70 ms +- 0.12 ms -> 7.17 ms +- 0.18 ms: 1.07x faster
- float: 130 ms +- 1 ms -> 122 ms +- 1 ms: 1.07x faster
- unpickle_pure_python: 350 us +- 4 us -> 327 us +- 3 us: 1.07x faster
- scimark_sparse_mat_mult: 6.86 ms +- 0.08 ms -> 6.47 ms +- 0.07 ms: 1.06x 
faster
- chaos: 122 ms +- 1 ms -> 115 ms +- 1 ms: 1.06x faster
- nqueens: 113 ms +- 1 ms -> 107 ms +- 1 ms: 1.06x faster
- json_dumps: 15.4 ms +- 0.1 ms -> 14.7 ms +- 0.1 ms: 1.05x faster
- pickle_pure_python: 505 us +- 6 us -> 480 us +- 5 us: 1.05x faster
- logging_simple: 9.88 us +- 0.17 us -> 9.40 us +- 0.18 us: 1.05x faster
- deltablue: 8.19 ms +- 0.13 ms -> 7.79 ms +- 0.15 ms: 1.05x faster
- scimark_lu: 190 ms +- 3 ms -> 182 ms +- 5 ms: 1.05x faster
- scimark_monte_carlo: 125 ms +- 2 ms -> 120 ms +- 2 ms: 1.05x faster
- pickle: 11.3 us +- 0.2 us -> 10.8 us +- 0.1 us: 1.05x faster
- logging_silent: 189 ns +- 4 ns -> 182 ns +- 4 ns: 1.04x faster
- pickle_dict: 31.7 us +- 0.2 us -> 30.5 us +- 0.1 us: 1.04x faster
- django_template: 51.6 ms +- 0.7 ms -> 49.8 ms +- 0.6 ms: 1.04x faster
- pyflate: 755 ms +- 7 ms -> 730 ms +- 7 ms: 1.03x faster

Benchmark hidden because not significant (30): 2to3, chameleon, genshi_text, 
genshi_xml, go, hexiom, json_loads, logging_format, meteor_contest, pathlib, 
pickle_list, python_startup, python_startup_no_site, regex_compile, richards, 
scimark_sor, sqlalchemy_declarative, sqlalchemy_imperative, sqlite_synth, 
sympy_expand, sympy_integrate, sympy_sum, sympy_str, tornado_http, unpickle, 
unpickle_list, xml_etree_parse, xml_etree_iterparse, xml_etree_generate, 
xml_etree_process

Geometric mean: 1.04x faster

--
nosy: +methane

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com