Antoine Pitrou added the comment:
Isn't pyport.h a public header? IMHO you should put in a private header and
make it an inline function there.
Here is for example what we do in Arrow:
https://github.com/apache/arrow/blob/master/cpp/src/arrow/util/bit_util.h#L48
--
nosy: +pitrou
Antoine Pitrou added the comment:
Steve is right.
--
___
Python tracker
<https://bugs.python.org/issue40255>
___
___
Python-bugs-list mailing list
Unsubscribe:
Antoine Pitrou added the comment:
I'll note that this "extremely common" model can break as soon as you have
hidden worker threads somewhere (this can happen in a third-party library).
For example, the libhdfs library is not
Change by Antoine Pitrou :
--
keywords: +patch
pull_requests: +18887
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/19539
___
Python tracker
<https://bugs.python.org/issu
Change by Antoine Pitrou :
--
nosy: +mark.dickinson
___
Python tracker
<https://bugs.python.org/issue40286>
___
___
Python-bugs-list mailing list
Unsubscribe:
Antoine Pitrou added the comment:
I agree I don't *need* it per se. However, I suspect that for non-exports it
would be easier than
`getrandbits(nbytes * 8).to_bytes(nbytes, 'endian')`.
As for `secrets.token_bytes()`, it's not really adequate for regular
pseudo-random data generation when
Change by Antoine Pitrou :
--
components: +Library (Lib)
___
Python tracker
<https://bugs.python.org/issue40282>
___
___
Python-bugs-list mailing list
Unsub
Change by Antoine Pitrou :
--
stage: -> needs patch
___
Python tracker
<https://bugs.python.org/issue40282>
___
___
Python-bugs-list mailing list
Unsubscrib
Antoine Pitrou added the comment:
About a hypothetical getrandbytes(), probably 90% of my uses of getrandbits()
have been to generate random bytestrings.
--
___
Python tracker
<https://bugs.python.org/issue40
Antoine Pitrou added the comment:
I think there's a misunderstanding: "wait_at_exit" will make the *executor*
forget about the threads, but Python itself still knows about them, and it
waits for them to end at interpreter shutdown.
These threads were daemon threads in 3.8, so
Antoine Pitrou added the comment:
I don't think there's much ThreadPoolExecutor can do. If you drop the
references to the threads, they still exist and they still be waited upon at
interpreter exit.
The solution is for you to avoid having hanging threads. In the particular
case of TCP
Change by Antoine Pitrou :
--
nosy: +aeros
___
Python tracker
<https://bugs.python.org/issue36780>
___
___
Python-bugs-list mailing list
Unsubscribe:
Antoine Pitrou added the comment:
I think you know the answer to your question ;-)
--
___
Python tracker
<https://bugs.python.org/issue40282>
___
___
Python-bug
New submission from Antoine Pitrou :
When creating variable-sized random binary strings with random.getrandbits(),
you currently have to special case when the number of bytes is 0, because
otherwise getrandbits() raises:
ValueError: number of bits must be greater than zero
It seems like
Antoine Pitrou added the comment:
As a separate discussion, I would be interested to know whether the original
use case (Eddie's) could be satisfied differently. It probably doesn't belong
to this issue, though.
(Eddie, if you want to discuss this, feel free to e-mail me privately. I think
Antoine Pitrou added the comment:
> I run the pyperformance test suite with PGO + LTO + full cpu isolation in the
> speed.python.org machine and these were the results:
Be mindful that synthetic benchmarks are probably a gentle case for branch
prediction. Real-world perfo
New submission from Antoine Pitrou :
"file" scheme URIs are normalized:
https://tools.ietf.org/html/rfc8089
Therefore it would be possible to provide a PurePath method that returns a
"file" scheme URI corresponding to the given path.
--
components: Library (Lib)
m
Change by Antoine Pitrou :
--
nosy: +barneygale
___
Python tracker
<https://bugs.python.org/issue40281>
___
___
Python-bugs-list mailing list
Unsubscribe:
Change by Antoine Pitrou :
--
versions: +Python 3.9 -Python 3.8
___
Python tracker
<https://bugs.python.org/issue39682>
___
___
Python-bugs-list mailin
Antoine Pitrou added the comment:
New changeset 2e6d8b0ccdb6e0d9e98a9a7f9c9edfdf1311 by Barney Gale in branch
'master':
bpo-39682: make `pathlib.Path` immutable by removing (undocumented) support for
"closing" a path by using it as a context manager (GH-18846)
https://github.
Change by Antoine Pitrou :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
type: -> behavior
___
Python tracker
<https://bugs.python
Change by Antoine Pitrou :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Antoine Pitrou added the comment:
New changeset b61b818d916942aad1f8f3e33181801c4a1ed14b by Kyle Stanley in
branch 'master':
bpo-39812: Remove daemon threads in concurrent.futures (GH-19149)
https://github.com/python/cpython/commit/b61b818d916942aad1f8f3e33181801c4a1ed14b
Change by Antoine Pitrou :
--
nosy: +pitrou
___
Python tracker
<https://bugs.python.org/issue39850>
___
___
Python-bugs-list mailing list
Unsubscribe:
Change by Antoine Pitrou :
--
stage: -> needs patch
type: crash -> behavior
versions: +Python 3.7, Python 3.8, Python 3.9 -Python 3.6
___
Python tracker
<https://bugs.python.org/i
Antoine Pitrou added the comment:
Note you could simply remove the "closed" flag and the context manager live.
That way, you don't have to deprecate anything.
--
___
Python tracker
<https://bugs.python.o
Antoine Pitrou added the comment:
As with the Accessor abstraction, the original idea was to support Path objects
backed by a directory file descriptor (for use with openat() and friends).
That idea was abandoned but it looks like the context manager stayed. It's
certainly not useful
Antoine Pitrou added the comment:
Yes, I don't think other weakref-supporting objects traverse the weakreflist in
their tp_traverse.
--
nosy: +pitrou
___
Python tracker
<https://bugs.python.org/issue39
New submission from Antoine Pitrou :
On Windows, Python 3.7.6 and 3.8.1:
```
>>> import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.pyd'
>>> from distutils import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.cp38-win_amd64.pyd'
```
Th
Antoine Pitrou added the comment:
Ok, I opened issue39812
--
___
Python tracker
<https://bugs.python.org/issue37266>
___
___
Python-bugs-list mailing list
Unsub
Antoine Pitrou added the comment:
Unfortunately, I didn't receive the notification :-(
--
___
Python tracker
<https://bugs.python.org/issue15012>
___
___
Pytho
Antoine Pitrou added the comment:
New changeset 0e89076247580ba0e570c4816f0e5628a7e36e83 by Thomas Moreau in
branch 'master':
bpo-39678: refactor queue manager thread (GH-18551)
https://github.com/python/cpython/commit/0e89076247580ba0e570c4816f0e5628a7e36e83
Change by Antoine Pitrou :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
type: -> enhancement
___
Python tracker
<https://bugs.python
Antoine Pitrou added the comment:
Could someone please post a dummy reply to this issue? I'm hoping to fix e-mail
notifications for me.
--
___
Python tracker
<https://bugs.python.org/issue15
Change by Antoine Pitrou :
--
nosy: +pitrou
___
Python tracker
<https://bugs.python.org/issue15012>
___
___
Python-bugs-list mailing list
Unsubscribe:
New submission from Antoine Pitrou :
Since issue37266 (which forbid daemon threads in subinterpreters), we probably
want to forego daemon threads in concurrent.futures. This means we also need a
way to run an atexit-like hook before non-daemon threads are joined on
(sub)interpreter shutdown
Antoine Pitrou added the comment:
@Elad:
> I'm not convinced that a multi-threaded fork()+exec() from C would be any
> better, unless the Python code goes to great lengths to avoid any
> non-async-signal-safe operations between the fork() and the exec().
That's exactly what t
Change by Antoine Pitrou :
--
priority: normal -> critical
versions: +Python 3.9
___
Python tracker
<https://bugs.python.org/issue39776>
___
___
Python-
Antoine Pitrou added the comment:
> To me, this seems best implemented as a new public function for the threading
> module, perhaps something like `threading.register_atexit()`. Would this be
> reasonable?
To me, yes.
--
___
Pytho
Antoine Pitrou added the comment:
Also cc'ing Kyle for the concurrent.futures issue.
--
nosy: +aeros
___
Python tracker
<https://bugs.python.org/issue37
Antoine Pitrou added the comment:
Perhaps the solution would be to have an additional kind of atexit hooks, which
get executed before threads are joined.
--
nosy: +ncoghlan
___
Python tracker
<https://bugs.python.org/issue37
Antoine Pitrou added the comment:
There will be a problem with `concurrent.futures.ProcessPoolExecutor`, which
currently launches its management thread as a daemon thread. The daemon thread
itself is not problematic, because ProcessPoolExecutor uses an atexit hook to
shutdown itself
Change by Antoine Pitrou :
--
nosy: +dstufft, eric.araujo, p-ganssle, vstinner
___
Python tracker
<https://bugs.python.org/issue39763>
___
___
Python-bugs-list m
Change by Antoine Pitrou :
--
title: Hang after fork due to logging trying to reacquire the module lock in an
atfork() handler -> distutils.spawn should use subprocess (hang in parallel
builds on QNX)
___
Python tracker
<https://bugs.pyth
Antoine Pitrou added the comment:
Note the original problem stems when you using "setup.py -j ".
The attached reproducer is a bit of a red herring, because indeed doing fork()
+ exec() in pure Python is too fragile. That is why the standard subprocess
does it in C. The pr
Antoine Pitrou added the comment:
The regression is fixed in git master and in the 3.8 branch. However, I would
still advice against the idiom of handling a mmap object as a regular file when
using pickle.
--
resolution: -> fixed
stage: patch review -> resolved
status
New submission from Antoine Pitrou :
In issue39681 we reestablished the fallback to read() when a file-like object
doesn't provide readinto() in _pickle.c. However, doing so leads to lower
performance and all file-like object should nowadays provide readinto() (simply
by deriving from
New submission from Antoine Pitrou :
In the doc for `-X` options (https://docs.python.org/3/using/cmdline.html#id5),
when an option can be triggered through an equivalent environment variable,
that variable is mentioned.
An exception to that is `-X dev`, which can also be triggered
Change by Antoine Pitrou :
--
assignee: -> pitrou
type: -> behavior
___
Python tracker
<https://bugs.python.org/issue39681>
___
___
Python-bugs-list
Change by Antoine Pitrou :
--
keywords: +patch
pull_requests: +17960
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/18592
___
Python tracker
<https://bugs.python.org/issu
Antoine Pitrou added the comment:
Well, in the mmap case, this is trivially fixed by writing:
```
with open("my_data.pkl", "r+b") as f_in:
mm = mmap.mmap(f_in.fileno(), 0)
print(pickle.loads(memoryview(mm)))
```
It will also be faster this way, since the pickle will b
Antoine Pitrou added the comment:
Sorry for the delay. I've now merged the PR for 3.9. Since this is a slight
behaviour change, and the original issue was easy to workaround, I won't
backport it.
--
resolution: -> fixed
stage: patch review -> resolved
status: open -&g
Antoine Pitrou added the comment:
New changeset c2ac4cf040ea950bf552d1e77bea613a1a5474fe by Christopher Hunt in
branch 'master':
bpo-35727: Use exit code 0 on sys.exit() in multiprocessing.Process. (GH-11538)
https://github.com/python/cpython/commit/c2ac4cf040ea950bf552d1e77bea613a1a5474fe
Antoine Pitrou added the comment:
Right, I think at some point during the PEP 574 implementation review, we
decided that the input object had to be a proper file implementation
(for example, deriving from io.BufferedIOBase).
I think it should be easy for PyTorch to fix this internally
Antoine Pitrou added the comment:
Thank you Thomas. Ii'd rather not backport delicate changes, so closing now.
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
type: -> behavior
versions: -Python 3.7
Change by Antoine Pitrou :
--
nosy: -pitrou
___
Python tracker
<https://bugs.python.org/issue1294959>
___
___
Python-bugs-list mailing list
Unsubscribe:
Antoine Pitrou added the comment:
New changeset 17236873392533ce0c5d7bbf52cbd61bca171c59 by Antoine Pitrou in
branch '3.8':
[3.8] bpo-39492: Fix a reference cycle between reducer_override and a Pickler
instance (GH-18266) (#18316)
https://github.com/python/cpython/commit
Antoine Pitrou added the comment:
Thank you for the PR!
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Change by Antoine Pitrou :
--
pull_requests: +17692
pull_request: https://github.com/python/cpython/pull/18316
___
Python tracker
<https://bugs.python.org/issue39
Antoine Pitrou added the comment:
New changeset 339fd46cb764277cbbdc3e78dcc5b45b156bb6ae by Kyle Stanley in
branch 'master':
bpo-39349: Add *cancel_futures* to Executor.shutdown() (GH-18057)
https://github.com/python/cpython/commit/339fd46cb764277cbbdc3e78dcc5b45b156bb6ae
Antoine Pitrou added the comment:
Thank you very muck, Kyle!
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Antoine Pitrou added the comment:
Reopening because it seems my reading of the doc was wrong (took a backslash
for a regular slash :-o).
--
resolution: not a bug ->
stage: resolved ->
status: closed -> open
___
Python tracke
Antoine Pitrou added the comment:
As mentioned on the attached PR, the first argument is positional, so it
doesn't matter that the name in the docs is not the same as the name in the
code.
The name "bytes_object" makes it clear which type of object is accepted, which
makes it a
Change by Antoine Pitrou :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Antoine Pitrou added the comment:
Good catch. Do you want to submit a PR?
--
___
Python tracker
<https://bugs.python.org/issue39426>
___
___
Python-bugs-list m
Antoine Pitrou added the comment:
It would certainly be better to start the worker processes on demand. It
probably also requires careful thought about how to detect that more workers
are required.
--
___
Python tracker
<https://bugs.python.
Antoine Pitrou added the comment:
Closing as won't fix. If you feel strongly about this, I would suggest to
bring the discussion on python-dev:
https://mail.python.org/mailman3/lists/python-dev.python.org/
--
resolution: -> wont fix
stage: -> resolved
status: open -&g
Antoine Pitrou added the comment:
Ahah. Nice catch!
Well, it's a pity this got overlooked when we added Path.link_to(). But I'm
afraid it's late to change it now, since this has been released, and changing
the argument order would break existing code in potentially dangerous ways.
Note
Antoine Pitrou added the comment:
I don't think so. I'll let other confirm.
--
nosy: +christian.heimes, vstinner
___
Python tracker
<https://bugs.python.org/issue39
Change by Antoine Pitrou :
--
nosy: +benjamin.peterson
___
Python tracker
<https://bugs.python.org/issue39193>
___
___
Python-bugs-list mailing list
Unsubscribe:
Antoine Pitrou added the comment:
It seems what you're saying is that one can crash Python by crafting
deliberately incorrect bytecode. If so, then I'm afraid it's something we
won't consider fixing.
--
nosy: +pitrou
___
Python tracker
<ht
Antoine Pitrou added the comment:
Jendrik, thank you for fixing this!
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Antoine Pitrou added the comment:
New changeset 5b9077134cd0535f21905d5f5195847526cac99c by Antoine Pitrou
(Jendrik Seipp) in branch 'master':
bpo-13601: always use line-buffering for sys.stderr (GH-17646)
https://github.com/python/cpython/commit/5b9077134cd0535f21905d5f5195847526cac99c
Antoine Pitrou added the comment:
I'm rejecting this, as explained on the Github PR.
--
nosy: +pitrou
resolution: -> not a bug
stage: -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Change by Antoine Pitrou :
--
nosy: +inada.naoki
___
Python tracker
<https://bugs.python.org/issue36051>
___
___
Python-bugs-list mailing list
Unsubscribe:
Antoine Pitrou added the comment:
If there is a large chunk (e.g. several MBs), dropping the GIL during the
memcpy of that chunk may be beneficial. This kind of optimization may be
applicable in other similar cases (such as extending a bytearray or a BytesIO
object).
--
nosy
Antoine Pitrou added the comment:
> 1. Python daemon threads
I think the answer is to document a bit more clearly that they can pose all
kinds of problems. Perhaps we could even display a visible warning when people
create daemon threads.
> 2. Python threads created in atexit handle
Antoine Pitrou added the comment:
(previous message deleted, I hadn't noticed the "reuse_port" parameter)
My preference for create_datagram_endpoint() would be:
- make the "reuse_address" parameter a no-op, and raise an error when
"reuse_address=True" is pa
Change by Antoine Pitrou :
--
Removed message: https://bugs.python.org/msg357067
___
Python tracker
<https://bugs.python.org/issue37228>
___
___
Python-bug
Antoine Pitrou added the comment:
Here is another suggestion:
- make the "reuse_address" parameter a no-op, and raise an error when
"reuse_address=True" is passed
- add a new "allow_multiple_bind=False" parameter that enables SO_REUSEPORT
- do that in 3.8
Antoine Pitrou added the comment:
Thank you Jake for the report and PR!
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.9
___
Python tracker
<https://bugs.python.or
Change by Antoine Pitrou :
--
nosy: +pablogsal
___
Python tracker
<https://bugs.python.org/issue38744>
___
___
Python-bugs-list mailing list
Unsubscribe:
Antoine Pitrou added the comment:
What happens if you instead write:
with Pool() as pool:
pool.map(sleep, [0.01] * 10)
--
___
Python tracker
<https://bugs.python.org/issue38
Change by Antoine Pitrou :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Change by Antoine Pitrou :
--
nosy: -miss-islington
versions: +Python 3.7, Python 3.8
___
Python tracker
<https://bugs.python.org/issue38422>
___
___
Python-bug
Antoine Pitrou added the comment:
I rejected the accompanying PR and the changes it proposed, therefore it seems
reasonable to close this issue as well.
--
nosy: +pitrou
resolution: -> rejected
stage: patch review -> resolved
status: open -&g
Antoine Pitrou added the comment:
Also I'm curious: why are you using O_DIRECT, and furthermore, why are you
using it to read into mmap'ed memory?
--
___
Python tracker
<https://bugs.python.org/issue38
Antoine Pitrou added the comment:
I agree with Josh. If you want to use O_DIRECT, use an unbuffered file object
and be sure to issue reads of the right size.
--
nosy: +pitrou
___
Python tracker
<https://bugs.python.org/issue38
Antoine Pitrou added the comment:
I agree with Serhiy. If you hit a rare situation that you don't want to
handle, use Py_FatalError(), not Py_UNREACHABLE().
Py_UNREACHABLE() is for situations which are *logically* impossible, not just
uncommon.
--
nosy: +pitrou
Antoine Pitrou added the comment:
Closing as not a Python bug.
--
resolution: -> not a bug
stage: -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Antoine Pitrou added the comment:
Python just exposes thin wrappers around the underlying libc calls, so you have
to understand how those work.
On Linux, the sigwaitinfo() man page says:
NOTES
In normal usage, the calling program blocks the signals in
set via a prior call
Antoine Pitrou added the comment:
Several things here:
- you can perform critical cleanup with the atexit module; using a signal
handler for that is extremely low-level and error-prone
- you can also try to switch to the "forkserver" method of multiprocessing,
perhaps that wil
Antoine Pitrou added the comment:
> Would it make any sense to add an opt-in option to emit a warning when a new
> type is created with Py_TPFLAGS_HAVE_GC but it doesn't implement tp_traverse?
Or even fatal error out? There's no reason to have Py_TPFLAGS_HAVE_GC but not
imp
Antoine Pitrou added the comment:
> What do you think of the followup PR to make use of the SEEK_* constants
> listed in the documentation? I think it would be useful to at least mention
> them in the tutorial, or even make use of them directly in the examples.
Yes, I think it woul
Antoine Pitrou added the comment:
Thanks you Kyle!
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.7, Python 3.8
___
Python tracker
<https://bugs.python.or
Antoine Pitrou added the comment:
Thanks for contributing this, Paul!
--
nosy: +pitrou
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Antoine Pitrou added the comment:
According to POSIX, the only system calls that can fail with ENOTEMPTY are
rmdir(), rename() and unlinkat().
--
___
Python tracker
<https://bugs.python.org/issue32
Antoine Pitrou added the comment:
I don't seen any significant difference here (Ubuntu 18.04):
$ time python2 sp.py
10.3433089256
real0m10,362s
user0m6,565s
sys 0m4,372s
$ time python3 sp.py
11.746907234191895
real0m11,781s
user0m7,356s
sys 0m5,239s
Antoine Pitrou added the comment:
Can you post a simple script showcasing the performance degradation?
--
nosy: +pitrou
___
Python tracker
<https://bugs.python.org/issue37
Change by Antoine Pitrou :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
201 - 300 of 16564 matches
Mail list logo
