Change by Alexey Izbyshev :
--
keywords: +patch
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue47260>
___
___
Python-bugs-list mai
Alexey Izbyshev added the comment:
> It's been years now and that hasn't happened, even with more recent flag
> additions. I think it's safe to say it won't, and such a fallback upon error
> won't put us back into a bogus pre-close_range situation where we're
> needlessly close(
Change by Alexey Izbyshev :
--
keywords: +patch
pull_requests: +30443
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/32418
___
Python tracker
<https://bugs.python.org/issu
New submission from Alexey Izbyshev :
After #40422 _Py_closerange() assumes that close_range() closes all file
descriptors even if it returns an error (other than ENOSYS):
if (close_range(first, last, 0) == 0 || errno != ENOSYS) {
/* Any errors encountered while closing file
Alexey Izbyshev added the comment:
> 3. We have to fix error-path in order not to change heap state (contents and
> allocations), possibly do not touch locks. During vfork() child execution -
> the only parent THREAD (not the process) is blocked. For example, it's not
> allowed
Alexey Izbyshev added the comment:
The preceding comment is wrong, see discussion in #47245 and
https://bugzilla.kernel.org/show_bug.cgi?id=215813#c14 for explanation of why
that bug report is irrelevant for CPython.
--
___
Python tracker
<ht
Alexey Izbyshev added the comment:
> As for glibc specifics, I'm mostly thinking of the calls we do in the child.
> According to the "Standard Description (POSIX.1)" calls to anything other
> than `_exit()` or `exec*()` are not allowed. But the longer "Linux
> D
Alexey Izbyshev added the comment:
In short: both this bug report and [1] are invalid.
The reason why doing syscall(SYS_vfork) is illegal is explained by Florian
Weimer in [2]:
>The syscall function in glibc does not protect the on-stack return address
>against overwriting, so it
Alexey Izbyshev added the comment:
> As a concrete example, we have a (non-Python) build system and task runner
> that orchestrates many tasks to run in parallel. Some of those tasks end up
> invoking Python scripts that use subprocess.run() to run other programs. Our
>
New submission from Alexey Izbyshev :
In PC/getpathp.c CPython uses buffers with length MAXPATHLEN+1, which is 257 on
Windows[1]. On Windows 7, where PathCch* functions are not available, CPython
<= 3.8 fallbacks to PathCombineW()/PathCanonicalizeW()[2]. Those functions
ass
Alexey Izbyshev added the comment:
> If we had a dedicated maintainer who was supporting Win7 and making releases
> for it, then we (i.e. they) could support it. But then, there's nothing to
> stop someone doing that already, and even to stop them charging money for it
> if they
Alexey Izbyshev added the comment:
> If Win8-only calls are not used, then presumably it should still build and
> run on Windows 7, presumably with the flag flipped back to Win7. And if there
> are Win8-only calls used and the flag is set to Win7+, I assume that the MSVC
&
Alexey Izbyshev added the comment:
As far as I understand, commit [1] explicitly prevents CPython from running on
Windows 7, and it's included into 3.9. So it seems to be too late to complain,
despite that, according to Wikipedia, more than 15% of all Windows PCs are
still running Windows 7
Change by Alexey Izbyshev :
--
nosy: +gregory.p.smith
___
Python tracker
<https://bugs.python.org/issue43308>
___
___
Python-bugs-list mailing list
Unsubscribe:
Alexey Izbyshev added the comment:
I generally agree, but getting a good, short error message seems to be the hard
part here. I previously complained[1] about the following proposal by @hroncok:
FileNotFoundError: [Errno 2] No such file or directory: Either './demo' or the
interpreter
Alexey Izbyshev added the comment:
How do you propose to approach documentation of such behavior? The underlying
cause is the ambiguity of ENOENT error code from execve() returned by the
kernel, so it applies to all places where Python can call execve(), including
os.posixspawn(), os.execve
Alexey Izbyshev added the comment:
> IMO the fix is simple: only create OSError from the errno, never pass a
> filename.
This will remove a normally helpful piece of the error message in exchange to
being marginally less confusing in a rare case of non-existing interpreter (the
Alexey Izbyshev added the comment:
> FileNotFoundError: [Errno 2] No such file or directory: Either './demo' or
> the interpreter of './demo' not found.
This doesn't sound good to me because a very probable and a very improbable
reasons are combined together without any distinction. A
Alexey Izbyshev added the comment:
> Ideally, the error would say:
> FileNotFoundError: ./demo: /usr/bin/hugo: bad interpreter: No such file or
> directory
The kernel simply returns ENOENT on an attempt to execve() a file with
non-existing hash-bang interpreter. The sa
Alexey Izbyshev added the comment:
I would suggest to start digging from the following piece of code in
`maybe_pyc_file()` (Python/pythonrun.c):
int ispyc = 0;
if (ftell(fp) == 0) {
if (fread(buf, 1, 2, fp) == 2 &&
((unsigned int)buf[1]<
Alexey Izbyshev added the comment:
> I think truncation via TRUNCATE_EXISTING (O_TRUNC, with O_WRONLY or O_RDWR)
> or overwriting with CREATE_ALWAYS (O_CREAT | O_TRUNC) is at least tolerable
> because the caller doesn't care about the existing data.
Yes, I had a thought that
Alexey Izbyshev added the comment:
> FYI, here are the access rights applicable to files
Thanks, I checked that mapping in headers when I was writing
_Py_wopen_noraise() as well. But I've found a catch via ProcessHacker:
CreateFile() with GENERIC_WRITE (or FILE_GENERIC_WRITE) additiona
Alexey Izbyshev added the comment:
> I don't know what you mean by default access rights.
I meant the access rights of the handle created by _wopen(). In my PR I
basically assume that _wopen() uses GENERIC_READ/GENERIC_WRITE access rights,
but _wopen() doesn't have a contractual obligat
Change by Alexey Izbyshev :
--
nosy: +izbyshev
___
Python tracker
<https://bugs.python.org/issue42969>
___
___
Python-bugs-list mailing list
Unsubscribe:
Alexey Izbyshev added the comment:
Thank you for testing. I've added a NEWS entry to the PR, so it's ready for
review by the core devs.
Note that PyThread_exit_thread() can still be called by daemon threads if they
try to take the GIL after Py_Finalize(), and also via C APIs like
Alexey Izbyshev added the comment:
> It's possible to query the granted access of a kernel handle via
> NtQueryObject: ObjectBasicInformation
Ah, thanks for the info. But it wouldn't help for option (1) that I had in mind
because open() and os.open() currently set only msvcrt-level O_
Alexey Izbyshev added the comment:
Could anybody provide their thoughts on this RFE? Thanks.
--
___
Python tracker
<https://bugs.python.org/issue42606>
___
___
Alexey Izbyshev added the comment:
I've made a PR to remove most calls to pthread_exit().
@xxm: could you test it in your environment?
--
___
Python tracker
<https://bugs.python.org/issue42
Change by Alexey Izbyshev :
--
keywords: +patch
pull_requests: +23063
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/24241
___
Python tracker
<https://bugs.python.org/issu
Alexey Izbyshev added the comment:
I've encountered this issue too. My use case was a 32-bit Python on a 64-bit
CentOS system, and my understanding of the issue was that 64-bit libgcc_s is
somehow counted as a "provider" of libgcc_s for 32-bit libc by the package
manager, so 32-bi
Change by Alexey Izbyshev :
--
components: +Library (Lib)
nosy: +vstinner
versions: -Python 3.6, Python 3.7
___
Python tracker
<https://bugs.python.org/issue42
Change by Alexey Izbyshev :
--
nosy: +izbyshev
___
Python tracker
<https://bugs.python.org/issue42736>
___
___
Python-bugs-list mailing list
Unsubscribe:
Change by Alexey Izbyshev :
--
nosy: +izbyshev
___
Python tracker
<https://bugs.python.org/issue38435>
___
___
Python-bugs-list mailing list
Unsubscribe:
Alexey Izbyshev added the comment:
> Using close_fds=False, subprocess can use posix_spawn() which is safer and
> faster than fork+exec. For example, on Linux, the glibc implements it as a
> function using vfork which is faster than fork if the parent allocated a lot
> of memo
Alexey Izbyshev added the comment:
> I've been struggling to understand today why a simple file redirection
> couldn't work properly today (encoding issues)
The core issue is that "working properly" is not defined in general when we're
talking about piping/redirec
Alexey Izbyshev added the comment:
This bug would have been caught at compile time if `_Py_Gid_Converter()` used
`gid_t *` instead of `void *`. I couldn't find any call sites where `void *`
would be needed, so probably `_Py_Gid_Converter()` should be fixed too (in a
separate PR/issue
Change by Alexey Izbyshev :
--
keywords: +patch
pull_requests: +22575
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/23712
___
Python tracker
<https://bugs.python.org/issu
New submission from Alexey Izbyshev :
On POSIX-conforming systems, O_APPEND flag for open() must ensure that no
intervening file modification occurs between changing the file offset and the
write operation[1]. In effect, two processes that independently opened the same
file with O_APPEND
Alexey Izbyshev added the comment:
Great approach :)
--
___
Python tracker
<https://bugs.python.org/issue42569>
___
___
Python-bugs-list mailing list
Unsub
Alexey Izbyshev added the comment:
Yes, despite that MSVCRT knows the type of the file descriptor because it calls
GetFileType() on its creation, it doesn't check it in lseek() implementation
and simply calls SetFilePointer(), which spuriously succeeds for pipes. MSDN
says the following[1
Change by Alexey Izbyshev :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
versions: -Python 3.7
___
Python tracker
<https://bugs.python.or
Alexey Izbyshev added the comment:
Thanks for the fix and backports!
--
resolution: fixed ->
stage: resolved -> patch review
status: closed -> open
versions: +Python 3.7
___
Python tracker
<https://bugs.python.or
Change by Alexey Izbyshev :
--
components: +Library (Lib)
nosy: +davin, pitrou
versions: -Python 3.6
___
Python tracker
<https://bugs.python.org/issue42
Alexey Izbyshev added the comment:
> So it should be, "if they fail and you're in a context where exceptions are
> allowed, raise an exception" (which will chain back to the one raised from an
> audit hook".
What exception should be raised if _Py_fopen() fails (retur
Alexey Izbyshev added the comment:
> To implement PEP 446: create non-inheritable file descriptors.
Yes, I understand that was the original role. But currently there is no easy
way to deal with errors from the helpers because of exception vs. errno
conundrum. Maybe they should be sp
Alexey Izbyshev added the comment:
> It seems like PyErr_ProgramText() is no longer used in Python.
Isn't it a part of the public API? I can't find it in the docs, but it seems to
be declared in the public header.
--
___
Python tracker
<
Alexey Izbyshev added the comment:
Thanks for the patch, Victor, it looks good.
Just so it doesn't get lost: the problem with the contract of
PyErr_ProgramText() which I mentioned in my dup 42568 is still there.
--
___
Python tracker
<ht
Alexey Izbyshev added the comment:
Thanks, Eryk, for catching the dup, I missed it somehow.
@ZackerySpytz: do you plan to proceed with your PR? If not, I can pick it up --
this issue broke the software I develop after upgrade to 3.8.
I filed issue 42569 to hopefully clarify the status
New submission from Alexey Izbyshev :
Before addition of audit hooks in 3.8, _Py_fopen() and _Py_wfopen() were simple
wrappers around corresponding C runtime functions. They didn't require GIL,
reported errors via errno and could be safely called during early interpreter
initialization
New submission from Alexey Izbyshev :
> python тест.pyc
python: Can't reopen .pyc file
The issue is caused by _Py_fopen() being used as though it can deal with paths
encoded in FS-default encoding (UTF-8 by default on Windows), but in fact it's
just a simple wrapper around fopen() from th
Change by Alexey Izbyshev :
--
nosy: +rhettinger
versions: -Python 3.6, Python 3.7
___
Python tracker
<https://bugs.python.org/issue42457>
___
___
Python-bug
Alexey Izbyshev added the comment:
> (probably can't even limit that to the case when `text` is used, since it was
> added in 3.7)
Well, actually, we can, since we probably don't need to preserve compatibility
with the AttributeError currently caused by `text=True` with `inpu
Alexey Izbyshev added the comment:
It seems that allowing `input=None` to mean "redirect stdin to a pipe and send
an empty string there" in `subprocess.check_output` was an accident(?), and
this behavior is inconsistent with `subprocess.run` and `communicate`, where
`input=None` ha
Alexey Izbyshev added the comment:
By the way, I don't see a direct relation between `test.py` (which doesn't use
`subprocess` directly) and your comment describing `subprocess` usage with
threads. So if you think that the bug in `test.py` is unrelated to the problem
you face, feel free
Alexey Izbyshev added the comment:
(Restored test.py attachment)
The issue happens due to an incorrect usage of `multiprocessing.Pool`.
```
# Set up multiprocessing pool, initialising logging in each subprocess
with multiprocessing.Pool(initializer=process_setup,
initargs=(get_queue
Change by Alexey Izbyshev :
Added file: https://bugs.python.org/file49531/test.py
___
Python tracker
<https://bugs.python.org/issue42097>
___
___
Python-bugs-list mailin
Alexey Izbyshev added the comment:
Thanks for merging! I've rebased PR 22970.
--
___
Python tracker
<https://bugs.python.org/issue42146>
___
___
Python-bug
Change by Alexey Izbyshev :
--
type: behavior -> resource usage
___
Python tracker
<https://bugs.python.org/issue42146>
___
___
Python-bugs-list mai
Change by Alexey Izbyshev :
--
keywords: +patch
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue42146>
___
___
Python-bugs-list mai
Alexey Izbyshev added the comment:
I've submitted both PRs.
Regarding PR 22970:
* I made it a draft since we'd probably want to fix the leak first, but then it
will have to be rebased.
* It fixes a bug with _enable_gc(): if it failed after fork(), we'd raise
OSError instead. Additionally
Change by Alexey Izbyshev :
--
pull_requests: +21885
pull_request: https://github.com/python/cpython/pull/22970
___
Python tracker
<https://bugs.python.org/issue42
Change by Alexey Izbyshev :
--
keywords: +patch
pull_requests: +21882
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/22966
___
Python tracker
<https://bugs.python.org/issu
New submission from Alexey Izbyshev :
The following test demonstrates the leak:
```
import subprocess
cwd = 'x' * 10**6
for __ in range(100):
try:
subprocess.call(['/xxx'], cwd=cwd, user=2**64)
except OverflowError:
pass
from resource import *
print(getrusage
Alexey Izbyshev added the comment:
@ronaldoussoren
> I'd prefer to not use vfork on macOS. For one I don't particularly trust that
> vfork would work reliably when using higher level APIs, but more importantly
> posix_spawn on macOS has some options that are hard to achieve
Alexey Izbyshev added the comment:
> regarding excluding the setsid() case: I was being conservative as I couldn't
> find a reference of what was and wasn't allowed after vfork.
Yes, there is no list of functions allowed after vfork(), except for the
conservative POSIX.1 list cons
Alexey Izbyshev added the comment:
> Thank you for taking this on! I'm calling it fixed for now as the buildbots
> are looking happy with it. If issues with it arise we can address them.
Thank you for reviewing and merging!
Using POSIX_CALL for pthread_sigmask() is incorrect, h
Change by Alexey Izbyshev :
--
pull_requests: +21862
pull_request: https://github.com/python/cpython/pull/22944
___
Python tracker
<https://bugs.python.org/issue35
Change by Alexey Izbyshev :
--
resolution: -> not a bug
___
Python tracker
<https://bugs.python.org/issue36034>
___
___
Python-bugs-list mailing list
Un
Alexey Izbyshev added the comment:
I've updated my PR.
* After a discussion with Alexander Monakov (a GCC developer), moved vfork()
into a small function to isolate it from both subprocess_fork_exec() and
child_exec(). This appears to be the best strategy to avoid -Wclobbered false
Alexey Izbyshev added the comment:
Well, much later than promised, but I'm picking it up. Since in the meantime
support for setting uid/gid/groups was merged, and I'm aware about potential
issues with calling corresponding C library functions in a vfork()-child, I
asked a question on musl
Alexey Izbyshev added the comment:
I'd really like to get this merged eventually because vfork()-based solution is
fundamentally more generic than posix_spawn(). Apart from having no issue with
close_fds=True, it will also continue to allow subprocess to add any process
context tweaks
Change by Alexey Izbyshev :
--
pull_requests: +16379
pull_request: https://github.com/python/cpython/pull/5812
___
Python tracker
<https://bugs.python.org/issue34
Change by Alexey Izbyshev :
--
nosy: +izbyshev
___
Python tracker
<https://bugs.python.org/issue31904>
___
___
Python-bugs-list mailing list
Unsubscribe:
Alexey Izbyshev added the comment:
> 1) This is intentional, this is for dropping privileges before running some
> (possibly untrusted) command, we do not want to leave a path for the
> subprocess to gain root back. If there is a subprocess that needs root for
> some operatio
Alexey Izbyshev added the comment:
Patrick, could you provide more background that would explain your choice of
setreuid/setregid functions and the desired handling of supplementary groups?
I'm not a security expert, so I may not have sufficient expertise to judge on
that, but maybe my
Change by Alexey Izbyshev :
--
assignee: -> gregory.p.smith
___
Python tracker
<https://bugs.python.org/issue36046>
___
___
Python-bugs-list mailing list
Un
Alexey Izbyshev added the comment:
> Interesting. Because both errors/conditions are mapped to
> ERROR_INVALID_HANDLE we need the creation time. I can work on a patch for
> that.
I don't understand why any patch for CPython is needed at all. Using invalid
handles is a serious pr
Change by Alexey Izbyshev :
--
nosy: +izbyshev
___
Python tracker
<https://bugs.python.org/issue36046>
___
___
Python-bugs-list mailing list
Unsubscribe:
Alexey Izbyshev added the comment:
I don't know what you mean by "in-line" pre-processing output, but you can use
-E option to get the normal preprocessor output. Line directives will tell you
where those functions come from on a system where there is no compilation error.
Alexey Izbyshev added the comment:
Thank you for your introduction about _xxsubinterpreters, Eric.
This particular leak is easy: it's right in _channel_send(). I've submitted a
PR.
I've also done a quick scan of neighboring code, and it seems there are other
leaks as well, e.g
Change by Alexey Izbyshev :
--
keywords: +patch
pull_requests: +11876
stage: test needed -> patch review
___
Python tracker
<https://bugs.python.org/issu
Alexey Izbyshev added the comment:
I'll look into it later today. An obvious guess is that my test simply exposed
an existing leak because the exception code path wasn't tested before AFAIK,
but I need to check it.
--
assignee: -> izbys
Alexey Izbyshev added the comment:
"long long" is mandated to be at least 64-bit by C99 (5.2.4.2.1 Sizes of
integer types). If it were 32-bit, no warnings would have been issued.
--
___
Python tracker
<https://bugs.python.o
Change by Alexey Izbyshev :
--
keywords: +patch
pull_requests: +11852
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue35972>
___
_
New submission from Alexey Izbyshev :
Victor Stinner pointed out that on x86 Gentoo Installed with X 3.x buildbot,
there is a compiler warning:
Python/pystate.c:1483:18: warning: cast to pointer from integer of
different size [-Wint-to-pointer-cast]
(https://buildbot.python.org/all
Alexey Izbyshev added the comment:
I've been struggling with fixing spurious -Wclobbered GCC warnings. Originally,
I've got the following:
/scratch2/izbyshev/cpython/Modules/_posixsubprocess.c: In function
‘subprocess_fork_exec’:
/scratch2/izbyshev/cpython/Modules/_posixsubprocess.c:612:15
Alexey Izbyshev added the comment:
Thank you for the review and your thoughts, Gregory.
> With this in place we may want to make the _use_posix_spawn() logic in
> subprocess.py stricter? That could be its own followup PR.
Yes, I think we can always use vfork() on Linux unless we fin
Alexey Izbyshev added the comment:
I've checked subprocess.Popen() error reporting in QEMU user-mode and WSL and
confirm that it works both with my patch (vfork/exec) and the traditional
fork/exec, but doesn't work with glibc's posix_spawn.
The first command below uses posix_spawn
Alexey Izbyshev added the comment:
> W.r.t. closing all file descriptors > 2: posix_spawn_file_actions_addclose
> can do this when using posix_spawn. That would have a performance cost, you'd
> basically have to resort to closing all possible file descriptors and cannot
>
Alexey Izbyshev added the comment:
> Is sys.platform equal to 'linux' on WSL? Sorry, I don't know WSL. If it's
> equal, is it possible to explicitly exclude WSL in the subprocess test,
> _use_posix_spawn()?
I don't have immediate access to WSL right now, but I'll try t
Change by Alexey Izbyshev :
--
keywords: +patch, patch
pull_requests: +11484, 11485
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issu
Change by Alexey Izbyshev :
--
keywords: +patch
pull_requests: +11484
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issue35823>
___
_
Change by Alexey Izbyshev :
--
keywords: +patch, patch, patch
pull_requests: +11484, 11485, 11486
stage: -> patch review
___
Python tracker
<https://bugs.python.org/issu
New submission from Alexey Izbyshev :
This issue is to propose a (complementary) alternative to the usage of
posix_spawn() in subprocess (see bpo-35537).
As mentioned by Victor Stinner in msg332236, posix_spawn() has the potential of
being faster and safer than fork()/exec() approach
Alexey Izbyshev added the comment:
Another problem with posix_spawn() on glibc: it doesn't report errors to the
parent process when run under QEMU user-space emulation and Windows Subsystem
for Linux. This is because starting with commit [1] (glibc 2.25) posix_spawn()
relies on address
Alexey Izbyshev added the comment:
>> * pass_fds: there is not API to mark a fd as inheritable (clear O_CLOEXEC
>> flag)
> POSIX has a bug for this [5]. It's marked fixed, but the current POSIX docs
> doesn't reflect the changes. The idea is to make
> posix_spawn
Alexey Izbyshev added the comment:
Thanks for the info on CS_PATH, Victor. IMHO it'd make sense to use the
libc-provided default PATH at least in shutil.which() since its intent is to
emulate "which" from the default shell.
--
___
Pyth
Alexey Izbyshev added the comment:
> It should be compared to the current code. Currently, _posixsubprocess uses a
> loop calling execv(). I don't think that calling posix_spawn() in a loop
> until one doesn't fail is more inefficient.
> The worst case would be when appl
Alexey Izbyshev added the comment:
Thank you for the answers, Kyle!
> I'll be preparing a patch for our posix_spawn's signal handling.
Great!
> My mistake in my setuid assessment was pointed out to me- it doesn't seem
> like a highly likely attack vector, but it is indeed
Alexey Izbyshev added the comment:
Would it make sense to use os.confstr('CS_PATH') instead of a hardcoded path,
or is identical behavior on all POSIX platforms preferred to that?
--
nosy: +izbyshev
___
Python tracker
<https://bugs.python.
1 - 100 of 331 matches
Mail list logo
