[issue43743] BlockingIOError: [Errno 11] Resource temporarily unavailable: on GPFS.

2022-03-13 Thread Marvin Poul


Marvin Poul  added the comment:

Here's the small patch.  Sadly I have no overview what the affected linux 
kernel version are.  I guess technically you can all this "working around a bug 
in specific linux version", but since it's a very minor change that saves one 
syscall even for non-breaking version, I feel it's justified.  Let me know if 
you'd like any modification done however.

--
keywords: +patch
Added file: https://bugs.python.org/file50672/shutil.patch

___
Python tracker 
<https://bugs.python.org/issue43743>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue43743] BlockingIOError: [Errno 11] Resource temporarily unavailable: on GPFS.

2022-03-13 Thread Marvin Poul


Marvin Poul  added the comment:

I hope you don't mind me necro posting, but I ran into this issue again and
have a small patch to solve it.

I attached an MWE that triggers the BlockingIOError reliably on ext4
filesystems in linux 4.12.14 and python 3.8.12.  Running under strace -e
sendfile gives the following output

# manually calling sendfile to check that it works
> sendfile(5, 4, [0] => [8388608], 8388608) = 8388608
# sendfile calls originating in shutil.copy
> sendfile(5, 4, [0] => [8388608], 8388608) = 8388608
> sendfile(5, 4, [8388608], 8388608)  = -1 EAGAIN (Resource temporarily 
> unavailable)
> Shutil Failed!
> [Errno 11] Resource temporarily unavailable: 
> '/cmmc/u/zora/scratch/sendfile_bug/tmpaqx2o4uj' -> 
> '/cmmc/u/zora/scratch/sendfile_bug/tmpb8rzg8rg'
> +++ exited with 0 +++

This shows that the first call to sendfile actually copies the whole file and
the EAGAIN is only triggered on the second, unnecessary, call.  I have tested
with a small C program that it's triggered whenever sendfile's offset + count
exceeds the file size of in_fd.  This is weird behaviour on the kernels side
that seems to have changed in newer kernel versions (issue is not present e.g.
on my 5.16.12 laptop).

Anyways my patch makes that second call not appear by keeping track of the file
size and the bytes written so far.  It's against the current python main
branch, but if I see correctly this part hasn't changed in years.  I have
checked the error is not thrown when the patch is applied.

(I can only attach one file, so patch is attached in a new one.)

--
nosy: +pmrv
Added file: https://bugs.python.org/file50671/sendfile.py

___
Python tracker 
<https://bugs.python.org/issue43743>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com