[issue37096] Add large-file tests for modules using sendfile(2)

STINNER Victor added the comment: I close the issue. The test was fixed in bpo-39488: commit b39fb8e847ac59b539ad7e93df91c1709815180e Author: Giampaolo Rodola Date: Wed Feb 5 18:20:52 2020 +0100 bpo-39488: Skip test_largefile tests if not enough disk space (GH-18261) --

[issue37096] Add large-file tests for modules using sendfile(2)

STINNER Victor added the comment: Right now, "AMD64 Fedora Rawhide Clang 3.x" worker is broken: "C compiler cannot create executables" https://bugs.python.org/issue39398 -- ___ Python tracker

[issue37096] Add large-file tests for modules using sendfile(2)

Charalampos Stratakis added the comment: Can we reopen the issue? Clearly this change modifies the expectations of free disk space for the temp files created by the tests. Or at least clarify that those tests require more than 6gb of free disk space in /tmp for unix --

[issue37096] Add large-file tests for modules using sendfile(2)

Petr Viktorin added the comment: The test passes with a 8GB /tmp, but not a 6GB one. That could suggest the test needs 3*2GB disk space for temporary files, plus something extra. -- ___ Python tracker

[issue37096] Add large-file tests for modules using sendfile(2)

Petr Viktorin added the comment: It seems that the test actually requires >4GB disk space, because it's copying a 2GB file from one place to another. Also, it's using a temporary directory. On Fedora /tmp is memory-backed by default, it doesn't use regular disk space. -- nosy:

[issue37096] Add large-file tests for modules using sendfile(2)

Charalampos Stratakis added the comment: Alright added some more disk space at the buildbots, however it seems that it is not related to that. The current Fedora rawhide buildbot has 19GB of free space and the test is still failing. I tested on the Fedora stable buildbot for which I reduced

[issue37096] Add large-file tests for modules using sendfile(2)

STINNER Victor added the comment: > checking whether to enable large file support... no That's only at the C level to decide how to pass a file position. It's unrelated to the regrtest "largefile" resource. -- ___ Python tracker

[issue37096] Add large-file tests for modules using sendfile(2)

Charalampos Stratakis added the comment: It seems that the -uall argument is passed to regrtest invocation for the buildbot run [0] which invokes the largefile tests (including all the resource intensive tests). However when configure is run you can see: checking whether to enable large

[issue37096] Add large-file tests for modules using sendfile(2)

STINNER Victor added the comment: Should we decorate the new files with @requires('largefile', '...')? Documentation: largefile - It is okay to run some test that may create huge files. These tests can take a long time and may consume >2 GiB of disk space

[issue37096] Add large-file tests for modules using sendfile(2)

Giampaolo Rodola' added the comment: Sorry about that. I'm attaching a patch which skips those tests if there's not enough free disk space. If you confirm it works I'll make a PR. -- Added file: https://bugs.python.org/file48636/disk-space.patch

[issue37096] Add large-file tests for modules using sendfile(2)

Charalampos Stratakis added the comment: After this change I get some disk space issues on the Fedora rawhide buildbot for the clang installed build only (and strangely enough not for the other jobs). There are currently around 9GB of free space there:

[issue37096] Add large-file tests for modules using sendfile(2)

STINNER Victor added the comment: TestCopyfile fails on 32-bit system: bpo-38319. -- nosy: +vstinner resolution: fixed -> status: closed -> open ___ Python tracker ___

[issue37096] Add large-file tests for modules using sendfile(2)

Change by Giampaolo Rodola' : -- components: +Tests -Library (Lib) resolution: -> fixed stage: patch review -> resolved status: open -> closed type: -> enhancement versions: +Python 3.9 -Python 3.8 ___ Python tracker

[issue37096] Add large-file tests for modules using sendfile(2)

Giampaolo Rodola' added the comment: New changeset 5bcc6d89bcb622a6786fff632fabdcaf67dbb4e2 by Giampaolo Rodola in branch 'master': bpo-37096: Add large-file tests for modules using sendfile(2) (GH-13676) https://github.com/python/cpython/commit/5bcc6d89bcb622a6786fff632fabdcaf67dbb4e2

[issue37096] Add large-file tests for modules using sendfile(2)

Change by Giampaolo Rodola' : -- keywords: +patch pull_requests: +13564 stage: -> patch review pull_request: https://github.com/python/cpython/pull/13676 ___ Python tracker

[issue37096] Add large-file tests for modules using sendfile(2)

New submission from Giampaolo Rodola' : The need for this emerged in: https://bugs.python.org/issue36610#msg343948. We currently use sendfile(2) syscall in high-level shutil.copyfile() on Linux and socket.sendfile() on UNIX. In addition this PR also tests shutil.copyfile() implementation on