On 12/04/2023 05:55, Sam James wrote:
Pádraig Brady <p...@draigbrady.com> writes:
On 10/04/2023 22:52, Sam James wrote:
Sam James <s...@gentoo.org> writes:
Pádraig Brady <p...@draigbrady.com> writes:
We plan to release coreutils-9.3 in the coming week
so any testing you can do on various different systems
between now and then would be most welcome.
This is a bug fix release coming about 3 weeks after the 9.2 release.
--------------------------------------
You can download the coreutils snapshot in xz format (5.7 MB) from:
https://pixelbeat.org/cu/coreutils-ss.tar.xz
And verify with gpg or md5sum with:
https://pixelbeat.org/cu/coreutils-ss.tar.xz.sig
MD5 (coreutils-ss.tar.xz) = d208d306026fb42c128a787dffcba17a
--------------------------------------
To test follow this standard procedure:
tar -xf coreutils-ss.tar.xz
cd coreutils-9.2.18-ffd62/
./configure && make check VERBOSE=yes
Failures are reported, and details are in tests/test-suite.log
Please report/attach any issues to coreutils@gnu.org
I get one failure in tests/cp/backup-dir.sh inside our packaging:
I see now you've caught this as a non-portable shell construct &
fixed -
thanks! So ignore this one, and see the failures below instead.
```
+ env cp --version
[...]
FAIL tests/cp/backup-dir.sh (exit status: 1)
```
These I can naturally still hit:
In another environment (same machine but this time running manually,
outside of our packaging, and on a ZFS filesystem instead of tmpfs),
I get two failures:
1. tests/cp/sparse-2.sh.log: http://sprunge.us/jOCSEr (unfortunately I
can't consistently reproduce this one)
This might be due to slightly different I/O patterns between cp and dd.
The log above suggests that cp is not inducing holes while dd is.
It would be useful to get both strace files from running the following on that
ZFS filesystem
cd coreutils-9.2.18-ffd62
printf x>k
dd bs=1k seek=1 of=k count=255 < /dev/zero
strace -e cp.strace src/cp --reflink=never --sparse=always k k2
hole_size=$(stat -c %o k2)
strace -o dd.strace src/dd if=k of=k2.dd bs=$hole_size conv=sparse
By the way, this time, I also got a tests/du/basic failure too:
http://sprunge.us/xEumX6.
For the commands you gave:
```
255+0 records in
255+0 records out
261120 bytes (261 kB, 255 KiB) copied, 0.00220926 s, 118 MB/s
2+0 records in
2+0 records out
262144 bytes (262 kB, 256 KiB) copied, 0.000550036 s, 477 MB/s
```
and:
cp.strace: http://sprunge.us/cng0Z9
dd.strace: http://sprunge.us/F0Vg2r
Note that I can't reproduce the original test failure every time,
only ~1/7, 1/8 times, so I might need to run these in a loop
to get you the output needed. Or these might be fine. Dunno. :)
The cp strace is interesting as it indicates the source file has holes detected,
which is unexpected. Perhaps zfs (with compression?) is detecting zeros
when SEEK_DATA/SEEK_HOLE are used? The interesting part of the strace is:
lseek(3, 0, SEEK_DATA) = 0
lseek(3, 0, SEEK_HOLE) = 131072
read(3, "x\0\0\0\0\0\0\0\0"..., 131072) = 131072
write(4, "x\0\0\0\0\0\0\0\"..., 131072) = 131072
lseek(3, 131072, SEEK_DATA) = -1 ENXIO (No such device or address)
ftruncate(4, 262144) = 0
fallocate(4, FALLOC_FL_KEEP_SIZE|FALLOC_FL_PUNCH_HOLE, 131072, 131072) = 0
In any case the output pattern for dd is very similar:
read(0, "x\0\0\0\0\0\0\0\0"..., 131072) = 131072
write(1, "x\0\0\0\0\0\0\0\"..., 131072) = 131072
read(0, "\0\0\0\0\0\0\0\0\"..., 131072) = 131072
lseek(1, 131072, SEEK_CUR) = 262144
read(0, "", 131072) = 0
lseek(1, 0, SEEK_CUR) = 262144
ftruncate(1, 262144) = 0
Note ZFS had this issue with SEEK_HOLE but I don't think you're hitting that:
https://github.com/openzfs/zfs/issues/11900
Given this is intermittent, I think this test may just be a little brittle,
by depending on file layout which is hard to match across all file systems.
The attached leverages our new --debug functionality to ensure
checking for zeros is enabled.
2. ./tests/misc/tty-eof.log: http://sprunge.us/yWNByM
perl's expect module seems to be getting spurious exitstatus for some commands.
I suspect a race, and there is a hardcoded 10s in that script.
There are various other issues in that script actually.
Hopefully the attached addresses this.
Interestingly, it doesn't!
tests/misc/tty-eof.log: http://sprunge.us/UwqooK.
I just applied the patch (and verified the changes were there) and
ran 'make check' again rather than a rebuild, but I assume that
shouldn't matter given it's a test script change.
Drats. It's awkward to debug further when I can't repro.
Given it happens to random commands, I still suspect an issue with your Expect
lib,
or with how we're using it, rather than an issue with the commands themselves.
cheers,
Pádraig
From 89c6e8c25bf01e21390dc67089f5de5822034fb2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?P=C3=A1draig=20Brady?= <p...@draigbrady.com>
Date: Wed, 12 Apr 2023 14:37:00 +0100
Subject: [PATCH] tests: avoid dependence on file layout for cp sparse check
* tests/cp/sparse-2.sh: Don't depend on the copy taking
<= allocation of the source. Instead leverage --debug
to check that zero detection is being enabled.
---
tests/cp/sparse-2.sh | 16 +++-------------
1 file changed, 3 insertions(+), 13 deletions(-)
diff --git a/tests/cp/sparse-2.sh b/tests/cp/sparse-2.sh
index 3024055e6..9ae317492 100755
--- a/tests/cp/sparse-2.sh
+++ b/tests/cp/sparse-2.sh
@@ -44,18 +44,8 @@ printf x > k || framework_failure_
dd bs=1k seek=1 of=k count=255 < /dev/zero || framework_failure_
# cp should detect the all-zero blocks and convert some of them to holes.
-# How many it detects/converts currently depends on io_blksize.
-# Currently, on my F14/ext4 desktop, this K file starts off with size 256KiB,
-# (note that the K in the preceding test starts off with size 4KiB).
-# cp from coreutils-8.9 with --sparse=always reduces the size to 32KiB.
-cp --reflink=never --sparse=always k k2 || fail=1
-if test $(stat -c %b k2) -ge $(stat -c %b k); then
- # If not sparse, then double check by creating with dd
- # as we're not guaranteed that seek will create a hole.
- # apfs on darwin 19.2.0 for example was seen to not to create holes < 16MiB.
- hole_size=$(stat -c %o k2) || framework_failure_
- dd if=k of=k2.dd bs=$hole_size conv=sparse || framework_failure_
- test $(stat -c %b k2) -eq $(stat -c %b k2.dd) || fail=1
-fi
+cp --debug --reflink=never --sparse=always k k2 >cp.out || fail=1
+cmp k k2 || fail=1
+grep 'sparse detection: zeros' cp.out || fail=1
Exit $fail
--
2.26.2
