libintl also missing in tests/Makefile.am
The fix from commit 8632df398b2f548465ebe68b8f494c0d6f8d913d is also required for tests/Makefile.am: LDADD = ../gnu/libgnu.a\ $(LIB_ACL) $(LIB_CLOCK_GETTIME) $(LIB_EACCESS)\ $(LIB_GETRANDOM) $(LIB_HARD_LOCALE) $(FILE_HAS_ACL_LIB) $(LIB_MBRTOWC)\ - $(LIB_SELINUX) $(LIB_SETLOCALE_NULL) + $(LIB_SELINUX) $(LIB_SETLOCALE_NULL) \ + $(LIBINTL) $(LIBICONV) -- Christian "naddy" Weisgerber na...@mips.inka.de
1.34: Accessing remote device now requires root?
As OpenBSD packager for GNU tar, I was contacted by a user who ran into a problem with 1.34: Accessing a remote device is now only possible as superuser. Otherwise, tar will error out with initgroups: Operation not permitted initgroups(3) is indeed limited to the superuser. The initgroups() call came in via changes to sys_reset_uid_gid(): https://git.savannah.gnu.org/cgit/paxutils.git/commit/lib/system.h?id=d247e3c2809a37b6d0c3067251d96bb7f12555e7 What does that even try to do? Getting rid of running as set[ug]id? -- Christian "naddy" Weisgerber na...@mips.inka.de
tar 1.33 TAR_OPTIONS parser bug
Commit 1ff0b63f4898 "Accept only position-sensitive (file-selection) options in file list files" has introduced a pointer indirection bug that breaks TAR_OPTIONS parsing. The problem shows up when running the test suite on FreeBSD 12.2 where it can be simplified to this reproducer: $ export TAR_OPTIONS='--pax-option=exthdr.name=%d/PaxHeaders/%f,delete=mtime,delete=atime,delete=ctime' $ tar Segmentation fault (core dumped) Here's the fix: --- src/tar.c.orig 2021-01-17 15:26:53 UTC +++ src/tar.c @@ -2248,7 +2248,7 @@ parse_default_options (struct tar_args *args) if (argp_parse (&argp, ws.ws_offs + ws.ws_wordc, ws.ws_wordv, - ARGP_IN_ORDER|ARGP_NO_EXIT, &idx, &args)) + ARGP_IN_ORDER|ARGP_NO_EXIT, &idx, args)) abort (); /* shouldn't happen */ args->loc = save_loc_ptr; if (name_more_files ()) -- Christian "naddy" Weisgerber na...@mips.inka.de
tar 1.33 regression on OpenBSD (extrac09 getcwd)
There is a regression in tar 1.33 on OpenBSD. Thanks to Stefan Hagen
for figuring out the details.
89: extracting even when . and .. are unreadableFAILED (extrac09.at:37)
The key part of the test:
mkdir dir
mkdir dir/sub
mkdir dir/sub/extract
genfile --file dir/sub/f
cd dir/sub
tar -cf archive.tar f
chmod a-r . ..
tar -xvf archive.tar -C extract f
And the failure of the "tar -xvf ..." command:
tar: .: Cannot getcwd: Permission denied
tar: Error is not recoverable: exiting now
The regression has been introduced with commit 66162927ebdf,
"Check return value from xgetcwd". Specifically, this chunk:
--->
@@ -922,6 +920,8 @@ chdir_arg (char const *dir)
{
wd[wd_count].name = ".";
wd[wd_count].abspath = xgetcwd ();
+ if (!wd[wd_count].abspath)
+ call_arg_fatal ("getcwd", ".");
wd[wd_count].fd = AT_FDCWD;
wd_count++;
}
<---
A trace shows that the native system call returns EACCES, which
conforms with POSIX.
13845 tar CALL __getcwd(0x3fd723c000,1024)
13845 tar RET __getcwd -1 errno 13 Permission denied
--
Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] 1.31: Test 162 (sparse07.at) fails on FreeBSD due to iconv() differences
GNU tar 1.31's regression test #162 (sparse07.at) fails on FreeBSD
11 and 12.
This has nothing to do with sparse files. It concerns unicode file
names in general. The underlying problem appears to be a difference
between GNU iconv() and FreeBSD's iconv(). For a conversion from
UTF-8 to ASCII, GNU iconv() will return -1 and signal an error if
the input contains any characters that cannot be represented in
ASCII. FreeBSD's iconv() replaces those characters with '?' and
returns the number of such substitutions. This latter behavior is
in agreement with my reading of the POSIX standard on iconv().
Tentative patch below.
I already reported this problem for 1.30.
https://lists.gnu.org/archive/html/bug-tar/2018-04/msg00019.html
When trying to reproduce this on FreeBSD, be careful that the tar
build does not pick up GNU libiconv if the latter happens to be
installed on the test system. To force use of the native iconv(3),
set CPPFLAGS='-DLIBICONV_PLUG -isystem /usr/local/include'.
--- src/utf8.c.orig 2019-01-11 17:46:41 UTC
+++ src/utf8.c
@@ -81,7 +81,7 @@ utf8_convert (bool to_utf, char const *input, char **o
outlen = inlen * MB_LEN_MAX + 1;
ob = ret = xmalloc (outlen);
ib = (char ICONV_CONST *) input;
- if (iconv (cd, &ib, &inlen, &ob, &outlen) == -1)
+ if (iconv (cd, &ib, &inlen, &ob, &outlen) != 0)
{
free (ret);
return false;
--
Christian "naddy" Weisgerber na...@mips.inka.de
# -*- compilation -*-
162. sparse07.at:21: testing sparse files with unicode names ...
./sparse07.at:24:
mkdir posix
(cd posix
TEST_TAR_FORMAT=posix
export TEST_TAR_FORMAT
TAR_OPTIONS="-H posix"
export TAR_OPTIONS
rm -rf *
genfile --sparse --file žluť --block-size 512 0 ABCD 1M EFGH 2000K IJKL || exit
77
tar -c -f archive --sparse žluť || exit 1
tar tf archive
)
--- - 2019-01-11 18:58:00.408827000 +0100
+++
/usr/ports/archivers/gtar/work/tar-1.31/tests/testsuite.dir/at-groups/162/stdout
2019-01-11 18:58:00.407853000 +0100
@@ -1,2 +1,2 @@
-\305\276lu\305\245
+?lu?
162. sparse07.at:21: 162. sparse files with unicode names (sparse07.at:21):
FAILED (sparse07.at:24)
[Bug-tar] 1.31: test 41 numeric.at broken with BSD filesystem semantics
GNU tar 1.31's regression test 41, numeric.at, is broken on filesystems
with BSD semantics (GID inherited from directory) if run in a
directory whose GID does not match the primary GID of the executing
user.
I already reported this problem for 1.29 and 1.30. A fix had been
suggested for 1.29, but was never committed.
https://lists.gnu.org/archive/html/bug-tar/2016-06/msg1.html
Testsuite.log from FreeBSD 12 attached.
(User naddy:naddy, directory naddy:wheel.)
--
Christian "naddy" Weisgerber na...@mips.inka.de
# -*- compilation -*-
41. numeric.at:18: testing --numeric-owner basic tests ...
./numeric.at:29:
mkdir posix
(cd posix
TEST_TAR_FORMAT=posix
export TEST_TAR_FORMAT
TAR_OPTIONS="-H posix"
export TAR_OPTIONS
rm -rf *
mkdir dir
genfile --file dir/file
MYUID=$(id -u) || exit 77
MYGID=$(id -g) || exit 77
MYUSR=$(id -un) || exit 77
MYGRP=$(id -gn) || exit 77
decho --create
tar --create -vvf a dir --numeric-owner | awk '2=="'"$MYUID/$MYGID"'" {print
"OK"; next} {print}'
tar --create -vvf a dir | awk '2=="'"$MYUSR/$MYGRP"'" {print "OK"; next}
{print}'
decho --list
tar --list -vvf a dir --numeric-owner | awk '2=="'"$MYUID/$MYGID"'" {print
"OK"; next} {print}'
tar --list -vvf a dir | awk '2=="'"$MYUSR/$MYGRP"'" {print "OK"; next} {print}'
decho --diff
tar --diff -vvf a dir --numeric-owner | awk '2=="'"$MYUID/$MYGID"'" {print
"OK"; next} {print}'
tar --diff -vvf a dir | awk '2=="'"$MYUSR/$MYGRP"'" {print "OK"; next} {print}'
decho --extract
tar --extract -vvf a dir --numeric-owner | awk '2=="'"$MYUID/$MYGID"'" {print
"OK"; next} {print}'
tar --extract -vvf a dir | awk '2=="'"$MYUSR/$MYGRP"'" {print "OK"; next}
{print}'
)
--- - 2019-01-11 16:33:18.634802000 +0100
+++
/usr/ports/archivers/gtar/work/tar-1.31/tests/testsuite.dir/at-groups/41/stdout
2019-01-11 16:33:18.633907000 +0100
@@ -1,21 +1,21 @@
--create
-OK
-OK
-OK
-OK
+drwxr-xr-x 1000/00 2019-01-11 16:33 dir/
+-rw-r--r-- 1000/00 2019-01-11 16:33 dir/file
+drwxr-xr-x naddy/wheel 0 2019-01-11 16:33 dir/
+-rw-r--r-- naddy/wheel 0 2019-01-11 16:33 dir/file
--list
-OK
-OK
-OK
-OK
+drwxr-xr-x 1000/00 2019-01-11 16:33 dir/
+-rw-r--r-- 1000/00 2019-01-11 16:33 dir/file
+drwxr-xr-x naddy/wheel 0 2019-01-11 16:33 dir/
+-rw-r--r-- naddy/wheel 0 2019-01-11 16:33 dir/file
--diff
-OK
-OK
-OK
-OK
+drwxr-xr-x 1000/00 2019-01-11 16:33 dir/
+-rw-r--r-- 1000/00 2019-01-11 16:33 dir/file
+drwxr-xr-x naddy/wheel 0 2019-01-11 16:33 dir/
+-rw-r--r-- naddy/wheel 0 2019-01-11 16:33 dir/file
--extract
-OK
-OK
-OK
-OK
+drwxr-xr-x 1000/00 2019-01-11 16:33 dir/
+-rw-r--r-- 1000/00 2019-01-11 16:33 dir/file
+drwxr-xr-x naddy/wheel 0 2019-01-11 16:33 dir/
+-rw-r--r-- naddy/wheel 0 2019-01-11 16:33 dir/file
41. numeric.at:18: 41. --numeric-owner basic tests (numeric.at:18): FAILED
(numeric.at:29)
[Bug-tar] 1.31: No default checkpoint action
The documentation for GNU tar 1.31 claims in section "3.8 Checkpoints": The simplest value of action is `echo'. [...] This is the default action, so running: $ tar -c --checkpoint=1000 --checkpoint-action=echo /var is equivalent to: $ tar -c --checkpoint=1000 /var However, this is manifestly not true. There is no default checkpoint action. Without an explicit --checkpoint-action=xxx argument, no checkpoint action is taken, no default message is written to stderr. I don't know if this is a documentation or implementation error. -- Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] tar-1.30: test 156 fails on FreeBSD due to iconv differences
On FreeBSD 11, I see this regression test failure with GNU tar 1.30: 156: sparse files with unicode names FAILED (sparse07.at:24) See the testsuite.log at the end of this message. This has nothing to do with sparse files. It concerns unicode file names in general. The underlying problem appears to be a difference between GNU iconv() and FreeBSD's iconv(). For a conversion from UTF-8 to ASCII, GNU iconv() will return -1 and signal an error if the input contains any characters that cannot be represented in ASCII. FreeBSD's iconv() replaces those characters with '?' and returns the number of such substitutions. This latter behavior is in agreement with my reading of the POSIX standard on iconv(). The difference can be papered over with something like this... --- src/utf8.c.orig 2018-04-30 17:25:34 UTC +++ src/utf8.c @@ -85,7 +85,7 @@ utf8_convert (bool to_utf, char const *i ib = (char ICONV_CONST *) input; rc = iconv (cd, &ib, &inlen, &ob, &outlen); *ob = 0; - return rc != -1; + return rc == 0; } ... but I'm uncertain if this doesn't have other consequences. The regression tests pass fine with it. # -*- compilation -*- 156. sparse07.at:21: testing sparse files with unicode names ... ./sparse07.at:24: mkdir posix (cd posix TEST_TAR_FORMAT=posix export TEST_TAR_FORMAT TAR_OPTIONS="-H posix" export TAR_OPTIONS rm -rf * genfile --sparse --file žluť --block-size 512 0 ABCD 1M EFGH 2000K IJKL || exit 77 tar -c -f archive --sparse žluť || exit 1 tar tf archive ) --- - 2018-04-30 19:16:35.785125184 +0200 +++ /usr/ports/archivers/gtar/work/tar-1.30/tests/testsuite.dir/at-groups/156/stdout 2018-04-30 19:16:35.784543000 +0200 @@ -1,2 +1,2 @@ -\305\276lu\305\245 +?lu? 156. sparse07.at:21: 156. sparse files with unicode names (sparse07.at:21): FAILED (sparse07.at:24) -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: [Bug-tar] tar-1.30 gnulib: error() and set_program_name() out of sync
Sergey Poznyakoff: > > For GNU tar 1.30, the regression tests remfiles01.at and remfiles02.at > > (175 176) fail. > > Pretty strange, but both tests work for me (the release would have been > impossible if they woudn't). Any specific conditions in order to > reproduce that? It fails for me on OpenBSD and FreeBSD. In February, Kiyoshi KANAZAWA reported it for Solaris 11.3 x86: https://lists.gnu.org/archive/html/bug-tar/2018-02/msg1.html Earlier this month, Ronnie Mainieri reported it, but I can't tell on which platform: https://lists.gnu.org/archive/html/bug-tar/2018-04/msg1.html Ah, I see, if a system has HAVE_DECL_PROGRAM_INVOCATION_NAME (glibc, I guess), then set_program_name() stores the name in program_invocation_name and getprogname() reads it from there. -- Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] tar-1.30 gnulib: error() and set_program_name() out of sync
For GNU tar 1.30, the regression tests remfiles01.at and remfiles02.at (175 176) fail. The immediate problem is that tar's child process fails to identify itself as "tar (child)" in the error message. The underlying cause is a change in gnulib, specifically this commit: https://git.savannah.gnu.org/cgit/gnulib.git/commit/lib/error.c?id=f29814b96c9737abc8193d57d630697700feebeb tar uses set_program_name() to set the program name in a variable program_name, but error() now calls getprogname(), which does not read program_name. I don't know at which level this should be fixed, but it seems odd that gnulib offers a set_program_name() that is ignored by the rest of the library. -- Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] tar HEAD: difflink.at portability issue
In the tar 1.30 release there was a problem with the difflink.at test; this has been fixed in HEAD. However, the test still fails on *BSD and there is an underlying portability issue. The preparatory part of the test runs this: mkdir a genfile -f a/x ln -s x a/y ln a/y a/z What is the expected result? ls -l a ==> OpenBSD, FreeBSD: total 0 -rw-r--r-- 2 naddy naddy 0 Apr 26 18:45 x lrwxr-xr-x 1 naddy naddy 1 Apr 26 18:45 y -> x -rw-r--r-- 2 naddy naddy 0 Apr 26 18:45 z ==> Linux: total 0 -rw-r--r-- 1 naddy unixag 0 Apr 26 18:46 x lrwxrwxrwx 2 naddy unixag 1 Apr 26 18:46 y -> x lrwxrwxrwx 2 naddy unixag 1 Apr 26 18:46 z -> x That's the difference between "ln -L a/y a/z" and "ln -P a/y a/z". POSIX says the default behavior is "implementation defined". I suggest the following fix: --- /home/naddy/difflink.at Thu Apr 26 18:58:04 2018 +++ difflink.at Thu Apr 26 18:52:59 2018 @@ -20,7 +20,7 @@ mkdir a genfile -f a/x ln -s x a/y -ln a/y a/z +ln -P a/y a/z tar cf a.tar a/x a/y a/z rm a/z ln -s x a/z -- Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] 1.30: test numeric.at still broken on BSD
The numeric.at test is broken on filesystems with BSD group semantics. I already reported this for 1.29 and this fix was suggested: https://lists.gnu.org/archive/html/bug-tar/2016-06/txt9bP0DVvYnu.txt Alas, it hasn't found its way into 1.30. -- Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] 1.30: info files are always rebuilt
There is a small distribution issue with GNU tar 1.30. Since doc/version.texi is out of date with respect to doc/stamp-vti, the *.info files are always rebuilt from the *.texi sources. This obviously fails is makeinfo is not available or too old. -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: [Bug-tar] tar 1.29: Test 30 numeric.at broken with BSD filesystem semantics
Pavel Raiskup: > Then we should be able to switch the group ownership back to $GID, does > the attached patch help? -snip- Yes, this looks correct... and successfully fixes the test. -- Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] tar 1.29: Test 30 numeric.at broken with BSD filesystem semantics
In GNU tar 1.29, this new regression test 30: numeric.at:18 --numeric-owner basic tests is itself broken on filesystems that have BSD group inheritance, i.e., where a newly created file gets the group of the directory rather than that of the user. Obviously, this affects (at least) all *BSD operating systems. -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: [Bug-tar] GNU tar 1.28: tests 161 163 fail on !Linux
(Replying to Nathan Treadway; I'm not subscribed to this list.) > > mkdir foo > > mkdir bar > > echo foo/foo_file > foo/foo_file > > echo bar/bar_file > bar/bar_file > > decho A > > tar -cvf foo.tar --remove-files -C foo . -C ../bar . > > decho B > > find . > Do you happen to still have tar 1.27.1 available on these test systems? > I am fairly certain that tar's behavior in these cases (i.e. > tar -cvf foo.tar --remove-files -C foo . -C ../bar . > and > tar -rvf foo.tar --remove-files -C foo . -C ../bar . > ) hasn't changed between 1.27.1 and 1.28, but it would be interesting to > know for sure if those operations fail in 1.27.1, too. Yes, they fail in 1.27.1, too. FWIW, I also tried building tar with ac_cv_func_unlinkat=no, but that does not circumvent the problem: ... 79712 gtar CALL fchdir(0x4) 79712 gtar RET fchdir 0 79712 gtar CALL rmdir(0x801451068) 79712 gtar NAMI "../bar" 79712 gtar RET rmdir -1 errno 2 No such file or directory ... -- Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] GNU tar 1.28: tests 161 163 fail on !Linux
On both FreeBSD and OpenBSD, these tests in GNU tar 1.28 fail:
161: remfiles08a.at:28 remove-files deleting two subdirs in -c/non-incr. mode
create remove-files remfiles08 remfiles08a
163: remfiles08c.at:28 remove-files deleting two subdirs in -r mode
create append remove-files remfiles08 remfiles08c
According to an earlier message to bug-tar, they also fail on Solaris.
I've looked at test #161 and it does this:
mkdir foo
mkdir bar
echo foo/foo_file > foo/foo_file
echo bar/bar_file > bar/bar_file
decho A
tar -cvf foo.tar --remove-files -C foo . -C ../bar .
decho B
find .
The failing result is that ./bar still exists.
A system call trace reveals this:
...
9370 gtar CALL unlinkat(0x4,0x801453100,0)
9370 gtar NAMI "foo_file"
9370 gtar RET unlinkat 0
9370 gtar CALL unlinkat(AT_FDCWD,0x801451060,0x800)
9370 gtar NAMI "foo"
9370 gtar RET unlinkat 0
9370 gtar CALL unlinkat(0x5,0x801453110,0)
9370 gtar NAMI "bar_file"
9370 gtar RET unlinkat 0
9370 gtar CALL unlinkat(0x4,0x801451068,0x800)
9370 gtar NAMI "../bar"
9370 gtar RET unlinkat -1 errno 2 No such file or directory
...
The problem is that "foo" is deleted and then the attempt fails to
delete a path relative to a filedescriptor referring to "foo".
The test program below reproduces this: The final unlinkat() fails
on FreeBSD/OpenBSD but succeeds on Linux.
(err.h may not be available everywhere.)
-
#include
#include
#include
#include
int main(void)
{
int fd;
if (mkdir("foo", 0777) == -1)
err(1, "can't create foo");
if (mkdir("bar", 0777) == -1)
err(1, "can't create bar");
if ((fd = open("foo", O_RDONLY)) == -1)
err(1, "can't open foo");
if (unlinkat(AT_FDCWD, "foo", AT_REMOVEDIR) == -1)
err(1, "can't remove foo");
if (unlinkat(fd, "../bar", AT_REMOVEDIR) == -1)
err(1, "can't remove ../bar");
return 0;
}
-
The BSD behavior appears to be in line with POSIX. unlinkat() with
AT_REMOVEDIR is equivalent to rmdir(), whose specification says:
If one or more processes have the directory open when the last
link is removed, the dot and dot-dot entries, if present, shall
be removed before rmdir() returns and no new entries may be created
in the directory, but the directory shall not be removed until
all references to the directory are closed.
Without "..", the path resolution of the subsequent unlinkat() call
should--or at least can--fail.
--
Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] tar and static linking
So you are on a system that has gettext in an extra libintl (and possibly libiconv, too). You want to link tar statically, say for recovery purposes, but you don't want to drop NLS support. How do you do this? (1) You configure with LDFLAGS=-static. You expect this to do the trick. (2) The configure script doesn't simply check for generic -lintl -liconv, but insists on something very specific, e.g.: checking how to link with libintl... /usr/local/lib/libintl.so /usr/local/lib/libiconv.so -Wl,-rpath -Wl,/usr/local/lib Result: You end up with -static /usr/local/lib/libintl.so [...]. This doesn't work. On FreeBSD 9, ld produces an error; configure already concludes that linking with libintl just doesn't work and disables NLS. (On OpenBSD, linking succeeds but you get a dynamic executable.) I guess this isn't so much a tar problem as a gettext/libiconv/gnulib one. -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: [Bug-tar] tar-1.25: test 39 fails on FreeBSD
Paul Eggert:
> If I understand things correctly, src/extract.c's extract_file
> is invoking open_output_file, which returns -1 with errno==ELOOP.
> extract_file then should invoke:
>
>maybe_recoverable ("./file1", true, &interdir_made)
>
> maybe_recoverable should execute this code:
>
> if (*interdir_made)
> return RECOVER_NO;
>
> switch (e)
> {
> case ELOOP:
The problem turns out to be very straightforward: On FreeBSD, the
open() call returns -1 with errno=EMLINK. This is documented
behavior in FreeBSD's open(2) man page:
[EMLINK] O_NOFOLLOW was specified and the target is a symbolic
link.
Adding EMLINK to the ELOOP case in maybe_recoverable fixes this (and
doesn't break any other regression tests).
--
Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] tar-1.25: test 39 fails on FreeBSD
On FreeBSD 7.3, test #39 "extract over symlinks" fails. Specifically,
it's this snippet of extrac13
ln -s target1 dst2/file1
echo target1 >dst2/target1
tar --overwrite -xf archive.tar -C dst2 --warning=no-timestamp
that errors out:
tar: ./file1: Cannot open: Too many links
ktrace shows that an open() call with O_NOFOLLOW set fails...
44351 tar CALL
open(0x800c1b100,O_WRONLY|O_NONBLOCK|O_NOFOLLOW|O_CREAT|O_TRUNC|O_NOCTTY,S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH)
44351 tar NAMI "./file1"
44351 tar RET open -1 errno 31 Too many links
... and tar proceeds to the error message. This open() behavior
is expected, so I'm unclear what GNU tar expects.
I compared with OpenBSD, where this test doesn't fail, and ktrace there
shows a sequence
open("./file1", ...) => ELOOP
lstat("./file1", ...)
unlink("./file1")
open("./file1", ...) => success
--
Christian "naddy" Weisgerber na...@mips.inka.de
Re: [Bug-tar] tar 1.24: OpenBSD testsuite failures
Paul Eggert: > Ah, OK. Does the following output "buggy shell", then? > > if (ulimit -n 10; : <&-) 2>/dev/null > then echo OK > else echo Houston we have a problem > fi Houston we have a problem on OpenBSD 4.8 (and older versions, nothing has changed there) FreeBSD 7.3 FreeBSD 8.1 -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: [Bug-tar] tar 1.24: OpenBSD testsuite failures
Paul Eggert: > One other thing. Can you please replace all instances > of "((ulimit -n" with "( (ulimit -n" in tests/testsuite? > This fixes a portability bug in that script that may > be relevant to the OpenBSD shell. This reflects the > following patch that I just pushed: When called as "ksh", OpenBSD's shell indeed stumbles over this, but when called as "sh"--as in the testsuite--it doesn't implement the (( )) arithmetic command, so this is not a problem. -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: [Bug-tar] tar 1.24: OpenBSD testsuite failures
Paul Eggert:
> The relevant shell source code (in testsuite) is:
>
> # Tar should work when there are few, but enough, file descriptors.
> ((ulimit -n 10 &&
> tar -cf archive3.tar a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
> tar -xf archive3.tar -C dest3 a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-
> ) &&
> diff -r a dest3/a >/dev/null 2>&1
> ) || { diff -r a dest3/a; exit 1; }
> )"
Yes. And I've set up the test directories manually and run the
above from the command line--where it works fine for a limit of 8
or more descriptors.
It only fails when called from the testsuite.
It succeeds from the testuite with -n 13 or more.
Very strange.
> For example, what does the following little shell script do on
> your host?
>
> #! /bin/sh
> if (ulimit -n 10 && true 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-) >/dev/null 2>&1
> then echo OK
> else echo buggy shell
> fi
It says OK.
> Or perhaps you can come up with a simple variant of the above test
> that will do the trick (maybe replace "true" with "cat /dev/null"?).
Also says OK.
> In reviewing the test, I see another possible portability problem, and
> I pushed the following patch. I doubt whether this will fix your
> issue, though.
No, it doesn't, same result.
> --- a/tests/extrac11.at
> +++ b/tests/extrac11.at
> @@ -24,6 +24,7 @@ AT_SETUP([scarce file descriptors])
> AT_KEYWORDS([extract extrac11])
>
> AT_TAR_CHECK([
> +exec dirs='a
>a/b
>a/b/c
--
Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] tar-1.24/gnulib: fdopendir.c C89 error
The gnu/fdopendir.c included with GNU tar 1.24 won't compile with a C89
compiler because it puts a declaration in between statements.
--- gnu/fdopendir.c.origTue Oct 26 06:54:19 2010
+++ gnu/fdopendir.c Tue Oct 26 06:54:52 2010
@@ -116,6 +116,7 @@ fdopendir_with_dup (int fd, int older_dupfd)
static DIR *
fd_clone_opendir (int fd)
{
+ struct saved_cwd saved_cwd;
int saved_errno;
DIR *dir;
@@ -152,7 +153,6 @@ fd_clone_opendir (int fd)
goto fail;
}
- struct saved_cwd saved_cwd;
if (save_cwd (&saved_cwd) != 0)
openat_save_fail (errno);
--
Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] tar 1.24: OpenBSD testsuite failures
Running the GNU tar 1.24 testsuite ("make check") on OpenBSD, I get
three failures:
34: extrac09.at:22 no need to save dir with unreadable . and ..
36: extrac11.at:23 scarce file descriptors
53: listed03.at:22 incremental dump when the parent directory is unreadable
#34 and #53 fail because on OpenBSD, getcwd(3) and the underlying
__getcwd() system call give an EACCES error when the parent directory
is unreadable.
#36 runs out of file descriptors for some reason. (ulimit -n 13 is
the minimum required for the test to complete successfully.)
Here are the interesting snippets of testsuite.log; I didn't want to
include the whole 1.6 MB.
## -- ##
## Detailed failed tests. ##
## -- ##
# -*- compilation -*-
34. extrac09.at:22: testing ...
./extrac09.at:25:
mkdir gnu
(cd gnu
TEST_TAR_FORMAT=gnu
export TEST_TAR_FORMAT
TAR_OPTIONS="-H gnu"
export TAR_OPTIONS
rm -rf *
echo test > 25656
chmod 0 25656
cat 25656 > /dev/null 2>&1
result=0
rm -f 25656
test -eq 0 && exit 77
mkdir dir
mkdir dir/sub
mkdir dir/sub/extract
genfile --file dir/sub/f
cd dir/sub
tar -cf archive.tar f
chmod a-r . ..
tar -xvf archive.tar -C extract f
status=$?
chmod a+r . ..
cmp f extract/f || status=$?
exit $status
)
Not enabling shell tracing (command contains an embedded newline)
--- /dev/null Tue Oct 26 11:13:30 2010
+++ /usr/ports/pobj/gtar-1.24/tar-1.24/tests/testsuite.dir/at-groups/34/stderr
Tue Oct 26 11:13:30 2010
@@ -0,0 +1,2 @@
+tar: unable to record current working directory: Permission denied
+cmp: extract/f: No such file or directory
./extrac09.at:25: exit code was 2, expected 0
34. extrac09.at:22: 34. no need to save dir with unreadable . and ..
(extrac09.at:22): FAILED (extrac09.at:25)
# -*- compilation -*-
36. extrac11.at:23: testing ...
./extrac11.at:26:
mkdir gnu
(cd gnu
TEST_TAR_FORMAT=gnu
export TEST_TAR_FORMAT
TAR_OPTIONS="-H gnu"
export TAR_OPTIONS
rm -rf *
dirs='a
a/b
a/b/c
a/b/c/d
a/b/c/d/e
a/b/c/d/e/f
a/b/c/d/e/f/g
a/b/c/d/e/f/g/h
a/b/c/d/e/f/g/h/i
a/b/c/d/e/f/g/h/i/j
a/b/c/d/e/f/g/h/i/j/k
'
files=
mkdir $dirs dest1 dest2 dest3 || exit
for dir in $dirs; do
for file in X Y Z; do
echo $file >$dir/$file || exit
files="$files $file"
done
done
# Check that "ulimit" itself works.
((ulimit -n 100 &&
tar -cf archive1.tar a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
tar -xf archive1.tar -C dest1 a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-
) &&
diff -r a dest1/a
) >/dev/null 2>&1 ||
exit 77
# Another test that "ulimit" itself works:
# tar should fail when completely starved of file descriptors.
((ulimit -n 4 &&
tar -cf archive2.tar a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
tar -xf archive2.tar -C dest2 a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-
) &&
diff -r a dest2/a
) >/dev/null 2>&1 &&
exit 77
# Tar should work when there are few, but enough, file descriptors.
((ulimit -n 10 &&
tar -cf archive3.tar a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
tar -xf archive3.tar -C dest3 a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-
) &&
diff -r a dest3/a >/dev/null 2>&1
) || { diff -r a dest3/a; exit 1; }
)
Not enabling shell tracing (command contains an embedded newline)
--- /dev/null Tue Oct 26 11:13:30 2010
+++ /usr/ports/pobj/gtar-1.24/tar-1.24/tests/testsuite.dir/at-groups/36/stderr
Tue Oct 26 11:13:30 2010
@@ -0,0 +1,2 @@
+./testsuite:
/usr/ports/pobj/gtar-1.24/tar-1.24/tests/testsuite.dir/at-groups/36/test-source[209]:
too many files open in shell
+diff: dest3/a: No such file or directory
./extrac11.at:26: exit code was 1, expected 0
36. extrac11.at:23: 36. scarce file descriptors (extrac11.at:23): FAILED
(extrac11.at:26)
# -*- compilation -*-
53. listed03.at:22: testing ...
./listed03.at:25:
mkdir gnu
(cd gnu
TEST_TAR_FORMAT=gnu
export TEST_TAR_FORMAT
TAR_OPTIONS="-H gnu"
export TAR_OPTIONS
rm -rf *
echo test > 25656
chmod 0 25656
cat 25656 > /dev/null 2>&1
result=0
rm -f 25656
test -eq 0 && exit 77
mkdir dir
mkdir dir/sub
mkdir dir/sub/a
genfile --file dir/sub/a/file
cd dir/sub
chmod a-r ..
tar -c -f archive.tar --listed-incremental=db.1 -v a
status=$?
chmod a+r ..
exit $status
)
Not enabling shell tracing (command contains an embedded newline)
--- - Tue Oct 26 11:14:29 2010
+++ /usr/ports/pobj/gtar-1.24/tar-1.24/tests/testsuite.dir/at-groups/53/stderr
Tue Oct 26 11:14:29 2010
@@ -1,2 +1,5 @@
+tar: Cannot get working directory: Permission denied
tar: a: Directory is new
+tar: Cannot get working directory: Permission denied
+tar: Cannot get working directory: Permission denied
53. listed03.at:22: 53. incremental dump when the parent directory is
unreadable (listed03.at:22): FAILED (listed03.at:25)
--
Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] 1.23: FreeBSD 7 bug affects GNU tar
There is a stupid, stupid bug in FreeBSD 7 (up to and including the 7.3 release): The fdopendir() function is present in libc, but the prototype is missing from . GNU tar's configure picks up fdopendir(), but since there is no prototype, it ends up typed as int fdopendir(); This is bad. fdopendir() returns a pointer, which is truncated from 64 to 32 bits on LP64 platforms. Running GNU tar's "make check" leaves a trail of coredumps behind... The macro complex around m4/dirent*.m4 and m4/fdopendir.m4 needs some sort of check if fdopendir() is declared, and if not, must provide a prototype. These macros are probably used in other GNU projects as well, and bug-tar may not be the right address, but I'm throwing it out there in the hope that it will reach somebody who knows how to deal with this properly. -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: [Bug-tar] 1.23: use after free()
Sergey Poznyakoff: > > The junk-fill malloc() debugging function on BSD reveals that there is > > some sort of use-after-free() bug in 1.23. > > Thanks. Please apply the attached patch. Yes, this fixes it. -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: [Bug-tar] 1.23: use after free()
Sergey Poznyakoff: > > The junk-fill malloc() debugging function on BSD reveals that there is > > some sort of use-after-free() bug in 1.23. > > To analyze this, I need to know the exact command line used to invoke > the tar. What were the options used? tar -c -f archive-1.1.tar -g db.1 -C dir -v --warning=no-new-dir . sub (This is test #44, incr06.at.) -- Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] 1.23: use after free()
The junk-fill malloc() debugging function on BSD reveals that there is some sort of use-after-free() bug in 1.23. Specifically, if you run the test suite with MALLOC_OPTIONS=J set, test 44 (incr06) fails because tar dumps core. (gdb) bt #0 strcmp ( s1=0x20392a0c0 "/usr/obj/gtar-1.23/tar-1.23/tests/testsuite.dir/44/gnu/dir/sub", s2=0xdfdfdfdfdfdfdfdf ) at /usr/src/lib/libc/string/strcmp.c:47 #1 0x0041c236 in name_compare (entry1=0x20af9a780, entry2=0x20af9ad00) at names.c:837 #2 0x0043a9be in hash_find_entry (table=0x20af9ac80, entry=0x20af9a780, bucket_head=0x7f7ef8d0, delete=false) at hash.c:828 #3 0x0043ae4f in hash_insert (table=0x20af9ac80, entry=0x20af9a780) at hash.c:1042 #4 0x0041c62f in collect_and_sort_names () at names.c:970 #5 0x0040bb73 in create_archive () at create.c:1283 #6 0x00424ccf in main (argc=12, argv=0x7f7efa80) at tar.c:2605 This is on OpenBSD where 0xdf is the pattern used to fill free()ed areas. The problem is equally reproducible on FreeBSD. Maybe a missing strdup() somewhere? -- Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] 1.23: link -> ln in tests?
For 1.23, the link02 and link03 regression tests misleadingly fail on OpenBSD, because that platform doesn't provide a link(8) command. Is there a good reason not to use ln(1)? --- tests/link02.at.origThu Mar 25 16:17:52 2010 +++ tests/link02.at Thu Mar 25 16:18:07 2010 @@ -34,9 +34,9 @@ AT_KEYWORDS([hardlinks link02]) AT_TAR_CHECK([ genfile -l 64 -f file1 -link file1 file2 -link file2 file3 -link file3 file4 +ln file1 file2 +ln file2 file3 +ln file3 file4 tar -c -f archive --remove-files file1 file2 file3 file4 tar tfv archive | sed -n 's/.*file[[2-4]] link to //p' ], --- tests/link03.at.origThu Mar 25 16:17:54 2010 +++ tests/link03.at Thu Mar 25 16:18:21 2010 @@ -26,9 +26,9 @@ AT_KEYWORDS([hardlinks link03]) m4_define([create_files],[ genfile -l 64 -f file1 -link file1 file2 -link file2 file3 -link file3 file4 +ln file1 file2 +ln file2 file3 +ln file3 file4 ]) AT_TAR_CHECK([ -- Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] Re: getline() differences FreeBSD / GNU libc
noordsij: > Dear GNU tar maintainer(s) / fBSD gtar port maintainer(s), > The problem appears to be a difference in getline() behaviour in the fBSD > and GNU libc. > So fBSD libc looks at the value of linecapp, whereas GNU libc looks at the > value of lineptr, to determine whether to allocate a new buffer or use a > provided one. In the tar source, linecapp (bufsize) is not initialized, > lineptr (buf) is. FWIW, getline() has been fixed in FreeBSD 9-CURRENT but the merge back to 8.0 was forgotten. > The fix is to simply initialize bufsize to 0 as well, to make (line > 1232-1233): > > char *buf = 0; > size_t bufsize = 0; read_incr_db_01() should have the same fix. -- Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] 1.21: shortrec test failure
For release 1.21, the included regression tests show a new failure: ... 54: working -C with --same-orderok 55: multiple -C options ok 56: short records FAILED (shortrec.at:28) This seems to be caused by unexpected stderr output: tar: Record size = 1 block That change is actually mentioned in NEWS: ** Fixed record size autodetection. If detected record size differs from the expected value (either default, or set on the command line), tar always prints a warning if verbosity level is set to 1 or greater, i.e. if either -t or -v option is given. So I guess the test just needs updating. -- Christian "naddy" Weisgerber na...@mips.inka.de
[Bug-tar] FreeBSD 8: openat() yes, fdopendir() no
GNU tar 1.19 fails to build on FreeBSD 8-CURRENT (starting from 2008-03-31). The problem is code in lib/getcwd.c and lib/savedir.c that mistakenly assumes if openat() is available, so will be fdopendir(). FreeBSD imported the Solaris-style *at() functions, but it does not have fdopendir(), and I don't see how the existence of the one should imply the existence of the other. Apparently the problematic code is shared across GNU projects. At least fileutils is affected as well. -- Christian "naddy" Weisgerber [EMAIL PROTECTED]
Re: [Bug-tar] src/names.c rev 1.64
Sergey Poznyakoff: > > Take tar 1.18, add the rev. 1.63/1.64 change to src/names.c, run > > make test. It will get stuck here: > > > > 19: extracting symlinks over an existing file > > I've tried that. It has passed the testsuite like a charm. My bad, and you are correct. The person who forwarded me that change left in the semicolon after the while(), which effectively created an infinite loop. I failed to notice this when I compared the patch with the Savannah CVS and even when I looked at the code. -- Christian "naddy" Weisgerber [EMAIL PROTECTED]
Re: [Bug-tar] src/names.c rev 1.64
Sergey Poznyakoff: > > really correct? It has been forwarded to me as a security fix for > > 1.18, but there it causes tar to hang when trying to extract any > > symlink from an archive. This can easily be confirmed by running > > the test suite. > > The test suite works for me. Any ways to reproduce? Take tar 1.18, add the rev. 1.63/1.64 change to src/names.c, run make test. It will get stuck here: 19: extracting symlinks over an existing file (I should try a complete HEAD version checked out from CVS, but I haven't gotten around yet to setting up the environment the bootstrap script expects.) -- Christian "naddy" Weisgerber [EMAIL PROTECTED]
[Bug-tar] src/names.c rev 1.64
Is the change in revision 1.64 of src/names.c (contains_dot_dot): Fix double-dot recognition in case of duplicate /. Patch by Dmitry V. Levin. really correct? It has been forwarded to me as a security fix for 1.18, but there it causes tar to hang when trying to extract any symlink from an archive. This can easily be confirmed by running the test suite. -- Christian "naddy" Weisgerber [EMAIL PROTECTED]
[Bug-tar] 1.16.1: T-null.at vs. echo backslash interpretation
The T-null regression test erroneously fails on OpenBSD (and probably some other platforms as well). The problem is that the expected output string ... [jeden\ndwa trzy ], ... is passed on to the echo(1) command in the generated testsuite script: echo >>"$at_stdout"; echo "jeden\\ndwa trzy " | $at_diff - "$at_stdout" || at_failed=: The builtin echo command of OpenBSD's sh performs backslash interpretation by default, producing jeden dwa trzy instead of the expected jeden\ndwa trzy -- Christian "naddy" Weisgerber [EMAIL PROTECTED]
[Bug-tar] 1.15.92: tests/shortrec.at fix
In tar 1.15.92, tests/shortrec.at still assumes that tar was built with DEFAULT_ARCHIVE=- (stdin/out). --- tests/shortrec.at.orig Fri Oct 20 17:35:12 2006 +++ tests/shortrec.at Fri Oct 20 17:35:28 2006 @@ -28,7 +28,7 @@ AT_KEYWORDS([shortrec.at]) AT_TAR_CHECK([ mkdir directory (cd directory && touch a b c d e f g h i j k l m n o p q r) -tar -c -b 1 directory | tar -t -f - >/dev/null +tar -c -b 1 -f - directory | tar -t -f - >/dev/null tar -c -b 1 -f archive directory tar -t -f archive >/dev/null tar -t -f - < archive >/dev/null -- Christian "naddy" Weisgerber [EMAIL PROTECTED] ___ Bug-tar mailing list Bug-tar@gnu.org http://lists.gnu.org/mailman/listinfo/bug-tar
