Re: -current tar(1) breakage

[RVP]

On Sat, 3 Apr 2021, RVP wrote:


For f_namemax, try a pathconf (which will fail on FUSE-ext2), then
return MAXNAMLEN if it fails.



Or, avoid allocating fixed-size structures altogether, and do what
other tree-walkers do: copy out the fields of interest, or use
ftw(3)/fts(3). But, this means rewriting the libarchive code quite
a bit...

-RVP

Re: -current tar(1) breakage

[RVP]

On Sat, 3 Apr 2021, Christos Zoulas wrote:


In article , Joerg Sonnenberger   wrote:

The infinite loops are perfectly reasonable behavior for broken kernel
input and found in other tools using the interface. IMO the kernel
should always do a sanity cap for puffs/fuse here.


Yes, but defensive programming is good.



For the block sizes the libarchive code itself hints at what should
be done. In setup_suitable_read_buffer (line 645) the comment says:

/* Some platform does not set a proper value to incr_xfer_size. */

0 is also not a reasonable value. We can just return 512 if we get
anything less that 512.

For f_namemax, try a pathconf (which will fail on FUSE-ext2), then
return MAXNAMLEN if it fails.

-RVP

Re: -current tar(1) breakage

[Christos Zoulas]

In article , Joerg Sonnenberger   wrote:
>On Sat, Apr 03, 2021 at 08:14:34PM -, Christos Zoulas wrote:
>> In article , Joerg Sonnenberger 
> wrote:
>> >On Sat, Apr 03, 2021 at 01:15:15AM -0400, Christos Zoulas wrote:
>> >> Yes, I think that the appropriate change is to make those assertions
>> >> so if there is a broken filesystem/syscall there is a more obvious
>> >> error (rather than infinite loop in read or core-dump in iconv), but let's
>> >> see what joerg thinks about all that.
>> >
>> >The infinite loops are perfectly reasonable behavior for broken kernel
>> >input and found in other tools using the interface. IMO the kernel
>> >should always do a sanity cap for puffs/fuse here.
>> 
>> Yes, but defensive programming is good.
>> 
>> >The iconv coredump is a buffer overflow, but nothing libarchive can do
>> >about. The memory allocated for the dirent is based on the namemax of
>> >the filesystem and we kind of have to trust the filesystem to do what it
>> >promised. The system call doesn't have a size argument...
>> 
>> Right, but it can check if the size makes sense before using it.
>
>The next filesystem will put one in the field and we are back...

There is a difference between sub-optimal performance and core-dumping
or infinite-looping.

christos

Re: -current tar(1) breakage

[Joerg Sonnenberger]

On Sat, Apr 03, 2021 at 08:14:34PM -, Christos Zoulas wrote:
> In article , Joerg Sonnenberger   
> wrote:
> >On Sat, Apr 03, 2021 at 01:15:15AM -0400, Christos Zoulas wrote:
> >> Yes, I think that the appropriate change is to make those assertions
> >> so if there is a broken filesystem/syscall there is a more obvious
> >> error (rather than infinite loop in read or core-dump in iconv), but let's
> >> see what joerg thinks about all that.
> >
> >The infinite loops are perfectly reasonable behavior for broken kernel
> >input and found in other tools using the interface. IMO the kernel
> >should always do a sanity cap for puffs/fuse here.
> 
> Yes, but defensive programming is good.
> 
> >The iconv coredump is a buffer overflow, but nothing libarchive can do
> >about. The memory allocated for the dirent is based on the namemax of
> >the filesystem and we kind of have to trust the filesystem to do what it
> >promised. The system call doesn't have a size argument...
> 
> Right, but it can check if the size makes sense before using it.

The next filesystem will put one in the field and we are back...

Joerg

Re: -current tar(1) breakage

[Christos Zoulas]

In article , Joerg Sonnenberger   wrote:
>On Sat, Apr 03, 2021 at 01:15:15AM -0400, Christos Zoulas wrote:
>> Yes, I think that the appropriate change is to make those assertions
>> so if there is a broken filesystem/syscall there is a more obvious
>> error (rather than infinite loop in read or core-dump in iconv), but let's
>> see what joerg thinks about all that.
>
>The infinite loops are perfectly reasonable behavior for broken kernel
>input and found in other tools using the interface. IMO the kernel
>should always do a sanity cap for puffs/fuse here.

Yes, but defensive programming is good.

>The iconv coredump is a buffer overflow, but nothing libarchive can do
>about. The memory allocated for the dirent is based on the namemax of
>the filesystem and we kind of have to trust the filesystem to do what it
>promised. The system call doesn't have a size argument...

Right, but it can check if the size makes sense before using it.

Yes, we expect the OS to return something reasonable in those cases,
but we can sanity check the values we compute (for buffers) so that
the program does not crash. An assertion in this case would have
saved a lot of debugging time.

christos

Re: -current tar(1) breakage

[Joerg Sonnenberger]

On Sat, Apr 03, 2021 at 01:15:15AM -0400, Christos Zoulas wrote:
> Yes, I think that the appropriate change is to make those assertions
> so if there is a broken filesystem/syscall there is a more obvious
> error (rather than infinite loop in read or core-dump in iconv), but let's
> see what joerg thinks about all that.

The infinite loops are perfectly reasonable behavior for broken kernel
input and found in other tools using the interface. IMO the kernel
should always do a sanity cap for puffs/fuse here.

The iconv coredump is a buffer overflow, but nothing libarchive can do
about. The memory allocated for the dirent is based on the namemax of
the filesystem and we kind of have to trust the filesystem to do what it
promised. The system call doesn't have a size argument...

Joerg

Re: -current tar(1) breakage

[Christos Zoulas]

Yes, I think that the appropriate change is to make those assertions
so if there is a broken filesystem/syscall there is a more obvious
error (rather than infinite loop in read or core-dump in iconv), but let's
see what joerg thinks about all that.

Best,

christos

> On Apr 2, 2021, at 5:13 PM, RVP  wrote:
> 
> On Fri, 2 Apr 2021, Christos Zoulas wrote:
> 
>>> f_iosize is also overridden (512 vs. 4k from FUSE-ext2).
>> 
>> I think all that initialization is done here:
>>  https://nxr.netbsd.org/xref/src/lib/libpuffs/subr.c#102
>> and here:
>>  https://nxr.netbsd.org/xref/src/sys/fs/puffs/puffs_vfsops.c#211
>> 
> 
> Yes, I noticed those.
> 
>> I am not sure if the puffs stavfs values are supposed to reflect the
>> values of the underlying filesystem or they are supposed to be standard
>> values for puffs filesystems (which seems to be the case).
>> 
> 
> OK.
> 
>> I believe I have fixed that.
>> 
> 
> Excellent! Where, though? Will you then be changing the libarchive fixes
> to something else? asserts or error-returns from
> setup_current_filesystem() maybe?
> 
> -RVP



signature.asc
Description: Message signed with OpenPGP

Re: -current tar(1) breakage

[RVP]

On Fri, 2 Apr 2021, Christos Zoulas wrote:


f_iosize is also overridden (512 vs. 4k from FUSE-ext2).


I think all that initialization is done here:
https://nxr.netbsd.org/xref/src/lib/libpuffs/subr.c#102
and here:
https://nxr.netbsd.org/xref/src/sys/fs/puffs/puffs_vfsops.c#211



Yes, I noticed those.


I am not sure if the puffs stavfs values are supposed to reflect the
values of the underlying filesystem or they are supposed to be standard
values for puffs filesystems (which seems to be the case).



OK.


I believe I have fixed that.



Excellent! Where, though? Will you then be changing the libarchive fixes
to something else? asserts or error-returns from
setup_current_filesystem() maybe?

-RVP

Re: -current tar(1) breakage

[Christos Zoulas]

Inline below...

> On Apr 2, 2021, at 3:14 AM, RVP  wrote:
> 
> On Thu, 1 Apr 2021, Christos Zoulas wrote:
> 
>>> On Apr 1, 2021, at 7:56 PM, RVP  wrote:
>>> 
>>> But, there is a problem here, though: copy_statvfs_info() overrides
>>> the FUSE-supplied f_namemax (EXT2_NAME_LEN = 255):
>> 
>> Yes, puffs is special... Let me think what's the best way to fix that.
>> 
> 
> f_iosize is also overridden (512 vs. 4k from FUSE-ext2).

I think all that initialization is done here:
https://nxr.netbsd.org/xref/src/lib/libpuffs/subr.c#102
and here:
https://nxr.netbsd.org/xref/src/sys/fs/puffs/puffs_vfsops.c#211

I am not sure if the puffs stavfs values are supposed to reflect the
values of the underlying filesystem or they are supposed to be standard
values for puffs filesystems (which seems to be the case).

> 
>>> 
>>> No idea--didn't look too deeply into this. Are you able to
>>> reproduce this issue?
>>> 
>> 
>> I am trying to follow the steps you described. I don't see the unmount
>> and remount though...
>> 
> 
> I can reliably get a f_namemax of 0 using this script:

I believe I have fixed that.

Best,

christos


signature.asc
Description: Message signed with OpenPGP

Re: -current tar(1) breakage

[RVP]

On Thu, 1 Apr 2021, Christos Zoulas wrote:


On Apr 1, 2021, at 7:56 PM, RVP  wrote:

But, there is a problem here, though: copy_statvfs_info() overrides
the FUSE-supplied f_namemax (EXT2_NAME_LEN = 255):


Yes, puffs is special... Let me think what's the best way to fix that.



f_iosize is also overridden (512 vs. 4k from FUSE-ext2).



No idea--didn't look too deeply into this. Are you able to
reproduce this issue?



I am trying to follow the steps you described. I don't see the unmount
and remount though...



I can reliably get a f_namemax of 0 using this script:

$ cat nfs-f_namemax-err.sh
---START---
#!/bin/sh

cat /etc/exports
# /mnt-network 127.0.0.0  -mask 255.0.0.0

# Mount some (an ext2) partition
sudo mount -r -t ext2fs /dev/dk11 /mnt

# Start NFS server
sudo sh -c 'umask 022; for s in rpcbind mountd nfsd nfslocking
do  /etc/rc.d/$s start
done'

sudo mount -r -t nfs localhost:/mnt /altroot
./a.out /altroot/work/dmesg.txt
# svfs.f_frsize: 512
# svfs.f_iosize: 32768
# svfs.f_bsize: 512
# svfs.f_namemax: 255
#
# OK the first time around...

# Unmount...
#
sudo umount /altroot
sudo umount /mnt
./a.out /altroot/work/dmesg.txt
# a.out: /altroot/work/dmesg.txt: statvfs failed: No such file or directory
#
# As expected...

# Mount again...
#
sudo mount -r -t ext2fs /dev/dk11 /mnt
sudo mount -r -t nfs localhost:/mnt /altroot
./a.out /altroot/work/dmesg.txt
# a.out: /altroot/work/dmesg.txt: statvfs failed: Permission denied
#
# Why `Permission denied' ?

# Notify mountd...
#
sudo pkill -HUP mountd
./a.out /altroot/work/dmesg.txt
# svfs.f_frsize: 512
# svfs.f_iosize: 32768
# svfs.f_bsize: 512
# svfs.f_namemax: 0
#
# Fails now...

sudo umount /altroot
sudo umount /mnt
--END--

Thanks,
RVP

Re: -current tar(1) breakage

[RVP]

On Thu, 1 Apr 2021, Christos Zoulas wrote:


RVP   wrote:


Indeed. Bug #56083 is caused by the wrong function being called:

diff -u /usr/src/sys/fs/puffs/puffs_vfsops.c{.orig,}
--- /usr/src/sys/fs/puffs/puffs_vfsops.c.orig   2020-02-27
22:12:53.0 +
+++ /usr/src/sys/fs/puffs/puffs_vfsops.c2021-03-30
12:02:35.402544154 +
@@ -503,7 +503,7 @@
 */
copy_statvfs_info(sbp, mp);
if (!error) {
-   statvfs_to_puffs_statvfs(sbp, _msg->pvfsr_sb);
+   puffs_statvfs_to_statvfs(_msg->pvfsr_sb, sbp);
}


This is not exactly right either, I fixed it.


Yes. What I did above is wrong. copy_statvfs_info() should come after
puffs_statvfs_to_statvfs(). Otherwise, you end up with a mount with
empty to and from names--yeesh.

But, there is a problem here, though: copy_statvfs_info() overrides
the FUSE-supplied f_namemax (EXT2_NAME_LEN = 255):

$ ./a.out /mnt/work/dmesg.txt 
svfs.f_frsize: 4096

svfs.f_iosize: 512
svfs.f_bsize: 4096
svfs.f_namemax: 511 <-- not EXT2_NAME_LEN


The statvfs-filling sequence starting at line 219 doesn't look right
either:

219 struct statvfs *sb = STATVFSBUF_GET();
220 puffs_statvfs_to_statvfs(>pa_svfsb, sb);
221 copy_statvfs_info(sb, mp);
222 STATVFSBUF_PUT(sb);
223
224 statvfs_to_puffs_statvfs(>mnt_stat, >pa_svfsb);


Why?



Well, because, it seems to me, that the values copied into `sb'
are just discarded. This confused me when I saw it first because
the puffs statvfs copy functions reverse the usual convention of
having a (dest, src) argument order--they have (src, dest).
copy_statvfs_info() has it the usual way.


The question is where does NFS return a 0 namelen...



No idea--didn't look too deeply into this. Are you able to
reproduce this issue?

-RVP

Re: -current tar(1) breakage

[Christos Zoulas]

> On Apr 1, 2021, at 7:56 PM, RVP  wrote:
> 
> On Thu, 1 Apr 2021, Christos Zoulas wrote:
> 
>> RVP   wrote:
>>> 
>>> Indeed. Bug #56083 is caused by the wrong function being called:
>>> 
>>> diff -u /usr/src/sys/fs/puffs/puffs_vfsops.c{.orig,}
>>> --- /usr/src/sys/fs/puffs/puffs_vfsops.c.orig   2020-02-27
>>> 22:12:53.0 +
>>> +++ /usr/src/sys/fs/puffs/puffs_vfsops.c2021-03-30
>>> 12:02:35.402544154 +
>>> @@ -503,7 +503,7 @@
>>> */
>>>copy_statvfs_info(sbp, mp);
>>>if (!error) {
>>> -   statvfs_to_puffs_statvfs(sbp, _msg->pvfsr_sb);
>>> +   puffs_statvfs_to_statvfs(_msg->pvfsr_sb, sbp);
>>>}
>> 
>> This is not exactly right either, I fixed it.
> 
> Yes. What I did above is wrong. copy_statvfs_info() should come after
> puffs_statvfs_to_statvfs(). Otherwise, you end up with a mount with
> empty to and from names--yeesh.
> 
> But, there is a problem here, though: copy_statvfs_info() overrides
> the FUSE-supplied f_namemax (EXT2_NAME_LEN = 255):

Yes, puffs is special... Let me think what's the best way to fix that.

> 
> $ ./a.out /mnt/work/dmesg.txt svfs.f_frsize: 4096
> svfs.f_iosize: 512
> svfs.f_bsize: 4096
> svfs.f_namemax: 511 <-- not EXT2_NAME_LEN
> 
>>> The statvfs-filling sequence starting at line 219 doesn't look right
>>> either:
>>> 
>>>219 struct statvfs *sb = STATVFSBUF_GET();
>>>220 puffs_statvfs_to_statvfs(>pa_svfsb, sb);
>>>221 copy_statvfs_info(sb, mp);
>>>222 STATVFSBUF_PUT(sb);
>>>223
>>>224 statvfs_to_puffs_statvfs(>mnt_stat, >pa_svfsb);
>> 
>> Why?
>> 
> 
> Well, because, it seems to me, that the values copied into `sb'
> are just discarded. This confused me when I saw it first because
> the puffs statvfs copy functions reverse the usual convention of
> having a (dest, src) argument order--they have (src, dest).
> copy_statvfs_info() has it the usual way.

Yes, when I wrote them I was debating about the order of the arguments:
follow the function name order or destination first...

>> The question is where does NFS return a 0 namelen...
>> 
> 
> No idea--didn't look too deeply into this. Are you able to
> reproduce this issue?
> 

I am trying to follow the steps you described. I don't see the unmount
and remount though...

christos



signature.asc
Description: Message signed with OpenPGP

Re: -current tar(1) breakage

[Christos Zoulas]

In article ,
RVP   wrote:
>On Thu, 1 Apr 2021, Joerg Sonnenberger wrote:
>
>> This should be fixed in the kernel. Seriously, stop adding more code to
>> deal with garbage from the kernel.
>>
>
>Indeed. Bug #56083 is caused by the wrong function being called:
>
>diff -u /usr/src/sys/fs/puffs/puffs_vfsops.c{.orig,}
>--- /usr/src/sys/fs/puffs/puffs_vfsops.c.orig   2020-02-27
>22:12:53.0 +
>+++ /usr/src/sys/fs/puffs/puffs_vfsops.c2021-03-30
>12:02:35.402544154 +
>@@ -503,7 +503,7 @@
>  */
> copy_statvfs_info(sbp, mp);
> if (!error) {
>-   statvfs_to_puffs_statvfs(sbp, _msg->pvfsr_sb);
>+   puffs_statvfs_to_statvfs(_msg->pvfsr_sb, sbp);
> }

This is not exactly right either, I fixed it.

>The statvfs-filling sequence starting at line 219 doesn't look right
>either:
>
> 219 struct statvfs *sb = STATVFSBUF_GET();
> 220 puffs_statvfs_to_statvfs(>pa_svfsb, sb);
> 221 copy_statvfs_info(sb, mp);
> 222 STATVFSBUF_PUT(sb);
> 223
> 224 statvfs_to_puffs_statvfs(>mnt_stat, >pa_svfsb);

Why?

The question is where does NFS return a 0 namelen...

christos

Re: -current tar(1) breakage

[Hauke Fath]

On Wed, 31 Mar 2021 17:56:16 -0400, Christos Zoulas wrote:
> What is the NFS server? Because on NetBSD it returns 255...

netbsd-9 (amd64).

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[RVP]

On Thu, 1 Apr 2021, Joerg Sonnenberger wrote:


This should be fixed in the kernel. Seriously, stop adding more code to
deal with garbage from the kernel.



Indeed. Bug #56083 is caused by the wrong function being called:

diff -u /usr/src/sys/fs/puffs/puffs_vfsops.c{.orig,}
--- /usr/src/sys/fs/puffs/puffs_vfsops.c.orig   2020-02-27 22:12:53.0 
+
+++ /usr/src/sys/fs/puffs/puffs_vfsops.c2021-03-30 12:02:35.402544154 
+
@@ -503,7 +503,7 @@
 */
copy_statvfs_info(sbp, mp);
if (!error) {
-   statvfs_to_puffs_statvfs(sbp, _msg->pvfsr_sb);
+   puffs_statvfs_to_statvfs(_msg->pvfsr_sb, sbp);
}

PUFFS_MSG_RELEASE(statvfs);


The statvfs-filling sequence starting at line 219 doesn't look right
either:

219 struct statvfs *sb = STATVFSBUF_GET();
220 puffs_statvfs_to_statvfs(>pa_svfsb, sb);
221 copy_statvfs_info(sb, mp);
222 STATVFSBUF_PUT(sb);
223
224 statvfs_to_puffs_statvfs(>mnt_stat, >pa_svfsb);

-RVP

Re: -current tar(1) breakage

[RVP]

On Wed, 31 Mar 2021, Christos Zoulas wrote:


What is the NFS server? Because on NetBSD it returns 255...



$ uname -a
NetBSD x202e.localdomain 9.99.81 NetBSD 9.99.81 (GENERIC) #0: Sat Mar 27 
02:58:39 UTC 2021  
mkre...@mkrepro.netbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC amd64

It doesn't happen usually. I was only able to reproduce Hauke's SEGV
by mounting an ext2 FS on /mnt; exporting /mnt via NFS as /altroot;
then unmounting and remounting /mnt:

$ sudo mount -r -t ext2fs /dev/dk11 /mnt
$ sudo mount -r -t nfs localhost:/mnt /altroot
$ mount
/dev/dk9 on / type ffs (log, noatime, local)
kernfs on /kern type kernfs (local)
ptyfs on /dev/pts type ptyfs (local)
procfs on /proc type procfs (local)
tmpfs on /var/shm type tmpfs (local)
tmpfs on /tmp type tmpfs (local)
/dev/dk11 on /mnt type ext2fs (read-only, local)
localhost:/mnt on /altroot type nfs (read-only)
$ /tmp/a.out /altroot/work/dmesg.txt
a.out: /altroot/work/dmesg.txt: statvfs failed: Permission denied
$ sudo pkill -HUP mountd
$ /tmp/a.out /altroot/work/dmesg.txt
svfs.f_frsize: 512
svfs.f_iosize: 32768
svfs.f_bsize: 512
svfs.f_namemax: 0
$

-RVP

Re: -current tar(1) breakage

[Joerg Sonnenberger]

On Wed, Mar 31, 2021 at 05:56:16PM -0400, Christos Zoulas wrote:
> What is the NFS server? Because on NetBSD it returns 255...

This should be fixed in the kernel. Seriously, stop adding more code to
deal with garbage from the kernel.

Joerg

Re: -current tar(1) breakage

[Christos Zoulas]

What is the NFS server? Because on NetBSD it returns 255...



statvfs.c
Description: Binary data


christos

> On Mar 31, 2021, at 5:21 PM, RVP  wrote:
> 
> On Sat, 27 Mar 2021, Hauke Fath wrote:
> 
>> [...]
>> Reading symbols from /bin/tar...
>> Reading symbols from /usr/libdata/debug//bin/tar.debug...
>> [New process 10317]
>> Core was generated by `tar'.
>> Program terminated with signal SIGSEGV, Segmentation fault.
>> #0  0x7f0a03467866 in _citrus_iconv_convert
>> (nresults=0x7f7fff902d18, flags=0, outbytes=0x7f7fff902d98,
>> out=0x7f7fff902d90,
>>   inbytes=0x7f7fff902d88, in=0x7f7fff902d80, cv=0x7f0a057c40e0) at
>> /usr/src/lib/libc/citrus/citrus_iconv.h:61
>> 61   _DIAGASSERT(cv && cv->cv_shared && cv->cv_shared->ci_ops &&
>> (gdb) bt
>> #0  0x7f0a03467866 in _citrus_iconv_convert
>> (nresults=0x7f7fff902d18, flags=0, outbytes=0x7f7fff902d98,
>> out=0x7f7fff902d90,
>>   inbytes=0x7f7fff902d88, in=0x7f7fff902d80, cv=0x7f0a057c40e0) at
>> /usr/src/lib/libc/citrus/citrus_iconv.h:61
>> #1  _iconv (handle=handle@entry=0x7f0a057c40e0,
>> in=in@entry=0x7f7fff902d80, szin=szin@entry=0x7f7fff902d88,
>> out=out@entry=0x7f7fff902d90,
>>   szout=szout@entry=0x7f7fff902d98) at
>> /usr/src/lib/libc/iconv/iconv.c:97
>> 
> 
> This iconv() trace is very misleading. The bug triggered by the
> NFS server returning 0 for .f_namemax in a statvfs() call.
> 
> Fix:
> ---START---
> diff -urN a/libarchive/dist/libarchive/archive_read_disk_posix.c 
> b/libarchive/dist/libarchive/archive_read_disk_posix.c
> --- a/libarchive/dist/libarchive/archive_read_disk_posix.c2019-07-24 
> 13:50:23.0 +
> +++ b/libarchive/dist/libarchive/archive_read_disk_posix.c2021-03-31 
> 12:01:37.437510048 +
> @@ -1713,7 +1713,7 @@
>   t->current_filesystem->noatime = 0;
> 
>   /* Set maximum filename length. */
> - t->current_filesystem->name_max = sfs.f_namemax;
> + t->current_filesystem->name_max = (sfs.f_namemax == 0) ? NAME_MAX : 
> sfs.f_namemax;
>   return (ARCHIVE_OK);
> }
> 
> ---END---
> 
> Isn't the NFS server supposed to return NFS_MAXNAMLEN?
> 
> -RVP



signature.asc
Description: Message signed with OpenPGP

Re: -current tar(1) breakage

[RVP]

On Sat, 27 Mar 2021, Hauke Fath wrote:


[...]
Reading symbols from /bin/tar...
Reading symbols from /usr/libdata/debug//bin/tar.debug...
[New process 10317]
Core was generated by `tar'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x7f0a03467866 in _citrus_iconv_convert
(nresults=0x7f7fff902d18, flags=0, outbytes=0x7f7fff902d98,
out=0x7f7fff902d90,
   inbytes=0x7f7fff902d88, in=0x7f7fff902d80, cv=0x7f0a057c40e0) at
/usr/src/lib/libc/citrus/citrus_iconv.h:61
61  _DIAGASSERT(cv && cv->cv_shared && cv->cv_shared->ci_ops &&
(gdb) bt
#0  0x7f0a03467866 in _citrus_iconv_convert
(nresults=0x7f7fff902d18, flags=0, outbytes=0x7f7fff902d98,
out=0x7f7fff902d90,
   inbytes=0x7f7fff902d88, in=0x7f7fff902d80, cv=0x7f0a057c40e0) at
/usr/src/lib/libc/citrus/citrus_iconv.h:61
#1  _iconv (handle=handle@entry=0x7f0a057c40e0,
in=in@entry=0x7f7fff902d80, szin=szin@entry=0x7f7fff902d88,
out=out@entry=0x7f7fff902d90,
   szout=szout@entry=0x7f7fff902d98) at
/usr/src/lib/libc/iconv/iconv.c:97



This iconv() trace is very misleading. The bug triggered by the
NFS server returning 0 for .f_namemax in a statvfs() call.

Fix:
---START---
diff -urN a/libarchive/dist/libarchive/archive_read_disk_posix.c 
b/libarchive/dist/libarchive/archive_read_disk_posix.c
--- a/libarchive/dist/libarchive/archive_read_disk_posix.c  2019-07-24 
13:50:23.0 +
+++ b/libarchive/dist/libarchive/archive_read_disk_posix.c  2021-03-31 
12:01:37.437510048 +
@@ -1713,7 +1713,7 @@
t->current_filesystem->noatime = 0;

/* Set maximum filename length. */
-   t->current_filesystem->name_max = sfs.f_namemax;
+   t->current_filesystem->name_max = (sfs.f_namemax == 0) ? NAME_MAX : 
sfs.f_namemax;
return (ARCHIVE_OK);
 }

---END---

Isn't the NFS server supposed to return NFS_MAXNAMLEN?

-RVP

Re: -current tar(1) breakage

[RVP]

On Tue, 30 Mar 2021, Joerg Sonnenberger wrote:

This is a bug in the FUSE layer. It should really sanitize the input and 
not force it on any consumer.




Returning 0 for some of the fields, esp. f_iosize--which is what's
the cause of bug #56083--is not wrong, I think. According to statvfs(5):

struct statvfs {
...
unsigned long f_iosize;
...
};

Returning a -1 for that field is probably not right, even if that's what
libarchive seems to expect. And, POSIX doesn't say anythng about these
implementation-specific fields.

-RVP

Re: -current tar(1) breakage

[Joerg Sonnenberger]

On Sun, Mar 28, 2021 at 06:49:33AM +, RVP wrote:
> This might, or might not, be related. There is another bug in
> tar. It hangs when reading files on FUSE-mounted filesystems:

This is a bug in the FUSE layer. It should really sanitize the input and
not force it on any consumer.

Joerg

Re: -current tar(1) breakage

[Hauke Fath]

On Sun, 28 Mar 2021 17:24:50 +0100, Robert Swindells wrote:
>> 
>> Interesting. Is that net-booted, or just the pkgsrc tree over nfs?
> 
> Just pkgsrc over NFS.
> 
> Does your netbooted machine have a swapfile on NFS ?

No, with 32 GB RAM there is no need, and the server is space limited.

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Robert Swindells]

Hauke Fath  wrote:
>On Sun, 28 Mar 2021 14:28:52 +0100, Robert Swindells wrote:
>> The unmodified package builds fine for me over NFS, with -current as
>> both client and server.
>
>Interesting. Is that net-booted, or just the pkgsrc tree over nfs?

Just pkgsrc over NFS.

Does your netbooted machine have a swapfile on NFS ?

Re: -current tar(1) breakage

[RVP]

On Sun, 28 Mar 2021, Hauke Fath wrote:


On Sun, 28 Mar 2021 06:49:33 + (UTC), RVP wrote:

Run the code on your NFS-mounted FS and check what FS sizes are
reported. If they are all 0, then it is the same bug (probably).


[hauke@i386-pxe] ~/src/tar-fssize > ./a.out 
/var/obj/pkgsrc/editors/xemacs-nox11/work/xemacs-21.4.24/lisp/console.el

sfs.f_frsize: 512
sfs.f_iosize: 32768
sfs.f_bsize: 512
[hauke@i386-pxe] ~/src/tar-fssize >



Ah...I found me a new bug then. Try the patch anyway.

The other machines I have access to all run Windows, so no NFS--
far less NetBSD--on them. Tried with a loopback NFS mount and tar 
worked just fine. Even with your xemacs-21.4 files. Other files with

Kanji/Hangul filenames also worked fine (even with a "C" locale).

-RVP

Re: -current tar(1) breakage

[Hauke Fath]

On Sun, 28 Mar 2021 14:28:52 +0100, Robert Swindells wrote:
> On Fri, 26 Mar 2021 15:16:13 +0100, Hauke Fath wrote:
>> building editors/xemacs-nox11 on a -current machine (net-booted, if it 
>> matters) fails [...]
> 
> Have you stated anywhere in the thread what the NFS server is running ?

netbsd-9 (Jan 5 userland and kernel of this week's sources).

> The unmodified package builds fine for me over NFS, with -current as
> both client and server.

Interesting. Is that net-booted, or just the pkgsrc tree over nfs?

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Robert Swindells]

On Fri, 26 Mar 2021 15:16:13 +0100, Hauke Fath wrote:
> building editors/xemacs-nox11 on a -current machine (net-booted, if it 
> matters) fails [...]

Have you stated anywhere in the thread what the NFS server is running ?

The unmodified package builds fine for me over NFS, with -current as
both client and server.

Robert Swindells

Re: -current tar(1) breakage

[RVP]

This might, or might not, be related. There is another bug in
tar. It hangs when reading files on FUSE-mounted filesystems:

$ sudo fuse-ext2 -o ro -o direct -o uid=$(id -u) -o gid=$(id -g) /dev/dk5 /mnt
$ tar -C /mnt/work -cvf /dev/null .
a .
a ./dmesg.txt^C <-- HANGS HERE
$

However, this works:
$ tar -cvf /dev/null /mnt/work
tar: Removing leading '/' from member names
a mnt/work
a mnt/work/dmesg.txt
a mnt/work/alc.txt
...
$

This bug is caused by {,f}statvfs() in setup_current_filesystem()
returning 0 sizes on FUSE-mounted FSes; and then setup_suitable_read_buffer()
only doing a if (var < 0) check in various places.

$ cat fssize.c
#include 
#include 
#include 
#include 
#include 

int
main(int argc, char* argv[])
{
struct statvfs sfs;
char* file;
int rc = EXIT_FAILURE;

if (argc != 2)
errx(rc, "no args. given");
file = argv[1];
if (statvfs(file, ) == -1)
err(rc, "%s: statvfs failed", file);

#define PR(v) printf("%s: %ld\n", #v, v)

PR(sfs.f_frsize);
PR(sfs.f_iosize);
PR(sfs.f_bsize);

return 0;
}
$ cc -Wall fssize.c
$ ./a.out /mnt/work/dmesg.txt
sfs.f_frsize: 0
sfs.f_iosize: 0
sfs.f_bsize: 0
$

Run the code on your NFS-mounted FS and check what FS sizes are
reported. If they are all 0, then it is the same bug (probably).

The patch below fixes my issue for me (on NetBSD--similar changes
may be needed on other OSes supported by libarchive)

---START PATCH---
diff -urN a/external/bsd/libarchive/dist/libarchive/archive_read_disk_posix.c 
b/external/bsd/libarchive/dist/libarchive/archive_read_disk_posix.c
--- a/external/bsd/libarchive/dist/libarchive/archive_read_disk_posix.c 
2019-07-24 13:50:23.0 +
+++ b/external/bsd/libarchive/dist/libarchive/archive_read_disk_posix.c 
2021-03-28 06:35:01.828041075 +
@@ -1690,14 +1690,14 @@
} else if (xr == 1) {
/* Usually come here unless NetBSD supports _PC_REC_XFER_ALIGN
 * for pathconf() function. */
-   t->current_filesystem->xfer_align = sfs.f_frsize;
+   t->current_filesystem->xfer_align = (sfs.f_frsize == 0) ? -1 : 
(long)sfs.f_frsize;
t->current_filesystem->max_xfer_size = -1;
 #if defined(HAVE_STRUCT_STATVFS_F_IOSIZE)
-   t->current_filesystem->min_xfer_size = sfs.f_iosize;
-   t->current_filesystem->incr_xfer_size = sfs.f_iosize;
+   t->current_filesystem->min_xfer_size = (sfs.f_iosize == 0) ? -1 
: (long)sfs.f_iosize;
+   t->current_filesystem->incr_xfer_size = (sfs.f_iosize == 0) ? 
-1 : (long)sfs.f_iosize;
 #else
-   t->current_filesystem->min_xfer_size = sfs.f_bsize;
-   t->current_filesystem->incr_xfer_size = sfs.f_bsize;
+   t->current_filesystem->min_xfer_size = (sfs.f_bsize == 0) ? -1 
: (long)sfs.f_bsize;
+   t->current_filesystem->incr_xfer_size = (sfs.f_bsize == 0) ? -1 
: (long)sfs.f_bsize;
 #endif
}
if (sfs.f_flag & ST_LOCAL)
---END PATCH---

-RVP

Re: -current tar(1) breakage

[Hauke Fath]

On Fri, 26 Mar 2021 15:16:13 +0100, Hauke Fath wrote:
> building editors/xemacs-nox11 on a -current machine (net-booted, if it 
> matters) fails [...]

See PR bin/56080.

As a workaround, I have adapted editors/xemacs* to use gtar, instead.

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Hauke Fath]

On Sun, 28 Mar 2021 06:49:33 + (UTC), RVP wrote:
> Run the code on your NFS-mounted FS and check what FS sizes are
> reported. If they are all 0, then it is the same bug (probably).

[hauke@i386-pxe] ~/src/tar-fssize > ./a.out 
/var/obj/pkgsrc/editors/xemacs-nox11/work/xemacs-21.4.24/lisp/console.el
sfs.f_frsize: 512
sfs.f_iosize: 32768
sfs.f_bsize: 512
[hauke@i386-pxe] ~/src/tar-fssize > 

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Hauke Fath]

On Sat, 27 Mar 2021 18:34:27 -0400, Christos Zoulas wrote:
> I didn't expect output, but I wanted to look at the pointer data. 
> Does it contain 0xa5 now instead of other random strings?

Ah. You mean *cv? No, same value:

[hauke@i386-pxe] /<6>xemacs-21.4.24/lisp > env MALLOC_CONF='junk:true' 
tar -cf /dev/null .
Segmentation fault (core dumped)
[hauke@i386-pxe] /<6>xemacs-21.4.24/lisp > gdb /bin/tar tar.core
GNU gdb (GDB) 11.0.50.20200914-git
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64--netbsd".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /bin/tar...
Reading symbols from /usr/libdata/debug//bin/tar.debug...
[New process 6992]
Core was generated by `tar'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x6fe46fc67866 in _citrus_iconv_convert 
(nresults=0x7f7fffe70f48, flags=0, outbytes=0x7f7fffe70fc8, 
out=0x7f7fffe70fc0, 
inbytes=0x7f7fffe70fb8, in=0x7f7fffe70fb0, cv=0x6fe471f0e0e0) at 
/usr/src/lib/libc/citrus/citrus_iconv.h:61
61  _DIAGASSERT(cv && cv->cv_shared && cv->cv_shared->ci_ops &&
(gdb) print *cv
$1 = {cv_shared = 0x6c652e73, cv_closure = 0x6fe471bcd050}
(gdb)  print *cv->cv_shared
Cannot access memory at address 0x6c652e73
(gdb) 


-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Christos Zoulas]

I didn't expect output, but I wanted to look at the pointer data. Does it 
contain 0xa5 now instead of other random strings?

christos

> On Mar 27, 2021, at 6:06 PM, Hauke Fath  
> wrote:
> 
> On Sat, 27 Mar 2021 17:04:58 -0400, Christos Zoulas wrote:
>> Maybe we are accessing freed memory? Can you run it with:
>> 
>> env MALLOC_CONF='junk:true'
> 
> No output:
> 
> [hauke@i386-pxe] /<6>xemacs-21.4.24/lisp > env MALLOC_CONF='junk:true'
> tar -cf /dev/null .
> Segmentation fault (core dumped)
> [hauke@i386-pxe] /<6>xemacs-21.4.24/lisp >
> 
> --
> Hauke Fath
> Linnéweg 7
> 64342 Seeheim-Jugenheim
> Germany



signature.asc
Description: Message signed with OpenPGP

Re: -current tar(1) breakage

[Hauke Fath]

On Sat, 27 Mar 2021 17:04:58 -0400, Christos Zoulas wrote:
> Maybe we are accessing freed memory? Can you run it with:
> 
> env MALLOC_CONF='junk:true'

No output:

[hauke@i386-pxe] /<6>xemacs-21.4.24/lisp > env MALLOC_CONF='junk:true' 
tar -cf /dev/null .
Segmentation fault (core dumped)
[hauke@i386-pxe] /<6>xemacs-21.4.24/lisp > 

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Havard Eidnes]

>> (gdb) print *cv
>> $1 = {cv_shared = 0x6c652e73, cv_closure = 0x761713184050}
>> (gdb) print *cv->shared
>> There is no member named shared.
>> (gdb) print *cv->cv_shared
>> Cannot access memory at address 0x6c652e73
>> (gdb) print *cv->cv_shared->ci_ops
>> Cannot access memory at address 0x6c652e73
>> (gdb)
>
> 0x6c652e73 == "s.el" (ascii) so it sounds like something
> uninitialized/overwritten.

Hmm...  I thought there had been made progress to make the
address sanitizer feature of gcc and/or clang work on NetBSD?
(ref. "gcc -fsanitize=address")

However, my previous attempt at using that feature on netbsd-9
was unfortunately not met with success (I tried with gcc) -- I
ended up with "rpl_malloc()" as undefined -- we don't define that
but it's supposedly defined in a Linux-based environment...

I didn't try with clang for the program I was looking at, and
I've also not tried on -current.

In principle, tar ought to be a simpler program to get to run
than the multi-threaded program I was trying to get going...

Regards,

- Håvard

Re: -current tar(1) breakage

[Christos Zoulas]

Maybe we are accessing freed memory? Can you run it with:

env MALLOC_CONF='junk:true'

christos

> On Mar 27, 2021, at 1:51 PM, Anders Magnusson  wrote:
> 
> 
>> (gdb) print *cv
>> $1 = {cv_shared = 0x6c652e73, cv_closure = 0x761713184050}
>> (gdb) print *cv->shared
>> There is no member named shared.
>> (gdb) print *cv->cv_shared
>> Cannot access memory at address 0x6c652e73
>> (gdb) print *cv->cv_shared->ci_ops
>> Cannot access memory at address 0x6c652e73
>> (gdb)
> 
> 0x6c652e73 == "s.el" (ascii) so it sounds like something 
> uninitialized/overwritten.
> 
> -- R



signature.asc
Description: Message signed with OpenPGP

re: -current tar(1) breakage

[matthew green]

> Joerg thinks that this is an nfs issue (a bug with nfs giving incorrect data).

even if true, tar shouldn't *core dump*.  is there a path
to RCE here some where?  it's clearly overwriting pointers
with strings, so unless someone can clearly show there is
no code exec vector here, it seems potentially problematic
and should be fixed.


.mrg.

Re: -current tar(1) breakage

[Hauke Fath]

On Sat, 27 Mar 2021 18:02:32 +0100, Hauke Fath wrote:
> On Sat, 27 Mar 2021 12:27:20 -0400, Christos Zoulas wrote:
>> Joerg thinks that this is an nfs issue (a bug with nfs giving 
>> incorrect data).
> 
> Doesn't do that in other locations, though: [/var/tmp]

Maybe it does:

[hauke@i386-pxe] /var/log > tar -cvf /dev/null .
a .
a ./rdist
a ./amdlog
a ./authlog[hauke@i386-pxe] /var/log >

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Anders Magnusson]

(gdb) print *cv
$1 = {cv_shared = 0x6c652e73, cv_closure = 0x761713184050}
(gdb) print *cv->shared
There is no member named shared.
(gdb) print *cv->cv_shared
Cannot access memory at address 0x6c652e73
(gdb) print *cv->cv_shared->ci_ops
Cannot access memory at address 0x6c652e73
(gdb)


0x6c652e73 == "s.el" (ascii) so it sounds like something 
uninitialized/overwritten.

-- R

Re: -current tar(1) breakage

[Hauke Fath]

On Sat, 27 Mar 2021 12:27:20 -0400, Christos Zoulas wrote:
> Joerg thinks that this is an nfs issue (a bug with nfs giving 
> incorrect data).

Doesn't do that in other locations, though:

[hauke@i386-pxe] /var/tmp > tar -cvf /dev/null .
a .
a ./vi.recover
a ./pkg_rr.out
a ./pizza-etc.tgz
[hauke@i386-pxe] /var/tmp > 

> Nevertheless can you
> 
> print *cv,
> print *cv->shared
> print *cv->shared->ci_ops

(gdb) print *cv
$1 = {cv_shared = 0x6c652e73, cv_closure = 0x761713184050}
(gdb) print *cv->shared
There is no member named shared.
(gdb) print *cv->cv_shared
Cannot access memory at address 0x6c652e73
(gdb) print *cv->cv_shared->ci_ops
Cannot access memory at address 0x6c652e73
(gdb) 

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Christos Zoulas]

Joerg thinks that this is an nfs issue (a bug with nfs giving incorrect data).
Nevertheless can you

print *cv,
print *cv->shared
print *cv->shared->ci_ops

Thanks,

christos

> On Mar 27, 2021, at 12:12 PM, Hauke Fath  
> wrote:
> 
> On Sat, 27 Mar 2021 09:22:16 -0400, Christos Zoulas wrote:
>> Can't you just download the pre-build ones? Assuming you are using
>> reproducible builds, we might get lucky?
> 
> [...]
> Reading symbols from /bin/tar...
> Reading symbols from /usr/libdata/debug//bin/tar.debug...
> [New process 10317]
> Core was generated by `tar'.
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0  0x7f0a03467866 in _citrus_iconv_convert
> (nresults=0x7f7fff902d18, flags=0, outbytes=0x7f7fff902d98,
> out=0x7f7fff902d90,
>inbytes=0x7f7fff902d88, in=0x7f7fff902d80, cv=0x7f0a057c40e0) at
> /usr/src/lib/libc/citrus/citrus_iconv.h:61
> 61_DIAGASSERT(cv && cv->cv_shared && cv->cv_shared->ci_ops &&
> (gdb) bt
> #0  0x7f0a03467866 in _citrus_iconv_convert
> (nresults=0x7f7fff902d18, flags=0, outbytes=0x7f7fff902d98,
> out=0x7f7fff902d90,
>inbytes=0x7f7fff902d88, in=0x7f7fff902d80, cv=0x7f0a057c40e0) at
> /usr/src/lib/libc/citrus/citrus_iconv.h:61
> #1  _iconv (handle=handle@entry=0x7f0a057c40e0,
> in=in@entry=0x7f7fff902d80, szin=szin@entry=0x7f7fff902d88,
> out=out@entry=0x7f7fff902d90,
>szout=szout@entry=0x7f7fff902d98) at
> /usr/src/lib/libc/iconv/iconv.c:97
> #2  0x7f0a05483834 in iconv_strncat_in_locale (as=0x7f0a05762158,
> _p=, length=, sc=0x7f0a056eb000)
>at
> /usr/src/external/bsd/libarchive/dist/libarchive/archive_string.c:2039
> #3  0x7f0a054844ba in archive_strncat_l
> (as=as@entry=0x7f0a05762158, _p=, n=,
> sc=)
>at
> /usr/src/external/bsd/libarchive/dist/libarchive/archive_string.c:1998
> #4  0x7f0a0548459a in archive_strncpy_l
> (as=as@entry=0x7f0a05762158, _p=, n=,
> sc=)
>at
> /usr/src/external/bsd/libarchive/dist/libarchive/archive_string.c:1944
> #5  0x7f0a054847d9 in archive_mstring_get_mbs_l
> (aes=0x7f0a05762110, p=0x7f7fff902eb8, length=0x7f7fff902ee8,
> sc=)
>at
> /usr/src/external/bsd/libarchive/dist/libarchive/archive_string.c:4005
> #6  0x7f0a0547b810 in _archive_entry_pathname_l (entry= out>, p=, len=, sc=)
>at
> /usr/src/external/bsd/libarchive/dist/libarchive/archive_entry.c:598
> #7  0x7f0a0541f051 in get_entry_pathname (a=0x7f0a057b6000,
> entry=, name=, length=,
>sc=) at
> /usr/src/external/bsd/libarchive/dist/libarchive/archive_write_set_format_pax.c:428
> #8  0x7f0a0541f5da in archive_write_pax_header (a=0x7f0a057b6000,
> entry_original=0x7f0a05761500)
>at
> /usr/src/external/bsd/libarchive/dist/libarchive/archive_write_set_format_pax.c:839
> #9  0x7f0a0546a1da in _archive_write_header (_a=0x7f0a057b6000,
> entry=0x7f0a05761500)
>at
> /usr/src/external/bsd/libarchive/dist/libarchive/archive_write.c:650
> #10 0x7b008ac2 in write_entry
> (bsdtar=bsdtar@entry=0x7f7fff9037f0, a=a@entry=0x7f0a057b6000,
> entry=0x7f0a05761500)
>at /usr/src/external/bsd/libarchive/dist/tar/write.c:974
> #11 0x7b008cac in write_file (entry=,
> a=0x7f0a057b6000, bsdtar=0x7f7fff9037f0)
>at /usr/src/external/bsd/libarchive/dist/tar/write.c:962
> #12 write_hierarchy (bsdtar=bsdtar@entry=0x7f7fff9037f0,
> a=a@entry=0x7f0a057b6000, path=path@entry=0x7f7fff9042f2 ".")
>at /usr/src/external/bsd/libarchive/dist/tar/write.c:941
> #13 0x7b00907e in write_archive (a=a@entry=0x7f0a057b6000,
> bsdtar=bsdtar@entry=0x7f7fff9037f0)
>at /usr/src/external/bsd/libarchive/dist/tar/write.c:513
> #14 0x7b0098a4 in tar_mode_c
> (bsdtar=bsdtar@entry=0x7f7fff9037f0) at
> /usr/src/external/bsd/libarchive/dist/tar/write.c:247
> #15 0x7b00ba4d in main (argc=, argv= out>) at /usr/src/external/bsd/libarchive/dist/tar/bsdtar.c:910
> (gdb)
> 
> HTH,
> Hauke
> 
> --
> Hauke Fath
> Linnéweg 7
> 64342 Seeheim-Jugenheim
> Germany



signature.asc
Description: Message signed with OpenPGP

Re: -current tar(1) breakage

[Hauke Fath]

On Sat, 27 Mar 2021 09:22:16 -0400, Christos Zoulas wrote:
> Can't you just download the pre-build ones? Assuming you are using 
> reproducible builds, we might get lucky?

Running 'ktrace -di tar -cf /dev/null .' from the same directory 
(ditors/xemacs-nox11/work/xemacs-21.4.24/lisp):

% kdump
[...]
 14308  14308 tar  GIO   fd 8 read 1728 bytes
   ";;; msw-select.el --- Lisp interface to mswindows 
selections.\n\n;; Copyright (C) 1990, 1997 Free Software Foundation, 
Inc.\n;; Copyright (C) \
1995 Sun Microsystems.\n\n;; Maintainer: XEmacs Development 
Team\n;; Keywords: extensions, dumped\n\n;; This file is part of 
XEmacs.\n\n;; XEm\
acs is free software; you can redistribute it and/or modify 
it\n;; under the terms of the GNU General Public License as published 
by\n;; the F\
ree Software Foundation; either version 2, or (at your 
option)\n;; any later version.\n\n;; XEmacs is distributed in the hope 
that it will be \
useful, but\n;; WITHOUT ANY WARRANTY; without even the implied 
warranty of\n;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  
See the G\
NU\n;; General Public License for more details.\n\n;; You 
should have received a copy of the GNU General Public License\n;; along 
with XEmacs;\
 see the file COPYING.  If not, write to the \n;; Free Software 
Foundation, Inc., 59 Temple Place - Suite 330,\n;; Boston, MA 
02111-1307, USA.\
\n\n;;; Synched up with: Not in FSF\n\n;;; Commentary:\n\n;; 
This file is dumped with XEmacs (when mswindows support is compiled 
in).\n;; \
 Only copes with copying/pasting text\n\n;;; Code:\n\n(defun 
mswindows-paste-clipboard ()\n  \"Insert the current contents of the 
mswindows cl\
ipboard at point,\nreplacing the active selection if there is 
one.\"\n  (interactive \"*\")\n  (setq last-command nil)\n  (setq 
this-command '\
yank) ; so that yank-pop works.\n  (let ((clip (get-clipboard)) 
(s (mark-marker)) (e (point-marker)))\n(or clip (error \"there is 
no text \
on the clipboard\"))\n(if s\n   (if 
mouse-track-rectangle-p\n   (delete-rectangle s e)\n  
(delete-region s e)))\n(push-mar\
k)\n(if mouse-track-rectangle-p\n   (insert-rectangle 
clip)\n  (insert clip\n\n\n\n\n"
 14308  14308 tar  RET   read 1728/0x6c0
 14308  14308 tar  CALL  close(8)
 14308  14308 tar  RET   close 0
 14308  14308 tar  CALL  
fstatat(6,0x761713209062,0x76171320a300,0x200)
 14308  14308 tar  NAMI  "gtk-widget-accessors.el"
 14308  14308 tar  RET   fstatat 0
 14308  14308 tar  CALL  fchdir(6)
 14308  14308 tar  RET   fchdir 0
 14308  14308 tar  CALL  openat(6,0x761713209100,4,0x761710eb0c9a)
 14308  14308 tar  NAMI  "gtk-widget-accessors.el"
 14308  14308 tar  RET   openat 8
 14308  14308 tar  CALL  __acl_get_fd(8,4,0x761712de2000)
 14308  14308 tar  RET   __acl_get_fd -1 errno 45 Operation not 
supported
 14308  14308 tar  CALL  __acl_get_fd(8,2,0x7617130d8000)
 14308  14308 tar  RET   __acl_get_fd -1 errno 45 Operation not 
supported
 14308  14308 tar  CALL  extattr_list_fd(8,1,0,0)
 14308  14308 tar  RET   extattr_list_fd -1 errno 45 Operation not 
supported
 14308  14308 tar  CALL  close(8)
 14308  14308 tar  RET   close 0
 14308  14308 tar  CALL  fchdir(4)
 14308  14308 tar  RET   fchdir 0
 14308  14308 tar  PSIG  SIGSEGV SIG_DFL: code=SEGV_MAPERR, 
addr=0x6c652e73, trap=6)
 14308  14308 tar  NAMI  "tar.core"
%

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Hauke Fath]

On Sat, 27 Mar 2021 09:22:16 -0400, Christos Zoulas wrote:
> Can't you just download the pre-build ones? Assuming you are using 
> reproducible builds, we might get lucky?

[...]
Reading symbols from /bin/tar...
Reading symbols from /usr/libdata/debug//bin/tar.debug...
[New process 10317]
Core was generated by `tar'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x7f0a03467866 in _citrus_iconv_convert 
(nresults=0x7f7fff902d18, flags=0, outbytes=0x7f7fff902d98, 
out=0x7f7fff902d90, 
inbytes=0x7f7fff902d88, in=0x7f7fff902d80, cv=0x7f0a057c40e0) at 
/usr/src/lib/libc/citrus/citrus_iconv.h:61
61  _DIAGASSERT(cv && cv->cv_shared && cv->cv_shared->ci_ops &&
(gdb) bt
#0  0x7f0a03467866 in _citrus_iconv_convert 
(nresults=0x7f7fff902d18, flags=0, outbytes=0x7f7fff902d98, 
out=0x7f7fff902d90, 
inbytes=0x7f7fff902d88, in=0x7f7fff902d80, cv=0x7f0a057c40e0) at 
/usr/src/lib/libc/citrus/citrus_iconv.h:61
#1  _iconv (handle=handle@entry=0x7f0a057c40e0, 
in=in@entry=0x7f7fff902d80, szin=szin@entry=0x7f7fff902d88, 
out=out@entry=0x7f7fff902d90, 
szout=szout@entry=0x7f7fff902d98) at 
/usr/src/lib/libc/iconv/iconv.c:97
#2  0x7f0a05483834 in iconv_strncat_in_locale (as=0x7f0a05762158, 
_p=, length=, sc=0x7f0a056eb000)
at 
/usr/src/external/bsd/libarchive/dist/libarchive/archive_string.c:2039
#3  0x7f0a054844ba in archive_strncat_l 
(as=as@entry=0x7f0a05762158, _p=, n=, 
sc=)
at 
/usr/src/external/bsd/libarchive/dist/libarchive/archive_string.c:1998
#4  0x7f0a0548459a in archive_strncpy_l 
(as=as@entry=0x7f0a05762158, _p=, n=, 
sc=)
at 
/usr/src/external/bsd/libarchive/dist/libarchive/archive_string.c:1944
#5  0x7f0a054847d9 in archive_mstring_get_mbs_l 
(aes=0x7f0a05762110, p=0x7f7fff902eb8, length=0x7f7fff902ee8, 
sc=)
at 
/usr/src/external/bsd/libarchive/dist/libarchive/archive_string.c:4005
#6  0x7f0a0547b810 in _archive_entry_pathname_l (entry=, p=, len=, sc=)
at 
/usr/src/external/bsd/libarchive/dist/libarchive/archive_entry.c:598
#7  0x7f0a0541f051 in get_entry_pathname (a=0x7f0a057b6000, 
entry=, name=, length=, 
sc=) at 
/usr/src/external/bsd/libarchive/dist/libarchive/archive_write_set_format_pax.c:428
#8  0x7f0a0541f5da in archive_write_pax_header (a=0x7f0a057b6000, 
entry_original=0x7f0a05761500)
at 
/usr/src/external/bsd/libarchive/dist/libarchive/archive_write_set_format_pax.c:839
#9  0x7f0a0546a1da in _archive_write_header (_a=0x7f0a057b6000, 
entry=0x7f0a05761500)
at 
/usr/src/external/bsd/libarchive/dist/libarchive/archive_write.c:650
#10 0x7b008ac2 in write_entry 
(bsdtar=bsdtar@entry=0x7f7fff9037f0, a=a@entry=0x7f0a057b6000, 
entry=0x7f0a05761500)
at /usr/src/external/bsd/libarchive/dist/tar/write.c:974
#11 0x7b008cac in write_file (entry=, 
a=0x7f0a057b6000, bsdtar=0x7f7fff9037f0)
at /usr/src/external/bsd/libarchive/dist/tar/write.c:962
#12 write_hierarchy (bsdtar=bsdtar@entry=0x7f7fff9037f0, 
a=a@entry=0x7f0a057b6000, path=path@entry=0x7f7fff9042f2 ".")
at /usr/src/external/bsd/libarchive/dist/tar/write.c:941
#13 0x7b00907e in write_archive (a=a@entry=0x7f0a057b6000, 
bsdtar=bsdtar@entry=0x7f7fff9037f0)
at /usr/src/external/bsd/libarchive/dist/tar/write.c:513
#14 0x7b0098a4 in tar_mode_c 
(bsdtar=bsdtar@entry=0x7f7fff9037f0) at 
/usr/src/external/bsd/libarchive/dist/tar/write.c:247
#15 0x7b00ba4d in main (argc=, argv=) at /usr/src/external/bsd/libarchive/dist/tar/bsdtar.c:910
(gdb) 

HTH,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Christos Zoulas]

Can't you just download the pre-build ones? Assuming you are using reproducible 
builds,
we might get lucky?

christos

> On Mar 27, 2021, at 9:09 AM, Hauke Fath  
> wrote:
> 
> On Fri, 26 Mar 2021 20:08:40 +0100, Hauke Fath wrote:
>> On Fri, 26 Mar 2021 14:41:56 -0400, Christos Zoulas wrote:
>>> Can you install the debug sets?
>> 
>> Will do - have to build them first, though.
> 
> Sorry, the build died. It wants more than 20 GB for objects, which I
> currently don't have on that machine...
> 
> Cheerio,
> Hauke
> 
> --
> Hauke Fath
> Linnéweg 7
> 64342 Seeheim-Jugenheim
> Germany



signature.asc
Description: Message signed with OpenPGP

Re: -current tar(1) breakage

[Hauke Fath]

On Fri, 26 Mar 2021 20:08:40 +0100, Hauke Fath wrote:
> On Fri, 26 Mar 2021 14:41:56 -0400, Christos Zoulas wrote:
>> Can you install the debug sets?
> 
> Will do - have to build them first, though.

Sorry, the build died. It wants more than 20 GB for objects, which I 
currently don't have on that machine...

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Hauke Fath]

On Fri, 26 Mar 2021 14:41:56 -0400, Christos Zoulas wrote:
> Can you install the debug sets?

Will do - have to build them first, though.

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Christos Zoulas]

On Mar 26,  6:52pm, ha...@espresso.rhein-neckar.de (Hauke Fath) wrote:
-- Subject: Re: -current tar(1) breakage

| On Fri, 26 Mar 2021 16:28:38 - (UTC), Christos Zoulas wrote:
| > What does the core file show?
| 
| Reading symbols from /usr/bin/tar...
| (No debugging symbols found in /usr/bin/tar)
| [New process 29527]
| Core was generated by `tar'.
| Program terminated with signal SIGSEGV, Segmentation fault.
| #0  0x74e0e8667866 in iconv () from /lib/libc.so.12
| (gdb) bt
| #0  0x74e0e8667866 in iconv () from /lib/libc.so.12
| #1  0x74e0ea683834 in ?? () from /usr/lib/libarchive.so.4
| #2  0x74e0ea6844ba in archive_strncat_l () from=20
| /usr/lib/libarchive.so.4
| #3  0x74e0ea6847d9 in archive_mstring_get_mbs_l () from=20
| /usr/lib/libarchive.so.4
| #4  0x74e0ea61f051 in ?? () from /usr/lib/libarchive.so.4
| #5  0x74e0ea61f5da in ?? () from /usr/lib/libarchive.so.4
| #6  0x74e0ea66a1da in ?? () from /usr/lib/libarchive.so.4
| #7  0x000155a08ac2 in write_entry ()
| #8  0x000155a08cac in write_hierarchy ()
| #9  0x000155a0907e in write_archive ()
| #10 0x000155a098a4 in tar_mode_c ()
| #11 0x000155a0ba4d in main ()
| (gdb)

Can you install the debug sets?

| Also, from a 'ktrace -di':
| 
| [...]
|  29527  29527 tar  CALL  __acl_get_fd(7,4,0x74e0ea5dc000)
|  29527  29527 tar  RET   __acl_get_fd -1 errno 45 Operation not=20
| supported
|  29527  29527 tar  CALL =20
| mmap(0,0x5000,PROT_READ|PROT_WRITE,0x1002,0=
| x,0,0)
|  29527  29527 tar  RET   mmap 128509353488384/0x74e0ea5d7000
|  29527  29527 tar  CALL  munmap(0x74e0ea5d7000,0x5000)
|  29527  29527 tar  RET   munmap 0
|  29527  29527 tar  CALL =20
| mmap(0,0x6000,PROT_READ|PROT_WRITE,0xd001002=
| ,0x,0,0)
|  29527  29527 tar  RET   mmap 128509353484288/0x74e0ea5d6000
|  29527  29527 tar  CALL  munmap(0x74e0ea5db000,0x1000)
|  29527  29527 tar  RET   munmap 0
|  29527  29527 tar  CALL  __acl_get_fd(7,2,0x74e0ea5d6000)
|  29527  29527 tar  RET   __acl_get_fd -1 errno 45 Operation not=20
| supported
|  29527  29527 tar  CALL  extattr_list_fd(7,1,0,0)
|  29527  29527 tar  RET   extattr_list_fd -1 errno 45 Operation not=20
| supported
|  29527  29527 tar  CALL  close(7)
|  29527  29527 tar  RET   close 0
|  29527  29527 tar  CALL  fchdir(3)
|  29527  29527 tar  RET   fchdir 0
|  29527  29527 tar  PSIG  SIGSEGV SIG_DFL: code=3DSEGV_MAPERR,=20
| addr=3D0x68, trap=3D6)
|  29527  29527 tar  NAMI  "tar.core"

I was hoping that this would show the filename but it does not.

Thanks,

christos

Re: -current tar(1) breakage

[Hauke Fath]

On Fri, 26 Mar 2021 18:56:09 +0100, Martin Husemann wrote:
> On Fri, Mar 26, 2021 at 06:52:04PM +0100, Hauke Fath wrote:
>> Program terminated with signal SIGSEGV, Segmentation fault.
>> #0  0x74e0e8667866 in iconv () from /lib/libc.so.12
>> (gdb) bt
>> #0  0x74e0e8667866 in iconv () from /lib/libc.so.12
> 
> What are your locale settings?

None set - this is a pretty plain -current amd64 installation with 
about a dozen packages installed, adapted for netboot.

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Hauke Fath]

On Fri, 26 Mar 2021 18:52:04 +0100, Hauke Fath wrote:
> Also, from a 'ktrace -di':

FTR, This is from a plain 'ktrace -di /usr/bin/tar -cf - . > /dev/null' 
on an nfs volume.

Works with GNU tar(1).

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Martin Husemann]

On Fri, Mar 26, 2021 at 06:52:04PM +0100, Hauke Fath wrote:
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0  0x74e0e8667866 in iconv () from /lib/libc.so.12
> (gdb) bt
> #0  0x74e0e8667866 in iconv () from /lib/libc.so.12

What are your locale settings?

Martin

Re: -current tar(1) breakage

[Hauke Fath]

On Fri, 26 Mar 2021 15:34:21 +, Robert Swindells wrote:
> The package builds fine for me on -current, what is unusual about your
> build host ?

It is net-booted. As soon as I put WRKOBJDIR on a local disk, I am fine.

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Hauke Fath]

On Fri, 26 Mar 2021 16:28:38 - (UTC), Christos Zoulas wrote:
> What does the core file show?

Reading symbols from /usr/bin/tar...
(No debugging symbols found in /usr/bin/tar)
[New process 29527]
Core was generated by `tar'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x74e0e8667866 in iconv () from /lib/libc.so.12
(gdb) bt
#0  0x74e0e8667866 in iconv () from /lib/libc.so.12
#1  0x74e0ea683834 in ?? () from /usr/lib/libarchive.so.4
#2  0x74e0ea6844ba in archive_strncat_l () from 
/usr/lib/libarchive.so.4
#3  0x74e0ea6847d9 in archive_mstring_get_mbs_l () from 
/usr/lib/libarchive.so.4
#4  0x74e0ea61f051 in ?? () from /usr/lib/libarchive.so.4
#5  0x74e0ea61f5da in ?? () from /usr/lib/libarchive.so.4
#6  0x74e0ea66a1da in ?? () from /usr/lib/libarchive.so.4
#7  0x000155a08ac2 in write_entry ()
#8  0x000155a08cac in write_hierarchy ()
#9  0x000155a0907e in write_archive ()
#10 0x000155a098a4 in tar_mode_c ()
#11 0x000155a0ba4d in main ()
(gdb)

Also, from a 'ktrace -di':

[...]
 29527  29527 tar  CALL  __acl_get_fd(7,4,0x74e0ea5dc000)
 29527  29527 tar  RET   __acl_get_fd -1 errno 45 Operation not 
supported
 29527  29527 tar  CALL  
mmap(0,0x5000,PROT_READ|PROT_WRITE,0x1002,0x,0,0)
 29527  29527 tar  RET   mmap 128509353488384/0x74e0ea5d7000
 29527  29527 tar  CALL  munmap(0x74e0ea5d7000,0x5000)
 29527  29527 tar  RET   munmap 0
 29527  29527 tar  CALL  
mmap(0,0x6000,PROT_READ|PROT_WRITE,0xd001002,0x,0,0)
 29527  29527 tar  RET   mmap 128509353484288/0x74e0ea5d6000
 29527  29527 tar  CALL  munmap(0x74e0ea5db000,0x1000)
 29527  29527 tar  RET   munmap 0
 29527  29527 tar  CALL  __acl_get_fd(7,2,0x74e0ea5d6000)
 29527  29527 tar  RET   __acl_get_fd -1 errno 45 Operation not 
supported
 29527  29527 tar  CALL  extattr_list_fd(7,1,0,0)
 29527  29527 tar  RET   extattr_list_fd -1 errno 45 Operation not 
supported
 29527  29527 tar  CALL  close(7)
 29527  29527 tar  RET   close 0
 29527  29527 tar  CALL  fchdir(3)
 29527  29527 tar  RET   fchdir 0
 29527  29527 tar  PSIG  SIGSEGV SIG_DFL: code=SEGV_MAPERR, 
addr=0x68, trap=6)
 29527  29527 tar  NAMI  "tar.core"

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Hauke Fath]

On Fri, 26 Mar 2021 16:03:53 +0100, Thomas Klausner wrote:
> On Fri, Mar 26, 2021 at 03:16:13PM +0100, Hauke Fath wrote:
>> building editors/xemacs-nox11 on a -current machine (net-booted, if it 
>> matters) fails with
> 
> Just as a data point, this packages fine for me on 9.99.81/amd64 (with
> a local file system).

Confirmed. So it's tar(1) on NFS...

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Robert Swindells]

Hauke Fath  wrote:
>On Fri, 26 Mar 2021 14:38:08 +, Robert Swindells wrote:
>>> The workaround was to symlink /usr/pkg/bin/gtar to .tools/bin/tar. Is 
>>> there any pkgsrc magic for making gtar pose as tar(1), or am I on my 
>>> own?
>> 
>> You can put a line in the pkgsrc Makefile of:
>> 
>> EXTRACT_USING=  gtar
>
>I've seen (and used) EXTRACT_USING. But does this just cover extracts 
>by the pkgsrc framework, or also any invocations of tar by the 
>package's build system?

Don't know.

The package builds fine for me on -current, what is unusual about your
build host ?

Re: -current tar(1) breakage

[Hauke Fath]

On Fri, 26 Mar 2021 14:38:08 +, Robert Swindells wrote:
>> The workaround was to symlink /usr/pkg/bin/gtar to .tools/bin/tar. Is 
>> there any pkgsrc magic for making gtar pose as tar(1), or am I on my 
>> own?
> 
> You can put a line in the pkgsrc Makefile of:
> 
> EXTRACT_USING=  gtar

I've seen (and used) EXTRACT_USING. But does this just cover extracts 
by the pkgsrc framework, or also any invocations of tar by the 
package's build system?

Cheerio,
Hauke

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany

Re: -current tar(1) breakage

[Thomas Klausner]

On Fri, Mar 26, 2021 at 03:16:13PM +0100, Hauke Fath wrote:
> building editors/xemacs-nox11 on a -current machine (net-booted, if it 
> matters) fails with

Just as a data point, this packages fine for me on 9.99.81/amd64 (with
a local file system).
 Thomas

Re: -current tar(1) breakage

[Robert Swindells]

Hauke Fath  wrote:
>building editors/xemacs-nox11 on a -current machine (net-booted, if it 
>matters) fails with

[snip]

>The workaround was to symlink /usr/pkg/bin/gtar to .tools/bin/tar. Is 
>there any pkgsrc magic for making gtar pose as tar(1), or am I on my 
>own?

You can put a line in the pkgsrc Makefile of:

EXTRACT_USING=  gtar

or:

EXTRACT_USING=  bsdtar

I think the 'bsdtar' option will build it from pkgsrc not use tar(1)
from the base OS.

-current tar(1) breakage

[Hauke Fath]

Hi,

building editors/xemacs-nox11 on a -current machine (net-booted, if it 
matters) fails with

[...]
Copying /var/obj/pkgsrc/editors/xemacs-nox11/work/xemacs-21.4.24/etc...
./TUTORIAL.ko: Truncated tar archive
tar: Error exit delayed from previous errors.
[1]   Segmentation fault (core dumped) (cd ${dir} && tar -cf - .) |
  Done(1) (cd ${dest} && umask 022 && tar -xf -)
Copying /var/obj/pkgsrc/editors/xemacs-nox11/work/xemacs-21.4.24/lisp...
./syntax.el: Truncated tar archive
tar: Error exit delayed from previous errors.
[1]   Segmentation fault (core dumped) (cd ${dir} && tar -cf - .) |
  Done(1) (cd ${dest} && umask 022 && tar -xf -)

This works on netbsd-9. What has changed in the -current tar?

The workaround was to symlink /usr/pkg/bin/gtar to .tools/bin/tar. Is 
there any pkgsrc magic for making gtar pose as tar(1), or am I on my 
own?

Cheerio,
Hauke 

-- 
Hauke Fath
Linnéweg 7
64342 Seeheim-Jugenheim
Germany