Re: -current tar(1) breakage
On Sat, Apr 03, 2021 at 01:15:15AM -0400, Christos Zoulas wrote: > Yes, I think that the appropriate change is to make those assertions > so if there is a broken filesystem/syscall there is a more obvious > error (rather than infinite loop in read or core-dump in iconv), but let's > see what joerg thinks about all that. The infinite loops are perfectly reasonable behavior for broken kernel input and found in other tools using the interface. IMO the kernel should always do a sanity cap for puffs/fuse here. The iconv coredump is a buffer overflow, but nothing libarchive can do about. The memory allocated for the dirent is based on the namemax of the filesystem and we kind of have to trust the filesystem to do what it promised. The system call doesn't have a size argument... Joerg
regarding the changes to kernel entropy gathering
So, I'm not sure what to say here. I'm very surprised, quite confused, more than a little perturbed, and even somewhat angry. It's taken me quite some time to write this. Now temper this with knowing that I do know I'm running -current, not a release, and that I accept the challenges this might cause (thus see the patch below). Updating a system, even on -current, shouldn't cause what I can only describe as _intentional_ breakage, even for matters so important as system security and integrity, and especially not without clear mention UPDATING, and perhaps also with documented and referenced tools to assist in undoing said breakage. Updating a system, even on -current, shouldn't create a long-lived situation where the system documentation and the behaviour and actions of system commands is completely out of sync with the behaviour of the kernel, and in fact lies to the administrator about the abilities of the system. In any case, the following patch (and in particular the last hunk) fixes all my problems and complaints in this domain. It is fully tested, and it works A-OK with Xen in both domU and dom0 kernels. My systems once again have consistent documentation, and tools that don't lie, and are able to function as before w.r.t. matters related to /dev/random and getrandom(2). Now I'm not proposing this as the final solution -- I think there's some middle ground to be found, but at least this gets things back to working. --- sys/kern/kern_entropy.c.~1.30.~ 2021-03-07 17:23:05.0 -0800 +++ sys/kern/kern_entropy.c 2021-04-03 11:25:31.667067667 -0700 @@ -1306,7 +1306,7 @@ /* Wait for some entropy to come in and try again. */ KASSERT(E->stage >= ENTROPY_WARM); - printf("entropy: pid %d (%s) blocking due to lack of entropy\n", + printf("entropy: pid %d (%s) blocking due to lack of entropy\n", /* xxx uprintf() instead/also? */ curproc->p_pid, curproc->p_comm); if (ISSET(flags, ENTROPY_SIG)) { @@ -1577,6 +1577,16 @@ KASSERT(i == __arraycount(extra)); entropy_enter(extra, sizeof extra, 0); explicit_memset(extra, 0, sizeof extra); + + aprint_verbose("entropy: %s attached as an entropy source (", rs->name); + if (!(flags & RND_FLAG_NO_COLLECT)) { + printf("collecting"); + if (flags & RND_FLAG_NO_ESTIMATE) + printf(" without estimation"); + } + else + printf("off"); + printf(")\n"); } /* @@ -1610,6 +1620,8 @@ /* Free the per-CPU data. */ percpu_free(rs->state, sizeof(struct rndsource_cpu)); + + aprint_verbose("entropy: %s detached as an entropy source\n", rs->name); } /* @@ -1754,21 +1766,21 @@ rnd_add_uint32(struct krndsource *rs, uint32_t value) { - rnd_add_data(rs, &value, sizeof value, 0); + rnd_add_data(rs, &value, sizeof value, sizeof value * NBBY); } void _rnd_add_uint32(struct krndsource *rs, uint32_t value) { - rnd_add_data(rs, &value, sizeof value, 0); + rnd_add_data(rs, &value, sizeof value, sizeof value * NBBY); } void _rnd_add_uint64(struct krndsource *rs, uint64_t value) { - rnd_add_data(rs, &value, sizeof value, 0); + rnd_add_data(rs, &value, sizeof value, sizeof value * NBBY); } /* -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms pgp_AompQk1f3.pgp Description: OpenPGP Digital Signature
Re: regarding the changes to kernel entropy gathering
Btw i track https://github.com/smuellerDD/jitterentropy-library.git for about two years, and i "never" (which is a couple of years at least) understood why something like this isn't simply used. For example in the myriads of times the scheduler runs each second, a little bit of that can be done on pretty compact local data. Quite honestly speaking, this random and bits-worth shit always annoyed me, not being mathematician, as most terrible pseudo chatter. Incorporating things like rdtsc with applied intermixing etc. to pimpen an entropy that as such is never revealed in order, not to talk about only serving bytes generated through it by cryptographical checked digest algorithms. I at least always mixed low-order/high-order bits. Wow. Yes it is unscientific. But whereas the new OpenSSL RNG mysteriously can fail (which it never did in the past), the Linux kernel now uses a pretty simple (last i looked) such wait-and-mix thing to overcome the seeding-blocks problem. So i (who still uses random-entropy devastating Python Mailman to serve minor MLs) have to use haveged, which, whereas the kernel with the I/O, the network, the process starts, the mapping( addresse)s, the (VM host served) timers etc., generates a bit of random in a second (or something like that, last i looked), generates thousands and thousands of bits of entropy at a glance. That is sick. Hopping off, --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: -current tar(1) breakage
In article , Joerg Sonnenberger wrote: >On Sat, Apr 03, 2021 at 01:15:15AM -0400, Christos Zoulas wrote: >> Yes, I think that the appropriate change is to make those assertions >> so if there is a broken filesystem/syscall there is a more obvious >> error (rather than infinite loop in read or core-dump in iconv), but let's >> see what joerg thinks about all that. > >The infinite loops are perfectly reasonable behavior for broken kernel >input and found in other tools using the interface. IMO the kernel >should always do a sanity cap for puffs/fuse here. Yes, but defensive programming is good. >The iconv coredump is a buffer overflow, but nothing libarchive can do >about. The memory allocated for the dirent is based on the namemax of >the filesystem and we kind of have to trust the filesystem to do what it >promised. The system call doesn't have a size argument... Right, but it can check if the size makes sense before using it. Yes, we expect the OS to return something reasonable in those cases, but we can sanity check the values we compute (for buffers) so that the program does not crash. An assertion in this case would have saved a lot of debugging time. christos
Re: -current tar(1) breakage
On Sat, Apr 03, 2021 at 08:14:34PM -, Christos Zoulas wrote: > In article , Joerg Sonnenberger > wrote: > >On Sat, Apr 03, 2021 at 01:15:15AM -0400, Christos Zoulas wrote: > >> Yes, I think that the appropriate change is to make those assertions > >> so if there is a broken filesystem/syscall there is a more obvious > >> error (rather than infinite loop in read or core-dump in iconv), but let's > >> see what joerg thinks about all that. > > > >The infinite loops are perfectly reasonable behavior for broken kernel > >input and found in other tools using the interface. IMO the kernel > >should always do a sanity cap for puffs/fuse here. > > Yes, but defensive programming is good. > > >The iconv coredump is a buffer overflow, but nothing libarchive can do > >about. The memory allocated for the dirent is based on the namemax of > >the filesystem and we kind of have to trust the filesystem to do what it > >promised. The system call doesn't have a size argument... > > Right, but it can check if the size makes sense before using it. The next filesystem will put one in the field and we are back... Joerg
Re: -current tar(1) breakage
In article , Joerg Sonnenberger wrote: >On Sat, Apr 03, 2021 at 08:14:34PM -, Christos Zoulas wrote: >> In article , Joerg Sonnenberger > wrote: >> >On Sat, Apr 03, 2021 at 01:15:15AM -0400, Christos Zoulas wrote: >> >> Yes, I think that the appropriate change is to make those assertions >> >> so if there is a broken filesystem/syscall there is a more obvious >> >> error (rather than infinite loop in read or core-dump in iconv), but let's >> >> see what joerg thinks about all that. >> > >> >The infinite loops are perfectly reasonable behavior for broken kernel >> >input and found in other tools using the interface. IMO the kernel >> >should always do a sanity cap for puffs/fuse here. >> >> Yes, but defensive programming is good. >> >> >The iconv coredump is a buffer overflow, but nothing libarchive can do >> >about. The memory allocated for the dirent is based on the namemax of >> >the filesystem and we kind of have to trust the filesystem to do what it >> >promised. The system call doesn't have a size argument... >> >> Right, but it can check if the size makes sense before using it. > >The next filesystem will put one in the field and we are back... There is a difference between sub-optimal performance and core-dumping or infinite-looping. christos
how do I mount a read-only filesystem from the "root device" prompt?
So with Xen one can export a "disk" (disk, file, LVM partiion, etc.) with "access=ro", and that is enforced. However if one tries to mount such a disk in a domU as root, it fails. When one first looks at the code which does the initial vfs_mountroot it would appear to be correct -- i.e. it is trying to open the root filesystem device for reading it uses VOP_OPEN() to open the root device with FREAD (which I think means "only for reading"): error = VOP_OPEN(rootvp, FREAD, FSCRED); if (error) { printf("vfs_mountroot: can't open root device, error = %d\n", error); return (error); } However something assumes that if it is like a disk (i.e. but not a CD-ROM/DVD) then it tries to open for write too as we get: root on dk1 vfs_mountroot: can't open root device, error = 30 cannot mount root, error = 30 (errno #30 is of course EROFS) I'm not even sure where this is happening. vfs_rootmountalloc() does indeed set MNT_RDONLY, but this error is happening before vfs_mountroot() calls ffs_mountroot (through the vfs_mountroot pointer). So I'm lost -- any hints? Is it from bounds_check_with_label()? How? -- Greg A. Woods Kelowna, BC +1 250 762-7675 RoboHack Planix, Inc. Avoncote Farms pgp1QhMimCiG1.pgp Description: OpenPGP Digital Signature
Re: -current tar(1) breakage
On Sat, 3 Apr 2021, Christos Zoulas wrote: In article , Joerg Sonnenberger wrote: The infinite loops are perfectly reasonable behavior for broken kernel input and found in other tools using the interface. IMO the kernel should always do a sanity cap for puffs/fuse here. Yes, but defensive programming is good. For the block sizes the libarchive code itself hints at what should be done. In setup_suitable_read_buffer (line 645) the comment says: /* Some platform does not set a proper value to incr_xfer_size. */ 0 is also not a reasonable value. We can just return 512 if we get anything less that 512. For f_namemax, try a pathconf (which will fail on FUSE-ext2), then return MAXNAMLEN if it fails. -RVP
Re: -current tar(1) breakage
On Sat, 3 Apr 2021, RVP wrote: For f_namemax, try a pathconf (which will fail on FUSE-ext2), then return MAXNAMLEN if it fails. Or, avoid allocating fixed-size structures altogether, and do what other tree-walkers do: copy out the fields of interest, or use ftw(3)/fts(3). But, this means rewriting the libarchive code quite a bit... -RVP
daily CVS update output
Updating src tree: P src/doc/3RDPARTY P src/doc/BRANCHES P src/doc/CHANGES P src/external/bsd/top/dist/machine/m_netbsd.c P src/external/mpl/bind/dist/lib/isc/app.c P src/external/mpl/bind/dist/lib/isc/timer.c P src/external/mpl/bind/dist/lib/isc/netmgr/netmgr.c P src/external/mpl/bind/dist/lib/isc/unix/socket.c P src/lib/librumpuser/rumpuser_port.h P src/sys/arch/alpha/alpha/pmap.c P src/sys/arch/alpha/include/lock.h P src/sys/arch/sparc64/doc/TODO P src/sys/arch/sparc64/include/cpu.h P src/sys/arch/sparc64/sparc64/cpu.c P src/sys/arch/sparc64/sparc64/genassym.cf P src/sys/arch/sparc64/sparc64/locore.s P src/sys/compat/common/kern_time_50.c P src/sys/dev/spkr.c P src/sys/dev/spkr_audio.c P src/sys/dev/spkrvar.h P src/sys/dev/isa/spkr_pcppi.c P src/sys/external/bsd/acpica/dist/changes.txt P src/sys/external/bsd/acpica/dist/common/acfileio.c P src/sys/external/bsd/acpica/dist/common/acgetline.c P src/sys/external/bsd/acpica/dist/common/adfile.c P src/sys/external/bsd/acpica/dist/common/adisasm.c P src/sys/external/bsd/acpica/dist/common/adwalk.c P src/sys/external/bsd/acpica/dist/common/ahids.c P src/sys/external/bsd/acpica/dist/common/ahpredef.c P src/sys/external/bsd/acpica/dist/common/ahtable.c P src/sys/external/bsd/acpica/dist/common/ahuuids.c P src/sys/external/bsd/acpica/dist/common/cmfsize.c P src/sys/external/bsd/acpica/dist/common/dmextern.c P src/sys/external/bsd/acpica/dist/common/dmrestag.c P src/sys/external/bsd/acpica/dist/common/dmswitch.c P src/sys/external/bsd/acpica/dist/common/dmtable.c P src/sys/external/bsd/acpica/dist/common/dmtables.c P src/sys/external/bsd/acpica/dist/common/dmtbdump.c P src/sys/external/bsd/acpica/dist/common/dmtbdump1.c P src/sys/external/bsd/acpica/dist/common/dmtbdump2.c P src/sys/external/bsd/acpica/dist/common/dmtbdump3.c P src/sys/external/bsd/acpica/dist/common/dmtbinfo.c P src/sys/external/bsd/acpica/dist/common/dmtbinfo1.c P src/sys/external/bsd/acpica/dist/common/dmtbinfo2.c P src/sys/external/bsd/acpica/dist/common/dmtbinfo3.c P src/sys/external/bsd/acpica/dist/common/getopt.c P src/sys/external/bsd/acpica/dist/compiler/aslallocate.c P src/sys/external/bsd/acpica/dist/compiler/aslanalyze.c P src/sys/external/bsd/acpica/dist/compiler/aslascii.c P src/sys/external/bsd/acpica/dist/compiler/aslbtypes.c P src/sys/external/bsd/acpica/dist/compiler/aslcache.c P src/sys/external/bsd/acpica/dist/compiler/aslcodegen.c P src/sys/external/bsd/acpica/dist/compiler/aslcompile.c P src/sys/external/bsd/acpica/dist/compiler/aslcompiler.h P src/sys/external/bsd/acpica/dist/compiler/aslcompiler.l P src/sys/external/bsd/acpica/dist/compiler/aslcstyle.y P src/sys/external/bsd/acpica/dist/compiler/asldebug.c P src/sys/external/bsd/acpica/dist/compiler/asldefine.h P src/sys/external/bsd/acpica/dist/compiler/aslerror.c P src/sys/external/bsd/acpica/dist/compiler/aslexternal.c P src/sys/external/bsd/acpica/dist/compiler/aslfileio.c P src/sys/external/bsd/acpica/dist/compiler/aslfiles.c P src/sys/external/bsd/acpica/dist/compiler/aslfold.c P src/sys/external/bsd/acpica/dist/compiler/aslglobal.h P src/sys/external/bsd/acpica/dist/compiler/aslhelp.c P src/sys/external/bsd/acpica/dist/compiler/aslhelpers.y P src/sys/external/bsd/acpica/dist/compiler/aslhex.c P src/sys/external/bsd/acpica/dist/compiler/aslkeywords.y P src/sys/external/bsd/acpica/dist/compiler/asllength.c P src/sys/external/bsd/acpica/dist/compiler/asllisting.c P src/sys/external/bsd/acpica/dist/compiler/asllistsup.c P src/sys/external/bsd/acpica/dist/compiler/aslload.c P src/sys/external/bsd/acpica/dist/compiler/asllookup.c P src/sys/external/bsd/acpica/dist/compiler/aslmain.c P src/sys/external/bsd/acpica/dist/compiler/aslmap.c P src/sys/external/bsd/acpica/dist/compiler/aslmapenter.c P src/sys/external/bsd/acpica/dist/compiler/aslmapoutput.c P src/sys/external/bsd/acpica/dist/compiler/aslmaputils.c P src/sys/external/bsd/acpica/dist/compiler/aslmessages.c P src/sys/external/bsd/acpica/dist/compiler/aslmessages.h P src/sys/external/bsd/acpica/dist/compiler/aslmethod.c P src/sys/external/bsd/acpica/dist/compiler/aslnamesp.c P src/sys/external/bsd/acpica/dist/compiler/asloffset.c P src/sys/external/bsd/acpica/dist/compiler/aslopcodes.c P src/sys/external/bsd/acpica/dist/compiler/asloperands.c P src/sys/external/bsd/acpica/dist/compiler/aslopt.c P src/sys/external/bsd/acpica/dist/compiler/asloptions.c P src/sys/external/bsd/acpica/dist/compiler/aslparseop.c P src/sys/external/bsd/acpica/dist/compiler/aslparser.y P src/sys/external/bsd/acpica/dist/compiler/aslpld.c P src/sys/external/bsd/acpica/dist/compiler/aslpredef.c P src/sys/external/bsd/acpica/dist/compiler/aslprepkg.c P src/sys/external/bsd/acpica/dist/compiler/aslprimaries.y P src/sys/external/bsd/acpica/dist/compiler/aslprintf.c P src/sys/external/bsd/acpica/dist/compiler/aslprune.c P src/sys/external/bsd/acpica/dist/compiler/aslresource.c P src/sys/external/bsd/acpica/dist/compiler/aslresources.y P src/sys/external/bsd/acpica/dist/compiler/aslrestype1.c P sr