On Mon, Jun 29, 2020 at 11:44 AM Bob Beck <b...@obtuse.com> wrote: > On Sun, Jun 28, 2020 at 12:18:06PM -0400, sven falempin wrote: > > On Sun, Jun 28, 2020 at 2:40 AM Bryan Linton <b...@shoshoni.info> wrote: > > > > > On 2020-06-27 19:29:31, Bob Beck <b...@obtuse.com> wrote: > > > > > > > > No. > > > > > > > > I know *exactly* what needbuf is but to attempt to diagnose what your > > > > problem is we need exact details. especially: > > > > > > > > 1) The configuration of your system including all the details of the > > > filesystems > > > > you have mounted, all options used, etc. > > > > > > > > 2) The script you are using to generate the problem (Not a > paraphrasing > > > of what > > > > you think the script does) What filesystems it is using. > > > > > > > > > > Not the OP, but this problem sounds almost exactly like the bug I > > > reported last year. > > > > > > There is a detailed list of steps I used to reproduce the bug in > > > the following bug report. > > > > > > https://marc.info/?l=openbsd-bugs&m=156412299418191 > > > > > > I was even able to bisect and identify the commit which first > > > caused the breakage for me. > > > > > > > > > ---8<--- > > > > > > CVSROOT: /cvs > > > Module name: src > > > Changes by: b...@cvs.openbsd.org 2019/05/08 06:40:57 > > > > > > Modified files: > > > sys/kern : vfs_bio.c vfs_biomem.c > > > > > > Log message: > > > Modify the buffer cache to always flip recovered DMA buffers high. > > > > > > This also modifies the backoff logic to only back off what is requested > > > and not a "mimimum" amount. Tested by me, benno@, tedu@ anda ports > build > > > by naddy@. > > > > > > ok tedu@ > > > > > > ---8<--- > > > > > > However, I have since migrated away from using vnd(4)s since I was > > > able to find other solutions that worked for my use cases. So I > > > may not be able to provide much additional information other than > > > what is contained in the above bug report. > > > > > > -- > > > Bryan > > > > > > > > > > > > > > > > > > > Reproduction of BUG. > > > > > > # optional > > mkdir /tmpfs > > mount_mfs -o rw -s 2500M swap /tmpfs # i mounted through fstab so this > line > > is not tested > > #the bug > > /bin/dd if=/dev/zero of=/tmpfs/img.dd count=0 bs=1 seek=2500000000 > > vnconfig vnd3 /tmpfs/img.dd > > printf "a a\n\n\n\nw\nq\n" | disklabel -E vnd3 > > newfs vnd3a > > mount /dev/vnd3a /mnt > > cd /tmp && ftp https://cdn.openbsd.org/pub/OpenBSD/6.7/amd64/base67.tgz > > cd /mnt > > #will occur here (the mkdir was ub beedbuf state for a while ... > > for v in 1 2 3 4 5 6 7 8 9; do mkdir /tmp/$v; tar xzvf /tmp/base67.tgz -C > > /mnt/$v; done > > > > Ready to test patches. > > > > > > So, your problem is that you have your vnd created in an mfs > filesystem, when I run your test with the vnd backed by a regular > filesystem (withe softdep even) it works fine. > > The trouble happens when your VND has buffers cached in it's > "filesystem" but then is not flushing them out to the underlyin file > (vnode) that you have your vnd backed by. On normal filesystems this > works fine, since vnd tells the lower layer to not cache the writes > and to do them syncrhonously, to avoid an explosion of delayed writes > and dependencies of buffers. > > The problem happens when we convert syncryonous bwrites to > asynchronous bdwrites if the fileystem is mounted ASYNC, which, > curiously, MFS always is (I don't know why, it doesn't really make any > sense, and I might even look at changing that) All the writes you do > end up being delayed anc chewing up more buffer space. And they are > all tied to one vnode (your image). once you exhaust the buffer > space, the cleaner runs, but as you have noticed can't clean out your > vnode until the syncer runs (every 60 seconds). This is why your > thing "takes a long time", and things stall in need buffer. softdep > has deep dark voodoo in it to avoid this problem and therefore when I > use a softdep filesystem instead of an ASYNC filesystem it works. > > Anyway, what's below fixes your issue on my machine. I'm not sure I'm > happy that it's the final fix but it does fix it. there are many other > dragons lurking in here. > > Index: sys/kern/vfs_bio.c > =================================================================== > RCS file: /cvs/src/sys/kern/vfs_bio.c,v > retrieving revision 1.200 > diff -u -p -u -p -r1.200 vfs_bio.c > --- sys/kern/vfs_bio.c 29 Apr 2020 02:25:48 -0000 1.200 > +++ sys/kern/vfs_bio.c 29 Jun 2020 15:18:21 -0000 > @@ -706,8 +706,14 @@ bwrite(struct buf *bp) > */ > async = ISSET(bp->b_flags, B_ASYNC); > if (!async && mp && ISSET(mp->mnt_flag, MNT_ASYNC)) { > - bdwrite(bp); > - return (0); > + /* > + * Don't convert writes from VND on async filesystems > + * that already have delayed writes in the upper layer. > + */ > + if (!ISSET(bp->b_flags, B_NOCACHE)) { > + bdwrite(bp); > + return (0); > + } > } > > /* >
Awesome, thanks! I will test that, ASAP, do not hesitate to slay dragon, i heard the bathing in the blood pool is good for the skin Little concern, I did the test without the MFS and ran into issues , anyway i get back to you (or list ?) when i have test report with patched kernel Again thanks for helping. -- -- --------------------------------------------------------------------------------------------------------------------- Knowing is not enough; we must apply. Willing is not enough; we must do
