> Date: Wed, 05 Feb 2014 23:03:09 -0500
> From: Ted Unangst <[email protected]>
>
> On Wed, Feb 05, 2014 at 17:53, Bob Beck wrote:
> > On Wed, Feb 5, 2014 at 3:17 PM, Ted Unangst <[email protected]> wrote:
> >> We are missing back pressure channels from uvm to the buf cache. The
> >> buf cache will happily sit on 9000 free pages while uvm churns around
> >> trying to scavenge up one more page.
>
> > Or are you in a situation here where the cache has *not* backed off?
>
> Talked to Bob and hashed out better ideas of the problem. The page
> daemon does tell the buffer cache to make some room, but...
>
> If you have a huge mmap file, the pdaemon will try to flush it out via
> VOP_WRITE, which circles back via ffs into buf_get, which eats those
> previously freed pages, and then some, as the pagedaemon continues
> pushing more and more of the mmap file out.
>
> We discussed some other changes and fixes that this situation has
> clearly highlighted, but here's a slightly revised diff. It now uses
> the correct bufbackoff() function to communicate uvm's needs. Any
> other fix is rather precarious for this release, but as stated before,
> this keeps the change to the deadlock paths. You were already dead,
> but now you have a second chance.
>
> (We don't currently use the pmemrange argument; we'll have to adjust
> accordingly when the bufcache becomes range aware.)
The approach taken here makes sense to me. And I don't really want to
hold up a fix for this. But there are some things I'd like you to
consider before committing this.
I believe the scenario you sketched should only land you in
uvm_wait_pla(), but not in uvm_wait(). Perhaps with the current code
we can end up in uvm_wait(), but I think those would be bugs where the
driver I/O paths are doing memory allocations when they really
shouldn't. Therefore, I'm not sure we should add the bufbackoff()
call in uvm_wait(). At the very least I'd like to have a printf
*before* the bufbackoff() call there.
I'd also like to see a comment on the bufbackoff() call in
uvm_wait_pla() explaining that even though we pushed back the buffer
cache in the page daemon, we might still consume the freed pages when
paging out dirty pages while scanning.
> Index: uvm_pdaemon.c
> ===================================================================
> RCS file: /cvs/src/sys/uvm/uvm_pdaemon.c,v
> retrieving revision 1.64
> diff -u -p -r1.64 uvm_pdaemon.c
> --- uvm_pdaemon.c 30 May 2013 16:29:46 -0000 1.64
> +++ uvm_pdaemon.c 6 Feb 2014 03:09:53 -0000
> @@ -117,6 +117,8 @@ uvm_wait(const char *wmsg)
> */
>
> if (curproc == uvm.pagedaemon_proc) {
> + if (bufbackoff(NULL, 4) == 0)
> + return;
> /*
> * now we have a problem: the pagedaemon wants to go to
> * sleep until it frees more memory. but how can it
> Index: uvm_pmemrange.c
> ===================================================================
> RCS file: /cvs/src/sys/uvm/uvm_pmemrange.c,v
> retrieving revision 1.36
> diff -u -p -r1.36 uvm_pmemrange.c
> --- uvm_pmemrange.c 29 Jan 2013 19:55:48 -0000 1.36
> +++ uvm_pmemrange.c 6 Feb 2014 03:10:32 -0000
> @@ -22,6 +22,7 @@
> #include <sys/malloc.h>
> #include <sys/proc.h> /* XXX for atomic */
> #include <sys/kernel.h>
> +#include <sys/mount.h>
>
> /*
> * 2 trees: addr tree and size tree.
> @@ -1883,6 +1884,13 @@ uvm_wait_pla(paddr_t low, paddr_t high,
> const char *wmsg = "pmrwait";
>
> if (curproc == uvm.pagedaemon_proc) {
> + uvm_unlock_fpageq();
> + if (bufbackoff(NULL, atop(size)) == 0) {
> + uvm_lock_fpageq();
> + return 0;
> + }
> + uvm_lock_fpageq();
> +
> /*
> * XXX detect pagedaemon deadlock - see comment in
> * uvm_wait(), as this is exactly the same issue.
>
>
