Hi, On Thu, Oct 19, 2023 at 11:20:02AM +0200, Mateusz Guzik wrote: > Running 20 find(1) instances, where each has a "private" tree with > million of files runs into trouble with the kernel killing them (and > others): > [ 785.194378] UVM: pid 1998.1998 (find), uid 0 killed: out of swap > [ 785.194378] UVM: pid 2010.2010 (find), uid 0 killed: out of swap > [ 785.224675] UVM: pid 1771.1771 (top), uid 0 killed: out of swap > [ 785.285291] UVM: pid 1960.1960 (zsh), uid 0 killed: out of swap > [ 785.376172] UVM: pid 2013.2013 (find), uid 0 killed: out of swap > [ 785.416572] UVM: pid 1760.1760 (find), uid 0 killed: out of swap > [ 785.416572] UVM: pid 1683.1683 (tmux), uid 0 killed: out of swap > > This should not be happening -- there is tons of reusable RAM as > virtually all of the vnodes getting here are immediately recyclable. > > $elsewhere I got a report of a workload with hundreds of millions of > files which get walked in parallel -- a number high enough that it > does not fit in RAM on boxes which run it. Out of curiosity I figured > I'll check how others are doing on the front, but key is that this is > not a made up problem.
I can second that. I have had UVM killing my X11 when visiting millions of files; it might have been using rump but I am not sure. What struck me was that swap was maxed out but systat showed something like 40gb as `File'. I haven't looked at the Meta percentage but it wouldn't surpise me if that was also high. Just some random snippet: Anon 2847644 4% zero Exec 378096 % wired 11540 File 41386972 63% inact14002564 Meta 8566989 % bufs 2397524 If a process really starts to take memory I'd say reduce the number of cached inodes and buffered file data first before starting to swap out. > > I'm running NetBSD 10, kernel built from this commit at top of the tree: > Author: andvar <and...@netbsd.org> > Date: Sat Oct 14 08:05:25 2023 +0000 > > fix various typos in comments and documentation, mainly in word "between". > > Specs are 24 cores, 24G of RAM and ufs2 with noatime. swap is *not* > configured. > > Test generates 20 separate trees, each has 1000 directories with 1000 > files (or 20 million files in total + some dirs). > > Repro instructions are here: > https://people.freebsd.org/~mjg/.junk/fstree.tgz I'll take a look in it. Cheers, Reinoud
