Re: Maxphys on -current?

[Thor Lancelot Simon]

On Fri, Aug 04, 2023 at 09:48:18PM +0200, Jarom??r Dole??ek wrote:
> 
> For the branch, I particularly disliked that there were quite a few
> changes which looked either unrelated, or avoidable.

There should not have been any "unrelated" changes.  I would not be
surprised if there were changes that could have been avoided.

It has been a very, very long time, but I think there are a few things
worth noting that I discovered in the course of the work I did on this
years ago.

1) It really is important to propagate maximum-transfer-size information
   down the bus hierarchy, because we have many cases where the same device
   could be connected to a different bus.

2) RAIDframe and its ilk are tough to get right, because there are many ugly
   corner cases such as someone trying to replace a failed component of a
   RAID set with a spare that is attached via a bus that has a smaller
   transfer size limit.  Ensuring both that this doesn't happen and that
   errors are propagated back correctly is pretty hard.  I have a vague
   recollection that this might be one source of the "unrelated" changes you
   mention.

3) With MAXPHYS increased to a very large value, we have filesystem code that
   can behave very poorly because it uses naive readahead or write clustering
   strategies that were only previously held in check by the 64K MAXPHYS
   limit.  I didn't even make a start at handling this, honestly, and the
   aparrent difficult of getting it right is one reason I eventually decided
   I didn't have time to finish the work I started on the tls-maxphys branch.
   Beware!  Don't trust linear-read or linear-write benchmarks that say your
   work in this area is done.  You may have massacred performance for real
   world use cases other than your own.

One thing we should probably do, if we have not already, is remove any ISA
DMA devices and some old things like the wdc and pciide IDE attachments from
the GENERIC kernels for ports like amd64, and then bump MAXPHYS to at least
128K, maybe 256K, for those kernels.  Beyond that, though, I think you will
quickly see the filesystem and paging misbehaviors I mention in #3 above.

Thor

pagedaemon spins due to KVM shortage on NetBSD/i386 10.0_BETA?

[Izumi Tsutsui]

Recently I observed excessive CPU loads due to pagedaemon spin
on NetBSD/i386 10.0_BETA GENERIC from daily snapshot:

---
% uname -a
NetBSD mirage 10.0_BETA NetBSD 10.0_BETA (GENERIC) #0: Thu Jul 27 18:10:25 UTC 
2023  mkre...@mkrepro.netbsd.org:/usr/src/sys/arch/i386/compile/GENERIC i386
% top -n
load averages:  1.23,  1.24,  1.10;   up 6+17:26:3722:13:37
107 processes: 18 runnable, 84 sleeping, 2 stopped, 3 on CPU
CPU states:  9.6% user,  0.0% nice, 27.2% system,  0.5% interrupt, 62.6% idle
Memory: 2001M Act, 980M Inact, 73M Wired, 170M Exec, 964M File, 27M Free
Swap: 8972M Total, 1403M Used, 7569M Free

  PID USERNAME PRI NICE   SIZE   RES STATE  TIME   WCPUCPU COMMAND
0 root 1260 0K   35M CPU/2423:09 99.02% 99.02% [system]
 7312 tsutsui   430  2250M  598M CPU/3100:13 14.89% 14.89% firefox
25088 tsutsui   410   712M  179M parked/1 123:01 10.50% 10.50% firefox
15769 tsutsui   430   815M  203M parked/0  17:03  0.49%  0.49% firefox
 8897 tsutsui   430   595M  139M parked/3   1:09  0.39%  0.39% firefox
16559 tsutsui   430   460M   37M parked/0   0:01  0.10%  0.10% firefox
  833 tsutsui   430   153M   58M RUN/3 61:31  0.00%  0.00% X
 2943 tsutsui   85082M 6488K select/0   6:49  0.00%  0.00% jwm
21097 tsutsui   850   728M  108M RUN/0  4:33  0.00%  0.00% firefox
 3434 tsutsui   85087M 4652K poll/1 3:31  0.00%  0.00% pulseaudio
[...]

% top -t -n
load averages:  0.89,  1.17,  1.12;   up 6+17:33:2522:20:25
839 threads: 20 idle, 4 runnable, 784 sleeping, 2 stopped, 27 zombie, 2 on CPU
CPU states:  7.5% user,  0.0% nice, 27.9% system,  0.2% interrupt, 64.2% idle
Memory: 2003M Act, 980M Inact, 80M Wired, 170M Exec, 964M File, 18M Free
Swap: 8972M Total, 1403M Used, 7569M Free

  PID   LID USERNAME PRI STATE  TIME   WCPUCPU NAME  COMMAND
0   170 root 126 CPU/2368:23 99.02% 99.02% pgdaemon  [system]
 7312 12379 tsutsui   42 parked/3  19:30  5.22%  5.22% WRRenderB firefox
 7312 25622 tsutsui   85 poll/029:34  4.98%  4.98% Renderer  firefox
 7312  7312 tsutsui   43 parked/3  24:02  4.10%  4.10% MainThrea firefox
25088 10023 tsutsui   42 parked/1  29:57  3.42%  3.42% - firefox
0   171 root 124 syncer/3  25:05  3.22%  3.22% ioflush   [system]
25088  7474 tsutsui   43 parked/0  30:13  2.83%  2.83% - firefox
  833   833 tsutsui   43 RUN/1 59:01  1.51%  1.51% - X
[...]

% vmstat -m
Memory resource pool statistics
NameSize Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
amappl52   4170200   380857   96085   875   960 0   inf0
anonpl20  69687350  6372592  5721   642  5079  5591 0   inf0
ataspl   104 124141720 124141721413 1 2 0   inf1
biopl17641315041310   316   315 144 0   inf0
buf16k  16896 1780  1672723 410 0   inf0
buf1k   1536   430   43 2 1 1 2 0   inf1
buf2k   2560   300   30 2 1 1 2 0   inf1
buf32k  32768   28212026146  5075  3980  1095  2481 0   inf0
buf4k   460869567068547  3488  3372   116  1626 0   inf0
buf512b 1024 36030 3602 3 2 1 3 0   inf0
buf64k  65536   800 9 0 9 9 0   inf1
buf8k   8704  2900  2751916 3 7 0   inf0
bufpl17660606057485  1519  1229   290  1112 0   inf0
cwdi 192 26230 251717 9 810 0   inf0
dirhash  340  6200  32230 42627 0   inf0
dirhashblk  2048 37360 1324  1767   561  1206  1206 0   inf0
ehcixfer 208302 1 0 1 1 0   inf0
ehcixfer 208302 1 0 1 1 0   inf0
execargs262144 1158600   115860   157   156 1 3 0161
extent24 10540  982 1 0 1 1 0   inf0
fcrpl108   500   46 2 0 2 2 2   inf1
fdfile6460424055279   232   14191   145 0   inf0
ffsdino2 260   186220085580  7533   197  7336  7347 0   inf0
ffsino   196   186240085600  560599  5506  5510 0   inf0
file 12841127039246   114417381 0   inf0
filedesc 704 26090 250399693041 0   inf0
fstlwp64 61320 546114 01414 0   inf0
icmp  20  8450  8453434 0 1 0   inf0
icmp6 2055706055706   441   440 1 1 0   inf1
in4pcbpl 192 50520 4954