On two occasions recently, vmstat has showed me that a
number of processes are blocked due to I/O. At the same
time, the number of disk transactions per second reported is
a small fraction of the disk's capability.
I am thinking some kind of VM config will help, but based on what
I have read, it is almost always best to let the default kernel
behavior do its thing.
Setup:
Dual PIII 1.0 GHz (256MB L1 and L2 cache)
3GB RAM
custom kernel (SMP + PF)
5.4-RELEASE
one intel fxp0 network interface used (second unused)
Case 1: (da0 36GB SCSI)
The first case is simple: running spamd-setup to load
the huge Composite Blacklist into spamd. The only other
things running are sshd and spamd itself.
When I run vmstat (vmstat -w 1 -c 10 output attached),
it shows ten processes blocked due to I/O, 1 runnable
and a total of four disk transactions during this
particular ten-second interval.
Accoring to postmark [1] da0 is capable of 250
transactions per second.
Interrupts look fine (200 of the 300/sec is the clock,
right?).
No swapping. No network traffic to speak of.
High CPU usage.
Paging looks really high to me. I have plenty of RAM
and swap--so why is it so high?
Is there a kernel param I can tweak to improve
performance?
Note: While this was running, I tried running vmstat
-m and got the kmem_map too small error. (I
read the FAQ entry and haven't yet rebuilt the
kernel using VM_KMEM_SIZE_MAX.)
Should I make VM_KMEM_SIZE_MAX 400MB as mentioned in the FAQ?
Any other kernel settings to switch from defaults when I
build the kernel?
Case 2: (mirror/gm0, da0 + da1 300GB SCSI)
This case is more complex and more important. This box
was heavily loaded, running SpamAssassin + ClamAv in one
jail, Courier-MTA + MySql in another, and dspamd + pf in
the jail host.
It was processing about one incoming email per second,
and had 20,000 entries in the spamd greylist and 4,000
in the whitelist. The black list was not loaded.
Courier-authlib was using MySQL to figure out where to
deliver the mail to for each of 20,000 virtual users.
Courier-MTA was calling the ClamAV daemon via TCP/IP
from a courier filter and calling SpamAssassin via
TCP/IP from maildrop.
Spamd (as in OpenBSD, not Apache) and courier were
hitting the disk hard--I watched top in "m" mode [2] and
over a five minute period these two apps had about
pretty much 50% each of the WRITES.
The only thing that jumps out at me is that when things
finally do get written to disk, the context switches go
really high.
Is this a VM issue of some sort? The disk transactions
per second are really low; this 300GB SCSI can do
730/sec according to postmark (well, this was testing on
a single disk not with the RAID1 gmirror).
Thanks for any tips.
m
[1] /usr/ports/benchmarks/postmark
[2] Where can I find docs on what the columns mean in top's
"m" mode?
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr da0 da1 in sy cs us sy
id
1 10 0 190424 1653924 1065 0 0 0 1337 0 0 0 796 2931 1971 3 8
89
1 10 0 190520 1653824 17407 0 0 0 17401 0 0 0 356 111 280 44 56
0
1 10 0 167376 1676992 23264 0 0 0 23264 0 0 0 359 119 301 40 23
37
1 10 0 190808 1659240 27728 0 0 0 29152 0 0 0 351 114 297 39 14
47
1 10 0 190936 1676864 18992 0 0 0 17544 0 4 0 352 115 286 39 45
17
1 10 0 191032 1676808 17586 0 0 0 17586 0 0 0 349 115 291 41 52
7
1 10 0 191160 1658572 28025 0 0 0 29374 0 0 0 347 114 270 42 10
47
1 10 0 191320 1676668 24949 0 0 0 23576 0 0 0 346 122 280 36 16
47
1 10 0 191384 1652968 17730 0 0 0 17724 0 0 0 350 111 278 41 59
0
1 10 0 191512 1665912 20409 0 0 0 23688 0 0 0 373 113 304 39 40
21
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr da0 da1 in sy cs us
sy id
0 27 0 345932 2622260 333 0 0 0 318 0 0 0 504 1261 842 8
3 89
0 25 0 341644 2622880 78 0 0 0 215 0 0 0 535 195 575 0
5 95
0 25 0 341644 2622880 0 0 0 0 0 0 0 0 455 159 458 0
6 94
0 25 0 341644 2622880 5 0 0 0 5 0 5 5 488 165 572 0
4 96
1 25 0 343856 2622408 270 0 0 0 126 0 3 4 472 867 706 1
7 92
0 28 0 355672 2620588 2356 0 0 0 1701 0 44 44 749 6224 1988 24
13 63
1 29 0 357616 2620356 1961 0 0 0 1757 0 44 44 819 7174 2333 46
15 39
1 29 0 357620 2620360 809 0 0 0 735 0 25 25 700 2694 1247 65
8 27
0 29 0 350148 2621460 79 0 0 0 349 0 18 17 654 1144 896 32
6 62
1 28 0 348808 2621640 140 0 0 0 175 0 2 2 438 433 509 0
5 95
1 29 0 351016 2621284 261 0 0 0 137 0 10 10 530 781 781 0
8 92
0 29 0 351196 2620928 1790 0 0 0 1540 0 32 32 724 4914 1799 19
13 68
7 27 0 366168 2618696 3657 0 0 0 2835 0 75 75 1003 9967 2644 58
21 21
4 28 0 377836 2616740 2383 0 0 0 1719 0 44 43 898 7199 1926 84
16 0
3 28 0 348996 2621212 2004 0 0 0 2851 0 74 75 948 7730 2200 62
15 23
2 28 0 351156 2620636 396 0 0 0 335 0 119 127 928 951 1849 3
11 86
1 29 0 358544 2619816 2006 0 0 0 1702 0 104 94 1046 7823 2761 33
17 49
1 29 0 352904 2620540 791 0 0 0 920 0 21 22 582 2467 1138 23
8 69
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr da0 da1 in sy cs us
sy id
0 30 0 353364 2620304 1004 0 0 0 837 0 16 15 652 3595 1202 33
10 57
3 28 0 358624 2619544 2025 0 0 0 1693 0 42 42 925 5461 2103 20
13 67
1 28 0 356484 2619684 1398 0 0 0 1396 0 28 28 1034 10604 4947 47
14 39
3 27 0 362108 2618540 2623 0 0 0 2138 0 151 151 1156 6026 2356 67
24 9
1 27 0 354620 2619880 1314 0 0 0 1615 0 39 40 808 5280 1401 83
12 4
1 29 0 359088 2618976 2360 0 0 0 1960 0 47 46 888 7626 2158 41
18 41
6 28 0 364524 2617952 2360 0 0 0 2031 0 37 37 1009 10575 3258 73
15 13
2 30 0 357952 2618624 3010 0 0 0 2987 0 77 77 1123 9732 2629 64
19 17
3 28 0 363976 2617728 2358 0 0 0 1962 0 43 43 987 7696 1977 73
13 14
5 28 0 371768 2616336 2283 0 0 0 1833 0 37 37 862 6972 1760 80
17 4
3 28 0 379872 2615124 3693 0 0 0 3158 0 77 78 1070 12627 3007 73
19 8
8 29 0 374012 2616044 638 0 0 0 845 0 30 30 759 3607 1310 90
10 0
2 31 0 363880 2617056 2532 0 0 0 2586 0 56 55 1100 8357 2252 74
17 9
3 30 0 370596 2616180 1959 0 0 0 1702 0 43 43 839 8104 2062 76
14 11
3 31 0 370756 2616252 173 0 0 0 152 0 12 12 596 606 854 96
4 0
1 30 0 361572 2617508 2026 0 0 0 2190 0 44 45 838 7504 1644 63
13 25
3 30 0 354344 2618260 1301 0 0 0 1433 0 30 29 716 4942 1371 50
10 41
2 28 0 360372 2617440 2233 0 0 0 1871 0 33 33 745 6363 1656 42
13 45
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr da0 da1 in sy cs us
sy id
4 29 0 369572 2615616 3118 0 0 0 2442 0 44 44 1104 11499 3007 61
18 20
5 30 0 364748 2616200 2856 0 0 0 2805 0 78 77 1100 9827 2589 81
19 0
2 29 0 373248 2614944 3685 0 0 0 3174 0 75 76 1095 13451 3296 70
26 4
2 29 0 367108 2615596 1974 0 0 0 2005 0 62 63 883 6594 2055 77
21 2
1 28 0 358020 2617080 1106 0 0 0 1392 0 34 33 667 6640 1427 57
38 4
0 28 0 350552 2618204 68 0 0 0 339 0 9 9 441 983 511 21
25 54
1 29 0 360044 2616796 1556 0 0 0 1104 0 17 17 612 3628 1252 5
12 82
0 30 0 352804 2617552 199 0 0 0 551 0 375 375 2084 1935 6164 26
18 56
1 28 0 358052 2616864 2037 0 0 0 1696 0 551 551 2999 6123 10286 47
32 20
0 28 0 357948 2617000 2011 0 0 0 1931 0 41 42 754 6509 1661 39
13 48
0 27 0 348212 2618536 237 0 0 0 578 0 13 13 498 1282 664 24
7 68
1 27 0 355664 2617472 1489 0 0 0 1081 0 12 12 547 2983 1044 13
10 76
2 28 0 350432 2618136 331 0 0 0 440 0 11 10 543 1486 846 7
5 88
1 28 0 350424 2618096 896 0 0 0 831 0 38 38 765 2849 1568 9
9 81
1 29 0 350640 2617780 524 0 0 0 418 0 170 182 1210 1469 2260 10
22 68
4 27 0 365384 2616412 2592 0 0 0 2178 0 80 68 967 8095 2464 64
16 20
0 27 0 322376 2633292 730 0 0 0 4882 0 34 34 611 3569 1248 65
10 25
1 28 0 332068 2631740 1081 0 0 0 621 0 12 12 556 2683 1099 7
9 84
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr da0 da1 in sy cs us
sy id
3 28 0 327040 2632400 570 0 0 0 750 0 25 26 868 3020 1662 15
9 76
4 28 0 339792 2629996 2594 0 0 0 1861 0 36 36 1175 8680 2634 65
19 16
2 29 0 341052 2629700 2682 0 0 0 2441 0 65 65 1130 8717 2452 76
16 8
2 28 0 326736 2631800 130 0 0 0 627 0 17 16 641 2223 1003 14
7 80
0 29 0 326804 2631596 311 0 0 0 243 0 7 7 659 1446 1135 3
10 88
1 29 0 326828 2631324 1086 0 0 0 961 0 25 25 835 3648 1619 14
9 77
1 29 0 334092 2629960 1032 0 0 0 830 0 17 18 904 4961 1147 63
12 25
0 29 0 326776 2630852 813 0 0 0 1048 0 13 12 1037 3148 1203 28
13 58
4 28 0 334316 2630088 2358 0 0 0 2581 0 58 58 1281 55999 20971 29
38 33
2 27 0 344276 2628704 2424 0 0 0 1960 0 50 51 887 8730 2223 77
15 8
6 26 0 335252 2630092 1008 0 0 0 1275 0 33 32 701 4373 1130 85
10 6
1 26 0 329360 2631012 1615 0 0 0 1713 0 40 40 746 4744 1592 34
12 53
0 26 0 321892 2632104 69 0 0 0 323 0 10 11 614 1045 833 6
7 88
0 26 0 321892 2632104 15 0 0 0 15 0 6 6 499 265 648 0
6 94
1 25 0 321812 2632172 326 0 0 0 330 0 8 8 549 761 746 1
4 94
2 25 0 327236 2631288 927 0 0 0 635 0 15 15 516 2279 922 24
9 67
0 25 0 319764 2632388 60 0 0 0 447 0 188 188 1111 864 2635 4
11 85
1 25 0 319764 2632388 0 0 0 0 0 0 2 2 389 107 366 0
4 96
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr da0 da1 in sy cs us
sy id
3 26 0 329384 2630900 1229 0 0 0 743 0 14 14 638 2543 1171 3
10 88
0 26 0 321988 2631920 960 0 0 0 1156 0 30 30 691 3809 1321 53
9 38
1 26 0 322024 2631852 44 0 0 0 29 0 10 10 591 450 901 0
7 93
2 27 0 338424 2629412 1875 0 0 0 1170 0 27 27 678 5395 1730 10
12 78
1 27 0 331624 2630436 346 0 0 0 547 0 9 8 462 1473 693 61
6 33
0 28 0 326352 2631132 698 0 0 0 789 0 13 13 618 2052 1007 9
6 85
1 28 0 326476 2630648 75 0 0 0 17 0 3 3 765 1518 1191 4
6 90
1 30 0 338916 2628664 1850 0 0 0 1556 0 100 107 1458 28808 12099 18
36 45
0 30 0 330780 2629956 379 0 0 0 732 0 84 78 837 2851 1877 13
11 76
0 29 0 328640 2630260 172 0 0 0 241 0 9 9 512 850 1012 0
8 92
0 29 0 328640 2630260 0 0 0 0 4 0 2 2 389 121 369 0
4 96
1 30 0 331940 2629672 884 0 0 0 681 0 13 12 607 2761 1047 17
7 76
0 29 0 328640 2630216 40 0 0 0 168 0 9 9 508 425 641 3
6 91
0 29 0 328640 2630212 11 0 0 0 10 0 4 4 548 230 644 0
5 95
0 29 0 328640 2630212 5 0 0 0 5 0 2 2 430 219 470 0
4 96
0 29 0 328640 2630208 6 0 0 0 5 0 2 2 437 256 481 1
4 96
0 28 0 326496 2630516 277 0 0 0 348 0 8 8 526 780 788 0
6 94
0 29 0 328728 2630052 274 0 0 0 126 0 2 2 451 802 604 1
7 92
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr da0 da1 in sy cs us
sy id
0 29 0 328636 2630204 842 0 0 0 807 0 27 27 619 2368 1140 12
8 80
0 29 0 328636 2630200 0 0 0 0 0 0 0 0 492 113 457 0
4 96
0 30 0 330732 2629900 333 0 0 0 233 0 2 2 503 687 607 0
6 94
1 30 0 330748 2629864 15 0 0 0 1 0 1 1 400 189 401 0
6 94
2 29 0 338296 2628512 1340 0 0 0 911 0 21 21 910 4186 1712 23
11 66
1 31 0 338024 2626652 1223 0 0 0 1364 0 51 51 2972 40356 18751 38
32 30
0 31 0 333028 2628012 318 0 0 0 1159 0 32 33 757 67408 35201 7
43 50
0 31 0 338336 2627192 1044 0 0 0 909 0 86 98 923 3573 2295 24
11 65
0 31 0 333088 2627968 1367 0 0 0 1407 0 65 53 734 4235 1442 20
9 71
0 30 0 330876 2628372 29 0 0 0 130 0 0 0 405 191 401 0
5 95
0 29 0 328680 2628740 152 0 0 0 246 0 2 2 428 459 495 1
6 93
2 29 0 330924 2628308 476 0 0 0 274 0 552 552 2776 1862 14249 3
24 72
0 30 0 337484 2627268 1119 0 0 0 877 0 18 18 875 11192 5634 16
15 69
3 30 0 345588 2625052 1285 0 0 0 675 0 13 13 748 2201 1059 53
7 40
1 29 0 336604 2626380 344 0 0 0 653 0 23 23 557 1605 911 54
8 38
0 29 0 329528 2627284 159 0 0 0 407 0 15 15 581 1541 679 12
5 83
0 29 0 329528 2627284 5 0 0 0 5 0 2 2 397 159 391 0
4 96
0 29 0 329528 2627300 5 0 0 0 9 0 3 3 474 177 539 0
4 96
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr da0 da1 in sy cs us
sy id
2 28 0 329548 2627280 5 0 0 0 5 0 12 12 539 246 678 0
5 94
0 30 0 331624 2627356 232 0 0 0 219 0 56 56 737 552 968 2
17 82
2 30 0 331724 2627304 300 0 0 0 246 0 2 2 402 676 518 1
6 93
1 29 0 337012 2626560 1654 0 0 0 1361 0 35 35 695 4538 1521 25
11 64
0 28 0 327388 2627956 96 0 0 0 429 0 12 12 464 1038 605 14
5 81
0 28 0 327388 2627956 5 0 0 0 5 0 2 2 473 191 526 0
4 96
2 28 0 329632 2627500 280 0 0 0 130 0 4 4 518 739 751 0
5 95
0 28 0 327388 2627916 877 0 0 0 934 0 24 24 695 3220 1214 21
10 70
0 28 0 327388 2627920 110 0 0 0 113 0 0 0 411 324 406 0
6 94
1 27 0 327388 2627920 0 0 0 0 4 0 2 2 404 128 387 0
4 96
2 26 0 325268 2628176 480 0 0 0 470 0 2 2 460 958 649 1
6 93
3 26 0 332712 2627008 934 0 0 0 641 0 19 19 831 10859 4034 16
13 71
_______________________________________________
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
