I wrote
Has anyone else noticed a significant zfs performance
deterioration when running recent opensolaris bits?
My 32-bit / 768 MB Toshiba Tecra S1 notebook was able
to do a full opensolaris release build in ~ 4 hours 45
minutes (gcc shadow compilation disabled; using an lzjb
compressed zpool / zfs on a single notebook hdd p-ata drive).
After upgrading to 2007-05-25 opensolaris release
bits (compiled from source), the same release build now
needs ~ 6 hours; that's ~ 25% slower.
It might be Bug ID 6469558
ZFS prefetch needs to be more aware of memory pressure:
http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6469558
Instead of compiling opensolaris for 4-6 hours, I've now used
the following find / grep test using on-2007-05-30 sources:
1st test using Nevada build 60:
% cd /files/onnv-2007-05-30
% repeat 10 /bin/time find usr/src/ -name *.[hc] -exec grep FooBar {} +
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 4:22.5
user3.3
sys 5.8
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 4:28.4
user3.3
sys 4.8
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 4:18.0
user3.3
sys 4.7
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 4:17.3
user3.3
sys 4.8
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 4:15.0
user3.3
sys 4.7
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 4:12.0
user3.3
sys 4.7
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 4:21.9
user3.3
sys 4.7
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 4:18.7
user3.3
sys 4.7
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 4:19.5
user3.3
sys 4.7
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 4:17.2
user3.3
sys 4.7
Same test, but running onnv-2007-05-30 release bits
(compiled from source). This is at least 25% slower
than snv_60:
(Note: zfs_prefetch_disable = 0 , the default value)
% repeat 10 /bin/time find usr/src/ -name *.[hc] -exec grep FooBar {} +
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 8:04.3
user7.3
sys13.2
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 6:34.4
user7.3
sys11.2
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 6:33.8
user7.3
sys11.1
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 5:35.6
user7.3
sys10.6
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 5:39.8
user7.3
sys10.6
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 5:37.8
user7.3
sys11.1
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 5:53.5
user7.3
sys11.0
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 5:45.2
user7.3
sys11.1
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 5:44.8
user7.3
sys11.0
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 5:49.1
user7.3
sys11.0
Then I patched zfs_prefetch_disable/W1, and now
the find grep test runs much faster on
onnv-2007-05-30 bits:
(Note: zfs_prefetch_disable = 1)
% repeat 10 /bin/time find usr/src/ -name *.[hc] -exec grep FooBar {} +
usr/src/lib/pam_modules/authtok_check/authtok_check.c: * user entering
FooBar1234 with PASSLENGTH=6, MINDIGIT=4, while
real 4:01.3
user7.2
sys 9.9