Re: [zfs-discuss] sub-optimal ZFS performance
hj == Henrik Johansson henr...@henkis.net writes: hj A überquota property for the whole pool would have been nice hj [to get out-of-space errors instead of fragmentation] just make an empty filesystem with a reservation. That's what I do. NAMEUSED AVAIL REFER MOUNTPOINT andaman3.71T 382G18K none andaman/arrchive 3.07T 382G 67.7G /arrchive andaman/balloon 18K 1010G18K none terabithia:/export/home/guest/Azureus Downloads# zfs get reservation andaman/balloon NAME PROPERTY VALUESOURCE andaman/balloon reservation 628G local pgprZjhXXryEg.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] sub-optimal ZFS performance
So the solution is to never get more than 90% full disk space, för fan? -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] sub-optimal ZFS performance
On Thu, 29 Oct 2009, Orvar Korvar wrote: So the solution is to never get more than 90% full disk space, för fan? Right. While UFS created artificial limits to keep the filesystem from getting so full that it became sluggish and sick, ZFS does not seem to include those protections. Don't ever run a ZFS pool for a long duration of time at very close to full since it will become excessively fragmented. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] sub-optimal ZFS performance
On Oct 29, 2009, at 5:23 PM, Bob Friesenhahn wrote: On Thu, 29 Oct 2009, Orvar Korvar wrote: So the solution is to never get more than 90% full disk space, för fan? Right. While UFS created artificial limits to keep the filesystem from getting so full that it became sluggish and sick, ZFS does not seem to include those protections. Don't ever run a ZFS pool for a long duration of time at very close to full since it will become excessively fragmented. Setting quotas for all dataset could perhaps be of use for some of us. A überquota property for the whole pool would have been nice until a real solution is available. Henrik http://sparcv9.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] sub-optimal ZFS performance
So the solution is to never get more than 90% full disk space while that's true, its not Henrik's main discovery. Henrik points out that 1/4 of the arc is used for metadata, and sometime that's not enough.. if echo ::arc | mdb -k | egrep ^size isn't reaching echo ::arc | mdb -k | egrep ^c and you are maxing out your metadata space, check: echo ::arc | mdb -k | grep meta_ one can set the metadata space (1G in this case) with: echo arc_meta_limit/Z 0x400 | mdb -kw So while Henrik's FS had some fragmentation, 1/4 of c_max wasn't enough metadata arc space for number of files in /var/pkg/download good find Henrik! Rob ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] sub-optimal ZFS performance
On Oct 29, 2009, at 15:08, Henrik Johansson wrote: On Oct 29, 2009, at 5:23 PM, Bob Friesenhahn wrote: On Thu, 29 Oct 2009, Orvar Korvar wrote: So the solution is to never get more than 90% full disk space, för fan? Right. While UFS created artificial limits to keep the filesystem from getting so full that it became sluggish and sick, ZFS does not seem to include those protections. Don't ever run a ZFS pool for a long duration of time at very close to full since it will become excessively fragmented. Setting quotas for all dataset could perhaps be of use for some of us. A überquota property for the whole pool would have been nice until a real solution is available. Or create lost+found with 'zfs create' and give it a reservation. The 'directory name' won't look too much out of place, and there'll be some space set aside. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] sub-optimal ZFS performance
On Oct 16, 2009, at 1:01 AM, Henrik Johansson wrote: My guess would be that this is due to fragmentation during some time when the filesystem might have been close to full, but it is still pretty terrible numbers even with 0.5M files in the structure. And while this is very bad I would at least expect the ARC to cache data and make a second run go faster: I solved this, the second run was also slow since the metadata part of the ARC was to small, raising arc_meta_limit helped, and turning off atime also helped much since this directory seem to be terrible fragmented. With these changes the ARC helps so that the second goes as fast as it should. The fragmentation can be solved by a copy if I would want to keep the files. I wrote some more details about what I did if anyone is interested: http://sparcv9.blogspot.com/2009/10/curious-case-of-strange-arc.html I'll make sure to keep some more free space in my pools at all times now ;) Regards Henrik http://sparcv9.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] sub-optimal ZFS performance
Hello,
ZFS is behaving strange on a OSOL laptop, your thoughts are welcome.
I am running OSOL on my laptop, currently b124 and i found that the
performance of ZFS is not optimal in all situations. If i check the
how much space the package cache for pkg(1) uses, it takes a bit
longer on this host than on comparable machine to which i transferred
all the data.
u...@host:/var/pkg$ time du -hs download
6.4Gdownload
real87m5.112s
user0m6.820s
sys 1m46.111s
My guess would be that this is due to fragmentation during some time
when the filesystem might have been close to full, but it is still
pretty terrible numbers even with 0.5M files in the structure. And
while this is very bad I would at least expect the ARC to cache data
and make a second run go faster:
u...@host:/var/pkg$ time du -hs download
6.4Gdownload
real94m14.688s
user0m6.708s
sys 1m27.105s
Two runs on the machine to which i have transferred the directory
structure:
$ time du -hs download
6.4Gdownload
real2m59.60s
user0m3.83s
sys 0m18.87s
This goes a bit faster after the initial run also:
$ time du -hs download
6.4Gdownload
real0m15.40s
user0m3.40s
sys 0m11.43s
The disk are of course very busy during the first runs on both
machines, but on the slow machine it has to do all the work again
while the disk in the fast machine gets to rest on the second run.
Slow system (OSOL b124, T61 Intel c2d laptop root pool on 2.5 disk):
memstat pre first run:
Kernel 162685 635 16%
ZFS File Data 81284 3178%
Anon57323 2236%
Exec and libs3248120%
Page cache 14924581%
Free (cachelist) 7881301%
Free (freelist)700315 2735 68%
Total 1027660 4014
Physical 1027659 4014
memstat post first run:
Page SummaryPagesMB %Tot
Kernel 461153 1801 45%
ZFS File Data 83598 3268%
Anon58389 2286%
Exec and libs3215120%
Page cache 14958581%
Free (cachelist) 6849261%
Free (freelist)399498 1560 39%
Total 1027660 4014
Physical 1027659 4014
arcstat first run:
Time read miss miss% dmis dm% pmis pm% mmis mm%
arcsz c
21:02:31 27919 7114 7 3011 10 439M
3G
21:12:31 19060 3152 28 8 9760 32 734M
3G
21:22:31 22558 2557 25 0 9458 25 873M
3G
21:32:31 20651 2451 24 0 2450 24 985M
3G
21:42:31 17543 2443 24 0 2942 24 1G
3G
21:52:31 16248 2948 29 0 5448 29 1G
3G
22:02:31 15955 3454 34 0 9055 34 1G
3G
22:12:31 16441 2541 24 0 6141 25 1G
3G
22:22:31 16140 2440 24 0 6840 24 1G
3G
arcstat second run:
Time read miss miss% dmis dm% pmis pm% mmis mm%
arcsz c
22:35:521K 447 24 429 2317 47 436 26 1G
3G
22:45:52 16340 2440 24 0 7540 24 1G
3G
22:55:52 16140 2540 24 0 8640 25 1G
3G
23:05:52 15940 2539 25 0 7140 25 1G
3G
23:15:52 15840 2540 25 0 8640 25 1G
3G
23:25:52 15840 2540 25 0 10040 25 1G
3G
23:35:52 15740 2540 25 0 10040 25 1G
3G
23:45:52 15840 2540 25 0 10040 25 1G
3G
23:55:52 16040 2540 25 0 10040 25 1G
3G
00:05:52 15640 2540 25 0 10040 25 1G
3G
Fast system (OSOL b124, AMD Athlon X2 server, tested on root pool on
2.5 SATA disk)
Memstat pre run:
Page SummaryPagesMB %Tot
Kernel 160338 6268%
ZFS File Data 44875 1752%
Anon24388951%
Exec and libs1295 50%
Page cache 6490250%
Free (cachelist) 4786180%
Free (freelist) 1753978 6851 88%
Balloon 0
