On Aug 4, 2010, at 9:03 AM, Eduardo Bragatto wrote: > On Aug 4, 2010, at 12:26 AM, Richard Elling wrote: > >> The tipping point for the change in the first fit/best fit allocation >> algorithm is >> now 96%. Previously, it was 70%. Since you don't specify which OS, build, >> or zpool version, I'll assume you are on something modern. > > I'm running Solaris 10 10/09 s10x_u8wos_08a, ZFS Pool version 15.
Then the first fit/best fit threshold is 96%. >> NB, "zdb -m" will show the pool's metaslab allocations. If there are no 100% >> free metaslabs, then it is a clue that the allocator might be working extra >> hard. > > On the first two VDEVs there are no allocations 100% free (most are nearly > full)... The two newer ones, however, do have several allocations of 128GB > each, 100% free. > > If I understand correctly in that scenario the allocator will work extra, is > that correct? Yes, and this can be measured, but... >> OK, so how long are they waiting? Try "iostat -zxCn" and look at the >> asvc_t column. This will show how the disk is performing, though it >> won't show the performance delivered by the file system to the >> application. To measure the latter, try "fsstat zfs" (assuming you are >> on a Solaris distro) > > Checking with iostat, I noticed the average wait time to be between 40ms and > 50ms for all disks. Which doesn't seem too bad. ... actually, that is pretty bad. Look for an average around 10 ms and peaks around 20ms. Solve this problem first -- the system can do a huge amount of allocations for any algorithm in 1ms. > And this is the output of fsstat: > > # fsstat zfs > new name name attr attr lookup rddir read read write write > file remov chng get set ops ops ops bytes ops bytes > 3.26M 1.34M 3.22M 161M 13.4M 1.36G 9.6M 10.5M 899G 22.0M 625G zfs Unfortunately, the first line is useless, it is the summary since boot. Try adding a sample interval to see how things are moving now. > > However I did have CPU spikes at 100% where the kernel was taking all cpu > time. Again, this can be analyzed using baseline performance analysis techniques. The "prstat" command should show how CPU is being used. I'm not running Solaris 10 10/09, but IIRC, it has the ZFS enhancement where CPU time is attributed to the pool, as seen in prstat. -- richard > > I have reduced my zfs_arc_max parameter as it seemed the applications were > struggling for RAM and things are looking better now > > Thanks for your time, > Eduardo Bragatto. > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Richard Elling rich...@nexenta.com +1-760-896-4422 Enterprise class storage for everyone www.nexenta.com _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
