Bob Friesenhahn wrote: > You may have one or more "slow" disk drives which slow down the whole > vdev due to long wait times. If you can identify those slow disk drives > and replace them, then overall performance is likely to improve. > > The problem is that under severe load, the vdev with the highest backlog > will be used the least. One or more slow disks in the vdev will slow > down the whole vdev. It takes only one slow disk to slow down the whole > vdev.
Hmm, since I only started with Solaris this year, is there a way to identify a "slow" disk? In principle these should all be identical Hitachi Deathstar^WDeskstar drives and should only have the standard deviation during production. > > ZFS commits the writes to all involved disks in a raidz2 before > proceeding with the next write. With so many disks, you are asking for > quite a lot of fortuitous luck in that everything must be working > optimally. Compounding the problem is that I understand that when the > stripe width exceeds the number of segmented blocks from the data to be > written (ZFS is only willing to dice to a certain minimum size), then > only a subset of the disks will be used, wasting potiential I/O > bandwidth. Your stripes are too wide. > Ah, ok, that's one of the first reasonable explanation (which I understand) why large zpools might be bad. So far I was not able to track that down and only found the standard "magic" rule not to exceed 10 drives - but our (synthetic) tests had not shown a significant drawbacks. But I guess we might be bitten by it now. >> (c) Would the use of several smaller vdev would help much? And which >> layout would be a good compromise for getting space as well as >> performance and reliability? 46 disks have so few prime factors > > Yes, more vdevs should definitely help quite a lot for dealing with > real-world muti-user loads. One raidz/raidz2 vdev provides (at most) > the IOPs of a single disk. > > There is a point of diminishing returns and your layout has gone far > beyond this limit. Thanks for the insight, I guess I need to experiment with empty boxes to get into a better state! Cheers Carsten _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
