Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
Darren: > > With all of the talk about performance problems due to > > ZFS doing a sync to force the drives to commit to data > > being on disk, how much of a benefit is this - especially > > for NFS? I would not call those things as problems, more like setting proper expectations. My understanding is that enabling write cache helps by providing I/O concurrency for drives that do not implement other form of Command Queuing. In other cases, WCE should not buy much if anything. I'd be interested in analysing any cases that shows otherwise... -r ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
Eric Schrock wrote: On Thu, Aug 03, 2006 at 10:24:12AM -0700, Marion Hakanson wrote: zpool create mirror c0t2d0 c0t3d0 mirror c0t0d0s5 c0t1d0s5 Is this allowed? Is it stupid? Will performance be so bad/bizarre that it should be avoided at all costs? Anybody tried it? Yes, it's allowed, but it's definitely not an ideal solution. Unless you manually tweak the write cache (as mentioned elsewhere, this could be bad for other slices), you'll end up with asymmetric performance on these two vdevs. The current ZFS allocation and I/O policies don't take latency into account (they're based solely on capacity), and so a slower vdev will slow down the entire pool. Is the fact that there are more spindles in general going to help at all? So in this case he's got a pool consisting of two mirrored stripes. Isn't performance still going to be better than if he dropped off the slower slices/stripe and had a single mirror? Won't the system be able to handle more i/os in general by taking advantage of the additional bandwidth provided by each controller? For example, while one block is being written to the slower stripe, it may also be able to write another block to the faster stripe concurrently? --joe ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
On Thu, Aug 03, 2006 at 10:24:12AM -0700, Marion Hakanson wrote: > > zpool create mirror c0t2d0 c0t3d0 mirror c0t0d0s5 c0t1d0s5 > > Is this allowed? Is it stupid? Will performance be so bad/bizarre that > it should be avoided at all costs? Anybody tried it? > Yes, it's allowed, but it's definitely not an ideal solution. Unless you manually tweak the write cache (as mentioned elsewhere, this could be bad for other slices), you'll end up with asymmetric performance on these two vdevs. The current ZFS allocation and I/O policies don't take latency into account (they're based solely on capacity), and so a slower vdev will slow down the entire pool. Future work will make ZFS behave better in the face of asymmetric performance, but in general this will still be a non-ideal configuration. You'll be fighting with other filesystems for control of the underlying disk, you won't be able to use hot spares effectively, and future FMA work (such as reading SMART data and predictive analysis) may not work well (if at all). - Eric -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
Folks, I realize this thread has run its course, but I've got a variant of the original question: What performance problems or anomalies might one see if mixing both whole disks _and_ slices within the same pool? I have in mind some Sun boxes (V440, T2000, X4200) with four internal drives. Typically, one would setup root/swap/var as an SVM/UFS mirror on one pair of drives, and the second pair as a mirrored ZFS pair. However, on the typical 72GB drives, much of that first pair (maybe half or more) would not be needed for system+swap. Suppose we use 24GB of the first pair as our SVM/UFS system mirror, with the free space left in slice 5 of each drive. We could then make maximum use of the remaining space as a zfs pool via something like: zpool create mirror c0t2d0 c0t3d0 mirror c0t0d0s5 c0t1d0s5 Is this allowed? Is it stupid? Will performance be so bad/bizarre that it should be avoided at all costs? Anybody tried it? BTW, in my experience the OS/swap/var filesystems get very little I/O unless RAM is in short supply. So on a well-sized system, there should be little contention between the "system" portion and "data" portion of those shared disks (regardless of whether or not ZFS is in use). Thanks and regards, Marion ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
Ahh, interesting information. Thanks folks, I'm have a better understanding of this now. --joe Jeff Bonwick wrote: is zfs any less efficient with just using a portion of a disk versus the entire disk? As others mentioned, if we're given a whole disk (i.e. no slice is specified) then we can safely enable the write cache. One other effect -- probably not huge -- is that the block placement algorithm is most optimal for an outer-to-inner track diameter ratio of about 2:1, which reflects typical platters. To quote the source: http://cvs.opensolaris.org/source/xref/on/usr/src/uts/common/fs/zfs/metaslab.c#m etaslab_weight /* * Modern disks have uniform bit density and constant angular velocity. * Therefore, the outer recording zones are faster (higher bandwidth) * than the inner zones by the ratio of outer to inner track diameter, * which is typically around 2:1. We account for this by assigning * higher weight to lower metaslabs (multiplier ranging from 2x to 1x). * In effect, this means that we'll select the metaslab with the most * free bandwidth rather than simply the one with the most free space. */ But like I said, the effect isn't huge -- the high-order bit that we have a preference for low LBAs. It's a second-order optimization to bias the allocation based on the maximum free bandwidth, which is currently based on an assumption about physical disk construction. In the future we'll do the smart thing and compute each metaslab's allocation bias based on its actual observed bandwidth. Jeff ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
On Aug 3, 2006, at 5:14 PM, Darren Dunham wrote: And it's portable. If you use whole disks, you can export the pool from one machine and import it on another. There's no way to export just one slice and leave the others behind... I got the impression that the export command exported the contents of the pool, not the underlying physical structure (minus the rust ;o). Seems that I was wrong. Correct. You could export a pool that existed on just one of the slices, but it would be difficult to make that useful. You couldn't for example put slice 1 in a new machine and leave slice 4 here. This also implies that the disk that receives the exported pool has to be the same size as the exported disk, or can it be bigger ??? Hmm? An 'exported' pool remains on the same media, but is no longer being used by the host. So the media can be transported and connected (imported) elsewhere. It doesn't move the data to new meda (that would be zfs send or the like). Oops, seems that I swapped the commands 'export' and 'send' in my head. Kaiser Jasse -- Authorized Stealth Oracle The axioms of wisdom: 1. You can't outstubborn a cat 2. You can't conquer the universe without the knowledge of FORTRAN 3. In the Unix realm, 10% of work fixes 90% of the problems ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
> > And it's portable. If you use whole disks, you can export the > > pool from one machine and import it on another. There's no way > > to export just one slice and leave the others behind... > > I got the impression that the export command exported the contents > of the pool, not the underlying physical structure (minus the rust ;o). > Seems that I was wrong. Correct. You could export a pool that existed on just one of the slices, but it would be difficult to make that useful. You couldn't for example put slice 1 in a new machine and leave slice 4 here. > This also implies that the disk that receives the exported pool > has to be the same size as the exported disk, or can it be bigger ??? Hmm? An 'exported' pool remains on the same media, but is no longer being used by the host. So the media can be transported and connected (imported) elsewhere. It doesn't move the data to new meda (that would be zfs send or the like). -- Darren Dunham [EMAIL PROTECTED] Senior Technical Consultant TAOShttp://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
Robert Milkowski <[EMAIL PROTECTED]> writes: > Additionally keep in mind that outer region of a disk is much faster. > So if you want to put OS and then designate rest of the disk for > application then probably putting ZFS on a slice beginning on cyl 0 is > best in most scenarios. This has the additional advantage that with the advent of ZFS boot, you can simply move / to a zfs file system and extend slice 0 to cover the whole disk. Rainer -- - Rainer Orth, Faculty of Technology, Bielefeld University ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
On Aug 3, 2006, at 8:17 AM, Jeff Bonwick wrote: ZFS will try to enable write cache if whole disks is given. Additionally keep in mind that outer region of a disk is much faster. And it's portable. If you use whole disks, you can export the pool from one machine and import it on another. There's no way to export just one slice and leave the others behind... I got the impression that the export command exported the contents of the pool, not the underlying physical structure (minus the rust ;o). Seems that I was wrong. This also implies that the disk that receives the exported pool has to be the same size as the exported disk, or can it be bigger ??? Kaiser Jasse -- Authorized Stealth Oracle The axioms of wisdom: 1. You can't outstubborn a cat 2. You can't conquer the universe without the knowledge of FORTRAN 3. In the Unix realm, 10% of work fixes 90% of the problems ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
> With all of the talk about performance problems due to > ZFS doing a sync to force the drives to commit to data > being on disk, how much of a benefit is this - especially > for NFS? It depends. For some drives it's literally 10x. > Also, if I was lucky enough to have a working prestoserv > card around, would ZFS be able to take advantage of > that at all? I'm working on the general lack-of-NVRAM-in-servers problem. As for using presto, I don't think it'd be too hard. We've already structured the code so that allocating intent log blocks from a different set of vdevs would be straightforward. It's probably a week's work to define the new metaslab class, new vdev type, and modify the ZIL to use it. Jeff ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
Jeff Bonwick wrote: is zfs any less efficient with just using a portion of a disk versus the entire disk? As others mentioned, if we're given a whole disk (i.e. no slice is specified) then we can safely enable the write cache. With all of the talk about performance problems due to ZFS doing a sync to force the drives to commit to data being on disk, how much of a benefit is this - especially for NFS? Also, if I was lucky enough to have a working prestoserv card around, would ZFS be able to take advantage of that at all? Darren ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
> is zfs any less efficient with just using a portion of a > disk versus the entire disk? As others mentioned, if we're given a whole disk (i.e. no slice is specified) then we can safely enable the write cache. One other effect -- probably not huge -- is that the block placement algorithm is most optimal for an outer-to-inner track diameter ratio of about 2:1, which reflects typical platters. To quote the source: http://cvs.opensolaris.org/source/xref/on/usr/src/uts/common/fs/zfs/metaslab.c#m etaslab_weight /* * Modern disks have uniform bit density and constant angular velocity. * Therefore, the outer recording zones are faster (higher bandwidth) * than the inner zones by the ratio of outer to inner track diameter, * which is typically around 2:1. We account for this by assigning * higher weight to lower metaslabs (multiplier ranging from 2x to 1x). * In effect, this means that we'll select the metaslab with the most * free bandwidth rather than simply the one with the most free space. */ But like I said, the effect isn't huge -- the high-order bit that we have a preference for low LBAs. It's a second-order optimization to bias the allocation based on the maximum free bandwidth, which is currently based on an assumption about physical disk construction. In the future we'll do the smart thing and compute each metaslab's allocation bias based on its actual observed bandwidth. Jeff ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
> ZFS will try to enable write cache if whole disks is given. > > Additionally keep in mind that outer region of a disk is much faster. And it's portable. If you use whole disks, you can export the pool from one machine and import it on another. There's no way to export just one slice and leave the others behind... Jeff ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
Hello Joseph, Thursday, August 3, 2006, 2:02:28 AM, you wrote: JM> I know this is going to sound a little vague but... JM> A coworker said he read somewhere that ZFS is more efficient if you JM> configure pools from entire disks instead of just slices of disks. I'm JM> curious if there is any merit to this? JM> The use case that we had been discussing was something to the effect of JM> building a 2 disk system, install the OS on slice 0 of disk 0 and make JM> the rest of the disk available for 1/2 of a zfs mirror. Then disk 1 JM> would probably be partitioned the same, but the only thing active would JM> be the other 1/2 of a zfs mirror. JM> Now clearly there is a contention issue between the OS and the data JM> partition, which would be there if SVM mirrors were used instead. But JM> besides this, is zfs any less efficient with just using a portion of a JM> disk versus the entire disk? ZFS will try to enable write cache if whole disks is given. Additionally keep in mind that outer region of a disk is much faster. So if you want to put OS and then designate rest of the disk for application then probably putting ZFS on a slice beginning on cyl 0 is best in most scenarios. -- Best regards, Robertmailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
Joseph Mocker wrote: I know this is going to sound a little vague but... A coworker said he read somewhere that ZFS is more efficient if you configure pools from entire disks instead of just slices of disks. I'm curious if there is any merit to this? If the entire disk is used in a zpool then the disk cache can, and in most cases is, enabled. This speeds operations up quite a bit in some scenarios. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS performance using slices vs. entire disk?
On Wed, 2 Aug 2006, Joseph Mocker wrote: > The use case that we had been discussing was something to the effect of > building a 2 disk system, install the OS on slice 0 of disk 0 and make the > rest of the disk available for 1/2 of a zfs mirror. Then disk 1 would probably > be partitioned the same, but the only thing active would be the other 1/2 of a > zfs mirror. Why wouldn't you mirror (using SVM) the OS slice on disk 1 too? Sorry, can't answer the ZFS bit of the question... -- Rich Teer, SCNA, SCSA, OpenSolaris CAB member President, Rite Online Inc. Voice: +1 (250) 979-1638 URL: http://www.rite-group.com/rich ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
