On 6 May 2010, at 13:18 , Edward Ned Harvey wrote: >> From: Michael Sullivan [mailto:michael.p.sulli...@mac.com] >> >> While it explains how to implement these, there is no information >> regarding failure of a device in a striped L2ARC set of SSD's. I have > > http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#Sepa > rate_Cache_Devices > > It is not possible to mirror or use raidz on cache devices, nor is it > necessary. If a cache device fails, the data will simply be read from the > main pool storage devices instead. >
I understand this. > I guess I didn't write this part, but: If you have multiple cache devices, > they are all independent from each other. Failure of one does not negate > the functionality of the others. > Ok, this is what I wanted to know. The that the L2ARC devices assigned to the pool are not striped but are independent. Loss of one drive will just cause a cache miss and force ZFS to go out to the pool for its objects. But then I'm not talking about using RAIDZ on a cache device. I'm talking about a striped device which would be RAID-0. If the SSD's are all assigned to L2ARC, then they are not striped in any fashion (RAID-0), but are completely independent and the L2ARC will continue to operate, just missing a single SSD. > >> I'm running 2009.11 which is the latest OpenSolaris. > > Quoi?? 2009.06 is the latest available from opensolaris.com and > opensolaris.org. > > If you want something newer, AFAIK, you have to go to developer build, such > as osol-dev-134 > > Sure you didn't accidentally get 2008.11? > My mistake… snv_111b which is 2009.06. I know it went up to 11 somewhere. > >> I am also well aware of the effect of losing a ZIL device will cause >> loss of the entire pool. Which is why I would never have a ZIL device >> unless it was mirrored and on different controllers. > > Um ... the log device is not special. If you lose *any* unmirrored device, > you lose the pool. Except for cache devices, or log devices on zpool >=19 > Well, if I've got a separate ZIL which is mirrored for performance, and mirrored because I think my data is valuable and important, I will have something more than RAID-0 on my main storage pool too. More than likely RAIDZ2 since I plan on using L2ARC to help improve performance along with separate SSD mirrored ZIL devices. > >> From the information I've been reading about the loss of a ZIL device, >> it will be relocated to the storage pool it is assigned to. I'm not >> sure which version this is in, but it would be nice if someone could >> provide the release number it is included in (and actually works), it >> would be nice. > > What the heck? Didn't I just answer that question? > I know I said this is answered in ZFS Best Practices Guide. > http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#Sepa > rate_Log_Devices > > Prior to pool version 19, if you have an unmirrored log device that fails, > your whole pool is permanently lost. > Prior to pool version 19, mirroring the log device is highly recommended. > In pool version 19 or greater, if an unmirrored log device fails during > operation, the system reverts to the default behavior, using blocks from the > main storage pool for the ZIL, just as if the log device had been gracefully > removed via the "zpool remove" command. > No need to get defensive here, all I'm looking for is the spool version number which supports it and the version of OpenSolaris which supports that ZPOOL version. I think that if you are building for performance, it would be almost intuitive to have a mirrored ZIL in the event of failure, and perhaps even a hot spare available as well. I don't like the idea of my ZIL being transferred back to the pool, but having it transferred back is better than the alternative which would be data loss or corruption. > >> Also, will this functionality be included in the >> mythical 2010.03 release? > > > Zpool 19 was released in build 125. Oct 16, 2009. You can rest assured it > will be included in 2010.03, or 04, or whenever that thing comes out. > Thanks, build 125. > >> So what you are saying is that if a single device fails in a striped >> L2ARC VDEV, then the entire VDEV is taken offline and the fallback is >> to simply use the regular ARC and fetch from the pool whenever there is >> a cache miss. > > It sounds like you're only going to believe it if you test it. Go for it. > That's what I did before I wrote that section of the ZFS Best Practices > Guide. > > In ZFS, there is no such thing as striping, although the term is commonly > used, because adding multiple devices creates all the benefit of striping, > plus all the benefit of concatenation, but colloquially, people think > concatenation is weird or unused or something, so people just naturally > gravitated to calling it a stripe in ZFS too, although that's not > technically correct according to the traditional RAID definition. But > nobody bothered to create a new term "stripecat" or whatever, for ZFS. > Ummm, yes you can create concatenated devices with ZFS, I have done this and even created mirrors of concatenated VDEVS in both configurations of RAID-0+1 and RAID-1+0. The only RAID level not supported by ZFS is RAID-4. > >> Or, does what you are saying here mean that if I have a 4 SSD's in a >> stripe for my L2ARC, and one device fails, the L2ARC will be >> reconfigured dynamically using the remaining SSD's for L2ARC. > > No reconfiguration necessary, because it's not a stripe. It's 4 separate > devices, which ZFS can use simultaneously if it wants to. > It's not a RAID-0 concatenated device either then? I mean that would be the most efficient way to balance the load across the four devices simultaneously, wouldn't it? Just want to make sure. I'm at a bit of a loss as to how the internals of the L2ARC work in the event of a single drive failure. My assumption was that if you used 4 SSD's as L2ARC, ZFS would be smart enough to spread the load and arrange the SSD's into a concatenated stripe. I'm not entirely clear on whether it actually does this or not, but from a performance standpoint, it would make sense. Additionally, from the VM model that ZFS is derived from, I would imagine these disks are arranged in a concat fashion. I understand that since the L2ARC is read only, a failure of a single device would result in a failure to validate the checksum, or even retrieve the block. The fall-back in this case would be like any other cache miss and go back out to the pool for the object. What I'd also like to know is a bit more of how the L2ARC is organized and used regardless of whether they are a concatenated device or independent devices (though I'd like verification of this, just for my own trivial pursuit), and how ZFS goes about reconfiguring or removing the failed device from the L2ARC. Mike --- Michael Sullivan michael.p.sulli...@me.com http://www.kamiogi.net/ Japan Mobile: +81-80-3202-2599 US Phone: +1-561-283-2034 _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
