Re: [zfs-discuss] Suggestions on 30 drive configuration?
Richard Elling wrote: Rob Logan wrote: an array of 30 drives in a RaidZ2 configuration with two hot spares I don't want to mirror 15 drives to 15 drives ok, so space over speed... and are willing to toss somewhere between 4 and 15 drives for protection. raidz splits the (up to 128k) write/read recordsize into each element of the raidz set.. (ie: all drives must be touched and all must finish before the block request is complete) so with a 9 disk raid1z set that's (8 data + 1 parity (8+1)) or 16k per disk for a full 128k write. or for a smaller 4k block, that a single 512b sector per disk. on a 26+2 raid2z set that 4k block would still use 8 disks, with the other 18 disks unneeded but allocated. It is not so easy to predict. ZFS will coalesce writes. A single transaction group may have many different writes in it. Also, raidz[12] is dynamic, and will use what it needs, unlike separate volume managers who do not have any understanding of the context of the data. There is a good slide which illustrates how stripe width is selected dynamically in RAID-Z. Please see slide 13 in this slide deck: http://www.snia.org/events/past/sdc2006/zfs_File_Systems-bonwick-moore.pdf Yes, some space may be wasted (marked by X on the slide), but there are guidelines for the number of the devices in the RAID-Z(2) vdev which allows to avoid this waste. Btw, I believe there's no link to this presentation on opensolaris.org. unfortunately... victor so perhaps three sets of 8+2 would let three blocks be read/written to at once with a total of 6 disks for protection. but for twice the speed, six sets of 4+1 would be the same size, (same number of disks for protection) but isn't quite as safe for its 2x speed. Yes, need to follow your priorities, there are just too many options otherwise. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Suggestions on 30 drive configuration?
On 28/06/2007, at 12:29 AM, Victor Latushkin wrote: It is not so easy to predict. ZFS will coalesce writes. A single transaction group may have many different writes in it. Also, raidz[12] is dynamic, and will use what it needs, unlike separate volume managers who do not have any understanding of the context of the data. There is a good slide which illustrates how stripe width is selected dynamically in RAID-Z. Please see slide 13 in this slide deck: http://www.snia.org/events/past/sdc2006/zfs_File_Systems-bonwick- moore.pdf [...] Btw, I believe there's no link to this presentation on opensolaris.org. unfortunately... Indeed. Is there any reason that the presentation at http:// www.opensolaris.org/os/community/zfs/docs/zfs_last.pdf Couldn't be updated to the one that Victor mentions? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Suggestions on 30 drive configuration?
an array of 30 drives in a RaidZ2 configuration with two hot spares I don't want to mirror 15 drives to 15 drives ok, so space over speed... and are willing to toss somewhere between 4 and 15 drives for protection. raidz splits the (up to 128k) write/read recordsize into each element of the raidz set.. (ie: all drives must be touched and all must finish before the block request is complete) so with a 9 disk raid1z set that's (8 data + 1 parity (8+1)) or 16k per disk for a full 128k write. or for a smaller 4k block, that a single 512b sector per disk. on a 26+2 raid2z set that 4k block would still use 8 disks, with the other 18 disks unneeded but allocated. so perhaps three sets of 8+2 would let three blocks be read/written to at once with a total of 6 disks for protection. but for twice the speed, six sets of 4+1 would be the same size, (same number of disks for protection) but isn't quite as safe for its 2x speed. Rob ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Suggestions on 30 drive configuration?
Dan Saul wrote: Good day ZFS-Discuss, I am planning to build an array of 30 drives in a RaidZ2 configuration with two hot spares. However I read on the internet that this was not ideal. So I ask those who are more experianced then me, what configuration would you recommend with ZFS, I would like to have some redundancy but still keeping as much disk space open for my uses as possible. I don't want to mirror 15 drives to 15 drives as that would drastically affect my storage capacity. There are hundreds of possible combinations of 30 drives. It really comes down to a trade-off of space vs performance vs RAS. http://blogs.sun.com/relling/entry/zfs_raid_recommendations_space_performance -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Suggestions on 30 drive configuration?
I care more about data integrity then performance. Of course if performance is so bad that one would not be able to, say stream a video off of it that wouldn't be acceptable. On 6/22/07, Richard Elling [EMAIL PROTECTED] wrote: Dan Saul wrote: Good day ZFS-Discuss, I am planning to build an array of 30 drives in a RaidZ2 configuration with two hot spares. However I read on the internet that this was not ideal. So I ask those who are more experianced then me, what configuration would you recommend with ZFS, I would like to have some redundancy but still keeping as much disk space open for my uses as possible. I don't want to mirror 15 drives to 15 drives as that would drastically affect my storage capacity. There are hundreds of possible combinations of 30 drives. It really comes down to a trade-off of space vs performance vs RAS. http://blogs.sun.com/relling/entry/zfs_raid_recommendations_space_performance -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Suggestions on 30 drive configuration?
Dan Saul wrote: I care more about data integrity then performance. Of course if performance is so bad that one would not be able to, say stream a video off of it that wouldn't be acceptable. The model I used in this blog deals with small, random reads, not streaming workloads. In part this is because the data needed to calculate small, random read performance is readily available on disk data sheets. For streaming workloads we can calculate the media bandwidth, which will be good when you have multiple drives. But there are other limitations in the system which will ultimately cap the bandwidth, and those limitations are not expressed in data sheets. The best way is to try it. Let us know how it works. -- richard On 6/22/07, Richard Elling [EMAIL PROTECTED] wrote: Dan Saul wrote: Good day ZFS-Discuss, I am planning to build an array of 30 drives in a RaidZ2 configuration with two hot spares. However I read on the internet that this was not ideal. So I ask those who are more experianced then me, what configuration would you recommend with ZFS, I would like to have some redundancy but still keeping as much disk space open for my uses as possible. I don't want to mirror 15 drives to 15 drives as that would drastically affect my storage capacity. There are hundreds of possible combinations of 30 drives. It really comes down to a trade-off of space vs performance vs RAS. http://blogs.sun.com/relling/entry/zfs_raid_recommendations_space_performance -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Suggestions on 30 drive configuration?
Dan Saul wrote: I care more about data integrity then performance. Of course if performance is so bad that one would not be able to, say stream a video off of it that wouldn't be acceptable. Any config I could imagine would be able to stream several videos at once (even 10Mbit/sec 1080p HD). For very high data integrity and plenty of performance for streaming a few videos, I'd try: 3 x 9-wide raidz-2 3 x hot spare For a total of 21 disks worth of usable space. If you need a bit more space, do: 2 x 14-wide raidz-2 2 x hot spare for a total of 24 disks worth of usable space. --matt On 6/22/07, Richard Elling [EMAIL PROTECTED] wrote: Dan Saul wrote: Good day ZFS-Discuss, I am planning to build an array of 30 drives in a RaidZ2 configuration with two hot spares. However I read on the internet that this was not ideal. So I ask those who are more experianced then me, what configuration would you recommend with ZFS, I would like to have some redundancy but still keeping as much disk space open for my uses as possible. I don't want to mirror 15 drives to 15 drives as that would drastically affect my storage capacity. There are hundreds of possible combinations of 30 drives. It really comes down to a trade-off of space vs performance vs RAS. http://blogs.sun.com/relling/entry/zfs_raid_recommendations_space_performance -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
