Re: [zfs-discuss] ZFS booting with Solaris (2007-08)
Sweet! Thank you! :) Bill This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] What would be the exact difference between import/mount and export/unmount ?
Hi All, What exactly import and export commands will do ? Are they similar to mount and unmount ? How import differs from mount and how export differs from umount ? If you run zpool import command, it will lists all theimportable zpools and the devices which are part of those zpools. How exactly import command works ? In my machine I have 10 to 15 zpools, and I am pumping IO on those zpools. IO pump tool is working fine for 2 to 3 hours after that some how my machine is going down. Can any one explain why it is happening ? I am not much aware of debug tools, Can any one explain how to debug coredump ? Your help is apprecialted. Thanks & Regards Masthan D - Fussy? Opinionated? Impossible to please? Perfect. Join Yahoo!'s user panel and lay it on us.___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS booting with Solaris (2007-08)
> 2. Back to Solaris Volume Manager (SVM), I guess. It's too bad too, because I > don't like it with 2 SATA disks either. There isn't enough drives to put the > State Database Replicas so that if either drive failed, the system would > reboot unattended. Unless there is a trick? There is a trick for this, not sure how long it's been around. Add to /etc/system: *Allow the system to boot if one of two rootdisks is missing set md:mirrored_root_flag=1 Good luck. --Kris -- Thomas Kris Kasner Qualcomm Inc. 5775 Morehouse Drive San Diego, CA 92121 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS booting with Solaris (2007-08)
k, Thanks for the tip. I spent a day trying it out, and this is what I learned ... 1. Solaris 10 (2007-08) doesn't have the "ZFS Boot Bits" installed 2. Only OpenSolaris version snv_62 or later has "ZFS Boot Bits" 3. Even if I got OpenSolaris working with ZFS booting the system, with a 2 disk SATA array, I would still have trouble; if one drive fails, ZFS won't boot. 4. Further there seems to be some issues with SATA framework and ZFS. Currently, it appears, it's best to use SAS or SCSI. It's too bad, I had high hopes that I could use ZFS to boot with the latest Solaris 10 build. With the "Live updating with ZFS running" problem solved, I was ready! On the other side, utilizing 2 filesystems on 2 drives is more complicated than I wanted to go. A couple questions ... 1. It appears that OpenSolaris has no way to get updates from Sun. (meaning Patch Manager doesn't work with it in CLI - #/usr/sbin/smpatch get -) So ... how do people "patch" OpenSolaris? 2. Back to Solaris Volume Manager (SVM), I guess. It's too bad too, because I don't like it with 2 SATA disks either. There isn't enough drives to put the State Database Replicas so that if either drive failed, the system would reboot unattended. Unless there is a trick? Thanks for the help, Bill This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS & array NVRAM cache?
Vincent Fox wrote: >> Solaris Cluster 3.2 supports Solaris 10 8/07 (aka >> u4). Where did you hear that >> it didn't? > > Took an Advanced Clustering class a few weeks before U4 came out. At that > time I think the instructor said U3 was the "supported" configuration and > he wasn't sure when U4 would be a verified and supported configuration on > which to run Cluster 3.2. Ah, ok that explains it. Prior to an OS release, Solaris Cluster will not be "supported" on it. While this seems intuitively obvious, the Office of Sales Prevention and Customer Disatisfaction insists upon promoting this fact :-( Solaris Cluster engineering tracks new Solaris releases very closely and will have support ready at release. Internal to Sun (and Sun partners) is the authoritative tome, "Sun Cluster 3 Configuration Guide," which contains the list of what is "supported." This is updated about once per month. But I'm not sure how that information is communicated externally... probably not well. It seems as though many docs say something like "ask your local Sun representative." In the long term, the Open HA Cluster community should be more responsive. http://opensolaris.org/os/community/ha-clusters/ > We have a Support Contract for production systems, and endeavour to stay > within the realm of what they will answer questions on when we have problems. > > So will this "nocache" option work in U4? It should be there. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS & array NVRAM cache?
On Sep 27, 2007, at 3:19 PM, Vincent Fox wrote: > So will this "nocache" option work in U4? Yes, it's in there: [EMAIL PROTECTED]/$ cat /etc/release Solaris 10 8/07 s10x_u4wos_12b X86 Copyright 2007 Sun Microsystems, Inc. All Rights Reserved. Use is subject to license terms. Assembled 16 August 2007 [EMAIL PROTECTED]/$ nm /kernel/fs/zfs | grep zfs_nocacheflush [1659] | 1892| 4|OBJT |GLOB |0|4 |zfs_nocacheflush ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS (and quota)
Roch - PAE wrote: > Pawel Jakub Dawidek writes: > > I'm CCing zfs-discuss@opensolaris.org, as this doesn't look like > > FreeBSD-specific problem. > > > > It looks there is a problem with block allocation(?) when we are near > > quota limit. tank/foo dataset has quota set to 10m: > > > > Without quota: > > > >FreeBSD: > ># dd if=/dev/zero of=/tank/test bs=512 count=20480 > >time: 0.7s > > > >Solaris: > ># dd if=/dev/zero of=/tank/test bs=512 count=20480 > >time: 4.5s > > > > With quota: > > > >FreeBSD: > ># dd if=/dev/zero of=/tank/foo/test bs=512 count=20480 > >dd: /tank/foo/test: Disc quota exceeded > >time: 306.5s > > > >Solaris: > ># dd if=/dev/zero of=/tank/foo/test bs=512 count=20480 > >write: Disc quota exceeded > >time: 602.7s > > > > CPU is almost entirely idle, but disk activity seems to be high. > > > > > Yes, as we are near quota limit, each transaction group > will accept a small amount as to not overshoot the limit. > > I don't know if we have the optimal strategy yet. > > -r Aside from the quota perf issue, has any analysis been done as to why FreeBSD is over 6X faster than Solaris without quotas? Do other perf tests show a similar disparity? Is there a difference in dd itself? I assume that it was identical hardware and pool config. Thanks: Neil. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation HP SAN, FC+SATA, ORACLE
>> 1. My understanding of the situation is having a zpool on a single LUN precludes some ZFS technology such as self-healing, hence is not best practice. Right? >For important data and when you have only one LUN (eg. arrays) using copies is a way to manage your redundancy policies for each file system. In general, we like to see ZFS have the ability to repair data. I'm not sure what you mean by copies. ZFS snapshots? Clones? >> 3. Suggestions for zpool, zfs file system layout >Follow the procedures for databases on ZFS at: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide Yeah, I read that. Thanks. I'm assuming he means something like: zfs create -o recsize=8k -o mountpoint=/ora01 san01/ora01 >Prior to Solaris 10u5 (spring '08?) you will want to put your Oracle redo log on a separate storage pool, preferrably on the fastest storage. But if you follow this advice, then I wouldn't try to mirror the redo log pool with a slow device because it is a write-mostly workload and will be sensitive to the latency of commits to both sides of the mirror. I eventually decided mirroring fast storage to slow was a stupid idea. Does the redo log filesystem need the same recsize as the filesystem ORACLE tables themselves? I thought they were more of a flat file. >NB since you only have one LUN, the performance won't be great anyway. Or to put this in perspective, competitive TPC-C benchmarks consume thousands of LUNs. In that case, you might consider investing in memory for a larger SGA. I would have assumed the SAN would be the bottleneck on this. Would having the SAN publish more LUNs buy me anything in terms of overall performance? Wouldn't the SAN be spinning its hamster wheel faster trying to deliver the data over multiple LUNs vs. one, losing us any performance gain on the ZFS end? This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS booting with Solaris (2007-08)
William Papolis wrote: > Hello there, > > I know this is an OpenSolaris forum, and likely if I used OpenSolaris I could > make this script work ... > > http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/ > > ..but I was trying to make it work with Solaris 10 (2007-08) and I got as far > as Step #5 before I encountered a bit of a problem. > > More specifically this line ... # zpool set bootfs=rootpool/rootfs rootpool > > As far as I can see, Solaris (2007-08) does not include the "ZFS Boot bits" > which means it doesn't support the boolean "bootfs". > > Now at this point I am a little stuck, because frankly I don't have a 100% > handle on exactly what I am doing, but I am thinking I can use this ... > > http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/mntroot-transition/ > > ... to boot ZFS the old "mountroot" way? > > Frankly I am guessing, and a better explanation would be much appreciated. > > Again, I know I am using Solaris 10 and not OpenSolaris. I was thinking > OpenSolaris wouldn't be good for a production system just yet, but I still > wanted to use ZFS if it's stable. Also, I figured if I posted here I would > get the best, quick response! Any help or explanation is much appreciated. > > You can certainly use ZFS on S10, but not as a root file system. None of that support has been backported and so zfs root is not yet ready for production use. Lori ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
A couple of Neptunes or a server with a Niagara T2 will fix that (10GigE) :^) On 9/27/07, David Dyer-Bennet <[EMAIL PROTECTED]> wrote: > > Well, as I said, I see no realistic risk of pushing the performance > limits of a home file server. I get *less* bandwidth through the > network than I do from a direct-connected drive, and that's just one > single drive. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
Solaris wrote: > You don't have to do it all at once... ZFS will function fine with 1 > large disk and 1 small disk in a mirror, it just means you will only > have the as much space as the smaller disk. > Sure, but you get no *benefit* until you've done it all, so you "have to" in terms of actually upgrading the vdev size. > As of things now, if you have multiple vdevs in a pool and they are of > diverse capacities, the striping becomes less effective (? not sure > what the right word here is). Each vdev will be capable of being used > to it's entire capacity. > > So suppose in your situation if you have 4 equal disks, configured in > two mirrors. In the future you reach 90% capacity and choose to > upgrade by doubling the size of one vdev. Your pool will use will > strip using the remaining 10% of the original capacity as expected and > then use only the larger vdev from there on. Further in the future if > you then choose to upgrade again by increasing the capacity of the > smaller vdev, the stiping will resume, but there is no way restripe > all of the data evenly across both disks with out copying it off, > removing it and copying it back again. > Well, as I said, I see no realistic risk of pushing the performance limits of a home file server. I get *less* bandwidth through the network than I do from a direct-connected drive, and that's just one single drive. > Overall, I'd would think that if being prepared to upgrade your > entire pool is a concern, that regardless of your zpool configuration > you would want to start saving from day 1 for the upgrade, wait until > the capacity becomes near critical and upgrade the entire pool. This > as opposed to being more reactionary and making a snap decision to buy > what you can afford to bandaid the situation. Afterall, by the time > you get around to upgrading the other vdev's the price/GB will have > dropped even further, assuming you don't replace them with yet even > larger disks than the other vdev. > I certainly expect each vdev to leapfrog the other when upgraded. That was the point. This way I have less paid-for but unused capacity lying around, and given the price and size of disk drives, that's a money-saver. > On 9/27/07, David Dyer-Bennet <[EMAIL PROTECTED]> wrote: > > >> Sure, that's the same process I have in mind, it's just that you have to >> replace all the disks in the vdev at once to get the new capacity, so I >> felt that sticking to mirrors (meaning only two disks in the vdev) was >> more suitable for my expected future history. >> >> -- >> David Dyer-Bennet, [EMAIL PROTECTED]; http://dd-b.net/ >> Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/ >> Photos: http://dd-b.net/photography/gallery/ >> Dragaera: http://dragaera.info >> >> >> > > -- David Dyer-Bennet, [EMAIL PROTECTED]; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Survivability of zfs root
Not long ago, I had a problem where I couldn't access a pool that was served up by a hardware raid card, after the raid card had been changed due to a hardware fault. As I recall, the devid of the devices in the pool didn't match once the hardware had been changed. The solution to this problem appears to be that you export the pool and import it again. Now, what if that system had been using ZFS root? I have a hardware failure, I replace the raid card, the devid of the boot device changes. Will the system still boot properly? -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] different/high atch/s, pflt/s, vflt/s on two systems
hello, a puzzle i hope you can shed some light on: I have two systems, built at different times, but running very similar software configurations (same hardware, patch rev). They *should* have near-identical load, as they are performing the same function in parallel behind a load balancer. However: system#2 is noticeably and consistently slower than system#1 (both are under high load): # sar -p 2 5 SunOS *system#1* 5.10 Generic_125100-10 sun4u09/26/2007 14:47:32 atch/s pgin/s ppgin/s pflt/s vflt/s slock/s 14:47:34 28.770.000.00 3439.18 196.590.00 14:47:36 23.690.000.00 2989.68 289.740.00 14:47:38 25.760.000.00 3312.57 270.750.00 14:47:40 17.610.000.00 1456.07 230.990.00 14:47:42 16.020.000.00 1474.72 140.620.00 Average22.370.000.00 2534.78 225.690.00 # sar -p 2 5 SunOS *system#2* 5.10 Generic_125100-10 sun4u09/26/2007 14:45:51 atch/s pgin/s ppgin/s pflt/s vflt/s slock/s 14:45:53 371.080.000.00 19009.31 1405.880.00 14:45:55 55.120.000.00 124.88 454.630.00 14:45:57 381.280.000.00 15695.57 993.600.00 14:45:59 149.510.000.00 19591.18 1829.900.00 14:46:01 125.000.000.00 12613.73 1639.710.00 Average 216.080.000.00 13391.67 1264.220.00 system #2 has a zfspool and a few (zfs) filesystems, which system#1 doesn't have. Otherwise, I'm unsure of what might be causing this. Thoughts on what I should be looking at? much thanks, Jonathan. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
You don't have to do it all at once... ZFS will function fine with 1 large disk and 1 small disk in a mirror, it just means you will only have the as much space as the smaller disk. As of things now, if you have multiple vdevs in a pool and they are of diverse capacities, the striping becomes less effective (? not sure what the right word here is). Each vdev will be capable of being used to it's entire capacity. So suppose in your situation if you have 4 equal disks, configured in two mirrors. In the future you reach 90% capacity and choose to upgrade by doubling the size of one vdev. Your pool will use will strip using the remaining 10% of the original capacity as expected and then use only the larger vdev from there on. Further in the future if you then choose to upgrade again by increasing the capacity of the smaller vdev, the stiping will resume, but there is no way restripe all of the data evenly across both disks with out copying it off, removing it and copying it back again. Overall, I'd would think that if being prepared to upgrade your entire pool is a concern, that regardless of your zpool configuration you would want to start saving from day 1 for the upgrade, wait until the capacity becomes near critical and upgrade the entire pool. This as opposed to being more reactionary and making a snap decision to buy what you can afford to bandaid the situation. Afterall, by the time you get around to upgrading the other vdev's the price/GB will have dropped even further, assuming you don't replace them with yet even larger disks than the other vdev. On 9/27/07, David Dyer-Bennet <[EMAIL PROTECTED]> wrote: > > > > Sure, that's the same process I have in mind, it's just that you have to > replace all the disks in the vdev at once to get the new capacity, so I > felt that sticking to mirrors (meaning only two disks in the vdev) was > more suitable for my expected future history. > > -- > David Dyer-Bennet, [EMAIL PROTECTED]; http://dd-b.net/ > Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/ > Photos: http://dd-b.net/photography/gallery/ > Dragaera: http://dragaera.info > > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Recommendation HP SAN, FC+SATA, ORACLE
Bruce Shaw wrote: > I've been presented with the following scenario: > > - this is to be used primarily for ORACLE, including usage of ORACLE > RMAN backups (to disk) > - HP SAN (will NOT do JBOD) > - 256 Gb disk available on high-speed Fibre Channel disk, currently on > one LUN (1) I presume this means a RAID array with nonvolatile write cache... > - 256 Gb disk available on slower-speed SATA disk, currently on one LUN > (2) > - the mount point must be /ora01 to match legacy scripts > > First proposed solution: > > - create one zpool using FC disk to hold ORACLE apps and data > - create one zpool using SATA disk to hold user directories, RMAN backup > files, etc. > > I think this lacks vision. > > My proposal (which I'm rethinking and request suggestions): > > - create one zpool out of both LUNs, perhaps a zpool mirror > - create /ora01, /ora_RMAN and /users zfs file systems underneath > > Hence: > > 1. My understanding of the situation is having a zpool on a single LUN > precludes some ZFS technology such as self-healing, hence is not best > practice. Right? Not completely correct, nor incorrect. You could use copies with a single LUN and be (mostly) protected against data loss. For important data and when you have only one LUN (eg. arrays) using copies is a way to manage your redundancy policies for each file system. In general, we like to see ZFS have the ability to repair data. > 2. My understanding of the situation is that ZFS is capable of > detecting the fast FC disk vs. the slow SATA disk and arranging the > underlying storage to best take advantage of this eg. shuffle the > snapshots off to slower disk. Right? No, not today. > 3. Suggestions for zpool, zfs file system layout I presume you'll use Solaris 10, so slogs are not available. Follow the procedures for databases on ZFS at: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide Prior to Solaris 10u5 (spring '08?) you will want to put your Oracle redo log on a separate storage pool, preferrably on the fastest storage. But if you follow this advice, then I wouldn't try to mirror the redo log pool with a slow device because it is a write-mostly workload and will be sensitive to the latency of commits to both sides of the mirror. NB since you only have one LUN, the performance won't be great anyway. Or to put this in perspective, competitive TPC-C benchmarks consume thousands of LUNs. In that case, you might consider investing in memory for a larger SGA. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
Solaris wrote: > I considered this as well, but that's the beauty of marrying ZFS with > a hotplug SATA backplane :) > > I chose the to use the 5-in-3 hot-swap chassis in order to give me a > way to upgrade capacity in place, though the 4-in-3 would be as easy, > though with higher risk. > > 1. hot-plug a new 500GB SATA disk into the 5th spot in the backplane. > > 2. zpool replace mypool c1t3d0 c1t4d0 > where c1t[0-3]d0 are my currently active 250GB drives > > 3. wait for resilver to complete. > > 4. hot-pull c1t3d0 and pull the drive from the chassis, replace with > new 500GB drive, hot-plug back into backplane. > > 5. zpool replace mypool c1t0d0 c1t3d0 > > 6. wait for resilver to complete, rinse and repeat. > > As soon as all disks have been replaced, my zpool will be 1TB, not 500GB. > Sure, that's the same process I have in mind, it's just that you have to replace all the disks in the vdev at once to get the new capacity, so I felt that sticking to mirrors (meaning only two disks in the vdev) was more suitable for my expected future history. -- David Dyer-Bennet, [EMAIL PROTECTED]; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
On Thu, 27 Sep 2007, Blake wrote: > For SATA support, try to stick to the Marvell/Supermicro Card, LSI, certain > Intel and Nvidia, and the Silicon Image 3112 and 3114 chipsets. The Sun HCL Avoid the 3112 and 3114. Last I looked they are supported by the legacy ata driver and performance is, for want of a better word, poor. A 3124 based card would be a good choice - as long as you use the fixed 3124 driver here: http://www.opensolaris.org/jive/servlet/JiveServlet/download/80-32437-138083-3390/si3124.tar.gz ... snip Al Hopper Logical Approach Inc, Plano, TX. [EMAIL PROTECTED] Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT OpenSolaris Governing Board (OGB) Member - Apr 2005 to Mar 2007 http://www.opensolaris.org/os/community/ogb/ogb_2005-2007/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
Christopher Gibbs wrote: > On 9/27/07, David Dyer-Bennet <[EMAIL PROTECTED]> wrote: > >> >> How long are you going to need this data? Do you have an easy and quick >> way to back it all up? Is the volume you need going to grow over time? >> For *my* home server, the need to expand over time ended up dominating >> the disk architecture, and I chose a less efficient (more space/money >> lost to redundant storage) architecture that was easier to upgrade in >> small increments, because that fit my intention to maintain the data >> long-term, and the lack of any efficient easy way to back up and restore >> the data (I *do* back it up to external firewire disks, but it takes 8 >> hours or so, so I don't want to have to have the system down for a full >> two-way copy when I need to upgrade the disk sizes). >> David, mind sharing the specifics of your configuration? >> >> I'm also about to re-configure my home file server so I'm interested >> in what configurations people are using. >> Asus M2n SLI Deluxe motherboard with AMD x2 processor and 2GB ram in a Chenbro case, maybe the 106? The pictures online show a different door than the one I have, so I may have the wrong model, but they may just have restyled the door. 8 hot-swap bays. Trouble is, hot-swap isn't recognized on the chipset on that motherboard, so I'm currently living with that. Currently using two bays for the boot disks, leaving my 6 bays (but only 4 controllers) for data disks. I picked up a bunch of 400GB spare SATA (Sun-branded Hitachi drives) from work when they were put out for disposal, so I've got two mirror pairs in there right now. Also I can't get RSA authentication to work with sshd on Solaris, and I keep asking here and elsewhere and nobody has been able to help yet, darn it (over a year now). I use RSA2 authentication to other servers just fine. -- David Dyer-Bennet, [EMAIL PROTECTED]; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
On 26/09/2007, Christopher <[EMAIL PROTECTED]> wrote: > I'm about to build a fileserver and I think I'm gonna use OpenSolaris and ZFS. > > I've got a 40GB PATA disk which will be the OS disk, Would be nice to remove that as a SPOF. I know ZFS likes whole disks, but I wonder how much would performance suffer if you SVMed up the first few Gb of a ZFS mirror pair for your root fs? I did it this week on Solaris 10 and it seemed to work pretty well ( http://number9.hellooperator.net/articles/2007/09/27/solaris-10-on-mirrored-disks ) Roll on ZFS root :) -- Rasputin :: Jack of All Trades - Master of Nuns http://number9.hellooperator.net/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
I did that - it was nice. Took forever though on my PIII 700mhz :^) blake/ On 9/27/07, Solaris <[EMAIL PROTECTED]> wrote: > > I considered this as well, but that's the beauty of marrying ZFS with > a hotplug SATA backplane :) > > I chose the to use the 5-in-3 hot-swap chassis in order to give me a > way to upgrade capacity in place, though the 4-in-3 would be as easy, > though with higher risk. > > 1. hot-plug a new 500GB SATA disk into the 5th spot in the backplane. > > 2. zpool replace mypool c1t3d0 c1t4d0 > where c1t[0-3]d0 are my currently active 250GB drives > > 3. wait for resilver to complete. > > 4. hot-pull c1t3d0 and pull the drive from the chassis, replace with > new 500GB drive, hot-plug back into backplane. > > 5. zpool replace mypool c1t0d0 c1t3d0 > > 6. wait for resilver to complete, rinse and repeat. > > As soon as all disks have been replaced, my zpool will be 1TB, not 500GB. > > On 9/27/07, [EMAIL PROTECTED] > <[EMAIL PROTECTED]> wrote: > > Date: Thu, 27 Sep 2007 13:20:15 -0500 > > From: David Dyer-Bennet <[EMAIL PROTECTED]> > > Subject: Re: [zfs-discuss] Best option for my home file server? > > To: zfs-discuss@opensolaris.org > > Message-ID: <[EMAIL PROTECTED]> > > Content-Type: text/plain; charset=UTF-8; format=flowed > > > > Blake wrote: > > >> Obviously from a cost AND size perspective it would be best/smart to > go > > >> for option 3 and have a raidz of 4x250 and one of 6x500. > > >> > > >> Comments? > > >> > > >> > > > > How long are you going to need this data? Do you have an easy and quick > > way to back it all up? Is the volume you need going to grow over time? > > For *my* home server, the need to expand over time ended up dominating > > the disk architecture, and I chose a less efficient (more space/money > > lost to redundant storage) architecture that was easier to upgrade in > > small increments, because that fit my intention to maintain the data > > long-term, and the lack of any efficient easy way to back up and restore > > the data (I *do* back it up to external firewire disks, but it takes 8 > > hours or so, so I don't want to have to have the system down for a full > > two-way copy when I need to upgrade the disk sizes). > > > > -- > > David Dyer-Bennet, [EMAIL PROTECTED]; http://dd-b.net/ > > Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/ > > Photos: http://dd-b.net/photography/gallery/ > > Dragaera: http://dragaera.info > > > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS & array NVRAM cache?
> > Solaris Cluster 3.2 supports Solaris 10 8/07 (aka > u4). Where did you hear that > it didn't? Took an Advanced Clustering class a few weeks before U4 came out. At that time I think the instructor said U3 was the "supported" configuration and he wasn't sure when U4 would be a verified and supported configuration on which to run Cluster 3.2. We have a Support Contract for production systems, and endeavour to stay within the realm of what they will answer questions on when we have problems. So will this "nocache" option work in U4? This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
I considered this as well, but that's the beauty of marrying ZFS with a hotplug SATA backplane :) I chose the to use the 5-in-3 hot-swap chassis in order to give me a way to upgrade capacity in place, though the 4-in-3 would be as easy, though with higher risk. 1. hot-plug a new 500GB SATA disk into the 5th spot in the backplane. 2. zpool replace mypool c1t3d0 c1t4d0 where c1t[0-3]d0 are my currently active 250GB drives 3. wait for resilver to complete. 4. hot-pull c1t3d0 and pull the drive from the chassis, replace with new 500GB drive, hot-plug back into backplane. 5. zpool replace mypool c1t0d0 c1t3d0 6. wait for resilver to complete, rinse and repeat. As soon as all disks have been replaced, my zpool will be 1TB, not 500GB. On 9/27/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Date: Thu, 27 Sep 2007 13:20:15 -0500 > From: David Dyer-Bennet <[EMAIL PROTECTED]> > Subject: Re: [zfs-discuss] Best option for my home file server? > To: zfs-discuss@opensolaris.org > Message-ID: <[EMAIL PROTECTED]> > Content-Type: text/plain; charset=UTF-8; format=flowed > > Blake wrote: > >> Obviously from a cost AND size perspective it would be best/smart to go > >> for option 3 and have a raidz of 4x250 and one of 6x500. > >> > >> Comments? > >> > >> > > How long are you going to need this data? Do you have an easy and quick > way to back it all up? Is the volume you need going to grow over time? > For *my* home server, the need to expand over time ended up dominating > the disk architecture, and I chose a less efficient (more space/money > lost to redundant storage) architecture that was easier to upgrade in > small increments, because that fit my intention to maintain the data > long-term, and the lack of any efficient easy way to back up and restore > the data (I *do* back it up to external firewire disks, but it takes 8 > hours or so, so I don't want to have to have the system down for a full > two-way copy when I need to upgrade the disk sizes). > > -- > David Dyer-Bennet, [EMAIL PROTECTED]; http://dd-b.net/ > Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/ > Photos: http://dd-b.net/photography/gallery/ > Dragaera: http://dragaera.info > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
I was recently evaluating much the same question but with out only a single pool and sizing my disks equally. I only need about 500GB of usable space and so I was considering the value of 4x 250GB SATA Drives versus 5x 160GB SATA drives. I had intended to use an AMS 5 disk in 3 5.25" bay hot-swap backplane. http://www.american-media.com/product/backplane/sata300/sata300.html I priced Seagate 250GB and 160GB SATA drives at $70 and $54 USD each, respectively. The backplane runs about $130. The config options I considered are: RZ1 +spare RZ2 -spare mirror(s) (+spare w/ 5 160GB disks) Using Richard's blogs to help my evaluation I made the following conclusions between the choices... - Mirrors cost you 50% of your total space, but give you the best performance and "average" fault tolerance. - RZ1+spare gives you the same space (2 data, 1 parity, 1 spare w/ 4 disks), middle of the road performance and the worst fault tolerance. - RZ2 gives you the same space as well (2 data, 2 parity w/ 4 disks), the worst performance and the best fault tolerance. - spares give you negligible benefit over time with respect to cost. What I mean by this is that for the "home" environment, if a disk costs $100 today, next year it will cost $80, the year after even less. Paying up front for a relatively small increase in reliability that declines quickly as time goes on is probably not a good "value" decision. In addition, they are readily available in most populated areas if you can't wait for warranty replacement...and Seagate offers 5 yr warranty (which is why I spec'd Seagate not WD). My conclusion was to go with 4x 250GB in a single pool with two mirror vdevs. Overall I'd suggest that you scale your disks to be in proportion with your target total usable size and keep it simple and to an even number of disks. I still have yet to purchase the system due to my issues with finding the right board with the right SATA controller. My desktop system at home runs an nVidia 590a chipset on a Foxconn motherboard and Solaris U3 will only recognize the DVD drive during installation. A side note to that compatibility issue... quick/dirty solution to the SATA controller issue... install VMWare Server and configure VMWare to supply the guest raw disks. VMWare Server provides a nice hardware abstraction layer that has turned my a partition on my SATA disk into a SCSI hard drive that Solaris interacts with swimmingly. I loose a little system resource to Host OS and VMWare overhead, but at least I have a fully time Solaris system running on my home network on hardware it would not otherwise agree with. On 9/27/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Date: Thu, 27 Sep 2007 10:10:24 PDT > From: Christopher <[EMAIL PROTECTED]> > Subject: Re: [zfs-discuss] Best option for my home file server? > To: zfs-discuss@opensolaris.org > Message-ID: <[EMAIL PROTECTED]> > Content-Type: text/plain; charset=UTF-8 > > Hmm.. Thanks for the input. I want to have the most space but still need a > raid in some way to have redundancy. > > I've added it up and found this: > ggendel - your suggestiong makes me "loose" 1TB - Loose 250GBx2 for the > raid-1 ones and then 500GB from a 3x500GB = 1TB > bonwick - your first suggestion makes me "loose" 1TB. The second 750GB. The > third, still 750GB but I gain 500GB more, since I know only loose 1/3 of 1500 > instead of 1/2 of 1000. > > ggendel - yeah I know you would degrade both pools, but you would still be > able to recover, unless our good friend Murphy comes around in between, as I > would expect him to :-/ > > So, as bonwick said - let's keep it simple :) No need to make it very complex. > > How is SATA support in OpenSolaris these days. I've read about ppl saying it > has poor support, but I believe it was blogs and suchs from 2006. I > downloaded the Developer Edition yesterday. > > I can have 10 SATA disks in my tower (8 onbord sata connections and 2 from a > controller card). Why not fill it :) > > I made a calculation, buying 1x500+3x750, 4x750 or 4x500 disks. The price/GB > doesn't differ much here in Norway. > > Option 1 - Buying 4x750GB disks: > 4x250 RaidZ - 750/250 (Raid size/lost to redundancy) > 2x500 Raid1 - 500/500 > 4x750 RaidZ - 2250/750 > Equals: 3500/1500 (3500GB space / 1500GB lost to redundancy) > Cost: 4x750 costs NOK6000 = US$1100 > > Option 2 - Buy 1x500 + 3x750 > 4x250 RaidZ - 750/250 > 3x500 Raid1 - 1000/500 > 3x750 RaidZ - 1500/750 > Equals 3250/1500 > 1x750 costs NOK 1500 = US$ 270 > 3x500 costs NOK 3000 = US$ 550 > Total US$ 820 > > Option 3 - Buying 4x500GB disks: > 4x250 RaidZ - 750/250 > 6x500 Raid1 - 2500/500 > Equals: 3250/750 > Cost: 4x500 costs NOK4000 = US$ 720 > > Option 2 is not winning in either cost or space, so thats out. > Option 1 gives me 250GB more space but costs me NOK 2000 / US$ 360 more than > option 3. For NOK 2000 I could get two more 500GB disks or one big 1TB disk. > > Obviously from a cost AND size perspective it would b
Re: [zfs-discuss] Best option for my home file server?
David, mind sharing the specifics of your configuration? I'm also about to re-configure my home file server so I'm interested in what configurations people are using. On 9/27/07, David Dyer-Bennet <[EMAIL PROTECTED]> wrote: > Blake wrote: > >> Obviously from a cost AND size perspective it would be best/smart to go > >> for option 3 and have a raidz of 4x250 and one of 6x500. > >> > >> Comments? > >> > >> > > How long are you going to need this data? Do you have an easy and quick > way to back it all up? Is the volume you need going to grow over time? > For *my* home server, the need to expand over time ended up dominating > the disk architecture, and I chose a less efficient (more space/money > lost to redundant storage) architecture that was easier to upgrade in > small increments, because that fit my intention to maintain the data > long-term, and the lack of any efficient easy way to back up and restore > the data (I *do* back it up to external firewire disks, but it takes 8 > hours or so, so I don't want to have to have the system down for a full > two-way copy when I need to upgrade the disk sizes). > > -- > David Dyer-Bennet, [EMAIL PROTECTED]; http://dd-b.net/ > Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/ > Photos: http://dd-b.net/photography/gallery/ > Dragaera: http://dragaera.info > > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > -- Christopher Gibbs Email / LDAP Administrator Web Integration & Programming Abilene Christian University ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ZFS booting with Solaris (2007-08)
Hello there, I know this is an OpenSolaris forum, and likely if I used OpenSolaris I could make this script work ... http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/ ..but I was trying to make it work with Solaris 10 (2007-08) and I got as far as Step #5 before I encountered a bit of a problem. More specifically this line ... # zpool set bootfs=rootpool/rootfs rootpool As far as I can see, Solaris (2007-08) does not include the "ZFS Boot bits" which means it doesn't support the boolean "bootfs". Now at this point I am a little stuck, because frankly I don't have a 100% handle on exactly what I am doing, but I am thinking I can use this ... http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/mntroot-transition/ ... to boot ZFS the old "mountroot" way? Frankly I am guessing, and a better explanation would be much appreciated. Again, I know I am using Solaris 10 and not OpenSolaris. I was thinking OpenSolaris wouldn't be good for a production system just yet, but I still wanted to use ZFS if it's stable. Also, I figured if I posted here I would get the best, quick response! Any help or explanation is much appreciated. Thanks, Bill This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] (no subject)
> David Runyon sun.com> writes: > > > > I'm trying to get maybe 200 MB/sec over NFS for > large movie files (need > > (I assume you meant 200 Mb/sec with a lower case > "b".) > > > large capacity to hold all of them). Are there any > rules of thumb on how > > much RAM is needed to handle this (probably RAIDZ > for all the disks) with > > zfs, and how large a server should be used ? > > If you have a handful of users streaming large movie > files over NFS, > RAM is not going to be a bottleneck. One of my ultra > low-end server > (Turion MT-37 2.0 GHz, 512 MB RAM, five 500-GB SATA > disk in a raidz1, > consumer-grade Nvidia GbE NIC) running an old Nevada > b55 install can > serve large files at about 650-670 Mb/sec over NFS. > CPU is the > bottleneck at this level. The same box with a > slightly better CPU > or a better NIC (with a less CPU-intensive driver > that doesn't generate > 45k interrupt/sec) would be capable of maxing out the > GbE link. > > -marc > > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discu > ss Thanks! This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
Blake wrote: >> Obviously from a cost AND size perspective it would be best/smart to go >> for option 3 and have a raidz of 4x250 and one of 6x500. >> >> Comments? >> >> How long are you going to need this data? Do you have an easy and quick way to back it all up? Is the volume you need going to grow over time? For *my* home server, the need to expand over time ended up dominating the disk architecture, and I chose a less efficient (more space/money lost to redundant storage) architecture that was easier to upgrade in small increments, because that fit my intention to maintain the data long-term, and the lack of any efficient easy way to back up and restore the data (I *do* back it up to external firewire disks, but it takes 8 hours or so, so I don't want to have to have the system down for a full two-way copy when I need to upgrade the disk sizes). -- David Dyer-Bennet, [EMAIL PROTECTED]; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
For SATA support, try to stick to the Marvell/Supermicro Card, LSI, certain Intel and Nvidia, and the Silicon Image 3112 and 3114 chipsets. The Sun HCL is invaluable here. Note that you want ZFS to be looking directly at the disks, not at some abstraction created by hardware RAID controllers. Therefore, you may have to set the chipset bios on these to JBOD or vanilla controller mode. Blake On 9/27/07, Christopher <[EMAIL PROTECTED]> wrote: > > Hmm.. Thanks for the input. I want to have the most space but still need a > raid in some way to have redundancy. > > I've added it up and found this: > ggendel - your suggestiong makes me "loose" 1TB - Loose 250GBx2 for the > raid-1 ones and then 500GB from a 3x500GB = 1TB > bonwick - your first suggestion makes me "loose" 1TB. The second 750GB. > The third, still 750GB but I gain 500GB more, since I know only loose 1/3 of > 1500 instead of 1/2 of 1000. > > ggendel - yeah I know you would degrade both pools, but you would still be > able to recover, unless our good friend Murphy comes around in between, as I > would expect him to :-/ > > So, as bonwick said - let's keep it simple :) No need to make it very > complex. > > How is SATA support in OpenSolaris these days. I've read about ppl saying > it has poor support, but I believe it was blogs and suchs from 2006. I > downloaded the Developer Edition yesterday. > > I can have 10 SATA disks in my tower (8 onbord sata connections and 2 from > a controller card). Why not fill it :) > > I made a calculation, buying 1x500+3x750, 4x750 or 4x500 disks. The > price/GB doesn't differ much here in Norway. > > Option 1 - Buying 4x750GB disks: > 4x250 RaidZ - 750/250 (Raid size/lost to redundancy) > 2x500 Raid1 - 500/500 > 4x750 RaidZ - 2250/750 > Equals: 3500/1500 (3500GB space / 1500GB lost to redundancy) > Cost: 4x750 costs NOK6000 = US$1100 > > Option 2 - Buy 1x500 + 3x750 > 4x250 RaidZ - 750/250 > 3x500 Raid1 - 1000/500 > 3x750 RaidZ - 1500/750 > Equals 3250/1500 > 1x750 costs NOK 1500 = US$ 270 > 3x500 costs NOK 3000 = US$ 550 > Total US$ 820 > > Option 3 - Buying 4x500GB disks: > 4x250 RaidZ - 750/250 > 6x500 Raid1 - 2500/500 > Equals: 3250/750 > Cost: 4x500 costs NOK4000 = US$ 720 > > Option 2 is not winning in either cost or space, so thats out. > Option 1 gives me 250GB more space but costs me NOK 2000 / US$ 360 more > than option 3. For NOK 2000 I could get two more 500GB disks or one big 1TB > disk. > > Obviously from a cost AND size perspective it would be best/smart to go > for option 3 and have a raidz of 4x250 and one of 6x500. > > Comments? > > > This message posted from opensolaris.org > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
Hmm.. Thanks for the input. I want to have the most space but still need a raid in some way to have redundancy. I've added it up and found this: ggendel - your suggestiong makes me "loose" 1TB - Loose 250GBx2 for the raid-1 ones and then 500GB from a 3x500GB = 1TB bonwick - your first suggestion makes me "loose" 1TB. The second 750GB. The third, still 750GB but I gain 500GB more, since I know only loose 1/3 of 1500 instead of 1/2 of 1000. ggendel - yeah I know you would degrade both pools, but you would still be able to recover, unless our good friend Murphy comes around in between, as I would expect him to :-/ So, as bonwick said - let's keep it simple :) No need to make it very complex. How is SATA support in OpenSolaris these days. I've read about ppl saying it has poor support, but I believe it was blogs and suchs from 2006. I downloaded the Developer Edition yesterday. I can have 10 SATA disks in my tower (8 onbord sata connections and 2 from a controller card). Why not fill it :) I made a calculation, buying 1x500+3x750, 4x750 or 4x500 disks. The price/GB doesn't differ much here in Norway. Option 1 - Buying 4x750GB disks: 4x250 RaidZ - 750/250 (Raid size/lost to redundancy) 2x500 Raid1 - 500/500 4x750 RaidZ - 2250/750 Equals: 3500/1500 (3500GB space / 1500GB lost to redundancy) Cost: 4x750 costs NOK6000 = US$1100 Option 2 - Buy 1x500 + 3x750 4x250 RaidZ - 750/250 3x500 Raid1 - 1000/500 3x750 RaidZ - 1500/750 Equals 3250/1500 1x750 costs NOK 1500 = US$ 270 3x500 costs NOK 3000 = US$ 550 Total US$ 820 Option 3 - Buying 4x500GB disks: 4x250 RaidZ - 750/250 6x500 Raid1 - 2500/500 Equals: 3250/750 Cost: 4x500 costs NOK4000 = US$ 720 Option 2 is not winning in either cost or space, so thats out. Option 1 gives me 250GB more space but costs me NOK 2000 / US$ 360 more than option 3. For NOK 2000 I could get two more 500GB disks or one big 1TB disk. Obviously from a cost AND size perspective it would be best/smart to go for option 3 and have a raidz of 4x250 and one of 6x500. Comments? This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Slow ZFS Performance
Pete Majka wrote: > I have a raidz zpool comprised of 5 320GB SATA drives and I am seeing the > following numbers. > >capacity operationsbandwidth > pool used avail read write read write > -- - - - - - - > vol01 123G 1.33T 66182 8.22M 9.85M > > The overall IO rate is very poor at the application level as the latency is > high as well. What are your goals? > Can anyone point me in a good direction, as all the roads I have gone down > have led to nowhere. I can't tell if your workload is small, random reads. It does not appear to be. However, if your workload is small, random reads then see: http://blogs.sun.com/relling/entry/zfs_raid_recommendations_space_performance http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide 66 iops looks like the performance I would expect from a 3.5" 7,200 rpm disk. However, there is no service time information from this output. You'd need to check with iostat to see whether the disks are actually busy. If they are not busy, then look elsewhere. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Best option for my home file server?
On Wed, 26 Sep 2007, Jeff Bonwick wrote: > I would keep it simple. Let's call your 250GB disks A, B, C, D, > and your 500GB disks X and Y. I'd either make them all mirrors: > >zpool create mypool mirror A B mirror C D mirror X Y > > or raidz the little ones and mirror the big ones: > >zpool create mypool raidz A B C D mirror X Y > > or, as you mention, get another 500GB disk, Z, and raidz like this: > >zpool create mypool raidz A B C D raidz X Y Z +1 All excellent solutions Also consider two pools, one a raidz and the 2nd a 2-way mirror. That way you can take advantage of the different operational characteristics of each pool: zpool create mypool raidz A B C D zpool create mirpool mirror X Y > Jeff > > On Wed, Sep 26, 2007 at 01:06:38PM -0700, Christopher wrote: >> I'm about to build a fileserver and I think I'm gonna use OpenSolaris and >> ZFS. >> >> I've got a 40GB PATA disk which will be the OS disk, and then I've got >> 4x250GB SATA + 2x500GB SATA disks. From what you are writing I would think >> my best option would be to slice the 500GB disks in two 250GB and then make >> two RAIDz with two 250 disks and one partition from each 500 disk, giving me >> two RAIDz of 4 slices of 250, equaling to 2 x 750GB RAIDz. >> >> How would the performance be with this? I mean, it would probably drop since >> I would have two raidz slices on one disk. >> >>> From what I gather, I would still be able to lose one of the 500 disks (or >>> 250) and still be able to recover, right? >> >> Perhaps I should just get another 500GB disk and run a RAIDz on the 500s and >> one RAIDz on the 250s? >> >> I'm also a bit of a noob when it comes to ZFS (but it looks like it's not >> that hard to admin) - Would I be able to join the two RAIDz together for one >> BIG volume altogether? And it will survive one disk failure? >> >> /Christopher >> >> >> This message posted from opensolaris.org >> ___ >> zfs-discuss mailing list >> zfs-discuss@opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > ___ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > Al Hopper Logical Approach Inc, Plano, TX. [EMAIL PROTECTED] Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT OpenSolaris Governing Board (OGB) Member - Apr 2005 to Mar 2007 http://www.opensolaris.org/os/community/ogb/ogb_2005-2007/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Balancing reads across mirror sets
On Wed, 26 Sep 2007, Jason P. Warr wrote: > Hi all, > > I have an interesting project that I am working on. It is a large > volume file download service that is in need of a new box. There > current systems are not able to handle the load because for various > reasons they have become very I/O limited. We currently run on > Debian Linux with 3ware hardware RAID5. I am not sure of the exact > disk config as it is a leased system that I have never seen. > > The usage patterns are usually pretty consistent in that out of > 2-300 concurrent downloads you will find between 15 and 20 different > files being fetched. You can guess that when the machine is trying > to push a total of 2-300Mbit the disks are going crazy and due to > file sizes and only having 4GB of ram on the system caching is of > little use. The systems will regularly get into a 80-90% I/O wait > mode. The disk write speed is of almost no concern as there are > only a few files added each day. > > The system we have come up with is pretty robust. 2 dual core > Opterons, 32GB of ram, 8 750GB SATA disks. The disks are going to > be paired off in 2 disk RAID0 sets each with a complete copy of the > data. Essentially a manually replicated 4 way mirror set. The > download manager would then use each set in a round robin fashion. > This should substantially reduce the amount of frantic disk head > dancing. The second item is to dedicate 50-75% of the ram to a > ramdisk that would be the fetch path for the top 10 and new, hot > downloads. This should again reduce the seeking on files that are > being downloaded many times concurrently. > > My question for this list is with ZFS is there a way to access the > individual mirror sets in a pool if I were to create a 4 way > mirrored stripe set with the 8 disks? Even better would be if zfs > would manage the mirror set "load balancing" by intelligently > splitting up the reads amongst the 4 sets. Either way would make > for a more elegant solution than replicating the sets with > cron/rsync. > Your basic requirement is for good/excellent random read performance with many concurrent IOPS (I/O Operations/Sec). The metric I use for disk IOPS are: disk driveIOPS -- 7,200 RPM SATA300 15k RPM 700 These numbers can be debated/argued - but that is what I use. So, to solve your problem, my first recommendation would be to use 15k RPM SAS drives, rather than SATA drives, for random I/O. Next, ZFS will automatically load balance read requests among the members of a multi-way mirror set. So if you were to form a pool like: zpool create fastrandom mirror disk1 disk2 disk3 disk4 you'd have a 4-way mirror that would sustain approx 1,200 (4 * 300) reads/Sec with SATA disks and 2,800 reads/Sec with 15k SAS disks. In addition, ZFS will make intelligent use of available RAM to cache data. I would suggest that you use the above config as a starting point and measure the resulting performance running the anticipated workload - without using any system memory for use as system buffering or as a RAM disk. Since its very convenient/fast to configure/reconfigure storage pools using the zfs interface, you can also experiment with 5-way, 6-way etc. pools - or form one pool using 4 devices for fast random access and assign the remaining disks as a raidz pool to maximize storage space. PS: If you take a look at genunix.org, you'll see I have some experience with this type of workload. We'll be deploying a zfs based storage system there next month - we had to resolve your type of issue when Belenix 0.6 was released - except for one twist - we had to use the existing infrastructure and would not accept any downtime. Feel free to email me offlist if I can help. Regards, Al Hopper Logical Approach Inc, Plano, TX. [EMAIL PROTECTED] Voice: 972.379.2133 Fax: 972.379.2134 Timezone: US CDT OpenSolaris Governing Board (OGB) Member - Apr 2005 to Mar 2007 http://www.opensolaris.org/os/community/ogb/ogb_2005-2007/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
