Re: [zfs-discuss] Suggested RaidZ configuration...
Thanks for all the replies :) My mindset is split in two now... Some detail - I'm using 4 1-to-5 Sata Port multipliers connected to a 4-port SATA raid card. I only need reliability and size, as long as my performance is the equivalent of one drive, Im happy. Im assuming all the data used in the group is read once when re-creating a lost drive. Also assuming space consumed is 50%. So option 1 - Stay with the 2 x 10drive RaidZ2. My concern is the stress on the drives when one drive fails and the others go crazy (read-wise) to re-create the new drive. Is there no way to reduce this stress? Maybe limit the data rate, so its not quite so stressful, even though it will end up taking longer? (quite acceptable) [Available Space: 16TB, Redundancy Space: 4TB, Repair data read: 4.5TB] And option 2 - Add a 21st drive to one of the motherboard sata ports. And then go with 3 x 7drive RaidZ2. [Available Space: 15TB, Redundancy Space: 6TB, Repair data read: 3TB] Sadly, SSD's wont go too well in a PM based setup like mine. I may add it directly onto the MB if I can afford it. But again, performance is not a prioity. Any further thoughts and ideas are much appreciated. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] onnv_142 - vfs_mountroot: cannot mount root
Did you run installgrub before rebooting? On Tue, 7 Sep 2010, Piotr Jasiukajtis wrote: Hi, After upgrade from snv_138 to snv_142 or snv_145 I'm unable to boot the system. Here is what I get. Any idea why it's not able to import rpool? I saw this issue also on older builds on a different machines. -- Piotr Jasiukajtis | estibi | SCA OS0072 http://estseg.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] onnv_142 - vfs_mountroot: cannot mount root
On 07.09.10 17:26, Piotr Jasiukajtis wrote: Hi, After upgrade from snv_138 to snv_142 or snv_145 I'm unable to boot the system. Here is what I get. Any idea why it's not able to import rpool? Provided output tells that it was able to read device labels, construct configuration and add it into spa namespace, but then spa_open() called as part of bootfs property processing returned EINVAL. I think you can step through spa_open() to find out where exactly and why. for that you'll have to place a breakpoint in rootconf() somewhere close but before call to vfs_mountroot() and continue, and when it stops in rootconf() you'll be able to set another breakpoint(s) anywhere you like in the ZFS code. regards victor I saw this issue also on older builds on a different machines. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- -- Victor Latushkin phone: x11467 / +74959370467 TSC-Kernel EMEAmobile: +78957693012 Sun Services, Moscow blog: http://blogs.sun.com/vlatushkin Sun Microsystems ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Suggested RaidZ configuration...
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of hatish I have just read the Best Practices guide, and it says your group shouldnt have 9 disks. I think the value you can take from this is: Why does the BPG say that? What is the reasoning behind it? Anything that is a rule of thumb either has reasoning behind it (you should know the reasoning) or it doesn't (you should ignore the rule of thumb, dismiss it as myth.) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Suggested RaidZ configuration...
may be 5x(3+1) use one disk from each controller, 15TB usable space, 3+1 raidz rebuild time should be reasonable On 9/7/2010 4:40 AM, hatish wrote: Thanks for all the replies :) My mindset is split in two now... Some detail - I'm using 4 1-to-5 Sata Port multipliers connected to a 4-port SATA raid card. I only need reliability and size, as long as my performance is the equivalent of one drive, Im happy. Im assuming all the data used in the group is read once when re-creating a lost drive. Also assuming space consumed is 50%. So option 1 - Stay with the 2 x 10drive RaidZ2. My concern is the stress on the drives when one drive fails and the others go crazy (read-wise) to re-create the new drive. Is there no way to reduce this stress? Maybe limit the data rate, so its not quite so stressful, even though it will end up taking longer? (quite acceptable) [Available Space: 16TB, Redundancy Space: 4TB, Repair data read: 4.5TB] And option 2 - Add a 21st drive to one of the motherboard sata ports. And then go with 3 x 7drive RaidZ2. [Available Space: 15TB, Redundancy Space: 6TB, Repair data read: 3TB] Sadly, SSD's wont go too well in a PM based setup like mine. I may add it directly onto the MB if I can afford it. But again, performance is not a prioity. Any further thoughts and ideas are much appreciated. attachment: laotsao.vcf___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] zpool create using whole disk - do I add p0? E.g. c4t2d0 or c42d0p0
I have seen conflicting examples on how to create zpools using full disks. The zpool(1M) page uses c0t0d0 but OpenSolaris Bible and others show c0t0d0p0. E.g.: zpool create tank raidz c0t0d0 c0t1d0 c0t2d0 c0t3d0 c0t4d0 c0t5d0 zpool create tank raidz c0t0d0p0 c0t1d0p0 c0t2d0p0 c0t3d0p0 c0t4d0p0 c0t5d0p0 I have not been able to find any discussion on whether (or when) to add the p0 to the disk name while creating whole disk zpools. In my case, I will have 9 disks in Raid-Z3 (home file server; requirements are adequate performance, maximum long-term storage reliability and minimal maintenance). I want to setup the zpools correct the first time to avoid any future issues. Thanks for you help and insight. -- Craig -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zpool create using whole disk - do I add p0? E.g. c4t2d0 or c42d0p0
Hi Craig, D'oh. I kept wondering where those p0 examples were coming from. Don't use the p* devices for your storage pools. They represent the larger fdisk partition. Use the d* devices instead, like this example below: zpool create tank raidz c0t0d0 c0t1d0 c0t2d0 c0t3d0 c0t4d0 c0t5d0 Thanks, Cindy On 09/07/10 09:59, Craig Stevenson wrote: I have seen conflicting examples on how to create zpools using full disks. The zpool(1M) page uses c0t0d0 but OpenSolaris Bible and others show c0t0d0p0. E.g.: zpool create tank raidz c0t0d0 c0t1d0 c0t2d0 c0t3d0 c0t4d0 c0t5d0 zpool create tank raidz c0t0d0p0 c0t1d0p0 c0t2d0p0 c0t3d0p0 c0t4d0p0 c0t5d0p0 I have not been able to find any discussion on whether (or when) to add the p0 to the disk name while creating whole disk zpools. In my case, I will have 9 disks in Raid-Z3 (home file server; requirements are adequate performance, maximum long-term storage reliability and minimal maintenance). I want to setup the zpools correct the first time to avoid any future issues. Thanks for you help and insight. -- Craig ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Configuration questions for Home File Server (CPU cores, dedup, checksum)?
I am working on a home file server. After reading a wide range of blogs and
forums, I have a few questions that are still not clear to me
1. Is there a benefit in having quad core CPU (e.g. Athlon II X4 vs X2)? All
of the web blogs seem to suggest using lower-wattage dual core CPUs. But; with
the recent advent of dedup, SHA256 checksum, etc., I am now wondering if
opensolaris is better served with quad core.
2. For checksum on ZFS datasets, is there value in using Fletcher4 or SHA256
instead of just using the default (which Evil Tuning Guide says is Fletcher2,
others say it is Fletcher4)? And, should I consider using different checksums
for different ZFS datasets (e.g. irreplaceable documents and digital photos vs.
backups from other machines). I think the answer was that for ZFS volumes, the
checksum is only to detect an error and not for reconstructing data and so the
default was fine. {Also, can you confirm the default is now Flectcher 4?}
3. Should I consider using dedup if my server has only 8Gb of RAM? Or, will
that not be enough to hold the DDT? In which case, should I add L2ARC / ZIL or
am I better to just skip using dedup on a home file server?
Thanks,
Craig
My config:
[u]Use:[/u] Home File Server supporting PC and Mac machines -- audio/video
files, home directories (documents photos), and backup images. Will not run
any other services.
[u]Objective:[/u] adequate performance for home use; maximize protection for
documents and photos; minimize administrative upkeep.
[u]Configuration:[/u]
OpenSolaris B134
AMD X2 260 / Asus MA4785TD-V EVO
8Gb ECC RAM (Crucial CT25672BA1339)
Intel EXPI9301CT NIC
LSI SAS9211-8i
Boot Disk: WD Scorpio Black 250Gb (2-way mirror)
Media Pool: WD Caviar Blue 640Gb (2-way mirror)
Mass Storage (File Backup) Pool: WD Carviar Blue 640Gb (9 disk Raid-Z3)
--
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Configuration questions for Home File Server (CPU cores, dedup, checksum)?
Craig,
I'm sure the other home file server users will comment on your gear
and any possible benefit of a L2ARC or separate log device...
Use the default checksum which is fletcher4, I fixed the tuning guide
reference, skip dedup for now. Keep things as simple as possible.
Thanks,
Cindy
On 09/07/10 14:58, Craig Stevenson wrote:
I am working on a home file server. After reading a wide range of blogs and
forums, I have a few questions that are still not clear to me
1. Is there a benefit in having quad core CPU (e.g. Athlon II X4 vs X2)? All
of the web blogs seem to suggest using lower-wattage dual core CPUs. But; with
the recent advent of dedup, SHA256 checksum, etc., I am now wondering if
opensolaris is better served with quad core.
2. For checksum on ZFS datasets, is there value in using Fletcher4 or SHA256
instead of just using the default (which Evil Tuning Guide says is Fletcher2,
others say it is Fletcher4)? And, should I consider using different checksums
for different ZFS datasets (e.g. irreplaceable documents and digital photos vs.
backups from other machines). I think the answer was that for ZFS volumes, the
checksum is only to detect an error and not for reconstructing data and so the
default was fine. {Also, can you confirm the default is now Flectcher 4?}
3. Should I consider using dedup if my server has only 8Gb of RAM? Or, will
that not be enough to hold the DDT? In which case, should I add L2ARC / ZIL or
am I better to just skip using dedup on a home file server?
Thanks,
Craig
My config:
[u]Use:[/u] Home File Server supporting PC and Mac machines -- audio/video files,
home directories (documents photos), and backup images. Will not run any
other services.
[u]Objective:[/u] adequate performance for home use; maximize protection for
documents and photos; minimize administrative upkeep.
[u]Configuration:[/u]
OpenSolaris B134
AMD X2 260 / Asus MA4785TD-V EVO
8Gb ECC RAM (Crucial CT25672BA1339)
Intel EXPI9301CT NIC
LSI SAS9211-8i
Boot Disk: WD Scorpio Black 250Gb (2-way mirror)
Media Pool: WD Caviar Blue 640Gb (2-way mirror)
Mass Storage (File Backup) Pool: WD Carviar Blue 640Gb (9 disk Raid-Z3)
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Configuration questions for Home File Server (CPU cores, dedup, checksum)?
On 09/07/2010 03:58 PM, Craig Stevenson wrote: I am working on a home file server. After reading a wide range of blogs and forums, I have a few questions that are still not clear to me 1. Is there a benefit in having quad core CPU (e.g. Athlon II X4 vs X2)? All of the web blogs seem to suggest using lower-wattage dual core CPUs. But; with the recent advent of dedup, SHA256 checksum, etc., I am now wondering if opensolaris is better served with quad core. With a big RAIDZ3, it's well worth having extra cores. A scrub on my eight-disk RAIDZ2 uses about 60% of all four cores on my Athlon II X4 630. With smaller pools, dual-core would be OK. If you're going to use dedup, you might want to go with an eight-disk RAIDZ2 and an SSD for L2ARC, instead of a nine-disk RAIDZ3. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Configuration questions for Home File Server (CPU cores, dedup, checksum)?
On Tue, Sep 7 at 17:13, Russ Price wrote: On 09/07/2010 03:58 PM, Craig Stevenson wrote: I am working on a home file server. After reading a wide range of blogs and forums, I have a few questions that are still not clear to me 1. Is there a benefit in having quad core CPU (e.g. Athlon II X4 vs X2)? All of the web blogs seem to suggest using lower-wattage dual core CPUs. But; with the recent advent of dedup, SHA256 checksum, etc., I am now wondering if opensolaris is better served with quad core. With a big RAIDZ3, it's well worth having extra cores. A scrub on my eight-disk RAIDZ2 uses about 60% of all four cores on my Athlon II X4 630. How are you measuring using 60% across all four cores? I kicked off a scrub just to see, and we're scrubbing at 200MB/s (2 vdevs) and the CPU is 94% idle, 6% kernel, 0% IOWAIT. zpool-tank is using 3.2% CPU as shown by 'ps aux | grep tank' Am I missing something? -- Eric D. Mudama edmud...@mail.bounceswoosh.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Configuration questions for Home File Server (CPU cores, dedup, checksum)?
Craig, 3. I do not think you will get much dedupe on video, music and photos. I would not bother. If you really wanted to know at some later stage, you could create a new file system, enable dedupe, and copy your data (or a subset) into it just to see. In my experience there is a significant CPU penalty as well. My four core (1.86GHz xeons, 4 yrs old) box nearly maxes out when putting a lot of data into a deduped file system. -Scott -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] iSCSI targets mapped to a VMWare ESX server
check fler.us Solaris 10 iSCSI Target for Vmware ESX -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Configuration questions for Home File Server (CPU cores, dedup, checksum)?
On 09/07/2010 05:58 PM, Eric D. Mudama wrote: How are you measuring using 60% across all four cores? I kicked off a scrub just to see, and we're scrubbing at 200MB/s (2 vdevs) and the CPU is 94% idle, 6% kernel, 0% IOWAIT. zpool-tank is using 3.2% CPU as shown by 'ps aux | grep tank' Whoops... I misspoke - it should have been about 23-25% per core. I'm getting old. :o) I am using gkrellm to watch the CPU usage. In any case, a scrub uses wildly different amounts of CPU at different times, and sometimes it uses far less (particularly early in the process, at least on my specific RAIDZ2). On the other hand, dd'ing a 23 GB video file on the RAIDZ2 to /dev/null will consistently get the 23-25% per core figure. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Suggested RaidZ configuration...
On Tue, Sep 7, 2010 at 4:59 PM, Edward Ned Harvey sh...@nedharvey.com wrote: I think the value you can take from this is: Why does the BPG say that? What is the reasoning behind it? Anything that is a rule of thumb either has reasoning behind it (you should know the reasoning) or it doesn't (you should ignore the rule of thumb, dismiss it as myth.) Let's examine the myth that you should limit the number of drives in a vdev because of resilver time. The myth goes something like this: You shouldn't use more than ___ drives in a vdev raidz_ configuration, because all the drives need to read during a resilver, so the more drives are present, the longer the resilver time. The truth of the matter is: Only the size of used data is read. Because this is ZFS, it's smarter than a hardware solution which would have to read all disks in their entirety. In ZFS, if you have a 6-disk raidz1 with capacity of 5 disks, and a total of 50G of data, then each disk has roughly 10G of data in it. During resilver, 5 disks will each read 10G of data, and 10G of data will be written to the new disk. If you have a 11-disk raidz1 with capacity of 10 disks, then each disk has roughly 5G of data. 10 disks will each read 5G of data, and 5G of data will be written to the new disk. If anything, more disks means a faster resilver, because you're more easily able to saturate the bus, and you have a smaller amount of data that needs to be written to the replaced disk. Let's examine the myth that you should limit the number of disks for the sake of redundancy. It is true that a carefully crafted system can survive things like SCSI controller or tray failure. Suppose you have 3 scsi cards. Suppose you construct a raidz2 device using 2 disks from controller 0, 2 disks from controller 1, and 2 disks from controller 2. Then if a controller dies, you have only lost 2 disks, and you are degraded but still functional as long as you don't lose another disk. But you said you have 20 disks all connected to a single controller. So none of that matters in your case. Personally, I can't imagine any good reason to generalize don't use more than ___ devices in a vdev. To me, a 12-disk raidz2 is just as likely to fail as a 6-disk raidz1. But a 12-disk raidz2 is slightly more reliable than having two 6-disk raidz1's. Perhaps, maybe, a 64bit processor is able to calculate parity on an 8-disk raidz set in a single operation, but requires additional operations to calculate parity if your raidz has 9 or more disks in it ... But I am highly skeptical of this line of reasoning, and AFAIK, nobody has ever suggested this before me. I made it up just now. I'm grasping at straws and stretching my imagination to find *any* merit in the statement, don't use more than ___ disks in a vdev. I see no reasoning behind it, and unless somebody can say anything to support it, I think it's bunk. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
