[zfs-discuss] Custom Jumpstart and RAID-10 ZFS rpool
Is it possible to create a custom Jumpstart profile to install Nevada on a RAID-10 rpool? From the ZFS Boot FAQ [1], you can create a profile to install Nevada with a RAID-1 rpool using the following line: pool newpool auto auto auto mirror c0t0d0s0 c0t1d0s0 Is there an equivalent line for RAID-10? I've tried using the following line, but the Jumpstart check script complains about it being invalid: pool newpool auto auto auto mirror c0t0d0s0 c0t1d0s0 mirror c0t2d0s0 c0t3d0s0 Thanks, Stephen Le [1] http://opensolaris.org/os/community/zfs/boot/zfsbootFAQ/#jumpinstall ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Custom Jumpstart and RAID-10 ZFS rpool
Stephen Le wrote: Is it possible to create a custom Jumpstart profile to install Nevada on a RAID-10 rpool? No, simple mirrors only. -- Ian. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [osol-bugs] what's the story wtih bug #6592835?
Hi Graham, (this message was posed on opensolaris-bugs initially, I am CC'ing and reply-to'ing zfs-discuss as it seems to be a more appropriate place to discuss this.) I'm surprised to see that the status of bug 6592835 hasn't moved beyond yes that's a problem. My understanding is that the resilver speed is tied to fact that the currenct resilver implementation follows the ZFS on disk structures, which needs random-like I/O operations while a traditional RAID rebuild issues sequential I/O only. Simply put, the former is very slow while the latter is very fast with respect to the amounts of data having to be touched. IIRC, this issue has been discussed on zfs-discuss several times already and my understanding it that it would be very difficult to implement some kind of sequential resilver (walking disk blocks sequentially rather than ZFS on disk structures), so I doubt if an improvement can be expected anytime soon. That said, the core developers on zfs-discuss will know more than me and might be willing to give more background on this. Nils ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] diagnosing read performance problem
On Tue, Oct 28, 2008 at 05:30:55PM -0700, Nigel Smith wrote: Hi Matt. Ok, got the capture and successfully 'unzipped' it. (Sorry, I guess I'm using old software to do this!) I see 12840 packets. The capture is a TCP conversation between two hosts using the SMB aka CIFS protocol. 10.194.217.10 is the client - Presumably Windows? 10.194.217.3 is the server - Presumably OpenSolaris - CIFS server? All correct so far Using WireShark, Menu: 'Statistics Endpoints' show: The Client has transmitted 4849 packets, and the Server has transmitted 7991 packets. Menu: 'Analyze Expert info Composite': The 'Errors' tab shows: 4849 packets with a 'Bad TCP checksum' error - These are all transmitted by the Client. (Apply a filter of 'ip.src_host == 10.194.217.10' to confirm this.) The 'Notes' tab shows: ..numerous 'Duplicate Ack's' For example, for 60 different ACK packets, the exact same packet was re-transmitted 7 times! Packet #3718 was duplicated 17 times. Packet #8215 was duplicated 16 times. packet #6421 was duplicated 15 times, etc. These bursts of duplicate ACK packets are all coming from the client side. This certainly looks strange to me - I've not seen anything like this before. It's not going to help the speed to unnecessarily duplicate packets like that, and these burst are often closely followed by a short delay, ~0.2 seconds. And as far as I can see, it looks to point towards the client as the source of the problem. If you are seeing the same problem with other client PC, then I guess we need to suspect the 'switch' that connects them. I have another switch on the way to move to. I will see if this helps. Thanks for your input Matt pgpPpxSVqiW79.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] HELP! SNV_97, 98, 99 zfs with iscsitadm and VMWare!
Hi Tano Great to hear that you've now got this working!! I understand you are using a Broadcom network card, from your previous posts I can see you are using the 'bnx' driver. I will raise this as a bug, but first please would you run '/usr/X11/bin/scanpci' to indentify the exact 'vendor id' and 'device id' for the Broadcom network chipset, and report that back here. I must admit that this is the first I have heard of 'I/OAT DMA', so I did some Googling on it, and found this links: http://opensolaris.org/os/community/arc/caselog/2008/257/onepager/ To quote from that ARC case: All new Sun Intel based platforms have Intel I/OAT (I/O Acceleration Technology) hardware. The first such hardware is an on-systemboard asynchronous DMA engine code named Crystal Beach. Through a set of RFEs Solaris will use this hardware to implement TCP receive side zero CPU copy via a socket. Ok, so I think that makes some sense, in the context of the problem we were seeing. It's referring to how the network adaptor transfers the data it has received, out of the buffer and onto the rest of the operating system. I've just looked to see if I can find the source code for the BNX driver, but I cannot find it. Digging deeper we find on this page: http://www.opensolaris.org/os/about/no_source/ ..on the 'ON' tab, that: Components for which there are currently no plans to release source bnx driver (B) Broadcom NetXtreme II Gigabit Ethernet driver So the bnx driver is closed source :-( Regards Nigel Smith -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] diagnosing read performance problem
On Tue, Oct 28, 2008 at 05:45:48PM -0700, Richard Elling wrote: I replied to Matt directly, but didn't hear back. It may be a driver issue with checksum offloading. Certainly the symptoms are consistent. To test with a workaround see http://bugs.opensolaris.org/view_bug.do?bug_id=6686415 Hi, Sorry for not replying, we had some problems with our email provider yesterday and I was up all night restoring backups. I did try the workaround, but it didn't have any effect, presumbably because it's not using the rge driver as you stated before. I'll try swapping the switch out and post back my results. Many Thanks Matt pgphN0KxBmHEz.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Cannot remove slog device from zpool
Hello Neil, Tuesday, October 28, 2008, 10:34:28 PM, you wrote: NP However, we wanted to make sure it fit into the framework for NP the removal of any device. This a much harder problem which we NP have made progress, but it's not there yet... I think a lot of people here would be interested in more details and any ETA (no commitments) on this - could you write couple of sentences on what you guys are actually doing re disk eviction? FOr example - would it be possible to change raidz2 - raidz1 or raid10 on a fly? -- Best regards, Robert Milkowskimailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [osol-bugs] what's the story wtih bug #6592835?
Hi Nils, thanks for the detailed info. I've tried searching the zfs-discuss archive for both the bug id and 'resilver', but in both cases the only result I can find from the whole history is this thread: http://www.opensolaris.org/jive/thread.jspa?messageID=276358#276358 Maybe the discussions you recall aren't fully indexed for searching on these keywords or they were in another forum, but thanks for giving me the gist of it. It is potentially quite an Achilles heel for ZFS though. I've argued locally to migrate our main online data archive (currently 3.5TB) to ZFS, but if the recovery time for disk failures keeps getting slower as the archive grows and accumulates snapshots etc., some questions might be asked about this policy. I've suggested we can do continuous data replication to a secondary server by sending incremental snapshot streams, but if the CDR had to be suspended for a significant time (days) to allow a resilver, this would be a real problem, at least until the 6343667 bugfix from snv_94 finds its way into a Solaris 10 patch (will this happen?). Does the severity of the problem depend on access / write patterns used, i.e. would a simple archiving system where data is only ever added in a sequential fashion be less susceptible to slow rebuilds than a system where data is written, snapshotted, moved, modified, deleted, etc. Does the time taken to scrub the pool give some indication of the likely resilvering time, or does that process walk a different kind of tree? Graham -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Lost Disk Space
No takers? :) benr. I'm quite curious about finding out about this too, to be honest :) And its not just ZFS on Solaris because I've filled up and imported pools into ZFS Fuse 0.5.0 (which is based on the latest ZFS code) in Linux, and on FreeBSD too. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS + OpenSolaris for home NAS?
Bob Friesenhahn wrote: AMD Athelon/Opteron dual core likely matches or exceeds Intel quad core for ZFS use due to a less bottlenecked memory channel. How big is the difference? Does anyone have benchmarking results (maybe even when using ZFS on Solaris 10)? Martti ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Strange result when syncing between SPARC and x86
Example is: [EMAIL PROTECTED] ls -la /data/zones/testfs/root/etc/services lrwxrwxrwx 1 root root 15 Oct 13 14:35 /data/zones/testfs/root/etc/services - ./inet/services [EMAIL PROTECTED] ls -la /data/zones/testfs/root/etc/services lrwxrwxrwx 1 root root 15 Oct 13 14:35 /data/zones/testfs/root/etc/services - s/teni/.ervices Ouch, thats a bad one. I downloaded and burnt b101 to dvd for x86 and solaris, i'm gonna install them tomorrow at work and try moving a pool between them to see what happens... -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS + OpenSolaris for home NAS?
Out of interest, and reasonably on-topic, can anyone predict performance comparison (CIFS) between these two setups? 1) Dedicated Windows 2003 Server, Intel hardware SATA RAID controller (single raid 5 array, 8 disks) 2) OpenSolaris+ZFS+CIFS, 8 drives with a SuperMicro controller ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [osol-bugs] what's the story wtih bug #6592835?
On Wed, 29 Oct 2008, Nils Goroll wrote: My understanding is that the resilver speed is tied to fact that the currenct resilver implementation follows the ZFS on disk structures, which needs random-like I/O operations while a traditional RAID rebuild issues sequential I/O only. Simply put, the former is very slow while the latter is very fast with respect to the amounts of data having to be touched. If this is indeed an issue, then the pool itself is severely fragmented and will be slow in normal use. This could happen if the pool is allowed to become overly full. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [osol-bugs] what's the story wtih bug #6592835?
On Wed, 29 Oct 2008, Graham McArdle wrote: Maybe the discussions you recall aren't fully indexed for searching on these keywords or they were in another forum, but thanks for giving me the gist of it. It is potentially quite an Achilles heel for ZFS though. I've argued locally to migrate our main online data archive (currently 3.5TB) to ZFS, but if the recovery time for disk failures keeps getting slower as the archive grows and accumulates snapshots etc., some questions might be asked about this policy. The simple solution is to use a reasonable pool design. Limit the maximum LUN size to a size which can be resilved in reasonable time. Don't do something silly like building a mirror across two 10TB LUNs. Huge disks will take longer to resilver, and it is more likely for a secondary failure to occur during resilvering. Manage your potential losses by managing the size of a loss, and therefore the time to recover. Another rules is to control how full a pool is allowed to become. If you fill it to 99% you can be assured that the pool will become fragmented and resilvers will take much longer. The pool will be slower in normal use as well. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Custom Jumpstart and RAID-10 ZFS rpool
Ian Collins wrote: Stephen Le wrote: Is it possible to create a custom Jumpstart profile to install Nevada on a RAID-10 rpool? No, simple mirrors only. Though a finish sscript could add additional simple mirrors to create the config his example would have created. Pretty sure that's still not RAID10 though. And any files laid down by the installer would be constrained to the first mirrored pair, only new files would have a chance at be distributed over the addtional pairs. -Kyle ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS + OpenSolaris for home NAS?
On Wed, 29 Oct 2008, Martti Kuparinen wrote: Bob Friesenhahn wrote: AMD Athelon/Opteron dual core likely matches or exceeds Intel quad core for ZFS use due to a less bottlenecked memory channel. How big is the difference? Does anyone have benchmarking results (maybe even when using ZFS on Solaris 10)? The big question would be what should be benchmarked. ZFS is like a big RAM cache. The more RAM the better. You would be surprised how little disk activity there can really be on systems with a lot of RAM as long as synchronous writes are avoided. As a result, some common scenarios mostly exercise RAM rather than the disk channel. Unless you need a higher power CPU for other purposes, a ZFS-based server should focus on maximizing installed RAM. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Strange result when syncing between SPARC and x86
Hi [EMAIL PROTECTED] ls -la /data/zones/testfs/root/etc/services lrwxrwxrwx 1 root root 15 Oct 13 14:35 /data/zones/testfs/root/etc/services - ./inet/services [EMAIL PROTECTED] ls -la /data/zones/testfs/root/etc/services lrwxrwxrwx 1 root root 15 Oct 13 14:35 /data/zones/testfs/root/etc/services - s/teni/.ervices Ouch, thats a bad one. I downloaded and burnt b101 to dvd for x86 and solaris, i'm gonna install them tomorrow at work and try moving a pool between them to see what happens... Would be interesting to know how it'll work if move whole zpool not just sync with send/recv. But I think all will be fine there as is seems the problem is in send/recv part on the file system itself on different architectures. Thanks Mike ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Hotplug issues on USB removable media.
I don't remember anyone saying they couldn't be stored, just that if they are stored it's not ZFS' fault if they go corrupt as it's outside of its control. I'm actually planning to store the zfs send dump on external USB devices myself, but since the USB device will be running on ZFS I expect to be able to find out if there are any corruption problems, at which point I'll just run the send / receive again. However, I don't think that's what they're talking about here. I think they're talking about a ZFS pool that consists of an external USB device, and doing a send / receive directly to that pool. That way the USB device is a true backup copy of your ZFS pool, and I think the idea is that you can then delete snapshots from your main system, confident that they are still present on the USB backup. If it works it's a nice idea, especially with an integrated restore interface. And yeah, auto mounting isn't something you're going to want to enable on production servers. But for desktop / development / small scale use, it's a great idea. And I agree 100% about hotplug stuff being untrusted input. You shouldn't be able to crash anything with a USB stick, especially not a bunch of unrelated ZFS filesystems. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] diagnosing read performance problem
Matt Harrison wrote: On Tue, Oct 28, 2008 at 05:45:48PM -0700, Richard Elling wrote: I replied to Matt directly, but didn't hear back. It may be a driver issue with checksum offloading. Certainly the symptoms are consistent. To test with a workaround see http://bugs.opensolaris.org/view_bug.do?bug_id=6686415 Hi, Sorry for not replying, we had some problems with our email provider yesterday and I was up all night restoring backups. I did try the workaround, but it didn't have any effect, presumbably because it's not using the rge driver as you stated before. The dohwcksum is not an rge option, it is an ip option (hence it is named ip:dohwcksum) and it transcends NIC drivers. But if that didn't fix anything, then you should be able to safely ignore it. I'll try swapping the switch out and post back my results. Yeah, I've seen this sort of thing before, too. I once had a switch that lost its mind and wouldn't let big packets through unscathed. We could telnet, ping, ftp, and do all sorts of things through the switch, but we couldn't push NFS through. These sorts of failures can be difficult to isolate. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Hotplug issues on USB removable media.
However, I don't think that's what they're talking about here. I think they're talking about a ZFS pool that consists of an external USB device, and doing a send / receive directly to that pool. That way the USB device is a true backup copy of your ZFS pool, and I think the idea is that you can then delete snapshots from your main system, confident that they are still present on the USB backup. That's exactly it. While it's great to have snapshots stored on the same device as the filesystem for convenience, this is limited in that it's not a full recovery solution since it doesn't offer any protection from physical hardware failure, especially on laptops and workstations where there will most likely be just one hard disk on the system. What we have right now is protection from user and software errors which is probably the source of most cases of data loss or corruption, but we need to extend this to cover hardware failure, and we need to be able to backup to secondary devices to acheive that. For most single users that means an external USB disk or thumb drive. If it works it's a nice idea, especially with an integrated restore interface. That would also be part of our plan. Cheers, Niall And yeah, auto mounting isn't something you're going to want to enable on production servers. But for desktop / development / small scale use, it's a great idea. And I agree 100% about hotplug stuff being untrusted input. You shouldn't be able to crash anything with a USB stick, especially not a bunch of unrelated ZFS filesystems. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Performance with tens of thousands of zfs filesystems
On Tue, Oct 28, 2008 at 9:27 AM, Morten-Christian Bernson [EMAIL PROTECTED] wrote: I have been reading this forum for a little while, and am interested in more information about the performance of ZFS when creating large amounts of filesystems. We are considering using ZFS for the user's home folders, and this could potentially be 30'000 filesystems, and if using snapshots that number would be multiplied by x snapshots as well. I am a bit nervous after reading this forum, that the performance when getting huge numbers of filesystems is not very good. Using hours to boot the server, and possibly weeks to make the filesystems seems not right. You'll have to solve this issue by using multiple ZFS servers so that the number of filesystems per server is a reasonably good fit with ZFS' current capabilities. OTOH ZFS is improving in this arena over time and, even with multiple servers, ZFS will still provide a cost effective solution. In any case, with 5 ZFS servers, would it not be better to have only 20% of your user community affected if a server were to have a catastrophic failure? (rhetorical question) Any official input on how this will be in the upcoming release of Solaris 10? You won't get any official input on Solaris (the commercial product) related topics on this list - which is dedicated to OpenSolaris. But, to give you something that resembles an answer, Sol10 Update 6 is based on Build 88 AFAIR. So looking at the features/performance of that release will give you a good feel for Sol 10U6 .. the usual disclaimers YMMV etc. Regards, -- Al Hopper Logical Approach Inc,Plano,TX [EMAIL PROTECTED] Voice: 972.379.2133 Timezone: US CDT OpenSolaris Governing Board (OGB) Member - Apr 2005 to Mar 2007 http://www.opensolaris.org/os/community/ogb/ogb_2005-2007/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS + OpenSolaris for home NAS?
On Wed, Oct 29, 2008 at 8:43 AM, Bob Friesenhahn [EMAIL PROTECTED] wrote: On Wed, 29 Oct 2008, Martti Kuparinen wrote: Bob Friesenhahn wrote: AMD Athelon/Opteron dual core likely matches or exceeds Intel quad core for ZFS use due to a less bottlenecked memory channel. How big is the difference? Does anyone have benchmarking results (maybe even when using ZFS on Solaris 10)? The big question would be what should be benchmarked. ZFS is like a big RAM cache. The more RAM the better. You would be surprised how little disk activity there can really be on systems with a lot of RAM as long as synchronous writes are avoided. As a result, some common scenarios mostly exercise RAM rather than the disk channel. Unless you need a higher power CPU for other purposes, a ZFS-based server should focus on maximizing installed RAM. Agreed 100% It's easy to find DDR2 RAM at around $20/gigabyte (based on 1Gb or 2Gb DIMMs) and I've seen some deals as low a $8/Gb for Kingston RAM. Regards, -- Al Hopper Logical Approach Inc,Plano,TX [EMAIL PROTECTED] Voice: 972.379.2133 Timezone: US CDT OpenSolaris Governing Board (OGB) Member - Apr 2005 to Mar 2007 http://www.opensolaris.org/os/community/ogb/ogb_2005-2007/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] zfs boot / root in Nevada build 101
This seems like a n00b question but I'm stuck. Nevada build 101. Doing fresh install (in vmware fusion). I don't see any way to select zfs as the root file system. Looks to me like UFS is the default, but I don't see any option box to allow that to be changed to zfs. What am I missing?! Thanks. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs boot / root in Nevada build 101
On Wed, 29 Oct 2008, Peter Baer Galvin wrote: Nevada build 101. Doing fresh install (in vmware fusion). I don't see any way to select zfs as the root file system. Looks to me like UFS is the default, but I don't see any option box to allow that to be changed to zfs. What am I missing?! Thanks. You have to use the text installer (rather than the default GUI one). Part way through the process you will be offered a choice of UFS or ZFS root file system (UFS is the default still). Select ZFS and away you go! -- Rich Teer, SCSA, SCNA, SCSECA CEO, My Online Home Inventory URLs: http://www.rite-group.com/rich http://www.linkedin.com/in/richteer http://www.myonlinehomeinventory.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs boot / root in Nevada build 101
Hi Peter, It's there, you just can't use the GUI installer. You have to choose the text interactive installer. It'll give you the choice there. Regards. Original Message Subject: [zfs-discuss] zfs boot / root in Nevada build 101 From: Peter Baer Galvin [EMAIL PROTECTED] To: zfs-discuss@opensolaris.org Date: Wed Oct 29 09:28:46 2008 This seems like a n00b question but I'm stuck. Nevada build 101. Doing fresh install (in vmware fusion). I don't see any way to select zfs as the root file system. Looks to me like UFS is the default, but I don't see any option box to allow that to be changed to zfs. What am I missing?! Thanks. begin:vcard fn:Daryl Doami n:Doami;Daryl org:Sun Microsystems Federal, Inc.;DoD, Intel, NASA West Regions adr;dom:;;222 N. Sepulveda Blvd. 10th Floor;El Segundo;CA;90245 email;internet:[EMAIL PROTECTED] title:Systems Engineer tel;work:310-242-6463 tel;fax:310-242-6463 x-mozilla-html:FALSE url:http://www.sun.com/government/ version:2.1 end:vcard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs boot / root in Nevada build 101
Hi Peter, You need to select the text-mode install option to select a ZFS root file system. Other ZFS root installation tips are described here: http://docs.sun.com/app/docs/doc/817-2271/zfsboot-1?a=view I'll be attending Richard Elling's ZFS workshop at LISA08. Hope to see you. :-) Cindy Peter Baer Galvin wrote: This seems like a n00b question but I'm stuck. Nevada build 101. Doing fresh install (in vmware fusion). I don't see any way to select zfs as the root file system. Looks to me like UFS is the default, but I don't see any option box to allow that to be changed to zfs. What am I missing?! Thanks. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs boot / root in Nevada build 101
* Peter Baer Galvin ([EMAIL PROTECTED]) wrote: This seems like a n00b question but I'm stuck. Nevada build 101. Doing fresh install (in vmware fusion). I don't see any way to select zfs as the root file system. Looks to me like UFS is the default, but I don't see any option box to allow that to be changed to zfs. What am I missing?! Thanks. You need to use the text mode installer, that's the only installer that has support for ZFS root installations in SXCE. Cheers, Glenn ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] [osol-bugs] what's the story wtih bug #6592835?
We have a 24-disk server, so the current design is 2-disk root mirror and 2x 11-disk RAIDZ2 vdevs. I suppose another solution could have been to have 3x 7-disk vdevs plus a hot spare, but the capacity starts to get compromised. Using 1TB disks in our current config will give us growth capacity to 16TiB. Obviously 3.5TiB is a small starting point, but we are facing an exponential growth curve. It seems like the recommendation is to keep expanding out in disk quantity rather than upgrading disk size to meet growth requirements. Perhaps 1TB SATA disk is already too big a lump for ZFS resilver. Looks like there is also some hope that if bp rewrite becomes a reality it will also help by allowing online defragmentation: http://www.opensolaris.org/jive/thread.jspa?messageID=186582 NEWS Actually I've just noticed that Matt Ahrens has updated the status of this bug to need more information and added a comment that the bugfix for 6343667 involved a major rewrite, which might have removed the problem. Has anyone out there got a large zpool running under snv_94 or higher, and if so have you had to rebuild a disk yet? If this has fixed both bugs then I'm definitely hoping for an early backport to Solaris 10. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs boot / root in Nevada build 101
Ah, thanks much all! Cindy I'll try to hit the Elling talk too but my time in San Diego will be short, unfortunately. I'll keep an eye out for you... -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs boot / root in Nevada build 101
Just occurred to me that S10U6 won't support ZFS root install via the GUI either? That will be confusing... -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] diagnosing read performance problem
Hi Matt Can you just confirm if that Ethernet capture file, that you made available, was done on the client, or on the server. I'm beginning to suspect you did it on the client. You can get a capture file on the server (OpenSolaris) using the 'snoop' command, as per one of my previous emails. You can still view the capture file with WireShark as it supports the 'snoop' file format. Normally it would not be too important where the capture was obtained, but here, where something strange is happening, it could be critical to understanding what is going wrong and where. It would be interesting to do two separate captures - one on the client and the one on the server, at the same time, as this would show if the switch was causing disruption. Try to have the clocks on the client server synchronised as close as possible. Thanks Nigel Smith -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs boot / root in Nevada build 101
Rats, text install hangs after printing out that very early set of ... More debugging...or I could wait for S10U6... -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs boot / root in Nevada build 101
Good point and we've tried to document this issue all over the place and will continue to publicize this fact. With the new ZFS boot and install features, it is a good idea to read the docs first. Tell you friends. I will send out a set of s10 10/08 doc pointers as soon as they are available. Thanks for reminding me. Cindy Peter Baer Galvin wrote: Just occurred to me that S10U6 won't support ZFS root install via the GUI either? That will be confusing... ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS + OpenSolaris for home NAS?
Al Hopper wrote: On Wed, Oct 29, 2008 at 8:43 AM, Bob Friesenhahn [EMAIL PROTECTED] wrote: On Wed, 29 Oct 2008, Martti Kuparinen wrote: Bob Friesenhahn wrote: AMD Athelon/Opteron dual core likely matches or exceeds Intel quad core for ZFS use due to a less bottlenecked memory channel. How big is the difference? Does anyone have benchmarking results (maybe even when using ZFS on Solaris 10)? The big question would be what should be benchmarked. ZFS is like a big RAM cache. The more RAM the better. You would be surprised how little disk activity there can really be on systems with a lot of RAM as long as synchronous writes are avoided. As a result, some common scenarios mostly exercise RAM rather than the disk channel. Unless you need a higher power CPU for other purposes, a ZFS-based server should focus on maximizing installed RAM. Agreed 100% It's easy to find DDR2 RAM at around $20/gigabyte (based on 1Gb or 2Gb DIMMs) and I've seen some deals as low a $8/Gb for Kingston RAM. ECC? -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs boot / root in Nevada build 101
Dunno about the text installer mentioned in other replies as I never use it. JumpStart installs working fine though with ZFS root. I am also in finish script pre-creating some additional filesystems etc. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS + OpenSolaris for home NAS?
ECC? $60 unbuffered 4GB 800MHz DDR2 ECC CL5 DIMM (Kit Of 2) http://www.provantage.com/kingston-technology-kvr800d2e5k2-4g~7KIN90H4.htm for Intel 32x0 north bridge like http://www.provantage.com/supermicro-x7sbe~7SUPM11K.htm ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Hotplug issues on USB removable media.
r == Ross [EMAIL PROTECTED] writes: r I don't remember anyone saying they couldn't be stored, just r that if they are stored it's not ZFS' fault if they go corrupt r as it's outside of its control. they can't be stored because they ``go corrupt'' in a transactional all-or-nothing way that other checksummed storable things like zpools, tarballs, zipfiles, do not. The software's written with the assumption it'll be used in a pipe, and I think fails appropriately for that use. But it's utterly without grace if you accidentally separate a stored 'zfs send' from your opportunity to re-send it. r I'm actually planning to store the zfs send dump on external r USB devices myself, but since the USB device will be running r on ZFS I expect to be able to find out if there are any r corruption problems, yeah I still think it's not a good idea. Our best practices are based on a feel for the statistics of bit-flips and their consequences If you flip one bit: in tar: you get the file back, with the flipped bit, and a warning from tar. in UFS: you get the file back with a bit flipped. in ZFS with redundancy: you get the file back with the bit unflipped. in ZFS without redundancy: you are warned about the flipped bit by losing the whole file in 'zfs send': you lose the ENTIRE FILESYSTEM because of one bit. 'zfs recv' will do a great job of warning you about corruption problems, whether it has ZFS underneath it or not. I don't think anyone could mistake the signal it sends for affection or negotiability. That's not the issue. It's the probability you'll be able to successfully restore your backup n years from now. We have a feel, not as good as we should but some, for this probability given 'tar' on tape, DVDR, disk, stick. I think 'zfs send' reduces this probability enough compared to other kinds of backup that you'd be crazy to use it. I also think this observation isn't adequately communicated to the average sysadmin by the disclaimery-sounding IT warning ``*BRRK* *BRRK* 'zfs send' is not a Backup Soloooshun!!'' It sounds like someone wants you to spend more money on CYAware and not, as it should, like they are begging you to use tar instead. There are other problems besides fragileness with storing 'zfs send' that are solved by both 'tar' and piping immediately to 'zfs recv': * stream format has not been as forward-portable across zfs versions and endiness as the pool structure itself. You could easily find yourself with a stream that won't restore, with no user-readable marker as to what it requires to restore itself. The choices are unstable Nevada builds you may not even be able to obtain two years from now much less get to boot on the hardware for sale at the time, and machine endynesses(!)---much harder to iteratively test than versions of GNU tar, if tar had this same problem which it doesn't. It may even demand to be restored ONTO a specific version of zpool, so your desperate restore attempt strategy is to reinstall an older Nevada, destroy and recreate a blank zpool, 'zfs recv', repeat. It's a bad situation. Store a zpool instead and you won't be in it because there's a commitment to forward-portability and endyness-independence. * no equivalent of 'zpool scrub' or 'tar t'. The only way to verify a 'zfs send' dump is to restore it. I think it's a best-practice when making backups to read back what you wrote. This catches medium errors, broken network pipes, filesystems you didn't quite sync before ejecting, whatever. You don't need to do silly Windows things like compare the backup to the files still on the disk and then scream OMG OMG something Changed!, but you ought to at least read it back now if you expect to restore it later. Is something missing from the toolbox? hell yes! Even beyond these inadequacies I'd like a way to restore a ZFS snapshot tree onto an LVM2 or Netapp snapshot tree. so I rather think it'll be some stream-storing rsync or deduplicating GNUtar instead of something coming from the ZFS project. (AIUI GNUtar incrementals include whole changed files, which is great for portability but useless for VM disk images) There are interesting questions like, would it drastically improve the efficiency of incremental backups if ZFS exposed some interface to its snapshot tree besides two copies of the file? Maybe the 'zfs send' stream itself is sufficient exposure, or maybe it needs to be some file-level rather than filesystem-level map. But this should probably come after we have userspace tools that do the job correctly but inefficiently given two copies of the file. And how do we design a storable incremental format that will take bit-flips gracefully, without seg-faulting and without killing the entire snapshot tree because one block is lost forever? Thirdly think it'd be good to write backups so one ``tape'' can be lost. ex:
Re: [zfs-discuss] Custom Jumpstart and RAID-10 ZFS rpool
I don't think this is possible. I already tried to add extra vdevs after install, but I got an error message telling me that multiple vdevs for rpool are not allowed. K -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Custom Jumpstart and RAID-10 ZFS rpool
kristof wrote: I don't think this is possible. I already tried to add extra vdevs after install, but I got an error message telling me that multiple vdevs for rpool are not allowed. K Oh. Ok. Good to know. I always put all my 'data' diskspace in a separate pool anyway to make migration to another host easier, so I haven't actually tried it. -Kyle ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] HELP! SNV_97, 98, 99 zfs with iscsitadm and VMWare!
ns == Nigel Smith [EMAIL PROTECTED] writes: ns the bnx driver is closed source :-( The GPL'd Linux driver is contributed by Broadcom: http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.27.y.git;a=blob;f=drivers/net/bnx2.c;h=2486a656f12d9f47ff27ead587e084a3c337a1a3;hb=HEAD and I believe the chip itself is newer than the Solaris 10 ``all new bits will be open-source'' pitch. pgpnZzPhLnb3Y.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] diagnosing read performance problem
On Wed, Oct 29, 2008 at 10:01:09AM -0700, Nigel Smith wrote: Hi Matt Can you just confirm if that Ethernet capture file, that you made available, was done on the client, or on the server. I'm beginning to suspect you did it on the client. That capture was done from the client You can get a capture file on the server (OpenSolaris) using the 'snoop' command, as per one of my previous emails. You can still view the capture file with WireShark as it supports the 'snoop' file format. I am uploading a snoop from the server to http://distfiles.genestate.com/snoop.zip Please note this snoop will include traffic to ssh as I can't work out how to filter that out :P Normally it would not be too important where the capture was obtained, but here, where something strange is happening, it could be critical to understanding what is going wrong and where. It would be interesting to do two separate captures - one on the client and the one on the server, at the same time, as this would show if the switch was causing disruption. Try to have the clocks on the client server synchronised as close as possible. Clocks are synced via ntp as we're using Active Directory with CIFS. On another note, I've just moved the offending network to another switch and it's even worse I think. I've noticed that under high load, the link light for the server's connection blinks on and off, not quite steadily but about every 2 seconds. This appears in /var/adm/messages: Oct 29 18:24:22 exodus mac: [ID 435574 kern.info] NOTICE: rtls0 link up, 100 Mbps, full duplex Oct 29 18:24:24 exodus mac: [ID 486395 kern.info] NOTICE: rtls0 link down Oct 29 18:24:25 exodus mac: [ID 435574 kern.info] NOTICE: rtls0 link up, 100 Mbps, full duplex Oct 29 18:24:27 exodus mac: [ID 486395 kern.info] NOTICE: rtls0 link down Oct 29 18:24:28 exodus mac: [ID 435574 kern.info] NOTICE: rtls0 link up, 100 Mbps, full duplex Oct 29 18:24:30 exodus mac: [ID 486395 kern.info] NOTICE: rtls0 link down Oct 29 18:24:31 exodus mac: [ID 435574 kern.info] NOTICE: rtls0 link up, 100 Mbps, full duplex I think it's got to be the NIC, the network runs full duplex quite happily so I don't think its an auto-neg problem. Thanks for sticking with this :) Matt pgpgEL3gweH2e.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs boot / root in Nevada build 101
Hi Cindy, I googled quite a lot before posting my question. This issue isn't mentioned in the ZFS boot FAQ for example or anywhere (that I saw) on the Opensolaris ZFS pages. Of course I could have read the ZFS Admin book at docs. sun.com... -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] zfs zpool recommendation
Hi all, I have been asked to build a new server and would like to get some opinions on how to setup a zfs pool for the application running on the server. The server will be exclusively for running netbackup application. Now which would be better? setting up a raidz pool with 6x146gig drives or setting up 3 mirrors of 2x146gig into a pool? Currently I have it setup with the mirrors.. but am wondering if the raidz possibility would be a better choice. Thanks in advance. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs zpool recommendation
On 29 October, 2008 - Mike sent me these 0,7K bytes:
Hi all,
I have been asked to build a new server and would like to get some
opinions on how to setup a zfs pool for the application running on the
server. The server will be exclusively for running netbackup
application.
Now which would be better? setting up a raidz pool with 6x146gig
drives or setting up 3 mirrors of 2x146gig into a pool?
Currently I have it setup with the mirrors.. but am wondering if the
raidz possibility would be a better choice.
Define better?
The raidz option will give you more storage at less performance.. The
mirror thing has the possibility of achieving higher reliability.. 1 to
3 disks can fail without interruptions, depending on how Murphy picks
them.. The raidz1 one can handle 1 disk only..
/Tomas
--
Tomas Ögren, [EMAIL PROTECTED], http://www.acc.umu.se/~stric/
|- Student at Computing Science, University of Umeå
`- Sysadmin at {cs,acc}.umu.se
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs boot / root in Nevada build 101
Hi Peter, It's mentioned here under Annoucements: http://opensolaris.org/os/community/zfs/boot/ It's just not very obvious. Original Message Subject: Re: [zfs-discuss] zfs boot / root in Nevada build 101 From: Peter Baer Galvin [EMAIL PROTECTED] To: zfs-discuss@opensolaris.org Date: Wed Oct 29 11:25:20 2008 Hi Cindy, I googled quite a lot before posting my question. This issue isn't mentioned in the ZFS boot FAQ for example or anywhere (that I saw) on the Opensolaris ZFS pages. Of course I could have read the ZFS Admin book at docs. sun.com... ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Custom Jumpstart and RAID-10 ZFS rpool
Kyle McDonald wrote: Ian Collins wrote: Stephen Le wrote: Is it possible to create a custom Jumpstart profile to install Nevada on a RAID-10 rpool? No, simple mirrors only. Though a finish sscript could add additional simple mirrors to create the config his example would have created. Pretty sure that's still not RAID10 though. No, this isn't possible. The attempt to add additional top-level vdevs will fail because it's not supported for root pools. One top-level vdev only. We hope to relax this restriction and support both multiple top-level vdevs and RAID-Z for root pools in the future. But that work hasn't started yet, so I have no estimates at all for when it might be available. lori And any files laid down by the installer would be constrained to the first mirrored pair, only new files would have a chance at be distributed over the addtional pairs. -Kyle ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Resilvering after drive failure - keeps restarting - any guesses why/how to fix?
All; I have a large zfs tank with four raidz2 groups in it. Each of these groups is 11 disks, and I have four hot spare disks in the system. The system is running Open Solaris build snv_90. One of these groups has had a disk failure, which the OS correctly detected, and replaced with one of the hot spares, and began rebuilding. Now it gets interesting. The resilver runs for about 1 hour, then stops. If I put zpool status v in a while loop with a 10 minute sleep, I see the repair proceed, then with no messages of ANY kind, it¹ll silently quit and start over. I¹m attaching the output of zpool status v from an hour ago and then from just now below. Has anyone seen this, or have any ideas as to the cause? Is there a timeout or priorty I need to change in a tuneable or something? --Mike One Hour Ago: [EMAIL PROTECTED]:~$ zpool status -v pool: tank1 state: DEGRADED status: One or more devices are faulted in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Replace the faulted device, or use 'zpool clear' to mark the device repaired. scrub: resilver in progress for 0h46m, 3.96% done, 18h39m to go config: NAME STATE READ WRITE CKSUM tank1 DEGRADED 0 0 0 raidz2 DEGRADED 0 0 0 c1t13d0ONLINE 0 0 0 spare DEGRADED 0 0 0 c1t14d0 FAULTED 0 0 0 too many errors c1t11d0 ONLINE 0 0 0 c1t15d0ONLINE 0 0 0 c1t16d0ONLINE 0 0 0 c1t17d0ONLINE 0 0 0 c1t18d0ONLINE 0 0 0 c1t19d0ONLINE 0 0 0 c1t20d0ONLINE 0 0 0 c1t21d0ONLINE 0 0 0 c1t22d0ONLINE 0 0 0 c1t23d0ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c1t0d0 ONLINE 0 0 0 c1t1d0 ONLINE 0 0 0 c1t2d0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 c1t4d0 ONLINE 0 0 0 c1t5d0 ONLINE 0 0 0 c1t6d0 ONLINE 0 0 0 c1t7d0 ONLINE 0 0 0 c1t8d0 ONLINE 0 0 0 c1t9d0 ONLINE 0 0 0 c1t10d0ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c2t13d0ONLINE 0 0 0 c2t14d0ONLINE 0 0 0 c2t15d0ONLINE 0 0 0 c2t16d0ONLINE 0 0 0 c2t17d0ONLINE 0 0 0 c2t18d0ONLINE 0 0 0 c2t19d0ONLINE 0 0 0 c2t20d0ONLINE 0 0 0 c2t21d0ONLINE 0 0 0 c2t22d0ONLINE 0 0 0 c2t23d0ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c2t0d0 ONLINE 0 0 0 c2t1d0 ONLINE 0 0 0 c2t2d0 ONLINE 0 0 0 c2t3d0 ONLINE 0 0 0 c1t24d0ONLINE 0 0 0 c2t5d0 ONLINE 0 0 0 c2t6d0 ONLINE 0 0 0 c2t7d0 ONLINE 0 0 0 c2t8d0 ONLINE 0 0 0 c2t9d0 ONLINE 0 0 0 c2t10d0ONLINE 0 0 0 spares c1t11d0 INUSE currently in use c2t24d0 AVAIL c2t11d0 AVAIL c2t4d0 AVAIL errors: No known data errors Just Now: [EMAIL PROTECTED]:~$ zpool status -v pool: tank1 state: DEGRADED status: One or more devices are faulted in response to persistent errors. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Replace the faulted device, or use 'zpool clear' to mark the device repaired. scrub: resilver in progress for 0h24m, 2.23% done, 17h51m to go config: NAME STATE READ WRITE CKSUM tank1 DEGRADED 0 0 0 raidz2 DEGRADED 0 0 0 c1t13d0ONLINE 0 0 0 spare DEGRADED 0 0 0 c1t14d0 FAULTED 0 0 0 too many errors c1t11d0 ONLINE 0 0 0 c1t15d0ONLINE 0 0 0 c1t16d0ONLINE 0 0 0 c1t17d0ONLINE 0 0 0 c1t18d0ONLINE
Re: [zfs-discuss] Hotplug issues on USB removable media.
Hi Miles, I probably should have explained that storing the zfs send on a USB device is just one part of the strategy, and in fact that's just our way of getting the backup off-site. Once off-site, we do zfs receive that into another pool, and in fact we plan to have two offsite zfs pools, plus standard tar backups on tape (just in case the zpools all get corrupted somehow). We're also considering streaming zfs send to file, and then ftp'ing that file to the remote servers. I don't fancy restarting an entire zfs send because a packet got lost 4/5 of the way through. I'd much rather use restartable FTP, and do the zfs receive later on. Ross -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs zpool recommendation
On Wed, 29 Oct 2008, Tomas Ögren wrote: The raidz option will give you more storage at less performance.. The mirror thing has the possibility of achieving higher reliability.. 1 to 3 disks can fail without interruptions, depending on how Murphy picks them.. The raidz1 one can handle 1 disk only.. Mirrors also offer ease of administration. Half the drives can be administratively removed and the pool will still work. The sizes of drives in individual pairs can be updated one by one in order to offer more space rather than requiring all of the drives in a raidz to be replaced before the space is seen. Resilvering will be faster when using mirrors since only one disk has to be read to reconstruct the data. If you feel uneasy about reliability, then you can easily use tripple mirroring. However raidz2 offers more reliability than dual-mirrors and raidz1. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs zpool recommendation
On Wed, 29 Oct 2008, Mike wrote: I am not seeing how using raidz would be a performance hit. Usually stripes perform faster than mirrors. The mirrors load-share and offer a lot more disk seeking capacity (more IOPS). Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs zpool recommendation
On Wed, Oct 29, 2008 at 3:42 PM, Mike [EMAIL PROTECTED] wrote: By Better I meant the best practice for a server running the Netbackup application. I am not seeing how using raidz would be a performance hit. Usually stripes perform faster than mirrors. raidz performs reads from all devices in parallel, so you get 1 drive's worth of I/O operations, not 6 drives' worth. With 3 mirrors, you'd get 6 drives' worth of reads and 3 drives' worth of writes. Using raidz might get you slightly better read and write bandwidth, though. Scott ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] HELP! SNV_97, 98, 99 zfs with iscsitadm and VMWare!
ns I will raise this as a bug, but first please would you run '/usr/X11/bin/scanpci' to indentify the exact 'vendor id' and 'device id' for the Broadcom network chipset, and report that back here Primary network interface Embedded NIC: pci bus 0x0005 cardnum 0x00 function 0x00: vendor 0x14e4 device 0x164c Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet Plus the two external add on Broadcom cards: (CURRENTLY NOT IN USE) pci bus 0x000b cardnum 0x00 function 0x00: vendor 0x1166 device 0x0103 Broadcom EPB PCI-Express to PCI-X Bridge pci bus 0x000c cardnum 0x00 function 0x00: vendor 0x14e4 device 0x164c Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet pci bus 0x000d cardnum 0x00 function 0x00: vendor 0x1166 device 0x0103 Broadcom EPB PCI-Express to PCI-X Bridge pci bus 0x000e cardnum 0x00 function 0x00: vendor 0x14e4 device 0x164c Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet I will submit the information that you had asked in email very soon. Tano -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS + OpenSolaris for home NAS?
Rob Logan wrote: ECC? $60 unbuffered 4GB 800MHz DDR2 ECC CL5 DIMM (Kit Of 2) http://www.provantage.com/kingston-technology-kvr800d2e5k2-4g~7KIN90H4.htm Geez, I have to move to the US for cheap hardware. I've paid 120€ for exactly that 4GB ECC kit (well, I bought two of these, so 240€) in Germany. -mg signature.asc Description: OpenPGP digital signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs shrinking swap
Karl Rossing wrote: $zfs list NAME USED AVAIL REFER MOUNTPOINT rpool 48.4G 10.6G31K /rpool rpool/ROOT36.4G 10.6G18K /rpool/ROOT rpool/ROOT/snv_90_zfs 29.6G 10.6G 29.3G /.alt.tmp.b-Ugf.mnt/ rpool/ROOT/[EMAIL PROTECTED] 319M - 29.6G - rpool/ROOT/snv_93 6.82G 10.6G 16.3G / rpool/dump3.95G 10.6G 3.95G - rpool/swap8.00G 17.6G 1.05G - I'm on build 93. I would like to shrink swap. Is it as simple as zfs set volsize=4G rpool/swap on a live system? Is there anything else I need to do? Yes. You need to remove it from use first. # swap -l swapfile devswaplo blocks free /dev/zvol/dsk/rpool/swap 182,2 8 4194296 4194296 # zfs get volsize rpool/swap NAMEPROPERTY VALUE SOURCE rpool/swap volsize 2G - # swap -d /dev/zvol/dsk/rpool/swap # swap -l No swap devices configured # zfs set volsize=1G rpool/swap # /sbin/swapadd # swap -l swapfile devswaplo blocks free /dev/zvol/dsk/rpool/swap 182,2 8 2097144 2097144 Note: in the first swap -l above, blocks == free, so the swap device was not actually being used. If your swap device is in use, then you might not be able to delete it, so you'll have to make another, then shuffle the data around. -- richard Thanks Karl CONFIDENTIALITY NOTICE: This communication (including all attachments) is confidential and is intended for the use of the named addressee(s) only and may contain information that is private, confidential, privileged, and exempt from disclosure under law. All rights to privilege are expressly claimed and reserved and are not waived. Any use, dissemination, distribution, copying or disclosure of this message and any attachments, in whole or in part, by anyone other than the intended recipient(s) is strictly prohibited. If you have received this communication in error, please notify the sender immediately, delete this communication from all data storage devices and destroy all hard copies. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] diagnosing read performance problem
Hi Matt In your previous capture, (which you have now confirmed was done on the Windows client), all those 'Bad TCP checksum' packets sent by the client, are explained, because you must be doing hardware TCP checksum offloading on the client network adaptor. WireShark will capture the packets before that hardware calculation is done, so the checksum all appear to be wrong, as they have not yet been calculated! http://wiki.wireshark.org/TCP_checksum_offload http://www.wireshark.org/docs/wsug_html_chunked/ChAdvChecksums.html Ok, so lets look at the new capture, 'snoop'ed on the OpenSolaris box. I was surprised how small that snoop capture file was - only 753400 bytes after unzipping. I soon realized why... The strange thing is that I'm only seeing half of the conversation! I see packets sent from client to server. That is from source: 10.194.217.10 to destination: 10.194.217.3 I can also see some packets from source: 10.194.217.5 (Your AD domain controller) to destination 10.194.217.3 But you've not capture anything transmitted from your OpenSolaris server - source: 10.194.217.3 (I checked, and I did not have any filters applied in WireShark that would cause the missing half!) Strange! I'm not sure how you did that. The half of the conversation that I can see looks fine - there does not seem to be any problem. I'm not seeing any duplication of ACK's from the client in this capture. (So again somewhat strange, unless you've fixed the problem!) I'm assuming your using a single network card in the Solaris server, but maybe you had better just confirm that. Regarding not capturing SSH traffic and only capturing traffic from ( hopefully to) the client, try this: # snoop -o test.cap -d rtls0 host 10.194.217.10 and not port 22 Regarding those 'link down', 'link up' messages, '/var/adm/messages'. I can tie up some of those events with your snoop capture file, but it just shows that no packets are being received while the link is down, which is exactly what you would expect. But dropping the link for a second will surely disrupt your video playback! If the switch is ok, and the cable from the switch is ok, then it does now point towards the network card in the OpenSolaris box. Maybe as simple as a bad mechanical connection on the cable socket BTW, just run '/usr/X11/bin/scanpci' and identify the 'vendor id' and 'device id' for the network card, just in case it turns out to be a driver bug. Regards Nigel Smith -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] diagnosing read performance problem
On Wed, Oct 29, 2008 at 05:32:39PM -0700, Nigel Smith wrote: Hi Matt In your previous capture, (which you have now confirmed was done on the Windows client), all those 'Bad TCP checksum' packets sent by the client, are explained, because you must be doing hardware TCP checksum offloading on the client network adaptor. WireShark will capture the packets before that hardware calculation is done, so the checksum all appear to be wrong, as they have not yet been calculated! I know that the client I was using has an nForce board with nVidia network controllers. There is an option to offload to hardware but I believe that was disabled. The strange thing is that I'm only seeing half of the conversation! I see packets sent from client to server. That is from source: 10.194.217.10 to destination: 10.194.217.3 I can also see some packets from source: 10.194.217.5 (Your AD domain controller) to destination 10.194.217.3 But you've not capture anything transmitted from your OpenSolaris server - source: 10.194.217.3 (I checked, and I did not have any filters applied in WireShark that would cause the missing half!) Strange! I'm not sure how you did that. I believe i was using the wrong filter expression...my bad :( The half of the conversation that I can see looks fine - there does not seem to be any problem. I'm not seeing any duplication of ACK's from the client in this capture. (So again somewhat strange, unless you've fixed the problem!) I'm assuming your using a single network card in the Solaris server, but maybe you had better just confirm that. Confirmed, there is a single PCI NIC that i'm using (there is the dual onboard but they don't work for me anymore). Regarding not capturing SSH traffic and only capturing traffic from ( hopefully to) the client, try this: # snoop -o test.cap -d rtls0 host 10.194.217.10 and not port 22 Much better thanks. I am attaching a second snoop from the server with the full conversation. http://distfiles.genestate.com/snoop2.zip Incidentally, this is talking to a different client, which although doesn't show checksum errors, does still have a load of duplicate ACKs. If this confuses the issue, I can do it from the old client as soon as it becomes free. Regarding those 'link down', 'link up' messages, '/var/adm/messages'. I can tie up some of those events with your snoop capture file, but it just shows that no packets are being received while the link is down, which is exactly what you would expect. But dropping the link for a second will surely disrupt your video playback! If the switch is ok, and the cable from the switch is ok, then it does now point towards the network card in the OpenSolaris box. Maybe as simple as a bad mechanical connection on the cable socket Very possible. I have an Intell Pro 1000 and a new GB switch on the way. BTW, just run '/usr/X11/bin/scanpci' and identify the 'vendor id' and 'device id' for the network card, just in case it turns out to be a driver bug. pci bus 0x0001 cardnum 0x06 function 0x00: vendor 0x10ec device 0x8139 Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ and the two onboards that no longer function: pci bus 0x cardnum 0x08 function 0x00: vendor 0x10de device 0x0373 nVidia Corporation MCP55 Ethernet pci bus 0x cardnum 0x09 function 0x00: vendor 0x10de device 0x0373 nVidia Corporation MCP55 Ethernet Thanks Matt pgpG9O5VYjefY.pgp Description: PGP signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Fwd: zpool import problem
oops, meant to reply-all... -- Forwarded message -- From: Terry Heatlie [EMAIL PROTECTED] Date: Wed, Oct 29, 2008 at 8:14 PM Subject: Re: [zfs-discuss] zpool import problem To: Eric Schrock [EMAIL PROTECTED] well, this does seem to be the case: bash-3.2# dtrace -s raidz_open2.d run 'zpool import' to generate trace 1145357764648 BEGIN RAIDZ OPEN 1145357764648 config asize = 1600340623360 1145357764648 config ashift = 9 1145358131986 child[0]: asize = 320071851520, ashift = 9 1145358861331 child[1]: asize = 400088457216, ashift = 9 1145396437606 child[2]: asize = 400088457216, ashift = 9 1145396891657 child[3]: asize = 320072933376, ashift = 9 1145397584944 child[4]: asize = 400087375360, ashift = 9 1145397920504 child[5]: asize = 400087375360, ashift = 9 1145398947963 asize = 1600335380480 1145398947963 ashift = 9 1145398947963 END RAIDZ OPEN But I still don't see a difference between the partition maps of the drive with only 2 labels and a good one... c2 is bad, c4 is good... # prtvtoc /dev/dsk/c2d0p0 /tmp/vtoc_c2 # prtvtoc /dev/dsk/c4d0p0 /tmp/vtoc_c4 # diff /tmp/vtoc_c2 /tmp/vtoc_c4 1c1 * /dev/dsk/c2d0p0 partition map --- * /dev/dsk/c4d0p0 partition map # On Tue, Oct 28, 2008 at 3:53 AM, Eric Schrock [EMAIL PROTECTED] wrote: These are the symptoms of a shrinking device in a RAID-Z pool. You can try to run the attached script during the import to see if this the case. There's a bug filed on this, but I don't have it handy. [...] ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
