Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
Frank Cusack wrote: yes I am an experienced Solaris admin and know all about devfsadm :-) and the older disks command. It doesn't help in this case. I think it's a BIOS thing. Linux and Windows can't see IDE drives that aren't there at boot time either, and on Solaris the SATA controller runs in some legacy mode so I guess that's why you can't see the newly added drive. Unfortunately all my x2100 hardware is in production and I can't readily retest this to verify. -frank This is exactly the issue; some of the simple SATA drives are used in PATA compatibility mode. The ide driver doesn't know a thing about hot anything, so we would need a proper SATA driver for these chips. Since they work (with the exception of hot *) it is difficult to prioritize this work above getting some other piece of hardware working under Solaris. In addition, switching drivers bios configs during upgrade is a non-trivial exercise. - Bart -- Bart Smaalders Solaris Kernel Performance [EMAIL PROTECTED] http://blogs.sun.com/barts ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
On January 22, 2007 10:03:14 AM -0800 Richard Elling [EMAIL PROTECTED] wrote: Toby Thain wrote: To be clear: the X2100 drives are neither hotswap nor hotplug under Solaris. Replacing a failed drive requires a reboot. I do not believe this is true, though I don't have one to test. Well if you won't accept multiple technically adept people's word on it, I highly suggest you get one to test instead of speculating. If this were true, then we would have had to rewrite the disk drivers to not allow us to open a device more than once, even if we also closed the device. I can't imagine anyone allowing such code to be written. Obviously you have not rewritten the disk drivers to do this, so this is the wrong line of reasoning. However, I don't believe this is the context of the issue. I believe that this release note deals with the use of NVRAID (NVidia's MCP RAID controller) which does not have a systems management interface under Solaris. The solution is to not use NVRAID for Solaris. Rather, use the proven techniques that we've been using for decades to manage hot plugging drives. No, the release note is not about NVRAID. In short, the release note is confusing, so ignore it. Use x2100 disks as hot pluggable like you've always used hot plug disks in Solaris. Again, NO these drives are not hot pluggable and the release note is accurate. PLEASE get a system to test. Or take our word for it. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
Hi Frank, I'm sure Richard will check it out. He's a very good guy and not trying to jerk you around. I'm sure the hostility isn't warranted. :-) Best Regards, Jason On 1/22/07, Frank Cusack [EMAIL PROTECTED] wrote: On January 22, 2007 10:03:14 AM -0800 Richard Elling [EMAIL PROTECTED] wrote: Toby Thain wrote: To be clear: the X2100 drives are neither hotswap nor hotplug under Solaris. Replacing a failed drive requires a reboot. I do not believe this is true, though I don't have one to test. Well if you won't accept multiple technically adept people's word on it, I highly suggest you get one to test instead of speculating. If this were true, then we would have had to rewrite the disk drivers to not allow us to open a device more than once, even if we also closed the device. I can't imagine anyone allowing such code to be written. Obviously you have not rewritten the disk drivers to do this, so this is the wrong line of reasoning. However, I don't believe this is the context of the issue. I believe that this release note deals with the use of NVRAID (NVidia's MCP RAID controller) which does not have a systems management interface under Solaris. The solution is to not use NVRAID for Solaris. Rather, use the proven techniques that we've been using for decades to manage hot plugging drives. No, the release note is not about NVRAID. In short, the release note is confusing, so ignore it. Use x2100 disks as hot pluggable like you've always used hot plug disks in Solaris. Again, NO these drives are not hot pluggable and the release note is accurate. PLEASE get a system to test. Or take our word for it. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
I certainly did NOT mean any hostility whatsoever. I highly value what Richard offers in this forum. I'm just frustrated at the misinformation which is being presented as authoritative. Repeatedly. But to be clear, in my mind Richard is one of the good ones and I eagerly read what he has to say -- to the point that when he chimes in on a thread I've been ignoring I start reading it, and always learn something. OK enough of that, back to the bashing! :-) -frank On January 22, 2007 12:02:03 PM -0700 Jason J. W. Williams [EMAIL PROTECTED] wrote: Hi Frank, I'm sure Richard will check it out. He's a very good guy and not trying to jerk you around. I'm sure the hostility isn't warranted. :-) Best Regards, Jason On 1/22/07, Frank Cusack [EMAIL PROTECTED] wrote: On January 22, 2007 10:03:14 AM -0800 Richard Elling [EMAIL PROTECTED] wrote: Toby Thain wrote: To be clear: the X2100 drives are neither hotswap nor hotplug under Solaris. Replacing a failed drive requires a reboot. I do not believe this is true, though I don't have one to test. Well if you won't accept multiple technically adept people's word on it, I highly suggest you get one to test instead of speculating. If this were true, then we would have had to rewrite the disk drivers to not allow us to open a device more than once, even if we also closed the device. I can't imagine anyone allowing such code to be written. Obviously you have not rewritten the disk drivers to do this, so this is the wrong line of reasoning. However, I don't believe this is the context of the issue. I believe that this release note deals with the use of NVRAID (NVidia's MCP RAID controller) which does not have a systems management interface under Solaris. The solution is to not use NVRAID for Solaris. Rather, use the proven techniques that we've been using for decades to manage hot plugging drives. No, the release note is not about NVRAID. In short, the release note is confusing, so ignore it. Use x2100 disks as hot pluggable like you've always used hot plug disks in Solaris. Again, NO these drives are not hot pluggable and the release note is accurate. PLEASE get a system to test. Or take our word for it. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
In short, the release note is confusing, so ignore it. Use x2100 disks as hot pluggable like you've always used hot plug disks in Solaris. Again, NO these drives are not hot pluggable and the release note is accurate. PLEASE get a system to test. Or take our word for it. hmm I think I may have just figured out the problem here. YES the x2100 is that bad. I too found it quite hard to believe that Sun would sell this without hot plug drives. It seems like a step backwards. (and of course I don't mean that the x2100 is awful, it's a great hardware and very well priced ... now if only hot plug worked!) My main issue is that the x2100 is advertised as hot plug working. You have to dig pretty deep -- deeper than would be expected of a typical buyer -- to find that Solaris does not support it. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
Hi Guys, The original X2100 was a pile of doggie doo-doo. All of our problems with it go back to the atrocious quality of the nForce 4 Pro chipset. The NICs in particular are just crap. The M2s are better, but the MCP55 chipset has not resolved all of its flakiness issues. That being said Sun designed that case with hot-plug bays, if Solaris isn't going to support it, then those shouldn't be there in my opinion. Best Regards, Jason On 1/22/07, Frank Cusack [EMAIL PROTECTED] wrote: In short, the release note is confusing, so ignore it. Use x2100 disks as hot pluggable like you've always used hot plug disks in Solaris. Again, NO these drives are not hot pluggable and the release note is accurate. PLEASE get a system to test. Or take our word for it. hmm I think I may have just figured out the problem here. YES the x2100 is that bad. I too found it quite hard to believe that Sun would sell this without hot plug drives. It seems like a step backwards. (and of course I don't mean that the x2100 is awful, it's a great hardware and very well priced ... now if only hot plug worked!) My main issue is that the x2100 is advertised as hot plug working. You have to dig pretty deep -- deeper than would be expected of a typical buyer -- to find that Solaris does not support it. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
On 22-Jan-07, at 5:28 PM, Frank Cusack wrote: In short, the release note is confusing, so ignore it. Use x2100 disks as hot pluggable like you've always used hot plug disks in Solaris. Won't work - some of us have tested it. Again, NO these drives are not hot pluggable and the release note is accurate. PLEASE get a system to test. Or take our word for it. hmm I think I may have just figured out the problem here. YES the x2100 is that bad. I too found it quite hard to believe that Sun would sell this without hot plug drives. It seems like a step backwards. (and of course I don't mean that the x2100 is awful, it's a great hardware and very well priced ... now if only hot plug worked!) My main issue is that the x2100 is advertised as hot plug working. Agree 100% with the above. In all other respects I like the X2100. I've come to accept the lack of hotswap as an indirect consequence of market segmentation, but it would be great if it worked (like Frank I saw nothing to indicate the contrary until someone pointed me to fine print in the release notes, long after we purchased). --Toby You have to dig pretty deep -- deeper than would be expected of a typical buyer -- to find that Solaris does not support it. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
On 22-Jan-07, at 4:03 PM, Richard Elling wrote: [EMAIL PROTECTED] wrote: That said, this definition is not always used consistently, as is the case with the x2100. I filed a bug against the docs in this case, and unfortunately it was closed as will not fix. :-( In the context of a hardware platform it makes little sense to distinguish between hot-plug and hot-swap. The distinction is purely based on the capabilities of the software. Agree. I filed the bug against the docs with the justification that it confuses customers. The bug was closed and we continue to have confused customers :-( Toby Thain wrote: To be clear: the X2100 drives are neither hotswap nor hotplug under Solaris. Replacing a failed drive requires a reboot. I do not believe this is true, though I don't have one to test. This error has been sufficiently addressed in later posts, I think... If this were true, then we would have had to rewrite the disk drivers to not allow us to open a device more than once, even if we also closed the device. I can't imagine anyone allowing such code to be written. However, I don't believe this is the context of the issue. I believe that this release note deals with the use of NVRAID (NVidia's MCP RAID controller) which does not have a systems management interface under Solaris. The solution is to not use NVRAID for Solaris. Rather, use the proven techniques that we've been using for decades to manage hot plugging drives. I have no interest in NVRAID whatsoever. I use SVM and ZFS. Furthermore, NVRAID is the only method that *does* allow hotswap on X2100! (Bizarrely, only with Windows, which is of course useless to me too.) --Toby In short, the release note is confusing, so ignore it. Use x2100 disks as hot pluggable like you've always used hot plug disks in Solaris. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
[EMAIL PROTECTED] wrote: That said, this definition is not always used consistently, as is the case with the x2100. I filed a bug against the docs in this case, and unfortunately it was closed as will not fix. :-( In the context of a hardware platform it makes little sense to distinguish between hot-plug and hot-swap. The distinction is purely based on the capabilities of the software. well, back when I tried (in vain) to apply some common terminology to this, there were SCSI backplanes that had sequenced logic-vs-power connections on insert vs. remove, and had generate an interrupt on insert or remove capabilityand there were backplanes that did not. The former class was maybe kinda practical to support unassisted surprise plugging. The latter made it impossible. I'm sure no one knows what their hardware capabilities ever are, because the industry has completely failed to come up with sane nomenclature for the hardware capabilities...and then we multiply that confusion by having no sane nomenclature for OS capabilities either, and the OS capabilities are never discussed as though they depend on the hardware, which, of course, they do. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
On January 22, 2007 12:12:19 PM -0600 Brian Hechinger [EMAIL PROTECTED] wrote: On Mon, Jan 22, 2007 at 09:39:19AM -0800, Frank Cusack wrote: On January 21, 2007 12:15:22 AM -0200 Toby Thain [EMAIL PROTECTED] wrote: To be clear: the X2100 drives are neither hotswap nor hotplug under Solaris. Replacing a failed drive requires a reboot. Also, adding a drive that wasn't present at boot requires a reboot. This couldn't possibly be true, unless we've taken major steps backwards as this has always been possible (at least on sparc) It is true. Try it. [Sorry to send a reply to a personal mail back to the list, but your email address bounces 450 [EMAIL PROTECTED]: Recipient address rejected: Domain not found] -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
Hi Frank, try man devfsadm, it will update devfs with your new disk drives. disks is an older command that does about the same thing. Cheers, Peter Frank Cusack wrote: On January 22, 2007 12:12:19 PM -0600 Brian Hechinger [EMAIL PROTECTED] wrote: On Mon, Jan 22, 2007 at 09:39:19AM -0800, Frank Cusack wrote: On January 21, 2007 12:15:22 AM -0200 Toby Thain [EMAIL PROTECTED] wrote: To be clear: the X2100 drives are neither hotswap nor hotplug under Solaris. Replacing a failed drive requires a reboot. Also, adding a drive that wasn't present at boot requires a reboot. This couldn't possibly be true, unless we've taken major steps backwards as this has always been possible (at least on sparc) It is true. Try it. [Sorry to send a reply to a personal mail back to the list, but your email address bounces 450 [EMAIL PROTECTED]: Recipient address rejected: Domain not found] -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
yes I am an experienced Solaris admin and know all about devfsadm :-) and the older disks command. It doesn't help in this case. I think it's a BIOS thing. Linux and Windows can't see IDE drives that aren't there at boot time either, and on Solaris the SATA controller runs in some legacy mode so I guess that's why you can't see the newly added drive. Unfortunately all my x2100 hardware is in production and I can't readily retest this to verify. -frank On January 23, 2007 12:02:48 PM +0700 Peter Karlsson [EMAIL PROTECTED] wrote: Hi Frank, try man devfsadm, it will update devfs with your new disk drives. disks is an older command that does about the same thing. Cheers, Peter Frank Cusack wrote: On January 22, 2007 12:12:19 PM -0600 Brian Hechinger [EMAIL PROTECTED] wrote: On Mon, Jan 22, 2007 at 09:39:19AM -0800, Frank Cusack wrote: On January 21, 2007 12:15:22 AM -0200 Toby Thain [EMAIL PROTECTED] wrote: To be clear: the X2100 drives are neither hotswap nor hotplug under Solaris. Replacing a failed drive requires a reboot. Also, adding a drive that wasn't present at boot requires a reboot. This couldn't possibly be true, unless we've taken major steps backwards as this has always been possible (at least on sparc) It is true. Try it. [Sorry to send a reply to a personal mail back to the list, but your email address bounces 450 [EMAIL PROTECTED]: Recipient address rejected: Domain not found] -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
That said, this definition is not always used consistently, as is the case with the x2100. I filed a bug against the docs in this case, and unfortunately it was closed as will not fix. :-( In the context of a hardware platform it makes little sense to distinguish between hot-plug and hot-swap. The distinction is purely based on the capabilities of the software. Casper ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
[EMAIL PROTECTED] said: I was talking about the huge gap in storage solutions from Sun for the middle-ground. While $24,000 is a wonderful deal, it's absolute overkill for what I'm thinking about doing. I was looking for more around 6-8 drives. How about a Sun V40z? It's available with up to 6 drives (300GB ea), and a low-end configuration (cpu/ram-wise) might not be out of your price range, depending on your discount. There are plenty of slots if you want to later add external enclosures, too. Of course, Dell probably has cheaper 64-bit systems with 6 internal drives available too. Regards, Marion ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
Frank Cusack wrote: On January 20, 2007 1:07:27 PM -0800 David J. Orman [EMAIL PROTECTED] wrote: On that note, I've recently read it might be the case that the 1u sun servers do not have hot-swappable disk drives... is this really true? Yes. Only for the x2100 (and x2100m2). It's not that the hardware isn't hot-swappable, it's that Solaris doesn't support it. If you run Windows you will get hot swap. No. To be clear, Sun defines hot swap as a device which can be inserted or removed without system administration tasks required. Sun defines hot plug as a device which can be inserted or removed without causing damage or interruption to a running system, but which may require system administration. The vast majority of the disks Sun sells are hot pluggable. That said, this definition is not always used consistently, as is the case with the x2100. I filed a bug against the docs in this case, and unfortunately it was closed as will not fix. :-( -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
On Sat, 20 Jan 2007, Richard Elling wrote: To be clear, Sun defines hot swap as a device which can be inserted or removed without system administration tasks required. Sun defines hot plug as a device which can be inserted or removed without causing damage or interruption to a running system, but which may require system administration. The vast majority of the disks Sun sells are hot pluggable. OK; given the above definitions, could you please confirm one way or another that the disks in the X2100 are hot pluggable? In other words, if I have a pair of mirrored drives in an X2100 and one of those drives dies, can I take out and replace the defective drive without down time? -- Rich Teer, SCSA, SCNA, SCSECA, OpenSolaris CAB member President, Rite Online Inc. Voice: +1 (250) 979-1638 URL: http://www.rite-group.com/rich ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
On 20-Jan-07, at 8:48 PM, Erik Trimble wrote: Frank Cusack wrote: On January 20, 2007 1:07:27 PM -0800 David J. Orman [EMAIL PROTECTED] wrote: On that note, I've recently read it might be the case that the 1u sun servers do not have hot-swappable disk drives... is this really true? Only for the x2100 (and x2100m2). It's not that the hardware isn't hot-swappable, it's that Solaris doesn't support it. If you run Windows you will get hot swap. -frank ___ I believe this also applies to the X2200 M2 as well. Essentially, all the low-end x64 servers using SATA have Nvidia chipsets which theoretically support Hot-swap of SATA; as noted, the Windows drivers do support this feature, while the Solaris 10 drivers don't (and, I don't know if there are plans to add this feature or not). Personally, I've always been a bit nervous of using chipset-based RAID and expecting Hot-swap to actually, particularly with SATA. I've been bitten on various different (non-Sun) hardware trying this, and it has made me gun-shy of thinking I can actually pull a SATA drive while its mirror is still mounted... Some of us don't give a damn about chipset RAID and want hotswap/ hotplug drives with SVM and/or ZFS. Richard Elling wrote: To be clear, Sun defines hot swap as a device which can be inserted or removed without system administration tasks required. Sun defines hot plug as a device which can be inserted or removed without causing damage or interruption to a running system, but which may require system administration. The vast majority of the disks Sun sells are hot pluggable. To be clear: the X2100 drives are neither hotswap nor hotplug under Solaris. Replacing a failed drive requires a reboot. --Toby -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800) ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: External drive enclosures + Sun Server for massstorage
On 21-Jan-07, at 12:12 AM, Rich Teer wrote: On Sat, 20 Jan 2007, Richard Elling wrote: To be clear, Sun defines hot swap as a device which can be inserted or removed without system administration tasks required. Sun defines hot plug as a device which can be inserted or removed without causing damage or interruption to a running system, but which may require system administration. The vast majority of the disks Sun sells are hot pluggable. OK; given the above definitions, could you please confirm one way or another that the disks in the X2100 are hot pluggable? In other words, if I have a pair of mirrored drives in an X2100 and one of those drives dies, can I take out and replace the defective drive without down time? NO - unless you're running Windows AND chipset RAID (or whatever you want to call it). This is easily verified by experiment with Solaris 10. More information via links I posted earlier in this thread, or buried in X2100 documentation. --Toby -- Rich Teer, SCSA, SCNA, SCSECA, OpenSolaris CAB member President, Rite Online Inc. Voice: +1 (250) 979-1638 URL: http://www.rite-group.com/rich ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
