Re: Current status of support for high end SAN hardware
Quoting Andy Kosela, who wrote on Mon, Jun 09, 2008 at 02:41:24PM +0200 .. > On Mon, Jun 9, 2008 at 12:36 PM, Russell Vincent <[EMAIL PROTECTED]> wrote: > > > > The FreeBSD support for multipath/SAN is fairly poor. It's fiddly > > to get to work and boot times are a little variable (into the > > minutes) as it tries to discover the devices. Once it is configured > > and booted, it just works as long as things don't go wrong. SAN > > outages cause the machine to hang up until the issue is resolved > > (in which case it just seems to continue) or it doesn't recover at > > all and requires a reboot. Note that I don't spend a significant > > amount of time on this, so it may be that I could do things a little > > better. I have also not tested the failover stuff very well (I > > only upgraded this machine to 7-STABLE fairly recently). Disk > > access seems to be restricted to a single path at a time. Problem > > solving is very tricky as there is very little information to trace > > which path/disk refers to which fabric/storage device/LUN. > > > > Russell, > Thank you for your insights. It's good to see you have no problems > with isp(4) and Qlogic HBAs. Though I'm concerned about > multipathing. We run 6.x-RELEASE releases so it seems we have > to upgrade to 7.0-RELEASE to achieve that goal. gmultipath(8) > code is fairly new so I suppose it's not that mature yet as in > Linux. Unfortunately it is only an active/passive approach with > no load balancing (the active path is active until a BIO request is > failed with EIO or ENXIO) Well, it is worse than that: HP EVA arrays have 2 different multipath behaviours. VCS 3.x firmware is active/passive, whereas VCS 4.x is active/active. Active/active as well are all XCS firmware versions. All newer EVAs run XCS. To properly do multipathing on active/active EVAs your multipathing software should be aware of the ALUA stuff. On active/active EVAs you should fire your read commands to the right HSV controller, if you take the other HSV things will work, but the controller you sent the read to will have to do a proxy read to the HSV controller that "owns" the LUN. This will be slower that taking the right HSV in one go. It also consumes bandwith on the HSV controller's mirror port(s). Writes do not matter, they are mirrored to the write cache in each HSV anyway. Linux devicemapper is sort-of getting there now. For the longest time HP did not support devicemapper, and for good reasons too. To make things interesting I think I remember Matt (the isp(4) author) telling me that some ispfw(4) versions not getting it right. If the adapter firmware does not properly inform the isp(4) driver of SAN disruptions you are obviously doomed as a multipathing driver that depends on the underlying HBA drivers to feed SAN problems/status upstream. In short: if you use isp(4), always load ispfw(4) too. Getting all the components in a FC SAN to DTRT is unfortunately still a major pain in the backside. hth Wilko -- Wilko Bulte [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Current status of support for high end SAN hardware
man gmultipath is your friend ;) MULTIPATH ARCHITECTURE This is an active/passive multiple path architecture with no device knowledge or presumptions other than size matching built in. Therefore the user must exercise some care in selecting providers that do indeed represent multiple paths to the same underlying disk device. The reason for this is that there are several criteria across multiple underlying transport types that can indicate identity, but in all respects such identity can rarely be considered definitive. Regards, Daniel Stefan Lambrev ha scritto: Daniel Ponticello wrote: On FreeBSD7, i'm succesfully using Qlogic 4gb fibre channel HBAs (ISP driver) attached to Fibre Brocade Switch and IBM DS4700 (14 disks array) using 4 way multipath with gmultipath. So far the support in gmultipath is active/passive only? I think in RH5 you can have active/active. Am I right? Regards, Daniel Andy Kosela ha scritto: Hi all, What is the current status of support for high end SAN hardware in FreeBSD? I'm especially interested in support for HP EVA/XP disk arrays, Qlogic HBAs, multipathing. How FreeBSD compares in this environment to RHEL 5? -- WBR, Cordiali Saluti, Daniel Ponticello, VP of Engineering Network Coordination Centre of Skytek --- - For further information about our services: - Please visit our website at http://www.Skytek.it --- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Current status of support for high end SAN hardware
On Mon, Jun 9, 2008 at 12:36 PM, Russell Vincent <[EMAIL PROTECTED]> wrote: > > The FreeBSD support for multipath/SAN is fairly poor. It's fiddly > to get to work and boot times are a little variable (into the > minutes) as it tries to discover the devices. Once it is configured > and booted, it just works as long as things don't go wrong. SAN > outages cause the machine to hang up until the issue is resolved > (in which case it just seems to continue) or it doesn't recover at > all and requires a reboot. Note that I don't spend a significant > amount of time on this, so it may be that I could do things a little > better. I have also not tested the failover stuff very well (I > only upgraded this machine to 7-STABLE fairly recently). Disk > access seems to be restricted to a single path at a time. Problem > solving is very tricky as there is very little information to trace > which path/disk refers to which fabric/storage device/LUN. > Russell, Thank you for your insights. It's good to see you have no problems with isp(4) and Qlogic HBAs. Though I'm concerned about multipathing. We run 6.x-RELEASE releases so it seems we have to upgrade to 7.0-RELEASE to achieve that goal. gmultipath(8) code is fairly new so I suppose it's not that mature yet as in Linux. Unfortunately it is only an active/passive approach with no load balancing (the active path is active until a BIO request is failed with EIO or ENXIO) Good support for high end SAN environment is essential in todays data centers, as most servers are connected to storage using FC based storage area network. I hope things will improve as 7.x-STABLE will be polished over time. Mark, I completely agree with you that ZFS is much better than Ext3+LVM2. Ext3 is still lacking internal snapshoting capability, so it's even inferior to UFS2. As a matter of fact I'm watching Oracle's btrfs development as it seems it will change many things on Linux filesystems scene. Though I still fear ZFS on FreeBSD is not as yet mature to the point of using it in a mission critical 24x7 production environments. But it's definetly something to watch out for. -- Andy Kosela ora et labora ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Current status of support for high end SAN hardware
Daniel Ponticello wrote: On FreeBSD7, i'm succesfully using Qlogic 4gb fibre channel HBAs (ISP driver) attached to Fibre Brocade Switch and IBM DS4700 (14 disks array) using 4 way multipath with gmultipath. So far the support in gmultipath is active/passive only? I think in RH5 you can have active/active. Am I right? Regards, Daniel Andy Kosela ha scritto: Hi all, What is the current status of support for high end SAN hardware in FreeBSD? I'm especially interested in support for HP EVA/XP disk arrays, Qlogic HBAs, multipathing. How FreeBSD compares in this environment to RHEL 5? -- Best Wishes, Stefan Lambrev ICQ# 24134177 ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Current status of support for high end SAN hardware
Andy I am currently using HP MSA1500cs SAN setups on FreeBSD 7 and 6.3 using qlogic cards in HP DL380G4 and G5 servers. I am not yet using multipath fiber channel which is supported in 7 and I want to test this out soon. As for Redhat ES 4 and 5 I am also using the same hardware setup , I have to say that RedHat ES4 works better for me the Enterprise 5 . ES5 has some odd ball networking issues, when you upgrade from say update 0 -> 1 or 1 -> 2. For some reason Redhat decided that it needed to remove your configs for eth0 as part of the upgrade. I would say to look at using 64Bit FreeBSD 7-RELEASE and ZFS as the filesystem on the SAN. ZFS is hands down better then EXT3+LVM . Andy Kosela wrote: Hi all, What is the current status of support for high end SAN hardware in FreeBSD? I'm especially interested in support for HP EVA/XP disk arrays, Qlogic HBAs, multipathing. How FreeBSD compares in this environment to RHEL 5? -- Andy Kosela ora et labora ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]" -- Mark Saad Managed UNIX Support DataPipe Managed Global IT Services () ascii ribbon campaign - against html e-mail /\ www.asciiribbon.org - against proprietary attachments This message may contain confidential or privileged information. If you are not the intended recipient, please advise us immediately and delete this message. See http://www.datapipe.com/emaildisclaimer.aspx for further information on confidentiality and the risks of non-secure electronic communication. If you cannot access these links, please notify us by reply message and we will send the contents to you. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Current status of support for high end SAN hardware
On FreeBSD7, i'm succesfully using Qlogic 4gb fibre channel HBAs (ISP driver) attached to Fibre Brocade Switch and IBM DS4700 (14 disks array) using 4 way multipath with gmultipath. Regards, Daniel Andy Kosela ha scritto: Hi all, What is the current status of support for high end SAN hardware in FreeBSD? I'm especially interested in support for HP EVA/XP disk arrays, Qlogic HBAs, multipathing. How FreeBSD compares in this environment to RHEL 5? -- WBR, Cordiali Saluti, Daniel Ponticello, VP of Engineering Network Coordination Centre of Skytek --- - For further information about our services: - Please visit our website at http://www.Skytek.it --- ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Current status of support for high end SAN hardware
Hi all, What is the current status of support for high end SAN hardware in FreeBSD? I'm especially interested in support for HP EVA/XP disk arrays, Qlogic HBAs, multipathing. How FreeBSD compares in this environment to RHEL 5? -- Andy Kosela ora et labora ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
