Re: [Veritas-vx] Strange DMPNODENAME
Only as of Solaris 10 update 4 can you boot from ZFS. AT least, I think you can. Zone roots can definitely go on ZFS, unsure about global zone. I'm researching that question currently. Raw devices = bad idea. PITA (as is preallocated table space, watch the DBAs suck up all available disk space for a 9MB database -- and as a sysadmin you can't see that their table space is 99% empty; that they have performance reasons to do some preallocation is granted but not 99%). As for ZFS and Oracle, interesting -- be sure to read the details on ZFS options to optimize record size of ZFS etc. they have some specific recommendations. Why would you let Oracle do mp is beyond me. I wouldn't let them do cluster IP address control either. It's a database *application*. Certainly I expect Oracle to manage locks on its database and handle the distributed lock problem for itselfusing standard APIs to do so. ZFS is not Storage Foundation equivalent. It is certainly easier to use than Solaris Volume Manager but it lacks the full power of Storage Foundation for UNIX. Beware the limitations even in u4 for ZFS. For example, once you assign a device to a raidz/raidz2 pool you can't take it back: you must copy out all your data, destroy original and copy back (all done with zfs send and recv -- and be VERY sure to use update 4 of Solaris 10 with current patches). Ignore the book's begging to get hold of individual SAN disk directly is my advice: let the SAN do its job of virtualizing storage and let ZFS carve up the LUN as it sees fit (but don't stripe or raidz multiple LUNs from the one SAN storage provider -- its fine to do that across multiple storage servers if you want to provide cross-SAN redundancy but within one server let -it- take care of the RAID on the other side of the SAN). Sun would really like to eat Veritas's lunch but neither SVM nor ZFS is ready to do that. I looked at the procedure, for example, of mirroring an existing filesystem with SVM. Way more complicated than Vx! ZFS has definitely got its place, especially with delegating storage to zones. The per-filesystem quota is the same thing as just making a bunch of VxVM volumes and putting a filesystem on it. With a full license for VxVM and VxFS go right ahead and make 1024 volumes for 1024 users' home directories 1 each. That's a hard 'quota', and just as you can expand your ZFS filesystem to the limit of the pool you can expand your VxVM volume to the limit of the disk group. The cost difference is quite significant and that's a big motivator especially on high-end servers. = Dana Hudes UNIX and Imaging group NYC-HRA MIS +1 718 510 8586 Nextel: 172*26*16684 = -Original Message- From: Jarkko Airaksinen [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 10, 2007 1:52 AM To: Hudes, Dana Subject: RE: [Veritas-vx] Strange DMPNODENAME Hello, Well, you got me convinced :) My next project is to try how Sol10 + Oracle10g perform. The plan is to use zfs on the system disks and either o10g's ASM or raw disk devices for oracle's data disks. Multipathing will be handled by oracle's mp driver. As my SAN hardware is already quite I/O optimized (i.e. the HP EVA already shares the I/O among each and every disk in the array) using ASM probably wouldn't give much performance gains, unless oracle really has developed better performing volume management multipathing drivers than Veritas. I'm expecting the use of raw disk devices to give more noticeable gains. Veritas has been the de facto standard for well performing disks for ages. Is this about to change now? Br, Jarkko -Original Message- From: Hudes, Dana [mailto:[EMAIL PROTECTED] Sent: martes, 09 de octubre de 2007 19:01 To: Jarkko Airaksinen Subject: RE: [Veritas-vx] Strange DMPNODENAME how could mpxio beat vxdmp in his example? 1. Its native, part of the Solaris Operating System. This gives better integration with the lower-level drivers which are the Leadville stack rather than the old scsi-over-fiber-channel drivers. 2. (1) also implies support from Sun including patches to stms if a patch to the underlying drivers needs it rather than waiting for Veritas to react. If you don't have a Sun support contract you might not have access to those patches; since my employer has such a contract I speak from that perspective. 3. You get only 1 device entry for each LUN or disk, not two as with vxdmp. 4. This is Solaris 10. It is new and improved in lots of respects; take advantage of them instead of treating it exactly as you did with solaris 2.6, 7, 8 and 9. = Dana Hudes UNIX and Imaging group NYC-HRA MIS +1 718 510 8586 Nextel: 172*26*16684 = -Original Message- From: Jarkko Airaksinen [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 03, 2007 10:57 AM To: Hudes, Dana Subject: RE: [Veritas-vx] Strange DMPNODENAME Hello, Just curious, how could mpxio beat vxdmp in his example? Br, Jarkko Airaksinen
Re: [Veritas-vx] Strange DMPNODENAME
James, As you correctly note in your email, the dmpnodename doesn't affect functionality. The dmpnodename is really just a 'label' for the DMP metanode that handles multipathing over the underlying subpaths to a given LUN. You have a choice between two namingschemes in SF: Enclosure Based Names (EBN) and OS Names (OSN). Your system is set to OSN, and so DMP picks 'one' of the OS device names of the underlying subpaths and uses it to name the dmp metanode. If your system was set to EBN, the dmpnodename would be something like AMS_WMS0_5 (or AMS_WMS0_6, you get the idea). The OS name really has no more meaning to DMP than the AMS_WMS0_5. It's just a label. In OSN, DMP picks which subpath name to use for the dmpnodename based on some kernel identifiers. That's how a straighforward algorithm based on kernel identifiers can lead to names that seem inconsistent to a user. We are changing the algorithm in 5.0 MP3 to avoid such outcomes: in 5.0 MP3, an OSN dmpnodename will be the subpath name with the smallest c#, then t# and then d#. That way if you have 50 devices presented through c5 and c6, their dmpnodenames will reliably all start with c5. In general, I would advise against enabling MPxIO in an SFHA environment. I hope this helps, Thomas -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of James Kelty Sent: Monday, October 01, 2007 9:08 AM To: Veritas-vx@mailman.eng.auburn.edu; Veritas-vx@mailman.eng.auburn.edu Subject: [Veritas-vx] Strange DMPNODENAME Hey there all, I am running (yes another one!) Storage Foundation HA Standard v5.0 on Solaris 10 X64 that is hooked up to an HDS AMS200. I have _one_ oddity in the DMP node name for a single LUN. Looking at c6t0d0s2 from the vxdisk list output, I see that the primary path is actually c5t0d0s2 (which I would have expected) and yet the DMPNODENAME is coming out as the controller 6 device. I don't think that this is causing any issues as it does have both paths, but it would be nice to know why this is happening. Any thoughts? I'm not running any other multi-path software (MPxIO has been disabled per the release notes), but this one disk is just behaving differently. Thanks a lot! -James bash-3.00# vxdisk list DEVICE TYPEDISK GROUPSTATUS c1t0d0s2 auto:none --online invalid c2t0d0s2 auto:none --online invalid c5t0d1 auto:none --online invalid c5t0d2 auto:none --online invalid c5t0d3s2 auto:none --online invalid c5t0d4 auto:none --online invalid c5t0d5 auto:none --online invalid c5t0d6s2 auto:none --online invalid c5t0d7 auto:none --online invalid c5t0d8 auto:none --online invalid c5t0d9 auto:none --online invalid c5t0d10 auto:none --online invalid c5t0d11 auto:none --online invalid c5t0d12 auto:none --online invalid c5t0d13 auto:none --online invalid c5t0d14 auto:none --online invalid c6t0d0s2 auto:none --online invalid bash-3.00# vxdisk list c6t0d0s2 Device:c6t0d0s2 devicetag: c6t0d0 type: auto info: format=none flags: online ready private autoconfig invalid pubpaths: block=/dev/vx/dmp/c6t0d0s2 char=/dev/vx/rdmp/c6t0d0s2 guid: - udid: HITACHI%5FDF600F%5F73014387%5F site: - Multipathing information: numpaths: 2 c6t0d0s2state=enabled type=secondary c5t0d0s2state=enabled type=primary bash-3.00# vxdmpadm getsubpaths ctlr=c6 NAME STATE[A] PATH-TYPE[M] DMPNODENAME ENCLR-TYPE ENCLR-NAME ATTRS == == c6t0d0s2 ENABLEDSECONDARYc6t0d0s2 AMS_WMS AMS_WMS0 - c6t0d1 ENABLEDSECONDARYc5t0d1 AMS_WMS AMS_WMS0 - c6t0d2 ENABLEDSECONDARYc5t0d2 AMS_WMS AMS_WMS0 - c6t0d3s2 ENABLED(A) PRIMARY c5t0d3s2 AMS_WMS AMS_WMS0 - c6t0d4 ENABLED(A) PRIMARY c5t0d4 AMS_WMS AMS_WMS0 - c6t0d5 ENABLED(A) PRIMARY c5t0d5 AMS_WMS AMS_WMS0 - c6t0d6s2 ENABLEDSECONDARYc5t0d6s2 AMS_WMS AMS_WMS0 - c6t0d7 ENABLEDSECONDARYc5t0d7 AMS_WMS AMS_WMS0 - c6t0d8 ENABLEDSECONDARYc5t0d8 AMS_WMS AMS_WMS0 - c6t0d9 ENABLED(A) PRIMARY
Re: [Veritas-vx] Strange DMPNODENAME
Assuming you meet the minimum BIOS levels for an Emulex or Qlogic HBA, you may well find that on Solaris 10 x64 you are happier disabling vxdmp on your SAN connections and instead using the native SAN suite which comes with the OS by issuing stmsboot -e (it'll reboot your system at the end of the configuration process). = Dana Hudes UNIX and Imaging group NYC-HRA MIS +1 718 510 8586 Nextel: 172*26*16684 = -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of James Kelty Sent: Monday, October 01, 2007 12:08 PM To: Veritas-vx@mailman.eng.auburn.edu; Veritas-vx@mailman.eng.auburn.edu Subject: [Veritas-vx] Strange DMPNODENAME Hey there all, I am running (yes another one!) Storage Foundation HA Standard v5.0 on Solaris 10 X64 that is hooked up to an HDS AMS200. I have _one_ oddity in the DMP node name for a single LUN. Looking at c6t0d0s2 from the vxdisk list output, I see that the primary path is actually c5t0d0s2 (which I would have expected) and yet the DMPNODENAME is coming out as the controller 6 device. I don't think that this is causing any issues as it does have both paths, but it would be nice to know why this is happening. Any thoughts? I'm not running any other multi-path software (MPxIO has been disabled per the release notes), but this one disk is just behaving differently. Thanks a lot! -James bash-3.00# vxdisk list DEVICE TYPEDISK GROUPSTATUS c1t0d0s2 auto:none --online invalid c2t0d0s2 auto:none --online invalid c5t0d1 auto:none --online invalid c5t0d2 auto:none --online invalid c5t0d3s2 auto:none --online invalid c5t0d4 auto:none --online invalid c5t0d5 auto:none --online invalid c5t0d6s2 auto:none --online invalid c5t0d7 auto:none --online invalid c5t0d8 auto:none --online invalid c5t0d9 auto:none --online invalid c5t0d10 auto:none --online invalid c5t0d11 auto:none --online invalid c5t0d12 auto:none --online invalid c5t0d13 auto:none --online invalid c5t0d14 auto:none --online invalid c6t0d0s2 auto:none --online invalid bash-3.00# vxdisk list c6t0d0s2 Device:c6t0d0s2 devicetag: c6t0d0 type: auto info: format=none flags: online ready private autoconfig invalid pubpaths: block=/dev/vx/dmp/c6t0d0s2 char=/dev/vx/rdmp/c6t0d0s2 guid: - udid: HITACHI%5FDF600F%5F73014387%5F site: - Multipathing information: numpaths: 2 c6t0d0s2state=enabled type=secondary c5t0d0s2state=enabled type=primary bash-3.00# vxdmpadm getsubpaths ctlr=c6 NAME STATE[A] PATH-TYPE[M] DMPNODENAME ENCLR-TYPE ENCLR-NAME ATTRS c6t0d0s2 ENABLEDSECONDARYc6t0d0s2 AMS_WMS AMS_WMS0 - c6t0d1 ENABLEDSECONDARYc5t0d1 AMS_WMS AMS_WMS0 - c6t0d2 ENABLEDSECONDARYc5t0d2 AMS_WMS AMS_WMS0 - c6t0d3s2 ENABLED(A) PRIMARY c5t0d3s2 AMS_WMS AMS_WMS0 - c6t0d4 ENABLED(A) PRIMARY c5t0d4 AMS_WMS AMS_WMS0 - c6t0d5 ENABLED(A) PRIMARY c5t0d5 AMS_WMS AMS_WMS0 - c6t0d6s2 ENABLEDSECONDARYc5t0d6s2 AMS_WMS AMS_WMS0 - c6t0d7 ENABLEDSECONDARYc5t0d7 AMS_WMS AMS_WMS0 - c6t0d8 ENABLEDSECONDARYc5t0d8 AMS_WMS AMS_WMS0 - c6t0d9 ENABLED(A) PRIMARY c5t0d9 AMS_WMS AMS_WMS0 - c6t0d10 ENABLED(A) PRIMARY c5t0d10 AMS_WMS AMS_WMS0 - c6t0d11 ENABLEDSECONDARYc5t0d11 AMS_WMS AMS_WMS0 - c6t0d12 ENABLEDSECONDARYc5t0d12 AMS_WMS AMS_WMS0 - c6t0d13 ENABLED(A) PRIMARY c5t0d13 AMS_WMS AMS_WMS0 - c6t0d14 ENABLED(A) PRIMARY c5t0d14 AMS_WMS AMS_WMS0 - bash-3.00# vxdmpadm getsubpaths ctlr=c5 NAME STATE[A] PATH-TYPE[M] DMPNODENAME ENCLR-TYPE ENCLR-NAME ATTRS c5t0d0s2 ENABLED(A) PRIMARY c6t0d0s2 AMS_WMS AMS_WMS0 - c5t0d1 ENABLED(A) PRIMARY c5t0d1 AMS_WMS AMS_WMS0 - c5t0d2 ENABLED(A) PRIMARY c5t0d2 AMS_WMS AMS_WMS0 - c5t0d3s2 ENABLEDSECONDARYc5t0d3s2 AMS_WMS AMS_WMS0 - c5t0d4 ENABLEDSECONDARYc5t0d4 AMS_WMS AMS_WMS0 - c5t0d5
