Hi All, As noted below, we have attempted to test the illumos based omniOS in our datacenter on a dell R610 connected through dual cisco fiber channel fabrics to a compellent SAN with dual controllers. We got pretty far with an out of the box setup until the last multipathing bit. I was hoping there might be some additional eyes on this list that may be able to provide some insight.

Thanks in advance for taking the time to read through my following notes on this topic,
Majeed Qulbain




On 10/18/2012 03:06 PM, Majeed wrote:
Thanks Theo, that makes sense. I will definitely do that.

Majeed

On 10/18/2012 02:58 PM, Theo Schlossnagle wrote:
This is a more general Illumos problem, you will open your
troubleshooting to a larger audience by interfacing with the
illumos-discuss list.  I encourage you to post there! I'm subscribed
there as well, but I haven't done multipathing against compellent and
it appears no one else on the omnios-discuss list has either.
Anything learned in the illumos-discuss group can be applied directly
to OmniOS.

On Thu, Oct 18, 2012 at 3:51 PM, Majeed<[email protected]>  wrote:
Hi Again, just wanted to update the list with my latest progress and
thoughts on this topic. I think the issue lies in libmpscsi_vhci.so's lack
of support for multipathing with compellent.

When I do "mpathadm show mpath-support libmpscsi_vhci.so" on omniOS I see many venders and models listed but not compellent. We are going to spin up a solaris 11 machine to see what it looks like. I wonder if this was forked just before SUN/oracle added support for compellent (i could be way off base
here).

When I do the same command on solaris 10 u10, at the very bottom of the list
i see.
                 Vendor:  COMPELNT
                 Product:  Compellent Vol
                 Revision:
                 Supported Load Balance Types:
                         round-robin

I have verified that multipathing is working properly on other arrays. I zoned up omniOS with all controllers on a 6140 (CSM200) that sit in multiple fabrics, mapped a lun, and it looks wonderful. mpatham reports as expected, and all paths are condensed to a single block device in format's output.

% mpathadm list LU
     /dev/rdsk/c0t600A0B800033816200001391507FF856d0s2
         Total Path Count: 4
         Operational Path Count: 4



It seems there are some options in /kernel/drv/scsi_vhci.conf to force
scsi_vhci to accept 3rd party devices (hopefully compellent) for failover,
but this is kinda getting over my head. It seems like if I had a proper
ddi-forceload ''misc/scsi_vhci/scsi_vhci_f_<x>" file for compellent and I knew the proper VenderID and ProductID for the compellent devices, i might be able to force it. Im not sure if can use the information from the format
inquiry to populate these values.

format>  inquiry
Vendor:   COMPELNT
Product:  Compellent Vol
Revision: 0505


Any thoughts on this would be excellent.

Thanks,
Majeed Qulbain



On 10/18/2012 09:30 AM, Majeed wrote:
I just noticed that we do have access to the mpathadm command by
installing the "pkg:/storage/[email protected]" package. After
installing I don't see any paths with "mpathadm list LU" which is what im used to on solaris 10 (i know I should be comparing to solaris 11, just havn't played with it much). I wonder if there are any other bits that I
need to install or other configuration tweaks to get this going.

Thanks in advance for any pointers,
Majeed

Some other related commands that might assist troubleshooting:

% uname -a
SunOS jhx011 5.11 omnios-eae537b i86pc i386 i86pc

% mpathadm list initiator-port
Initiator Port:  w5d4ae520937bbb00
Initiator Port: iqn.1986-03.com.sun:01:b06a64f606ff.507f35b3,4000002a00ff
Initiator Port:  w5d4ae520937bbb00
Initiator Port:  2101001b32b40767
Initiator Port:  2100001b32940767

% luxadm -e port
/devices/pci@0,0/pci8086,3410@9/pci1077,143@0/fp@0,0:devctl
CONNECTED
/devices/pci@0,0/pci8086,3410@9/pci1077,143@0,1/fp@0,0:devctl
CONNECTED

% luxadm -v probe

Found Fibre Channel device(s):
   Node WWN:5000d310004a0602  Device Type:Disk device
     Logical Path:/dev/rdsk/c16t5000D310004A064Bd1s2
   Node WWN:5000d310004a0602  Device Type:Disk device
     Logical Path:/dev/rdsk/c17t5000D310004A064Cd1s2

% luxadm disp /dev/rdsk/c16t5000D310004A064Bd1s2
DEVICE PROPERTIES for disk: /dev/rdsk/c16t5000D310004A064Bd1s2
   Vendor:        COMPELNT
   Product ID:        Compellent Vol
   Revision:        0505
   Serial Num:        00004a06-00000014
   Unformatted capacity:    512000.000 MBytes
   Read Cache:        Enabled
     Minimum prefetch:    0x100
     Maximum prefetch:    0xf0
   Device Type:        Disk device
   Path(s):

   /dev/rdsk/c16t5000D310004A064Bd1s2

/devices/pci@0,0/pci8086,3410@9/pci1077,143@0/fp@0,0/disk@w5000d310004a064b,1:c,raw
    Controller          /dev/cfg/c16
     Device Address        5000d310004a064b,1
     Host controller port WWN    2100001b32940767
    Controller          /dev/cfg/c17
     Device Address        5000d310004a064c,1
     Host controller port WWN    2101001b32b40767


On 10/17/2012 06:25 PM, Majeed wrote:
Hi All, We are doing some testing with omniOS in our datacenter and would love connect it up to our compellent san via fc connectivity. I slapped omniOS onto a dell r610 that has a qlogic dual port hba (each port is pathed to separate fabrics), performed zoning in each fabric, and assigned some luns. Now im not sure if compellent supports Solaris 11 (i can contact them and see about that), but it definately support solaris 10 which is what we
use on other production systems.

I'v actually made it pretty far through the process which is really
exciting, except for the last multipathing bit. After mapping a volume to
the host I can't seem to condense all the devices down to a single
mulipathed device. It seems multipathing is enabled for the fp driver. I
noticed there is not a mpathadm command.

Anyone have any ideas? I'll post some output from the commands i think of
below. Let me know if you would like to see any other information.

Thanks much,
Majeed


On solaris 10 the main thing we have to do is add the following to
/kernel/drv/scsi_vhci.conf (reconfigure reboot)
device-type-scsi-options-list =
"COMPELNTCompellent Vol", "symmetric-option";
symmetric-option = 0x1000000;


Here is the format output:
AVAILABLE DISK SELECTIONS:
0. c13t5000CCA0213ED1E1d0<HITACHI-HUC106030CSS600-A360 cyl 36470
alt 2 hd 255 sec 63>

/pci@0,0/pci8086,2940@1c/pci1028,1f1e@0/iport@40/disk@w5000cca0213ed1e1,0 1. c14t5000CCA0213EFC7Dd0<HITACHI-HUC106030CSS600-A360 cyl 36469
alt 2 hd 255 sec 63>

/pci@0,0/pci8086,2940@1c/pci1028,1f1e@0/iport@80/disk@w5000cca0213efc7d,0 2. c16t5000D310004A064Bd1<COMPELNT-CompellentVol-0505 cyl 65268
alt 2 hd 255 sec 63>

/pci@0,0/pci8086,3410@9/pci1077,143@0/fp@0,0/disk@w5000d310004a064b,1
3. c17t5000D310004A064Cd1<COMPELNT-CompellentVol-0505 cyl 65268
alt 2 hd 255 sec 63>

/pci@0,0/pci8086,3410@9/pci1077,143@0,1/fp@0,0/disk@w5000d310004a064c,1
Specify disk (enter its number): Specify disk (enter its number):



% fcinfo hba-port
HBA Port WWN: 2100001b32940767
     Port Mode: Initiator
     Port ID: 91c00
     OS Device Name: /dev/cfg/c16
     Manufacturer: QLogic Corp.
     Model: 375-3356-02
     Firmware Version: 05.03.01
     FCode/BIOS Version:  BIOS: 2.02; fcode: 2.01; EFI: 2.00;
     Serial Number: 0402H00-0941774287
     Driver Name: qlc
     Driver Version: 20100408-3.01
     Type: N-port
     State: online
     Supported Speeds: 1Gb 2Gb 4Gb
     Current Speed: 4Gb
     Node WWN: 2000001b32940767
     Max NPIV Ports: 127
     NPIV port list:
HBA Port WWN: 2101001b32b40767
     Port Mode: Initiator
     Port ID: a1900
     OS Device Name: /dev/cfg/c17
     Manufacturer: QLogic Corp.
     Model: 375-3356-02
     Firmware Version: 05.03.01
     FCode/BIOS Version:  BIOS: 2.02; fcode: 2.01; EFI: 2.00;
     Serial Number: 0402H00-0941774287
     Driver Name: qlc
     Driver Version: 20100408-3.01
     Type: N-port
     State: online
     Supported Speeds: 1Gb 2Gb 4Gb
     Current Speed: 4Gb
     Node WWN: 2001001b32b40767
     Max NPIV Ports: 127
     NPIV port list:


% cfgadm -al -o show_SCSI_LUN
Ap_Id                          Type         Receptacle   Occupant
Condition
c16                            fc-fabric    connected    configured
unknown
c16::5000d310004a0649          unknown      connected    unconfigured
unknown
c16::5000d310004a064b,1        disk         connected    configured
unknown
c17                            fc-fabric    connected    configured
unknown
c17::5000d310004a064a          unknown      connected    unconfigured
unknown
c17::5000d310004a064c,1        disk         connected    configured
unknown


_______________________________________________
--
Majeed Abu-Qulbain
Lead Unix Admin - CISS
309-438-3455
Illinois State University



-------------------------------------------
illumos-discuss
Archives: https://www.listbox.com/member/archive/182180/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be
Modify Your Subscription: 
https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4
Powered by Listbox: http://www.listbox.com

Reply via email to