I just had this with another SAN vendor yesterday. The issue I ran into
was that almost all the documentation I saw told me to modify
/kernel/drv/scsi_vhci.conf by adding a device-type-scsi-options-list
section, but that's wrong!
You need to modify that file, alright, but it needs a
scsi-vhci-failover-override section, like this:
scsi-vhci-failover-override =
"COMPELNTCompellent", "f_sym";
Note that the second half of the identifier is the product name, but only
up to the first space!
Good luck.
On Thu, Oct 18, 2012 at 4:49 PM, Majeed <[email protected]> wrote:
> Hi All, As noted below, we have attempted to test the illumos based omniOS
> in our datacenter on a dell R610 connected through dual cisco fiber channel
> fabrics to a compellent SAN with dual controllers. We got pretty far with
> an out of the box setup until the last multipathing bit. I was hoping there
> might be some additional eyes on this list that may be able to provide some
> insight.
>
> Thanks in advance for taking the time to read through my following notes
> on this topic,
> Majeed Qulbain
>
>
>
>
> On 10/18/2012 03:06 PM, Majeed wrote:
>
>> Thanks Theo, that makes sense. I will definitely do that.
>>
>> Majeed
>>
>> On 10/18/2012 02:58 PM, Theo Schlossnagle wrote:
>>
>>> This is a more general Illumos problem, you will open your
>>> troubleshooting to a larger audience by interfacing with the
>>> illumos-discuss list. I encourage you to post there! I'm subscribed
>>> there as well, but I haven't done multipathing against compellent and
>>> it appears no one else on the omnios-discuss list has either.
>>> Anything learned in the illumos-discuss group can be applied directly
>>> to OmniOS.
>>>
>>> On Thu, Oct 18, 2012 at 3:51 PM, Majeed<[email protected]> wrote:
>>>
>>>> Hi Again, just wanted to update the list with my latest progress and
>>>> thoughts on this topic. I think the issue lies in libmpscsi_vhci.so's
>>>> lack
>>>> of support for multipathing with compellent.
>>>>
>>>> When I do "mpathadm show mpath-support libmpscsi_vhci.so" on omniOS I
>>>> see
>>>> many venders and models listed but not compellent. We are going to spin
>>>> up a
>>>> solaris 11 machine to see what it looks like. I wonder if this was
>>>> forked
>>>> just before SUN/oracle added support for compellent (i could be way off
>>>> base
>>>> here).
>>>>
>>>> When I do the same command on solaris 10 u10, at the very bottom of the
>>>> list
>>>> i see.
>>>> Vendor: COMPELNT
>>>> Product: Compellent Vol
>>>> Revision:
>>>> Supported Load Balance Types:
>>>> round-robin
>>>>
>>>> I have verified that multipathing is working properly on other arrays. I
>>>> zoned up omniOS with all controllers on a 6140 (CSM200) that sit in
>>>> multiple
>>>> fabrics, mapped a lun, and it looks wonderful. mpatham reports as
>>>> expected,
>>>> and all paths are condensed to a single block device in format's output.
>>>>
>>>> % mpathadm list LU
>>>> /dev/rdsk/**c0t600A0B800033816200001391507**FF856d0s2
>>>> Total Path Count: 4
>>>> Operational Path Count: 4
>>>>
>>>>
>>>>
>>>> It seems there are some options in /kernel/drv/scsi_vhci.conf to force
>>>> scsi_vhci to accept 3rd party devices (hopefully compellent) for
>>>> failover,
>>>> but this is kinda getting over my head. It seems like if I had a proper
>>>> ddi-forceload ''misc/scsi_vhci/scsi_vhci_f_<**x>" file for compellent
>>>> and I
>>>> knew the proper VenderID and ProductID for the compellent devices, i
>>>> might
>>>> be able to force it. Im not sure if can use the information from the
>>>> format
>>>> inquiry to populate these values.
>>>>
>>>> format> inquiry
>>>> Vendor: COMPELNT
>>>> Product: Compellent Vol
>>>> Revision: 0505
>>>>
>>>>
>>>> Any thoughts on this would be excellent.
>>>>
>>>> Thanks,
>>>> Majeed Qulbain
>>>>
>>>>
>>>>
>>>> On 10/18/2012 09:30 AM, Majeed wrote:
>>>>
>>>>> I just noticed that we do have access to the mpathadm command by
>>>>> installing the "pkg:/storage/[email protected]**0.151002" package.
>>>>> After
>>>>> installing I don't see any paths with "mpathadm list LU" which is what
>>>>> im
>>>>> used to on solaris 10 (i know I should be comparing to solaris 11, just
>>>>> havn't played with it much). I wonder if there are any other bits that
>>>>> I
>>>>> need to install or other configuration tweaks to get this going.
>>>>>
>>>>> Thanks in advance for any pointers,
>>>>> Majeed
>>>>>
>>>>> Some other related commands that might assist troubleshooting:
>>>>>
>>>>> % uname -a
>>>>> SunOS jhx011 5.11 omnios-eae537b i86pc i386 i86pc
>>>>>
>>>>> % mpathadm list initiator-port
>>>>> Initiator Port: w5d4ae520937bbb00
>>>>> Initiator Port: iqn.1986-03.com.sun:01:**b06a64f606ff.507f35b3,**
>>>>> 4000002a00ff
>>>>> Initiator Port: w5d4ae520937bbb00
>>>>> Initiator Port: 2101001b32b40767
>>>>> Initiator Port: 2100001b32940767
>>>>>
>>>>> % luxadm -e port
>>>>> /devices/pci@0,0/pci8086,3410@**9/pci1077,143@0/fp@0,0:devctl
>>>>> CONNECTED
>>>>> /devices/pci@0,0/pci8086,3410@**9/pci1077,143@0,1/fp@0,0:**devctl
>>>>> CONNECTED
>>>>>
>>>>> % luxadm -v probe
>>>>>
>>>>> Found Fibre Channel device(s):
>>>>> Node WWN:5000d310004a0602 Device Type:Disk device
>>>>> Logical Path:/dev/rdsk/**c16t5000D310004A064Bd1s2
>>>>> Node WWN:5000d310004a0602 Device Type:Disk device
>>>>> Logical Path:/dev/rdsk/**c17t5000D310004A064Cd1s2
>>>>>
>>>>> % luxadm disp /dev/rdsk/**c16t5000D310004A064Bd1s2
>>>>> DEVICE PROPERTIES for disk: /dev/rdsk/**c16t5000D310004A064Bd1s2
>>>>> Vendor: COMPELNT
>>>>> Product ID: Compellent Vol
>>>>> Revision: 0505
>>>>> Serial Num: 00004a06-00000014
>>>>> Unformatted capacity: 512000.000 MBytes
>>>>> Read Cache: Enabled
>>>>> Minimum prefetch: 0x100
>>>>> Maximum prefetch: 0xf0
>>>>> Device Type: Disk device
>>>>> Path(s):
>>>>>
>>>>> /dev/rdsk/**c16t5000D310004A064Bd1s2
>>>>>
>>>>> /devices/pci@0,0/pci8086,3410@**9/pci1077,143@0/fp@0,0/disk@**w5000d310004a064b,1:c,raw
>>>>>
>>>>> Controller /dev/cfg/c16
>>>>> Device Address 5000d310004a064b,1
>>>>> Host controller port WWN 2100001b32940767
>>>>> Controller /dev/cfg/c17
>>>>> Device Address 5000d310004a064c,1
>>>>> Host controller port WWN 2101001b32b40767
>>>>>
>>>>>
>>>>> On 10/17/2012 06:25 PM, Majeed wrote:
>>>>>
>>>>>> Hi All, We are doing some testing with omniOS in our datacenter and
>>>>>> would
>>>>>> love connect it up to our compellent san via fc connectivity. I
>>>>>> slapped
>>>>>> omniOS onto a dell r610 that has a qlogic dual port hba (each port is
>>>>>> pathed
>>>>>> to separate fabrics), performed zoning in each fabric, and assigned
>>>>>> some
>>>>>> luns. Now im not sure if compellent supports Solaris 11 (i can
>>>>>> contact them
>>>>>> and see about that), but it definately support solaris 10 which is
>>>>>> what we
>>>>>> use on other production systems.
>>>>>>
>>>>>> I'v actually made it pretty far through the process which is really
>>>>>> exciting, except for the last multipathing bit. After mapping a
>>>>>> volume to
>>>>>> the host I can't seem to condense all the devices down to a single
>>>>>> mulipathed device. It seems multipathing is enabled for the fp
>>>>>> driver. I
>>>>>> noticed there is not a mpathadm command.
>>>>>>
>>>>>> Anyone have any ideas? I'll post some output from the commands i
>>>>>> think of
>>>>>> below. Let me know if you would like to see any other information.
>>>>>>
>>>>>> Thanks much,
>>>>>> Majeed
>>>>>>
>>>>>>
>>>>>> On solaris 10 the main thing we have to do is add the following to
>>>>>> /kernel/drv/scsi_vhci.conf (reconfigure reboot)
>>>>>> device-type-scsi-options-list =
>>>>>> "COMPELNTCompellent Vol", "symmetric-option";
>>>>>> symmetric-option = 0x1000000;
>>>>>>
>>>>>>
>>>>>> Here is the format output:
>>>>>> AVAILABLE DISK SELECTIONS:
>>>>>> 0. c13t5000CCA0213ED1E1d0<**HITACHI-HUC106030CSS600-A360 cyl
>>>>>> 36470
>>>>>> alt 2 hd 255 sec 63>
>>>>>>
>>>>>> /pci@0,0/pci8086,2940@1c/**pci1028,1f1e@0/iport@40/disk@**w5000cca0213ed1e1,0
>>>>>>
>>>>>> 1. c14t5000CCA0213EFC7Dd0<**HITACHI-HUC106030CSS600-A360 cyl
>>>>>> 36469
>>>>>> alt 2 hd 255 sec 63>
>>>>>>
>>>>>> /pci@0,0/pci8086,2940@1c/**pci1028,1f1e@0/iport@80/disk@**w5000cca0213efc7d,0
>>>>>>
>>>>>> 2. c16t5000D310004A064Bd1<**COMPELNT-CompellentVol-0505 cyl
>>>>>> 65268
>>>>>> alt 2 hd 255 sec 63>
>>>>>>
>>>>>> /pci@0,0/pci8086,3410@9/**pci1077,143@0/fp@0,0/disk@**
>>>>>> w5000d310004a064b,1
>>>>>> 3. c17t5000D310004A064Cd1<**COMPELNT-CompellentVol-0505 cyl
>>>>>> 65268
>>>>>> alt 2 hd 255 sec 63>
>>>>>>
>>>>>> /pci@0,0/pci8086,3410@9/**pci1077,143@0,1/fp@0,0/disk@**w5000d310004a064c,1
>>>>>>
>>>>>> Specify disk (enter its number): Specify disk (enter its number):
>>>>>>
>>>>>>
>>>>>>
>>>>>> % fcinfo hba-port
>>>>>> HBA Port WWN: 2100001b32940767
>>>>>> Port Mode: Initiator
>>>>>> Port ID: 91c00
>>>>>> OS Device Name: /dev/cfg/c16
>>>>>> Manufacturer: QLogic Corp.
>>>>>> Model: 375-3356-02
>>>>>> Firmware Version: 05.03.01
>>>>>> FCode/BIOS Version: BIOS: 2.02; fcode: 2.01; EFI: 2.00;
>>>>>> Serial Number: 0402H00-0941774287
>>>>>> Driver Name: qlc
>>>>>> Driver Version: 20100408-3.01
>>>>>> Type: N-port
>>>>>> State: online
>>>>>> Supported Speeds: 1Gb 2Gb 4Gb
>>>>>> Current Speed: 4Gb
>>>>>> Node WWN: 2000001b32940767
>>>>>> Max NPIV Ports: 127
>>>>>> NPIV port list:
>>>>>> HBA Port WWN: 2101001b32b40767
>>>>>> Port Mode: Initiator
>>>>>> Port ID: a1900
>>>>>> OS Device Name: /dev/cfg/c17
>>>>>> Manufacturer: QLogic Corp.
>>>>>> Model: 375-3356-02
>>>>>> Firmware Version: 05.03.01
>>>>>> FCode/BIOS Version: BIOS: 2.02; fcode: 2.01; EFI: 2.00;
>>>>>> Serial Number: 0402H00-0941774287
>>>>>> Driver Name: qlc
>>>>>> Driver Version: 20100408-3.01
>>>>>> Type: N-port
>>>>>> State: online
>>>>>> Supported Speeds: 1Gb 2Gb 4Gb
>>>>>> Current Speed: 4Gb
>>>>>> Node WWN: 2001001b32b40767
>>>>>> Max NPIV Ports: 127
>>>>>> NPIV port list:
>>>>>>
>>>>>>
>>>>>> % cfgadm -al -o show_SCSI_LUN
>>>>>> Ap_Id Type Receptacle Occupant
>>>>>> Condition
>>>>>> c16 fc-fabric connected configured
>>>>>> unknown
>>>>>> c16::5000d310004a0649 unknown connected unconfigured
>>>>>> unknown
>>>>>> c16::5000d310004a064b,1 disk connected configured
>>>>>> unknown
>>>>>> c17 fc-fabric connected configured
>>>>>> unknown
>>>>>> c17::5000d310004a064a unknown connected unconfigured
>>>>>> unknown
>>>>>> c17::5000d310004a064c,1 disk connected configured
>>>>>> unknown
>>>>>>
>>>>>>
>>>>>> ______________________________**_________________
>>>>>>
>>>>> --
> Majeed Abu-Qulbain
> Lead Unix Admin - CISS
> 309-438-3455
> Illinois State University
>
>
>
> ------------------------------**-------------
> illumos-discuss
> Archives:
> https://www.listbox.com/**member/archive/182180/=now<https://www.listbox.com/member/archive/182180/=now>
> RSS Feed: https://www.listbox.com/**member/archive/rss/182180/**
> 22222335-76d00b28<https://www.listbox.com/member/archive/rss/182180/22222335-76d00b28>
> Modify Your Subscription: https://www.listbox.com/**
> member/?&id_**secret=22222335-230ebb49<https://www.listbox.com/member/?&;>
> Powered by Listbox: http://www.listbox.com
>
--
Joe Hartley, Software Engineer
GreenBytes, Inc.
401.315.5580
-------------------------------------------
illumos-discuss
Archives: https://www.listbox.com/member/archive/182180/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be
Modify Your Subscription:
https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4
Powered by Listbox: http://www.listbox.com
