Thank you very much Joe, thats a win!
Multipathing against compellent now works. Format's output is condense
and mpathadm list LU shows the paths.
% echo | format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c0t600A0B800033816200001391507FF856d0 <SUN-CSM200_R-0750 cyl
13052 alt 2 hd 255 sec 63>
/scsi_vhci/disk@g600a0b800033816200001391507ff856
1. c0t6000D310004A06000000000000000014d0
<COMPELNT-CompellentVol-0505 cyl 65268 alt 2 hd 255 sec 63>
/scsi_vhci/disk@g6000d310004a06000000000000000014
% mpathadm list lU
/dev/rdsk/c0t6000D310004A06000000000000000014d0s2
Total Path Count: 2
Operational Path Count: 2
/dev/rdsk/c0t600A0B800033816200001391507FF856d0s2
Total Path Count: 4
Operational Path Count: 4
Also, i did have device-type-scsi-options-list specified in the
scsi_vhci.conf following the documentation as you mentioned. I commented
that out and all is well! (it didn't seem to hurt having it there, but
as you mentioned, definately wrong and not needed)
Thanks again,
Majeed
On 10/18/2012 04:12 PM, Joe Hartley wrote:
I just had this with another SAN vendor yesterday. The issue I ran
into was that almost all the documentation I saw told me to modify
/kernel/drv/scsi_vhci.conf by adding a device-type-scsi-options-list
section, but that's wrong!
You need to modify that file, alright, but it needs a
scsi-vhci-failover-override section, like this:
scsi-vhci-failover-override =
"COMPELNTCompellent", "f_sym";
Note that the second half of the identifier is the product name, but
only up to the first space!
Good luck.
On Thu, Oct 18, 2012 at 4:49 PM, Majeed <[email protected]
<mailto:[email protected]>> wrote:
Hi All, As noted below, we have attempted to test the illumos
based omniOS in our datacenter on a dell R610 connected through
dual cisco fiber channel fabrics to a compellent SAN with dual
controllers. We got pretty far with an out of the box setup until
the last multipathing bit. I was hoping there might be some
additional eyes on this list that may be able to provide some insight.
Thanks in advance for taking the time to read through my following
notes on this topic,
Majeed Qulbain
On 10/18/2012 03:06 PM, Majeed wrote:
Thanks Theo, that makes sense. I will definitely do that.
Majeed
On 10/18/2012 02:58 PM, Theo Schlossnagle wrote:
This is a more general Illumos problem, you will open your
troubleshooting to a larger audience by interfacing with the
illumos-discuss list. I encourage you to post there! I'm
subscribed
there as well, but I haven't done multipathing against
compellent and
it appears no one else on the omnios-discuss list has either.
Anything learned in the illumos-discuss group can be
applied directly
to OmniOS.
On Thu, Oct 18, 2012 at 3:51 PM, Majeed<[email protected]
<mailto:[email protected]>> wrote:
Hi Again, just wanted to update the list with my
latest progress and
thoughts on this topic. I think the issue lies in
libmpscsi_vhci.so's lack
of support for multipathing with compellent.
When I do "mpathadm show mpath-support
libmpscsi_vhci.so" on omniOS I see
many venders and models listed but not compellent. We
are going to spin up a
solaris 11 machine to see what it looks like. I wonder
if this was forked
just before SUN/oracle added support for compellent (i
could be way off base
here).
When I do the same command on solaris 10 u10, at the
very bottom of the list
i see.
Vendor: COMPELNT
Product: Compellent Vol
Revision:
Supported Load Balance Types:
round-robin
I have verified that multipathing is working properly
on other arrays. I
zoned up omniOS with all controllers on a 6140
(CSM200) that sit in multiple
fabrics, mapped a lun, and it looks wonderful. mpatham
reports as expected,
and all paths are condensed to a single block device
in format's output.
% mpathadm list LU
/dev/rdsk/c0t600A0B800033816200001391507FF856d0s2
Total Path Count: 4
Operational Path Count: 4
It seems there are some options in
/kernel/drv/scsi_vhci.conf to force
scsi_vhci to accept 3rd party devices (hopefully
compellent) for failover,
but this is kinda getting over my head. It seems like
if I had a proper
ddi-forceload ''misc/scsi_vhci/scsi_vhci_f_<x>" file
for compellent and I
knew the proper VenderID and ProductID for the
compellent devices, i might
be able to force it. Im not sure if can use the
information from the format
inquiry to populate these values.
format> inquiry
Vendor: COMPELNT
Product: Compellent Vol
Revision: 0505
Any thoughts on this would be excellent.
Thanks,
Majeed Qulbain
On 10/18/2012 09:30 AM, Majeed wrote:
I just noticed that we do have access to the
mpathadm command by
installing the
"pkg:/storage/[email protected]" package. After
installing I don't see any paths with "mpathadm
list LU" which is what im
used to on solaris 10 (i know I should be
comparing to solaris 11, just
havn't played with it much). I wonder if there are
any other bits that I
need to install or other configuration tweaks to
get this going.
Thanks in advance for any pointers,
Majeed
Some other related commands that might assist
troubleshooting:
% uname -a
SunOS jhx011 5.11 omnios-eae537b i86pc i386 i86pc
% mpathadm list initiator-port
Initiator Port: w5d4ae520937bbb00
Initiator Port:
iqn.1986-03.com.sun:01:b06a64f606ff.507f35b3,4000002a00ff
Initiator Port: w5d4ae520937bbb00
Initiator Port: 2101001b32b40767
Initiator Port: 2100001b32940767
% luxadm -e port
/devices/pci@0,0/pci8086,3410@9/pci1077,143@0/fp@0,0:devctl
CONNECTED
/devices/pci@0,0/pci8086,3410@9/pci1077,143@0,1/fp@0,0:devctl
CONNECTED
% luxadm -v probe
Found Fibre Channel device(s):
Node WWN:5000d310004a0602 Device Type:Disk device
Logical Path:/dev/rdsk/c16t5000D310004A064Bd1s2
Node WWN:5000d310004a0602 Device Type:Disk device
Logical Path:/dev/rdsk/c17t5000D310004A064Cd1s2
% luxadm disp /dev/rdsk/c16t5000D310004A064Bd1s2
DEVICE PROPERTIES for disk:
/dev/rdsk/c16t5000D310004A064Bd1s2
Vendor: COMPELNT
Product ID: Compellent Vol
Revision: 0505
Serial Num: 00004a06-00000014
Unformatted capacity: 512000.000 MBytes
Read Cache: Enabled
Minimum prefetch: 0x100
Maximum prefetch: 0xf0
Device Type: Disk device
Path(s):
/dev/rdsk/c16t5000D310004A064Bd1s2
/devices/pci@0,0/pci8086,3410@9/pci1077,143@0/fp@0,0/disk@w5000d310004a064b,1:c,raw
Controller /dev/cfg/c16
Device Address 5000d310004a064b,1
Host controller port WWN 2100001b32940767
Controller /dev/cfg/c17
Device Address 5000d310004a064c,1
Host controller port WWN 2101001b32b40767
On 10/17/2012 06:25 PM, Majeed wrote:
Hi All, We are doing some testing with omniOS
in our datacenter and would
love connect it up to our compellent san via
fc connectivity. I slapped
omniOS onto a dell r610 that has a qlogic dual
port hba (each port is pathed
to separate fabrics), performed zoning in each
fabric, and assigned some
luns. Now im not sure if compellent supports
Solaris 11 (i can contact them
and see about that), but it definately support
solaris 10 which is what we
use on other production systems.
I'v actually made it pretty far through the
process which is really
exciting, except for the last multipathing
bit. After mapping a volume to
the host I can't seem to condense all the
devices down to a single
mulipathed device. It seems multipathing is
enabled for the fp driver. I
noticed there is not a mpathadm command.
Anyone have any ideas? I'll post some output
from the commands i think of
below. Let me know if you would like to see
any other information.
Thanks much,
Majeed
On solaris 10 the main thing we have to do is
add the following to
/kernel/drv/scsi_vhci.conf (reconfigure reboot)
device-type-scsi-options-list =
"COMPELNTCompellent Vol", "symmetric-option";
symmetric-option = 0x1000000;
Here is the format output:
AVAILABLE DISK SELECTIONS:
0.
c13t5000CCA0213ED1E1d0<HITACHI-HUC106030CSS600-A360
cyl 36470
alt 2 hd 255 sec 63>
/pci@0,0/pci8086,2940@1c/pci1028,1f1e@0/iport@40/disk@w5000cca0213ed1e1,0
1.
c14t5000CCA0213EFC7Dd0<HITACHI-HUC106030CSS600-A360
cyl 36469
alt 2 hd 255 sec 63>
/pci@0,0/pci8086,2940@1c/pci1028,1f1e@0/iport@80/disk@w5000cca0213efc7d,0
2.
c16t5000D310004A064Bd1<COMPELNT-CompellentVol-0505
cyl 65268
alt 2 hd 255 sec 63>
/pci@0,0/pci8086,3410@9/pci1077,143@0/fp@0,0/disk@w5000d310004a064b,1
3.
c17t5000D310004A064Cd1<COMPELNT-CompellentVol-0505
cyl 65268
alt 2 hd 255 sec 63>
/pci@0,0/pci8086,3410@9/pci1077,143@0,1/fp@0,0/disk@w5000d310004a064c,1
Specify disk (enter its number): Specify disk
(enter its number):
% fcinfo hba-port
HBA Port WWN: 2100001b32940767
Port Mode: Initiator
Port ID: 91c00
OS Device Name: /dev/cfg/c16
Manufacturer: QLogic Corp.
Model: 375-3356-02
Firmware Version: 05.03.01
FCode/BIOS Version: BIOS: 2.02; fcode:
2.01; EFI: 2.00;
Serial Number: 0402H00-0941774287
Driver Name: qlc
Driver Version: 20100408-3.01
Type: N-port
State: online
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 4Gb
Node WWN: 2000001b32940767
Max NPIV Ports: 127
NPIV port list:
HBA Port WWN: 2101001b32b40767
Port Mode: Initiator
Port ID: a1900
OS Device Name: /dev/cfg/c17
Manufacturer: QLogic Corp.
Model: 375-3356-02
Firmware Version: 05.03.01
FCode/BIOS Version: BIOS: 2.02; fcode:
2.01; EFI: 2.00;
Serial Number: 0402H00-0941774287
Driver Name: qlc
Driver Version: 20100408-3.01
Type: N-port
State: online
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: 4Gb
Node WWN: 2001001b32b40767
Max NPIV Ports: 127
NPIV port list:
% cfgadm -al -o show_SCSI_LUN
Ap_Id Type
Receptacle Occupant
Condition
c16 fc-fabric
connected configured
unknown
c16::5000d310004a0649 unknown
connected unconfigured
unknown
c16::5000d310004a064b,1 disk
connected configured
unknown
c17 fc-fabric
connected configured
unknown
c17::5000d310004a064a unknown
connected unconfigured
unknown
c17::5000d310004a064c,1 disk
connected configured
unknown
_______________________________________________
--
Majeed Abu-Qulbain
Lead Unix Admin - CISS
309-438-3455 <tel:309-438-3455>
Illinois State University
-------------------------------------------
illumos-discuss
Archives: https://www.listbox.com/member/archive/182180/=now
RSS Feed:
https://www.listbox.com/member/archive/rss/182180/22222335-76d00b28
Modify Your Subscription:
https://www.listbox.com/member/?&;
<https://www.listbox.com/member/?&;>
Powered by Listbox: http://www.listbox.com
--
Joe Hartley, Software Engineer
GreenBytes, Inc.
401.315.5580
*illumos-discuss* | Archives
<https://www.listbox.com/member/archive/182180/=now>
<https://www.listbox.com/member/archive/rss/182180/23508103-b88c366c>
| Modify
<https://www.listbox.com/member/?&;>
Your Subscription [Powered by Listbox] <http://www.listbox.com>
--
Majeed Abu-Qulbain
Lead Unix Admin - CISS
309-438-3455
Illinois State University
-------------------------------------------
illumos-discuss
Archives: https://www.listbox.com/member/archive/182180/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be
Modify Your Subscription:
https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4
Powered by Listbox: http://www.listbox.com
