Re: Sun Fire 880 phantom disks

[phil]

Hi,

Sorry I'm wrong for setting device boot in openfirmrware, I have checked 
on old power pc, you can set more than one boot disk. On my system I can 
set 4 bootable devices no more but on your sun it probably different.

If OpenBSD as the begining of multi-path support you are ok to install 
with the both active loop on your sun. This is a really good features 
for OpenBSD but this is a really hard works to write a multi-pathing driver.

Question for Ken : with FCAL the multi-path driver use serial number of 
disk or world wide name of disk to identify a unique disk with multi-path ?
Because in Fabric mode we use world wide name and NportID ( wwn of port 
switch ).

I'am not a really good with C but I work for a long time with Fibre 
Channel storage, Ken are you interesting by some help or some test, I 
have some old hardware not used and I can install an openBSD 4.7 or 
current to do some test.

Phil

Kenneth R Westerback wrote:
On Mon, Mar 01, 2010 at 10:58:31PM +0100, Pete Vickers wrote:
  
Hi,

The 880 is stock from Sun. I've done no hardware plumbing on it.

According to http://docs.sun.com/app/docs/doc/806-7338-10/6jg7hm79b?a=view

"You can use multipathing software to switch I/O operations from one I/O controller 
to another to prepare for DR operations. With a combination of DR and multipathing 
software, you can remove, replace, or deactivate a PCI controller card with no 
interruption to system operation. Note that this requires redundant hardware; that is, 
the system must contain an alternate I/O controller that is connected to the same 
device(s) as the card being removed or replaced"

So the disk bus is connected to two controllers for redundancy, and Solaris 
obviously deals with this accordingly. I guess I should config> a controller 
away to stop OpenBSD seeing it ? In the longer term perhaps OpenBSD scsi layer 
could examine disk serial numbers, and avoid assigning device IDs to subsequent 
disks with the same serial number ?
    

OpenBSD already has the beginning of multi-path support, but it is
early days. man mpath(4) on -current. But at the moment if the box is
configured to allow both ports to see all the disks then you will
have to take manual action of some kind to suppress the 2nd set.

.... Ken

  
{2} ok devalias
cdrom                    /p...@8,700000/s...@1/d...@6,0:f
tape                     /p...@8,700000/s...@1/t...@4,0
scsix                    /p...@8,700000/s...@1
disk                     /p...@8,600000/SUNW,q...@2/f...@0,0/d...@0,0
disk0                    /p...@8,600000/SUNW,q...@2/f...@0,0/d...@0,0
disk1                    /p...@8,600000/SUNW,q...@2/f...@0,0/d...@1,0
disk2                    /p...@8,600000/SUNW,q...@2/f...@0,0/d...@2,0
disk3                    /p...@8,600000/SUNW,q...@2/f...@0,0/d...@3,0
disk4                    /p...@8,600000/SUNW,q...@2/f...@0,0/d...@4,0
disk5                    /p...@8,600000/SUNW,q...@2/f...@0,0/d...@5,0
disk6                    /p...@8,600000/SUNW,q...@2/f...@0,0/d...@8,0
disk7                    /p...@8,600000/SUNW,q...@2/f...@0,0/d...@9,0
disk8                    /p...@8,600000/SUNW,q...@2/f...@0,0/d...@a,0
disk9                    /p...@8,600000/SUNW,q...@2/f...@0,0/d...@b,0
disk10                   /p...@8,600000/SUNW,q...@2/f...@0,0/d...@c,0
disk11                   /p...@8,600000/SUNW,q...@2/f...@0,0/d...@d,0
scsi                     /p...@8,600000/SUNW,q...@2
net                      /p...@9,700000/netw...@1,1
gem                      /p...@8,600000/netw...@1
flash                    /p...@9,700000/e...@1/flashp...@0,0
idprom                   /p...@9,700000/e...@1/i...@1,500030/idp...@0,a0
nvram                    /p...@9,700000/e...@1/i...@1,500030/nv...@0,a0
i2c3                     /p...@9,700000/e...@1/i...@1,500030
i2c2                     /p...@9,700000/e...@1/i...@1,50002e
bbc1                     /p...@9,700000/e...@1/b...@1,500000
i2c1                     /p...@9,700000/e...@1/i...@1,30
i2c0                     /p...@9,700000/e...@1/i...@1,2e
bbc0                     /p...@9,700000/e...@1/b...@1,0
rsc-console              /p...@9,700000/e...@1/rsc-cons...@1,3083f8
rsc-control              /p...@9,700000/e...@1/rsc-cont...@1,3062f8
ttyb                     /p...@9,700000/e...@1/ser...@1,400000:b
ttya                     /p...@9,700000/e...@1/ser...@1,400000:a
pci9b                    /p...@9,700000
pci9a                    /p...@9,600000
pci8b                    /p...@8,700000
pci8a                    /p...@8,600000
ebus                     /p...@9,700000/e...@1
name                     aliases




/Pete




On 1. mars 2010, at 19.40, Kenneth R Westerback wrote:

    
On Mon, Mar 01, 2010 at 03:56:22PM +0100, Pete Vickers wrote:
      
Hei,


Upon booting either 4.6-RELEASE or 4.7-BETA on my SunFire 880 causes the
kernel it to 'see' twice the correct number of physical disk. Further if I
install the o/s  using bsd.rd on to sd0, then upon reboot the kernel can't
find the root disk. However if I install on sd12 then booting etc is fine.

the machine has physically 12 disks (36GB FC-AL), which appear to be
duplicated as sd0-sd11 and sd12-23.
        
Do you have the disk in a loop that includes both ports on the 2200? That
could explain why the disks are seen twice. And I believe only one WWN
is saved from the boot so if it sees the same disk on the other port
it may be lost.

.... Ken

      
r...@sf880 ~>sysctl hw.disknames
hw.disknames=cd0,sd0,sd1,sd2,sd3,sd4,sd5,sd6,sd7,sd8,sd9,sd10,sd11,sd12,sd13,
sd14,sd15,sd16,sd17,sd18,sd19,sd20,sd21,sd22,sd23



compare disk serial numbers :

r...@sf880 ~>bioctl  sd0
sd0: <SEAGATE, ST336605FSUN36G, 0438>, serial 3FP0N1K600007214DE8J
r...@sf880 ~>bioctl  sd1
sd1: <SEAGATE, ST336605FSUN36G, 0438>, serial 3FP0JHHX00007214DDNK

r...@sf880 ~>bioctl  sd12
sd12: <SEAGATE, ST336605FSUN36G, 0438>, serial 3FP0N1K600007214DE8J
r...@sf880 ~>bioctl  sd13
sd13: <SEAGATE, ST336605FSUN36G, 0438>, serial 3FP0JHHX00007214DDNK





full dmesg:


console is /p...@9,700000/e...@1/ser...@1,400000:a
Copyright (c) 1982, 1986, 1989, 1991, 1993
       The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2009 OpenBSD. All rights reserved.  http://www.OpenBSD.org

OpenBSD 4.6 (GENERIC.MP) #53: Thu Jul  9 21:50:16 MDT 2009
   dera...@sparc64.openbsd.org:/usr/src/sys/arch/sparc64/compile/GENERIC.MP
real mem = 4294967296 (4096MB)
avail mem = 4140875776 (3949MB)
mainbus0 at root: Sun Fire 880
cpu0 at mainbus0: SUNW,UltraSPARC-III (rev 5.4) @ 750 MHz
cpu0: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external
(512 b/l)
cpu1 at mainbus0: SUNW,UltraSPARC-III (rev 5.4) @ 750 MHz
cpu1: physical 32K instruction (32 b/l), 64K data (32 b/l), 8192K external
(512 b/l)
"memory-controller" at mainbus0 not configured
"memory-controller" at mainbus0 not configured
schizo0 at mainbus0: "Schizo", version 4, ign 200, bus B 0 to 1
schizo0: dvma map c0000000-ffffffff
pci0 at schizo0
siop0 at pci0 dev 1 function 0 "Symbios Logic 53c875" rev 0x04: ivec 0x21c,
using 4K of on-board RAM
scsibus0 at siop0: 16 targets, initiator 7
cd0 at scsibus0 targ 6 lun 0: <TOSHIBA, DVD-ROM SD-M1401, 1009> SCSI2 5/cdrom
removable
ppb0 at pci0 dev 5 function 0 "DEC 21154 PCI-PCI" rev 0x05
pci1 at ppb0 bus 1
isp0 at pci1 dev 4 function 0 "QLogic ISP2200" rev 0x05: ivec 0x218
isp0: Board Type 2200, Chip Revision 0x5, loaded F/W Revision 2.2.6
scsibus1 at isp0: 512 targets
sd0 at scsibus1 targ 0 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd0: 34732MB, 512 bytes/sec, 71132959 sec total
sd1 at scsibus1 targ 1 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd1: 34732MB, 512 bytes/sec, 71132959 sec total
sd2 at scsibus1 targ 2 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd2: 34732MB, 512 bytes/sec, 71132959 sec total
sd3 at scsibus1 targ 3 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd3: 34732MB, 512 bytes/sec, 71132959 sec total
sd4 at scsibus1 targ 4 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd4: 34732MB, 512 bytes/sec, 71132959 sec total
sd5 at scsibus1 targ 5 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd5: 34732MB, 512 bytes/sec, 71132959 sec total
ses0 at scsibus1 targ 6 lun 0: <SUNW, SUNWGS INT FCBPL, 9222> SCSI3
13/enclosure services fixed
sd6 at scsibus1 targ 7 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd6: 34732MB, 512 bytes/sec, 71132959 sec total
sd7 at scsibus1 targ 8 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd7: 34732MB, 512 bytes/sec, 71132959 sec total
sd8 at scsibus1 targ 9 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd8: 34732MB, 512 bytes/sec, 71132959 sec total
sd9 at scsibus1 targ 10 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd9: 34732MB, 512 bytes/sec, 71132959 sec total
sd10 at scsibus1 targ 11 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3
0/direct fixed
sd10: 34732MB, 512 bytes/sec, 71132959 sec total
sd11 at scsibus1 targ 12 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3
0/direct fixed
sd11: 34732MB, 512 bytes/sec, 71132959 sec total
isp1 at pci1 dev 5 function 0 "QLogic ISP2200" rev 0x05: ivec 0x219
isp1: Board Type 2200, Chip Revision 0x5, loaded F/W Revision 2.2.6
scsibus2 at isp1: 512 targets
schizo1 at mainbus0: "Schizo", version 4, ign 200, bus A 0 to 0
schizo1: dvma map c0000000-ffffffff
pci2 at schizo1
gem0 at pci2 dev 1 function 0 "Sun GEM" rev 0x01: ivec 0x200, address
00:03:ba:08:de:01
gentbi0 at gem0 phy 0: Generic ten-bit interface, rev. 0
isp2 at pci2 dev 2 function 0 "QLogic ISP2200" rev 0x05: ivec 0x204
isp2: Board Type 2200, Chip Revision 0x5, loaded F/W Revision 2.2.6
scsibus3 at isp2: 512 targets
sd12 at scsibus3 targ 0 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd12: 34732MB, 512 bytes/sec, 71132959 sec total
sd13 at scsibus3 targ 1 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd13: 34732MB, 512 bytes/sec, 71132959 sec total
sd14 at scsibus3 targ 2 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd14: 34732MB, 512 bytes/sec, 71132959 sec total
sd15 at scsibus3 targ 3 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd15: 34732MB, 512 bytes/sec, 71132959 sec total
sd16 at scsibus3 targ 4 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd16: 34732MB, 512 bytes/sec, 71132959 sec total
sd17 at scsibus3 targ 5 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd17: 34732MB, 512 bytes/sec, 71132959 sec total
ses1 at scsibus3 targ 6 lun 0: <SUNW, SUNWGS INT FCBPL, 9222> SCSI3
13/enclosure services fixed
sd18 at scsibus3 targ 7 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd18: 34732MB, 512 bytes/sec, 71132959 sec total
sd19 at scsibus3 targ 8 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd19: 34732MB, 512 bytes/sec, 71132959 sec total
sd20 at scsibus3 targ 9 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3 0/direct
fixed
sd20: 34732MB, 512 bytes/sec, 71132959 sec total
sd21 at scsibus3 targ 10 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3
0/direct fixed
sd21: 34732MB, 512 bytes/sec, 71132959 sec total
sd22 at scsibus3 targ 11 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3
0/direct fixed
sd22: 34732MB, 512 bytes/sec, 71132959 sec total
sd23 at scsibus3 targ 12 lun 0: <SEAGATE, ST336605FSUN36G, 0438> SCSI3
0/direct fixed
sd23: 34732MB, 512 bytes/sec, 71132959 sec total
schizo2 at mainbus0: "Schizo", version 4, ign 240, bus B 0 to 0
schizo2: dvma map c0000000-ffffffff
pci3 at schizo2
ebus0 at pci3 dev 1 function 0 "Sun RIO EBus" rev 0x01
"flashprom" at ebus0 addr 0-1fffff not configured
bbc0 at ebus0 addr 0-fffff: AID 0x00
power0 at ebus0 addr 30002e-30002f, 300600-300607
pcfiic0 at ebus0 addr 2e-2f, 2d-2d ivec 0x23
iic0 at pcfiic0
admtemp0 at iic0 addr 0x218: max1617
admtemp1 at iic0 addr 0x219: max1617
admtemp2 at iic0 addr 0x21a: max1617
admtemp3 at iic0 addr 0x229: max1617
admtemp4 at iic0 addr 0x22a: max1617
admtemp5 at iic0 addr 0x22b: max1617
admtemp6 at iic0 addr 0x24c: max1617
pcfiic1 at ebus0 addr 30-31 ivec 0x23
iic1 at pcfiic1
"ssc100" at iic1 addr 0xb not configured
"smbus-ara" at iic1 addr 0xc not configured
"ssc100" at iic1 addr 0xd not configured
"ssc100" at iic1 addr 0xe not configured
"ssc100" at iic1 addr 0xf not configured
admtemp7 at iic1 addr 0x18: max1617
admtemp8 at iic1 addr 0x1a: max1617
"pcf8574" at iic1 addr 0x23 not configured
"x9241" at iic1 addr 0x2c not configured
"ltc1427" at iic1 addr 0x2d not configured
"x9241" at iic1 addr 0x2e not configured
"ltc1427" at iic1 addr 0x2f not configured
"pcf8574" at iic1 addr 0x38 not configured
"pcf8574" at iic1 addr 0x39 not configured
"pcf8574" at iic1 addr 0x3a not configured
"ssc050" at iic1 addr 0x40 not configured
"ssc050" at iic1 addr 0x41 not configured
"ssc050" at iic1 addr 0x44 not configured
"ssc050" at iic1 addr 0x45 not configured
"ssc050" at iic1 addr 0x46 not configured
"ssc050" at iic1 addr 0x47 not configured
"pcf8591" at iic1 addr 0x48 not configured
"pcf8591" at iic1 addr 0x49 not configured
"pcf8591" at iic1 addr 0x4a not configured
"pcf8591" at iic1 addr 0x4b not configured
lmtemp0 at iic1 addr 0x4e: lm75
lmtemp1 at iic1 addr 0x4f: lm75
"ds1307" at iic1 addr 0x68 not configured
"hpc3130" at iic1 addr 0x71 not configured
"hpc3130" at iic1 addr 0x73 not configured
"hpc3130" at iic1 addr 0x74 not configured
"hpc3130" at iic1 addr 0x76 not configured
bbc1 at ebus0 addr 500000-5fffff: AID 0x04
pcfiic2 at ebus0 addr 50002e-50002f, 50002d-50002d ivec 0x28
iic2 at pcfiic2
admtemp9 at iic2 addr 0x229: max1617
admtemp10 at iic2 addr 0x22a: max1617
admtemp11 at iic2 addr 0x22b: max1617
pcfiic3 at ebus0 addr 500030-500031 ivec 0x28
iic3 at pcfiic3
rtc0 at ebus0 addr 300070-300071 ivec 0x24: ds1287
"gpio" at ebus0 addr 300600-300607 not configured
pmc0 at ebus0 addr 300700-300701
"rsc-control" at ebus0 addr 3062f8-3062ff ivec 0x2e not configured
com0 at ebus0 addr 3083f8-3083ff ivec 0x2d: ns16550a, 16 byte fifo
sab0 at ebus0 addr 400000-40007f ivec 0x22: rev 3.2
sabtty0 at sab0 port 0: console
sabtty1 at sab0 port 1
gem1 at pci3 dev 1 function 1 "Sun ERI Ether" rev 0x01: ivec 0x25d, address
00:03:ba:08:de:00
luphy0 at gem1 phy 1: LU6612 10/100 PHY, rev. 1
ohci0 at pci3 dev 1 function 3 "Sun USB" rev 0x01: ivec 0x25f, version 1.0,
legacy support
usb0 at ohci0: USB revision 1.0
uhub0 at usb0 "Sun OHCI root hub" rev 1.00/1.00 addr 1
schizo3 at mainbus0: "Schizo", version 4, ign 240, bus A 0 to 0
schizo3: dvma map c0000000-ffffffff
pci4 at schizo3
softraid0 at root
bootpath: /p...@8,600000/SUNW,q...@2,0/f...@0,0/d...@21000004cf27d31b,0
root on sd12a swap on sd12b dump on sd12b