Hi David,
I see some inconsistencies between the mirrored pool tank info below
and the device info that you included.
1. The zpool status for tank shows some remnants of log devices (?),
here:
tank FAULTED corrupted data
logs
Generally, the log devices are listed after the pool devices.
Did this pool have log devices at one time? Are they missing?
# zpool status datap
pool: datap
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
datap ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c1t2d0 ONLINE 0 0 0
mirror-1 ONLINE 0 0 0
c1t3d0 ONLINE 0 0 0
c1t4d0 ONLINE 0 0 0
logs
mirror-2 ONLINE 0 0 0
c1t5d0 ONLINE 0 0 0
c1t8d0 ONLINE 0 0 0
I would like to see this output:
# zpool history tank
2. Can you include the zdb -l output for c9t57d0 because
the zdb -l device output below is from a RAIDZ config, not
a mirrored config, although the pool GUIDs match so I'm
confused.
I don't think this has anything to do with moving from s10u9 to S11
express.
My sense is that if you have remnants of the same pool name on some of
your devices but as different pools, then you will see device problems
like these.
Thanks,
Cindy
On 06/22/11 20:28, David W. Smith wrote:
On Wed, Jun 22, 2011 at 06:32:49PM -0700, Daniel Carosone wrote:
On Wed, Jun 22, 2011 at 12:49:27PM -0700, David W. Smith wrote:
# /home/dws# zpool import
pool: tank
id: 13155614069147461689
state: FAULTED
status: The pool metadata is corrupted.
action: The pool cannot be imported due to damaged devices or data.
see: http://www.sun.com/msg/ZFS-8000-72
config:
tank FAULTED corrupted data
logs
mirror-6 ONLINE
c9t57d0 ONLINE
c9t58d0 ONLINE
mirror-7 ONLINE
c9t59d0 ONLINE
c9t60d0 ONLINE
Is there something else I can do to see what is wrong.
Can you tell us more about the setup, in particular the drivers and
hardware on the path? There may be labelling, block size, offset or
even bad drivers or other issues getting in the way, preventing ZFS
from doing what should otherwise be expected to work. Was there
something else in the storage stack on the old OS, like a different
volume manager or some multipathing?
Can you show us the zfs labels with zdb -l /dev/foo ?
Does import -F get any further?
Original attempt when specifying the name resulted in:
# /home/dws# zpool import tank
cannot import 'tank': I/O error
Some kind of underlying driver problem odour here.
--
Dan.
The system is an x4440 with two dual port Qlogic 8 Gbit FC cards connected to a
DDN 9900 storage unit. There are 60 luns configured from the storage unit we using
raidz1 across these luns in a 9+1 configuration. Under Solaris 10U9
multipathing
is enabled.
For example here is one of the devices:
# luxadm display /dev/rdsk/c8t60001FF010DC50AA2E000800001D1BF1d0s2
DEVICE PROPERTIES for disk: /dev/rdsk/c8t60001FF010DC50AA2E000800001D1BF1d0s2
Vendor: DDN
Product ID: S2A 9900
Revision: 6.11
Serial Num: 10DC50AA002E
Unformatted capacity: 15261576.000 MBytes
Write Cache: Enabled
Read Cache: Enabled
Minimum prefetch: 0x0
Maximum prefetch: 0x0
Device Type: Disk device
Path(s):
/dev/rdsk/c8t60001FF010DC50AA2E000800001D1BF1d0s2
/devices/scsi_vhci/disk@g60001ff010dc50aa2e000800001d1bf1:c,raw
Controller /dev/cfg/c5
Device Address 24000001ff051232,2e
Host controller port WWN 2101001b32bfe1d3
Class secondary
State ONLINE
Controller /dev/cfg/c7
Device Address 28000001ff0510dc,2e
Host controller port WWN 2101001b32bd4f8f
Class primary
State ONLINE
Here is the output of the zdb command:
# zdb -l /dev/dsk/c8t60001FF010DC50AA2E000800001D1BF1d0s0
--------------------------------------------
LABEL 0
--------------------------------------------
version=22
name='tank'
state=0
txg=402415
pool_guid=13155614069147461689
hostid=799263814
hostname='Chaiten'
top_guid=7879214599529115091
guid=9439709931602673823
vdev_children=8
vdev_tree
type='raidz'
id=5
guid=7879214599529115091
nparity=1
metaslab_array=35
metaslab_shift=40
ashift=12
asize=160028491776000
is_log=0
create_txg=22
children[0]
type='disk'
id=0
guid=15738823520260019536
path='/dev/dsk/c8t60001FF01232528037000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232528037000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232528037000800001d1bf1:a'
whole_disk=1
DTL=166
create_txg=22
children[1]
type='disk'
id=1
guid=7241121769141495862
path='/dev/dsk/c8t60001FF010DC50C536000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50c536000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50c536000800001d1bf1:a'
whole_disk=1
DTL=165
create_txg=22
children[2]
type='disk'
id=2
guid=2777230007222012140
path='/dev/dsk/c8t60001FF01232527935000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232527935000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232527935000800001d1bf1:a'
whole_disk=1
DTL=164
create_txg=22
children[3]
type='disk'
id=3
guid=5525323314985659974
path='/dev/dsk/c8t60001FF010DC50BE34000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50be34000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50be34000800001d1bf1:a'
whole_disk=1
DTL=163
create_txg=22
children[4]
type='disk'
id=4
guid=9152185867089638590
path='/dev/dsk/c8t60001FF01232527233000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232527233000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232527233000800001d1bf1:a'
whole_disk=1
DTL=162
create_txg=22
children[5]
type='disk'
id=5
guid=15506884896416107740
path='/dev/dsk/c8t60001FF010DC50B832000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50b832000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50b832000800001d1bf1:a'
whole_disk=1
DTL=161
create_txg=22
children[6]
type='disk'
id=6
guid=13655161443342419149
path='/dev/dsk/c8t60001FF01232526B31000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232526b31000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232526b31000800001d1bf1:a'
whole_disk=1
DTL=160
create_txg=22
children[7]
type='disk'
id=7
guid=8658338305352581764
path='/dev/dsk/c8t60001FF010DC50B030000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50b030000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50b030000800001d1bf1:a'
whole_disk=1
DTL=159
create_txg=22
children[8]
type='disk'
id=8
guid=16757152450233754713
path='/dev/dsk/c8t60001FF0123252632F000800001D1BF1d0s0'
devid='id1,sd@n60001ff0123252632f000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff0123252632f000800001d1bf1:a'
whole_disk=1
DTL=158
create_txg=22
children[9]
type='disk'
id=9
guid=9439709931602673823
path='/dev/dsk/c8t60001FF010DC50AA2E000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50aa2e000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50aa2e000800001d1bf1:a'
whole_disk=1
DTL=157
create_txg=22
rewind_txg_ts=1308690257
bad config type 7 for seconds_of_rewind
verify_data_errors=0
--------------------------------------------
LABEL 1
--------------------------------------------
version=22
name='tank'
state=0
txg=402415
pool_guid=13155614069147461689
hostid=799263814
hostname='Chaiten'
top_guid=7879214599529115091
guid=9439709931602673823
vdev_children=8
vdev_tree
type='raidz'
id=5
guid=7879214599529115091
nparity=1
metaslab_array=35
metaslab_shift=40
ashift=12
asize=160028491776000
is_log=0
create_txg=22
children[0]
type='disk'
id=0
guid=15738823520260019536
path='/dev/dsk/c8t60001FF01232528037000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232528037000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232528037000800001d1bf1:a'
whole_disk=1
DTL=166
create_txg=22
children[1]
type='disk'
id=1
guid=7241121769141495862
path='/dev/dsk/c8t60001FF010DC50C536000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50c536000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50c536000800001d1bf1:a'
whole_disk=1
DTL=165
create_txg=22
children[2]
type='disk'
id=2
guid=2777230007222012140
path='/dev/dsk/c8t60001FF01232527935000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232527935000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232527935000800001d1bf1:a'
whole_disk=1
DTL=164
create_txg=22
children[3]
type='disk'
id=3
guid=5525323314985659974
path='/dev/dsk/c8t60001FF010DC50BE34000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50be34000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50be34000800001d1bf1:a'
whole_disk=1
DTL=163
create_txg=22
children[4]
type='disk'
id=4
guid=9152185867089638590
path='/dev/dsk/c8t60001FF01232527233000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232527233000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232527233000800001d1bf1:a'
whole_disk=1
DTL=162
create_txg=22
children[5]
type='disk'
id=5
guid=15506884896416107740
path='/dev/dsk/c8t60001FF010DC50B832000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50b832000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50b832000800001d1bf1:a'
whole_disk=1
DTL=161
create_txg=22
children[6]
type='disk'
id=6
guid=13655161443342419149
path='/dev/dsk/c8t60001FF01232526B31000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232526b31000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232526b31000800001d1bf1:a'
whole_disk=1
DTL=160
create_txg=22
children[7]
type='disk'
id=7
guid=8658338305352581764
path='/dev/dsk/c8t60001FF010DC50B030000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50b030000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50b030000800001d1bf1:a'
whole_disk=1
DTL=159
create_txg=22
children[8]
type='disk'
id=8
guid=16757152450233754713
path='/dev/dsk/c8t60001FF0123252632F000800001D1BF1d0s0'
devid='id1,sd@n60001ff0123252632f000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff0123252632f000800001d1bf1:a'
whole_disk=1
DTL=158
create_txg=22
children[9]
type='disk'
id=9
guid=9439709931602673823
path='/dev/dsk/c8t60001FF010DC50AA2E000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50aa2e000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50aa2e000800001d1bf1:a'
whole_disk=1
DTL=157
create_txg=22
rewind_txg_ts=1308690257
bad config type 7 for seconds_of_rewind
verify_data_errors=0
--------------------------------------------
LABEL 2
--------------------------------------------
version=22
name='tank'
state=0
txg=402415
pool_guid=13155614069147461689
hostid=799263814
hostname='Chaiten'
top_guid=7879214599529115091
guid=9439709931602673823
vdev_children=8
vdev_tree
type='raidz'
id=5
guid=7879214599529115091
nparity=1
metaslab_array=35
metaslab_shift=40
ashift=12
asize=160028491776000
is_log=0
create_txg=22
children[0]
type='disk'
id=0
guid=15738823520260019536
path='/dev/dsk/c8t60001FF01232528037000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232528037000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232528037000800001d1bf1:a'
whole_disk=1
DTL=166
create_txg=22
children[1]
type='disk'
id=1
guid=7241121769141495862
path='/dev/dsk/c8t60001FF010DC50C536000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50c536000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50c536000800001d1bf1:a'
whole_disk=1
DTL=165
create_txg=22
children[2]
type='disk'
id=2
guid=2777230007222012140
path='/dev/dsk/c8t60001FF01232527935000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232527935000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232527935000800001d1bf1:a'
whole_disk=1
DTL=164
create_txg=22
children[3]
type='disk'
id=3
guid=5525323314985659974
path='/dev/dsk/c8t60001FF010DC50BE34000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50be34000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50be34000800001d1bf1:a'
whole_disk=1
DTL=163
create_txg=22
children[4]
type='disk'
id=4
guid=9152185867089638590
path='/dev/dsk/c8t60001FF01232527233000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232527233000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232527233000800001d1bf1:a'
whole_disk=1
DTL=162
create_txg=22
children[5]
type='disk'
id=5
guid=15506884896416107740
path='/dev/dsk/c8t60001FF010DC50B832000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50b832000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50b832000800001d1bf1:a'
whole_disk=1
DTL=161
create_txg=22
children[6]
type='disk'
id=6
guid=13655161443342419149
path='/dev/dsk/c8t60001FF01232526B31000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232526b31000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232526b31000800001d1bf1:a'
whole_disk=1
DTL=160
create_txg=22
children[7]
type='disk'
id=7
guid=8658338305352581764
path='/dev/dsk/c8t60001FF010DC50B030000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50b030000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50b030000800001d1bf1:a'
whole_disk=1
DTL=159
create_txg=22
children[8]
type='disk'
id=8
guid=16757152450233754713
path='/dev/dsk/c8t60001FF0123252632F000800001D1BF1d0s0'
devid='id1,sd@n60001ff0123252632f000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff0123252632f000800001d1bf1:a'
whole_disk=1
DTL=158
create_txg=22
children[9]
type='disk'
id=9
guid=9439709931602673823
path='/dev/dsk/c8t60001FF010DC50AA2E000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50aa2e000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50aa2e000800001d1bf1:a'
whole_disk=1
DTL=157
create_txg=22
rewind_txg_ts=1308690257
bad config type 7 for seconds_of_rewind
verify_data_errors=0
--------------------------------------------
LABEL 3
--------------------------------------------
version=22
name='tank'
state=0
txg=402415
pool_guid=13155614069147461689
hostid=799263814
hostname='Chaiten'
top_guid=7879214599529115091
guid=9439709931602673823
vdev_children=8
vdev_tree
type='raidz'
id=5
guid=7879214599529115091
nparity=1
metaslab_array=35
metaslab_shift=40
ashift=12
asize=160028491776000
is_log=0
create_txg=22
children[0]
type='disk'
id=0
guid=15738823520260019536
path='/dev/dsk/c8t60001FF01232528037000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232528037000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232528037000800001d1bf1:a'
whole_disk=1
DTL=166
create_txg=22
children[1]
type='disk'
id=1
guid=7241121769141495862
path='/dev/dsk/c8t60001FF010DC50C536000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50c536000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50c536000800001d1bf1:a'
whole_disk=1
DTL=165
create_txg=22
children[2]
type='disk'
id=2
guid=2777230007222012140
path='/dev/dsk/c8t60001FF01232527935000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232527935000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232527935000800001d1bf1:a'
whole_disk=1
DTL=164
create_txg=22
children[3]
type='disk'
id=3
guid=5525323314985659974
path='/dev/dsk/c8t60001FF010DC50BE34000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50be34000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50be34000800001d1bf1:a'
whole_disk=1
DTL=163
create_txg=22
children[4]
type='disk'
id=4
guid=9152185867089638590
path='/dev/dsk/c8t60001FF01232527233000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232527233000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232527233000800001d1bf1:a'
whole_disk=1
DTL=162
create_txg=22
children[5]
type='disk'
id=5
guid=15506884896416107740
path='/dev/dsk/c8t60001FF010DC50B832000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50b832000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50b832000800001d1bf1:a'
whole_disk=1
DTL=161
create_txg=22
children[6]
type='disk'
id=6
guid=13655161443342419149
path='/dev/dsk/c8t60001FF01232526B31000800001D1BF1d0s0'
devid='id1,sd@n60001ff01232526b31000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff01232526b31000800001d1bf1:a'
whole_disk=1
DTL=160
create_txg=22
children[7]
type='disk'
id=7
guid=8658338305352581764
path='/dev/dsk/c8t60001FF010DC50B030000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50b030000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50b030000800001d1bf1:a'
whole_disk=1
DTL=159
create_txg=22
children[8]
type='disk'
id=8
guid=16757152450233754713
path='/dev/dsk/c8t60001FF0123252632F000800001D1BF1d0s0'
devid='id1,sd@n60001ff0123252632f000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff0123252632f000800001d1bf1:a'
whole_disk=1
DTL=158
create_txg=22
children[9]
type='disk'
id=9
guid=9439709931602673823
path='/dev/dsk/c8t60001FF010DC50AA2E000800001D1BF1d0s0'
devid='id1,sd@n60001ff010dc50aa2e000800001d1bf1/a'
phys_path='/scsi_vhci/disk@g60001ff010dc50aa2e000800001d1bf1:a'
whole_disk=1
DTL=157
create_txg=22
rewind_txg_ts=1308690257
bad config type 7 for seconds_of_rewind
verify_data_errors=0
When I tried out Solaris 11, I just exported the pool prior to the install of
Solaris 11. I was lucky in that I had mirrored the boot drive, so after I had
installed Solaris 11 I still had the other disk in the mirror with Solaris 10
still
installed.
I didn't install any additional software in either environments with regards to
volume management, etc.
From the format command, I did remember seeing 60 luns coming from the DDN and
as I recall I disk see multiple paths as well under Solaris 11. I think you are
correct however in that for some reason Solaris 11 could not read the devices.
Let me know if you need any further output.
David
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
