I have a Solaris Express snv_130 box that imports a zpool from two iSCSI targets, and after some power problems I cannot import the pool.
When I found the machine, the pool was FAULTED with half of most
mirrors shoring CORRUPTED DATA and half showing UNAVAIL. One of
the two iSCSI enclosures was on, while the other was off. When I
brought the other iSCSI enclosure up, bringing all the devices in each
of the seven mirror vdev's online, the box paniced.
It went into a panic loop every time it tried to import the problem
pool at boot. I disabled all the iSCSI targets that make up the
problem pool and brought the box up, then saved a copy of
/etc/zfs/zpool.cache and exported the UNAVAIL pool. Then I turned the
host off, brought back all the iSCSI targets, and booted without a
crash, hoping I could 'zpool import' the problem pool.
(Another mirrored pool on the same pair of iSCSI enclosures came back
fine and scrubbed with no errors. <shrug>)
Here is what I get typing some basic commands:
-----8<-----
terabithia:/# zpool import
pool: andaman
id: 7400719929021713582
state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:
andaman ONLINE
mirror-0 ONLINE
c3t43d0 ONLINE
c3t48d0 ONLINE
mirror-1 ONLINE
c3t45d0 ONLINE
c3t47d0 ONLINE
mirror-2 ONLINE
c3t52d0 ONLINE
c3t59d0 ONLINE
mirror-3 ONLINE
c3t46d0 ONLINE
c3t49d0 ONLINE
mirror-4 ONLINE
c3t50d0 ONLINE
c3t44d0 ONLINE
mirror-5 ONLINE
c3t57d0 ONLINE
c3t53d0 ONLINE
mirror-6 ONLINE
c3t54d0 ONLINE
c3t51d0 ONLINE
terabithia:/# zpool import andaman
cannot import 'andaman': I/O error
Destroy and re-create the pool from
a backup source.
terabithia:/# zpool status
pool: aboveground
state: ONLINE
status: The pool is formatted using an older on-disk format. The pool can
still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'. Once this is done, the
pool will no longer be accessible on older software versions.
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
aboveground ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c3t10d0 ONLINE 0 0 0
c3t16d0 ONLINE 0 0 0
errors: No known data errors
pool: rpool
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c1t0d0s0 ONLINE 0 0 0
c1t1d0s0 ONLINE 0 0 0
errors: No known data errors
terabithia:/# zpool import -F andaman
cannot import 'andaman': I/O error
Destroy and re-create the pool from
a backup source.
terabithia:/# zdb -ve andaman
Configuration for import:
version: 22
pool_guid: 7400719929021713582
name: 'andaman'
state: 0
hostid: 2200768359
hostname: 'terabithia.th3h.inner.chaos'
vdev_children: 7
vdev_tree:
type: 'root'
id: 0
guid: 7400719929021713582
children[0]:
type: 'mirror'
id: 0
guid: 337393226491877361
whole_disk: 0
metaslab_array: 14
metaslab_shift: 33
ashift: 9
asize: 1000191557632
is_log: 0
children[0]:
type: 'disk'
id: 0
guid: 1781150413433362160
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,0:a'
whole_disk: 1
DTL: 91
path: '/dev/dsk/c3t43d0s0'
devid:
'id1,sd@t49455400000000000000000002000000591000000f000000/a'
children[1]:
type: 'disk'
id: 1
guid: 7841235598547702997
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,1:a'
whole_disk: 1
DTL: 215
path: '/dev/dsk/c3t48d0s0'
devid:
'id1,sd@t49455400000000000000000002000000880e00000f000000/a'
children[1]:
type: 'mirror'
id: 1
guid: 1953060080997571723
whole_disk: 0
metaslab_array: 210
metaslab_shift: 33
ashift: 9
asize: 1000191557632
is_log: 0
children[0]:
type: 'disk'
id: 0
guid: 14567891505260370684
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,3:a'
whole_disk: 1
DTL: 1395
path: '/dev/dsk/c3t45d0s0'
devid:
'id1,sd@t49455400000000000000000002000000731000000f000000/a'
children[1]:
type: 'disk'
id: 1
guid: 12913430974955856922
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,0:a'
whole_disk: 1
DTL: 217
path: '/dev/dsk/c3t47d0s0'
devid:
'id1,sd@t494554000000000000000000020000005c0f00000f000000/a'
children[2]:
type: 'mirror'
id: 2
guid: 17555572171866652488
metaslab_array: 330
metaslab_shift: 31
ashift: 9
asize: 1000191557632
is_log: 0
children[0]:
type: 'disk'
id: 0
guid: 10644297178963807622
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,7:a'
whole_disk: 1
DTL: 1167
path: '/dev/dsk/c3t52d0s0'
devid:
'id1,sd@t49455400000000000000000002000000090f00000f000000/a'
children[1]:
type: 'disk'
id: 1
guid: 2378339860349039724
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,10:a'
whole_disk: 1
DTL: 1168
path: '/dev/dsk/c3t59d0s0'
devid:
'id1,sd@t49455400000000000000000002000000061100000f000000/a'
children[3]:
type: 'mirror'
id: 3
guid: 5261242283366420554
metaslab_array: 332
metaslab_shift: 31
ashift: 9
asize: 250046054400
is_log: 0
children[0]:
type: 'disk'
id: 0
guid: 17534037132307391717
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,7:a'
whole_disk: 1
DTL: 335
path: '/dev/dsk/c3t46d0s0'
devid:
'id1,sd@t494554000000000000000000020000008a1000000f000000/a'
children[1]:
type: 'disk'
id: 1
guid: 2758124488221280115
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,2:a'
whole_disk: 1
DTL: 336
path: '/dev/dsk/c3t49d0s0'
devid:
'id1,sd@t49455400000000000000000002000000af0e00000f000000/a'
children[4]:
type: 'mirror'
id: 4
guid: 2126473073638606315
whole_disk: 0
metaslab_array: 510
metaslab_shift: 31
ashift: 9
asize: 750142881792
is_log: 0
children[0]:
type: 'disk'
id: 0
guid: 7272191256105073467
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,3:a'
whole_disk: 1
DTL: 660
path: '/dev/dsk/c3t50d0s0'
devid:
'id1,sd@t49455400000000000000000002000000c20e00000f000000/a'
children[1]:
type: 'disk'
id: 1
guid: 5437265705837840703
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,1:a'
whole_disk: 1
DTL: 513
path: '/dev/dsk/c3t44d0s0'
devid:
'id1,sd@t49455400000000000000000002000000c91000000f000000/a'
children[5]:
type: 'mirror'
id: 5
guid: 6278268859805291262
whole_disk: 0
metaslab_array: 509
metaslab_shift: 30
ashift: 9
asize: 2000385474560
is_log: 0
children[0]:
type: 'disk'
id: 0
guid: 2444845073767152521
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,5:a'
whole_disk: 1
DTL: 4188
path: '/dev/dsk/c3t57d0s0'
devid:
'id1,sd@t49455400000000000000000002000000a11000000f000000/a'
children[1]:
type: 'disk'
id: 1
guid: 7384627338137907176
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,8:a'
whole_disk: 1
DTL: 4192
path: '/dev/dsk/c3t53d0s0'
devid:
'id1,sd@t49455400000000000000000002000000990f00000f000000/a'
children[6]:
type: 'mirror'
id: 6
guid: 16251593073754009033
metaslab_array: 961
metaslab_shift: 31
ashift: 9
asize: 320059473920
is_log: 0
children[0]:
type: 'disk'
id: 0
guid: 2494043808582069183
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,11:a'
whole_disk: 1
DTL: 5289
path: '/dev/dsk/c3t54d0s0'
devid:
'id1,sd@t49455400000000000000000002000000ef1000000f000000/a'
children[1]:
type: 'disk'
id: 1
guid: 12236206099593455135
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,6:a'
whole_disk: 1
DTL: 964
path: '/dev/dsk/c3t51d0s0'
devid:
'id1,sd@t494554000000000000000000020000004d0f00000f000000/a'
[long pause]
zdb: can't open 'andaman': I/O error
terabithia:/# format -e
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c1t0d0 <NETAPP-X231_SCHT5036F10-NA07 cyl 28837 alt 2 hd 4 sec 607>
boot-a
/pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w2100002037f86b53,0
1. c1t1d0 <NETAPP-X231_SCHT5036F10-NA07 cyl 28837 alt 2 hd 4 sec 607>
boot-b
/pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w2100002037f87b4c,0
2. c3t10d0 <IET-VIRTUAL-DISK-0-298.09GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd00001,0
3. c3t16d0 <IET-VIRTUAL-DISK-0-298.09GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd30001,0
4. c3t43d0 <IET-VIRTUAL-DISK-0-931.51GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,0
5. c3t44d0 <IET-VIRTUAL-DISK-0-698.64GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,1
6. c3t45d0 <IET-VIRTUAL-DISK-0-931.51GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,3
7. c3t46d0 <IET-VIRTUAL-DISK-0-233.76GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,7
8. c3t47d0 <IET-VIRTUAL-DISK-0-931.51GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,0
9. c3t48d0 <IET-VIRTUAL-DISK-0-931.51GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,1
10. c3t49d0 <IET-VIRTUAL-DISK-0-232.89GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,2
11. c3t50d0 <IET-VIRTUAL-DISK-0-698.64GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,3
12. c3t51d0 <IET-VIRTUAL-DISK-0-298.09GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,6
13. c3t52d0 <IET-VIRTUAL-DISK-0-931.51GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,7
14. c3t53d0 <IET-VIRTUAL-DISK-0-1.82TB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,8
15. c3t54d0 <IET-VIRTUAL-DISK-0-931.51GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,11
16. c3t57d0 <IET-VIRTUAL-DISK-0-1.82TB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,5
17. c3t59d0 <IET-VIRTUAL-DISK-0-931.51GB>
/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,10
Specify disk (enter its number): ^C
terabithia:/# exit
-----8<-----
Here is 'zdb -C' before I exported the problem pool:
-----8<-----
aboveground:
version: 8
name: 'aboveground'
state: 0
txg: 27093860
pool_guid: 14919333615317928292
hostid: 2200768359
hostname: 'terabithia.th3h.inner.chaos'
vdev_children: 1
vdev_tree:
type: 'root'
id: 0
guid: 14919333615317928292
children[0]:
type: 'mirror'
id: 0
guid: 16297033706428251175
whole_disk: 0
metaslab_array: 14
metaslab_shift: 31
ashift: 9
asize: 320059473920
is_log: 0
children[0]:
type: 'disk'
id: 0
guid: 6677977776636178990
path: '/dev/dsk/c3t10d0s0'
devid:
'id1,sd@t49455400000000000000000001000000541100000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd00001,0:a'
whole_disk: 1
DTL: 300
children[1]:
type: 'disk'
id: 1
guid: 8268759604762550338
path: '/dev/dsk/c3t16d0s0'
devid:
'id1,sd@t49455400000000000000000004000000680e00000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd30001,0:a'
whole_disk: 1
DTL: 19
andaman:
version: 22
name: 'andaman'
state: 0
txg: 27092168
pool_guid: 7400719929021713582
hostid: 2200768359
hostname: 'terabithia.th3h.inner.chaos'
vdev_children: 7
vdev_tree:
type: 'root'
id: 0
guid: 7400719929021713582
stats[0]: 1085420200
stats[1]: 4
stats[2]: 3
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 0
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[0]:
type: 'mirror'
id: 0
guid: 337393226491877361
whole_disk: 0
metaslab_array: 14
metaslab_shift: 33
ashift: 9
asize: 1000191557632
is_log: 0
stats[0]: 1085422400
stats[1]: 4
stats[2]: 3
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 996432412672
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[0]:
type: 'disk'
id: 0
guid: 1781150413433362160
path: '/dev/dsk/c3t43d0s0'
devid:
'id1,sd@t49455400000000000000000002000000521000000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,0:a'
whole_disk: 1
DTL: 91
stats[0]: 1085420200
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 996437131264
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[1]:
type: 'disk'
id: 1
guid: 7841235598547702997
path: '/dev/dsk/c3t48d0s0'
devid:
'id1,sd@t494554000000000000000000020000009b0e00000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,1:a'
whole_disk: 1
DTL: 215
stats[0]: 1085415000
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 996437131264
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[1]:
type: 'mirror'
id: 1
guid: 1953060080997571723
whole_disk: 0
metaslab_array: 210
metaslab_shift: 33
ashift: 9
asize: 1000191557632
is_log: 0
stats[0]: 1085431600
stats[1]: 4
stats[2]: 3
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 996432412672
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[0]:
type: 'disk'
id: 0
guid: 14567891505260370684
path: '/dev/dsk/c3t45d0s0'
devid:
'id1,sd@t49455400000000000000000002000000601000000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,3:a'
whole_disk: 1
DTL: 1395
stats[0]: 1085428200
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 996437131264
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[1]:
type: 'disk'
id: 1
guid: 12913430974955856922
path: '/dev/dsk/c3t47d0s0'
devid:
'id1,sd@t49455400000000000000000002000000960f00000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,0:a'
whole_disk: 1
DTL: 217
stats[0]: 1085270000
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 996437131264
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[2]:
type: 'mirror'
id: 2
guid: 17555572171866652488
metaslab_array: 330
metaslab_shift: 31
ashift: 9
asize: 1000191557632
is_log: 0
stats[0]: 1085290200
stats[1]: 4
stats[2]: 3
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 998579896320
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[0]:
type: 'disk'
id: 0
guid: 10644297178963807622
path: '/dev/dsk/c3t52d0s0'
devid:
'id1,sd@t49455400000000000000000002000000310f00000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,7:a'
whole_disk: 1
DTL: 1167
stats[0]: 1085289600
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 998584614912
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[1]:
type: 'disk'
id: 1
guid: 2378339860349039724
path: '/dev/dsk/c3t59d0s0'
devid:
'id1,sd@t49455400000000000000000002000000a41000000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,10:a'
whole_disk: 1
DTL: 1168
stats[0]: 1085287600
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 998584614912
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[3]:
type: 'mirror'
id: 3
guid: 5261242283366420554
metaslab_array: 332
metaslab_shift: 31
ashift: 9
asize: 250046054400
is_log: 0
stats[0]: 1085311200
stats[1]: 4
stats[2]: 3
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 249108103168
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[0]:
type: 'disk'
id: 0
guid: 17534037132307391717
path: '/dev/dsk/c3t46d0s0'
devid:
'id1,sd@t494554000000000000000000020000003a1000000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,7:a'
whole_disk: 1
DTL: 335
stats[0]: 1085276000
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 249112821760
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[1]:
type: 'disk'
id: 1
guid: 2758124488221280115
path: '/dev/dsk/c3t49d0s0'
devid:
'id1,sd@t49455400000000000000000002000000b00e00000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,2:a'
whole_disk: 1
DTL: 336
stats[0]: 1085275800
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 249112821760
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[4]:
type: 'mirror'
id: 4
guid: 2126473073638606315
whole_disk: 0
metaslab_array: 510
metaslab_shift: 31
ashift: 9
asize: 750142881792
is_log: 0
stats[0]: 1085300800
stats[1]: 4
stats[2]: 3
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 749471793152
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[0]:
type: 'disk'
id: 0
guid: 7272191256105073467
path: '/dev/dsk/c3t50d0s0'
devid:
'id1,sd@t494554000000000000000000020000000b0f00000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,3:a'
whole_disk: 1
DTL: 660
stats[0]: 1085299800
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 749476511744
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[1]:
type: 'disk'
id: 1
guid: 5437265705837840703
path: '/dev/dsk/c3t44d0s0'
devid:
'id1,sd@t494554000000000000000000020000007f1000000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,1:a'
whole_disk: 1
DTL: 513
stats[0]: 1085299800
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 749476511744
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[5]:
type: 'mirror'
id: 5
guid: 6278268859805291262
whole_disk: 0
metaslab_array: 509
metaslab_shift: 30
ashift: 9
asize: 2000385474560
is_log: 0
stats[0]: 1085291400
stats[1]: 4
stats[2]: 3
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 2000381018112
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[0]:
type: 'disk'
id: 0
guid: 2444845073767152521
path: '/dev/dsk/c3t57d0s0'
devid:
'id1,sd@t49455400000000000000000002000000211000000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,5:a'
whole_disk: 1
DTL: 4188
stats[0]: 1085290200
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 2000385736704
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[1]:
type: 'disk'
id: 1
guid: 7384627338137907176
path: '/dev/dsk/c3t53d0s0'
devid:
'id1,sd@t49455400000000000000000002000000b80f00000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,8:a'
whole_disk: 1
DTL: 4192
stats[0]: 1085289400
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 2000385736704
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[6]:
type: 'mirror'
id: 6
guid: 16251593073754009033
metaslab_array: 961
metaslab_shift: 31
ashift: 9
asize: 320059473920
is_log: 0
stats[0]: 1085390000
stats[1]: 4
stats[2]: 3
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 319975063552
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[0]:
type: 'disk'
id: 0
guid: 2494043808582069183
path: '/dev/dsk/c3t54d0s0'
devid:
'id1,sd@t49455400000000000000000002000000271100000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h.fishstick%3Asd-andaman0001,11:a'
whole_disk: 1
DTL: 5289
stats[0]: 1085390800
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 319979782144
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
children[1]:
type: 'disk'
id: 1
guid: 12236206099593455135
path: '/dev/dsk/c3t51d0s0'
devid:
'id1,sd@t49455400000000000000000002000000750f00000f000000/a'
phys_path:
'/iscsi/disk@0000iqn.2006-11.chaos.inner.th3h%3Aoldfishstick%3Asd-andaman0001,6:a'
whole_disk: 1
DTL: 964
stats[0]: 1085358200
stats[1]: 4
stats[2]: 1
stats[3]: 0
stats[4]: 0
stats[5]: 0
stats[6]: 319979782144
stats[7]: 0
stats[8]: 0
stats[9]: 0
stats[10]: 0
stats[11]: 0
stats[12]: 0
stats[13]: 0
stats[14]: 0
stats[15]: 0
stats[16]: 0
stats[17]: 0
stats[18]: 0
stats[19]: 0
stats[20]: 0
stats[21]: 0
stats[22]: 0
stats[23]: 0
stats[24]: 0
stats[25]: 0
stats[26]: 0
stats[27]: 0
stats[28]: 0
stats[29]: 0
rpool:
version: 22
name: 'rpool'
state: 0
txg: 18614338
pool_guid: 16346529704285805676
hostid: 2200768359
hostname: ''
vdev_children: 1
vdev_tree:
type: 'root'
id: 0
guid: 16346529704285805676
create_txg: 4
children[0]:
type: 'mirror'
id: 0
guid: 15963611323458091535
whole_disk: 0
metaslab_array: 23
metaslab_shift: 28
ashift: 9
asize: 35843473408
is_log: 0
create_txg: 4
children[0]:
type: 'disk'
id: 0
guid: 10495950740462619950
path: '/dev/dsk/c1t0d0s0'
devid: 'id1,ssd@n2000002037f86b53/a'
phys_path:
'/pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w2100002037f86b53,0:a'
whole_disk: 0
DTL: 102
create_txg: 4
children[1]:
type: 'disk'
id: 1
guid: 12455868191384207452
path: '/dev/dsk/c1t1d0s0'
devid: 'id1,ssd@n2000002037f87b4c/a'
phys_path:
'/pci@8,600000/SUNW,qlc@4/fp@0,0/ssd@w2100002037f87b4c,0:a'
whole_disk: 0
DTL: 180
create_txg: 4
-----8<-----
I have a coredump from the first panic, but for some reason dumpadm is
not writing 'unix.5' for me. I don't know how to get a backtrace with
'mdb -k'---any advice?
-----8<-----
terabithia:/# ls -l /var/crash/terabithia.th3h.inner.chaos/
total 5854958
-rw-r--r-- 1 root root 2 Apr 4 02:28 bounds
-rw-r--r-- 1 root root 686161920 Aug 1 2010 vmdump.0
-rw-r--r-- 1 root root 684130304 Aug 8 2010 vmdump.1
-rw-r--r-- 1 root root 684261376 Aug 22 2010 vmdump.2
-rw-r--r-- 1 root root 680525824 Sep 5 2010 vmdump.3
-rw-r--r-- 1 root root 178847744 Oct 31 12:39 vmdump.4
-rw-r--r-- 1 root root 132055040 Apr 4 02:28 vmdump.5
-----8<-----
Are there things I might try to recover the data? Backups exist, but
are several years old.
I can buy and house enough drives to make a copy of the pool, if
needed. I thought I might try 'zpool import -V andaman' from solaris
express 11 or openindiana b148, but I wonder if there's something
smarter I should be doing.
--
00
(--)
( || )
^^~~^^
pgprH9TPwCFaD.pgp
Description: PGP signature
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
