On Mon, 5 Aug 2024 at 13:50, Carsten Grzemba via illumos-discuss
<[email protected]> wrote:
> I have a problem shutting down a server with Zpool on iscsi devices. The
> shutdown gets stuck. In the console log I see the message:
> WARNING: Pool 'zones' has encountered an uncorrectable I/O failure and has
> been suspended; `zpool clear` will be required before the pool can be written
> to.
> The message looks to me like the IP connection to the storage server was
> already terminated before the Zpool was unmounted.
> I forced the shutdown with an NMI reset so that I have a crash dump. In the
> crash dump I see that a zone on the pool in question is still in
> shutting_down.
> Is it possible to get information from crash dump like *svcs* of a running
> system?
Not really easily, no, but you can at least see the tree of processes
that is running; e.g.,
> ::ptree
fffffffffbc93760 sched
fffffeb1cedab008 zpool-extra
fffffeb1c8c2c000 fsflush
fffffeb1c8c2f020 pageout
fffffeb1c8c34018 init
fffffeb38ccb4020 screen
fffffeb5aaddd020 bash
fffffeb1f123e010 init
fffffeb3a3070000 image-builder
fffffeb263501020 pkg
fffffeb1cea71018 sac
fffffeb1ca8a7018 ttymon
...
> How can I see in the crash dump whether the IP interfaces are still working?
> Normally the SMF
> svc:/network/iscsi/initiator:default
> should wait for the zones to shutdown and reach the state installed.
I'm not sure that it does do that. Looking at the manifest, I don't
see any interaction with zones at all:
https://github.com/illumos/illumos-gate/blob/master/usr/src/cmd/iscsid/iscsi-initiator.xml
There is some special handling in the shutdown code today for
unmounting file systems that are listed in vfstab(5) as being mounted
from iSCSI disks. That obviously doesn't help you with ZFS pools,
though, and I expect the real bug here is that the "umount_iscsi()"
function in that method script doesn't do anything to export ZFS pools
that live on iSCSI disks:
https://github.com/illumos/illumos-gate/blob/master/usr/src/cmd/iscsid/iscsi-initiator#L172-L199
How do you _import_ these pools today? It seems likely that pools
that come from iSCSI devices should actually not appear in the global
zpool cache file (i.e., "/etc/zfs/zpool.cache"), but rather should get
imported transiently somehow by the iSCSI initiator service in
"mount_iscsi()". Then they should get exported on the way to
shutdown. In short: importing and exporting those pools should
probably work like vfstab(5) entries with "iscsi" set in the automount
field. Otherwise there does not appear to be any mechanism in place
to handle the network going away as part of shutdown.
Cheers.
--
Joshua M. Clulow
http://blog.sysmgr.org
------------------------------------------
illumos: illumos-discuss
Permalink:
https://illumos.topicbox.com/groups/discuss/T494e7618fdacd18b-Mffbe5a8bca2b03942d98d534
Delivery options: https://illumos.topicbox.com/groups/discuss/subscription
