I've pursued the idea of a separate SMF service which would take
care of local ZFS-based split-root systems instead of hacking into
existing scripts (network or filesystem). All the logic that I've
earlier added into fs-root and fs-minimal has now moved into the
new script and service fs-root-zfs. This service declares as its
dependants the networking services as well as filesystem/root, to
ensure that it runs before consumers of /usr and other filesystems
that make up a root hierarchy. In fact, it mounts all of the ZFS
filesystems which are children of the current bootfs or of the
$rpool/SHARED dataset, and /var/run in particular, fulfilling
much of filesystem/minimal in one early blow. This should be
appreciated by NWAM in particular :)
It does check for filesystems listed in /etc/vfstab and provided
by non-ZFS technologies - this script should skip mounting those
mountpoints and any under them. Also, it only runs for a ZFS root.
The large changes I proposed before to the older scripts are no
longer needed for the local-ZFS split-root setups; though smaller
changes (provided in new patches) are still added - to skip
zfs-mounting in case that the filesystem in question has already
been mounted.
My local tests were quite successful, so I posted the update at
the Wiki:
http://wiki.openindiana.org/oi/Advanced+-+Split-root+installation
http://wiki.openindiana.org/download/attachments/27230229/fs-root-zfs
http://wiki.openindiana.org/download/attachments/27230229/fs-root-zfs.xml
Slight but important fixes to the older fs-root and fs-minimal
scripts (as well as the whole of the new fs-root-zfs script) can
be reviewed in this patch:
http://wiki.openindiana.org/download/attachments/27230229/fs-root-zfs.patch
An example of console output for booting a BE with bad mountpoint
due to untimely reset while this BE was mounted for administrative
tasks from another running BE (with console debugging enabled by
"touch /$BEMOUNT/.debug_mnt"):
OpenIndiana Build oi_151a8 64-bit (illumos 7256a34efe)
SunOS Release 5.11 - Copyright 1983-2010 Oracle and/or its affiliates.
All rights reserved. Use is subject to license terms.
Rootfs mountpoint not '/' but '/tmp/tmp.a5aGOb', trying to fix.
Fixing 'rpool/ROOT/test/opt' to use '/opt' mountpoint instead of
'/tmp/tmp.a5aGOb/opt': shifted in same root hierarchy
Fixing 'rpool/ROOT/test/usr' to use '/usr' mountpoint instead of
'/tmp/tmp.a5aGOb/usr': shifted in same root hierarchy
Fixing 'rpool/ROOT/test/usr/local' to use '/usr/local' mountpoint
instead of '/tmp/tmp.a5aGOb/usr/local': shifted in same root hierarchy
Fixing 'rpool/ROOT/test/var' to use '/var' mountpoint instead of
'/tmp/tmp.a5aGOb/var': shifted in same root hierarchy
Mounting '/usr': use 'rpool/ROOT/test/usr': in same root hierarchy
Mounting '/var': use 'rpool/ROOT/test/var': in same root hierarchy
Mounting '/var/adm': use 'rpool/SHARED/var/adm': the only option
Not ZFS-mounting '/tmp': equal or under a non-ZFS mountpoint '/tmp'
Mounting '/opt': use 'rpool/ROOT/test/opt': in same root hierarchy
Not mounting: '/usr' from 'rpool/ROOT/test/usr': something already mounted
Mounting '/usr/local': use 'rpool/ROOT/test/usr/local': in same root
hierarchy
Not mounting: '/var' from 'rpool/ROOT/test/var': something already mounted
Not mounting: '/var' from 'rpool/SHARED/var': canmount!=on
Not mounting: '/var/adm' from 'rpool/SHARED/var/adm': something already
mounted
Mounting '/var/cores': use 'rpool/SHARED/var/cores': in shared root
hierarchy
Mounting '/var/crash': use 'rpool/SHARED/var/crash': in shared root
hierarchy
Mounting '/var/log': use 'rpool/SHARED/var/log': in shared root hierarchy
Mounting '/var/mail': use 'rpool/SHARED/var/mail': in shared root hierarchy
Not mounting: '/var/spool' from 'rpool/SHARED/var/spool': canmount!=on
Mounting '/var/spool/clientmqueue': use
'rpool/SHARED/var/spool/clientmqueue': in shared root hierarchy
Mounting '/var/spool/mqueue': use 'rpool/SHARED/var/spool/mqueue': in
shared root hierarchy
Mounting '/var/tmp': use 'rpool/SHARED/var/tmp': in shared root hierarchy
fs-root-zfs: completed without fatal errors
Hostname: openindiana
...
The filesystem tree on this box is:
# df -k
Filesystem 1K-blocks Used Available Use% Mounted on
rpool/ROOT/test 2065321 314150 1751171 16% /
swap 1347972 1112 1346860 1% /etc/svc/volatile
rpool/ROOT/test/usr 2576515 825344 1751171 33% /usr
rpool/ROOT/test/var 1809675 58504 1751171 4% /var
rpool/SHARED/var/adm 1751283 112 1751171 1% /var/adm
swap 1346912 52 1346860 1% /var/run
rpool/ROOT/test/opt 1752589 1418 1751171 1% /opt
rpool/ROOT/test/usr/local
1751202 31 1751171 1% /usr/local
rpool/SHARED/var/cores
1762117 10946 1751171 1% /var/cores
rpool/SHARED/var/crash
1751202 31 1751171 1% /var/crash
rpool/SHARED/var/log 1751229 58 1751171 1% /var/log
rpool/SHARED/var/mail
1751203 32 1751171 1% /var/mail
rpool/SHARED/var/spool/clientmqueue
1751203 32 1751171 1%
/var/spool/clientmqueue
rpool/SHARED/var/spool/mqueue
1751202 31 1751171 1% /var/spool/mqueue
rpool/SHARED/var/tmp 1751223 52 1751171 1% /var/tmp
/usr/lib/libc/libc_hwcap2.so.1
2576515 825344 1751171 33% /lib/libc.so.1
swap 1346868 8 1346860 1% /tmp
rpool/export 1751203 32 1751171 1% /export
rpool/export/home 1751203 32 1751171 1% /export/home
rpool/export/home/admin
1751846 675 1751171 1% /export/home/admin
rpool 1751217 46 1751171 1% /rpool
/export/home/admin 1751846 675 1751171 1% /home/admin
Hope for comments,
//Jim Klimov
On 2013-11-30 10:25, Jim Klimov wrote:
2) I think one more valid approach to unroll these dependencies
via SMF in a packageable manner has emerged, and a rather apparent
one: to move (or duplicate, or invoke) the code from fs-root which
mounts a zfs-based /usr filesystem into a service of its own, on
which consumers of the /usr namespace would depend (optional_all).
At start this service would check if current root is zfs, and
if a child dataset or legacy-mounted ZFS /usr are known and
available - it would mount the dataset if yes. Otherwise it
would exit without an error. As a result, the networking
scripts in my split-zfs-based-root cause would be guaranteed
to have a /usr before they run.
It would (should) have no impact on systems that use monoroots
on ZFS, or that use other roots (networked, metadevice, etc.) -
these would work or fail the same as they do today.
3) Similarly, such a service can mount ZFS-based datasets of
the rest of the root hierarchy if available (/var, children
of bootfs, SHARED/*) and as a result of this, even the NWAM
method on systems with local storage would have a complete
environment to work in (for its LDAP/NIS interaction), all
without major rehaul of SMF dependencies and method code.
But in this extended case there is a possible though improbable
loophole: if some parts of the operating environment including
the rootfs are mounted from ZFS, but some major components
like /var work from nfs/cachefs/ufs/... and then some datasets
like /var/adm would be mounted on top of that. A script that
only mounts a ZFS hierarchy in order to avoid dependencies
on networking and metadevices would apparently ignore these
other options; at most it can detect them in /etc/vfstab and
stop mounting stuff under the involved mountpoint (this would
come in later via filesystem service chains that exist today).
And the current filesystem service methods should need to check
that they don't mount the same (zfs) filesystem twice, so as
to not bail out on "zfs mount" errors due to this.
_______________________________________________
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss
