Re: [OpenWrt-Devel] [PATCH] mdadm: add new init features; documentation; bug fixes
On Wed, Jan 9, 2019 at 8:31 PM Joseph Tingiris wrote: > > This is a significant revision of /etc/init.d/mdadm. It adds new > features, support for new configuration options, safer error > handling, (configurable) verbose output, and contains multiple bug > fixes. > > Most notably, mdadm was being started with the --config flag and > that prevented it from using its built in Auto Assembly features. > Users were required to put a correct uuid in /etc/config/mdadm. > Configuring arrays is not for the feint of heart. > > The previous /etc/config/mdadm options left uncommented, that prevented > mdadm from starting successfully, were corrected and the default startup > mode is now to automatically assemble all RAID arrays attached to the > machine using device scans rather than configuation options and a > persistent state file. > > A new UCI section, config mdadm global, was added with new options that are > supported by the accompanying /etc/init.d/mdadm. Documentation for all > new (and previous) options was added as well. See the > /etc/config/mdadmin or mdadm.init file itself for more details. > > Additionally, a new stateful 'auto' feature was added that functions > similarly to the stateless Auto Assembly feature. The benefits of > stateful auto assembly are to support features that mdadm 4.0 will only > read from a configuration file, such as setting the MAILFROM value. The > new mdadm_conf_auto() function will also greatly aid users in > troubleshooting. When verbose is turned on it provides tips, better > visibility for what's actually happening, and sets the stage for > moderately advanced users to take more control over their RAIDs. > > Stateful UCI only configurations are still supported and will still work. > All previously existing configurations will work in this mode. However, > these users will now have to explicitly turn it on. Documentation is > provided in the new mdadm.config file. I'd expect very few regressions. > Anyone using the old init had to be an expert. The mdadm 4.0-5 package > doesn't function out of the box. For more detail about these parts, > see FS2043 here https://bugs.openwrt.org/ > > A new reload_service() function was added to prevent reloads from > stopping mdadm. That caused issues when filesystems were mounted. > Reloads will now be ignored, though the stage is set for reloads to > trigger scans for new devices. Explicit restarts still work as expected. > > The start_service() function was enhanced to query new UCI mdadm.global > options: alert_program, config, email, email_from, monitor_frequency, > and verbose. Each is documented in /etc/mdadm/config (config.init) and > some additional code comments were added. As well, error handling and > verbose output was greatly enhanced. It's not going to do anything > without users knowing what's going on (if verbose is turned on). > Backward compatibility was retained. Strict reliance on a shell global > ($CONF) was removed and replaced with a single global ($TMP_FILE) that's > for development convenience. When/if a config file is not specified in > the UCI config, it will fall back to using $TMP_FILE as the > configuration file. > > Incremented PKG_RELEASE from 5 to 6 > > For more details, see: https://github.com/openwrt/openwrt/pull/1713/ Any progress on this? I've been using this for a while. I recently ran into an issue (segfault) with mdadm that required me to bump the version. > > Signed-off-by: Joseph Tingiris > --- > package/utils/mdadm/Makefile | 2 +- > package/utils/mdadm/files/mdadm.config | 162 ++- > package/utils/mdadm/files/mdadm.init | 491 > +++-- > 3 files changed, 615 insertions(+), 40 deletions(-) > > diff --git a/package/utils/mdadm/Makefile b/package/utils/mdadm/Makefile > index ba74997..3fabc8a 100644 > --- a/package/utils/mdadm/Makefile > +++ b/package/utils/mdadm/Makefile > @@ -9,7 +9,7 @@ include $(TOPDIR)/rules.mk > > PKG_NAME:=mdadm > PKG_VERSION:=4.0 > -PKG_RELEASE:=5 > +PKG_RELEASE:=6 > > PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.xz > PKG_SOURCE_URL:=@KERNEL/linux/utils/raid/mdadm > diff --git a/package/utils/mdadm/files/mdadm.config > b/package/utils/mdadm/files/mdadm.config > index 50afbc2..88e0bb9 100644 > --- a/package/utils/mdadm/files/mdadm.config > +++ b/package/utils/mdadm/files/mdadm.config > @@ -1,18 +1,154 @@ > -config mdadm > +# > +# The mdadm 'global' section is for options that apply to all sections. > +# > + > +config mdadm global > + > + # > + # option 'alert_program' values may be a path to a valid, executable > binary. > + # > + # The default 'alert_program' is not set. > + # > + # When mdadm detects an event it will execute this program with 3 > arguments, see mdadm(8) > + # $1 = will be the event > + # $2 = will be the meta device > + # $3 = may be a related component (if one exists) > + # > + # * alert_program runs independently
Re: [OpenWrt-Devel] [PATCH] mdadm: add new init features; documentation; bug fixes
Hi,
comments inline.
> [...]
> + # partition - stateless, mdadm --assemble --scan
> --config=partition; see mdadm(8)
> + # uci - stateful, dynamically generated mdadm.conf via uci
> array values (below),
> + # stored in /var/etc/mdadm.conf
> + #- stateful, manually generated mdadm.conf file(s),
> + # must be preceded by a / and may be a readable
> filename
These file tags look wrong.
> [...]
> +#
> +# The mdadm 'array' section(s) are for stateful, manual configurations.
> Experts only. Use with caution.
> +#
> +#
> +# The use of multiple 'array' sections is supported by /etc/init.d/mdadm.
> +# They must all be named 'array'.
> +#
> +# As of this writing unnamed 'mdadm' sections are still allowed, but
> deprecated. Do not use.
> +#
Config "array" sections were used before as well? I do not understand
the deprecation remark.
> +
> +#config array
> + #
> + # example 'array' options may be a valid mix of:
> + #
> + # bitmap
> + # container
> + # device
> + # devices
> + # member
> + # name
> + # spare_group
> + # spares
> + # super_minor
> + # uuid
> + #
> # option bitmap /bitmap.md
> # option container :::
> + # option device /dev/md0
> + # -and/or a devices list-
> + # list devices /dev/hd* # mdadm allows glob, see glob(7)
> + # list devices /dev/sd*
> + # list devices /dev/sda1
> + # list devices /dev/sdb1
> + # list devices containers
> + # list devices partitions
> # option member 1
> + # option name raid:0
> + # option spare_group spares
> + # option spares 0
> + # option super_minor 0
> + # use uuid from block info (preferred), or mdadm --misc --detail
> /dev/md0
> + # option uuid 2084de11-70c4-4521-8f95-6113e75f1fe9
> + #
> + # These options directly translate to mdadm -- options, see man mdadm(8)
Consider linking to https://linux.die.net/man/8/mdadm, "man" is rarely
available on OpenWrt.
> diff --git a/package/utils/mdadm/files/mdadm.init
> b/package/utils/mdadm/files/mdadm.init
> index 64a50b3..5453f7d 100644
> --- a/package/utils/mdadm/files/mdadm.init
> +++ b/package/utils/mdadm/files/mdadm.init
> @@ -1,13 +1,21 @@
> #!/bin/sh /etc/rc.common
>
> -START=13
> -STOP=98
> +# 20190106, joseph.tingi...@gmail.com, significant revision; new features,
> +# safer error handling, code formatting,
> +# & multiple bug fixes
Please drop this comment, changes are recorded in Git & commit message.
> +
> +START=12
> +STOP=X99 # X99? seems to work; passed service enable/disable tests & boot
> tests
Please stick to plain 99, will change the umount prio in a separate commit.
>
> USE_PROCD=1
> PROG=/sbin/mdadm
> NAME=mdadm
>
> -CONF="/var/etc/mdadm.conf"
> +VERBOSE=0 # off
> +
> +TMP_FILE="/var/etc/mdadm.conf" # /var/etc is on /tmp; used for temporary
> state, to enable stateful only mdadm features
Please remove this comment.
> +
> +[ ! -x "$PROG" ] && exit 1
>
> append_list_item() {
> append "$2" "$1" "$3"
> @@ -30,30 +38,165 @@ append_option() {
> [ -n "$str" ] && append "$var" $(printf "%s=%s" "${name:-${opt//_/-}}"
> "$str")
> }
>
> -mdadm_common() {
> - local cfg="$1"
> - local email devices
> +verbose() {
> + local msg="$1"
> + local level="$2"
>
> - if [ -x /usr/sbin/sendmail ]; then
> - config_get email "$cfg" email
> - [ -n "$email" ] && printf "MAILADDR %s\n" "$email" >> $CONF
> + [ -z "$level" ] && level="INFO"
> +
> + if [ "$VERBOSE" == "1" ]; then
> + if [ ${#SSH_TTY} -gt 0 ]; then
> + printf "$NAME: init %7s - %b\n" "$level" "$msg"
> + else
> + # no SSH_TTY goes to logger
> + printf "$NAME: init %7s - %b\n" "$level" "$msg" |
> logger -t mdadm
> + fi
> fi
User sessions may not always happen via SSH, could be telnet, uart,
screen. Stick to either variant, independently of $SSH_TTY.
> +}
>
> - config_list_foreach "$cfg" devices append_list_item devices " "
> - [ -n "$devices" ] && printf "DEVICE %s\n" "$devices" >> $CONF
> +mdadm_conf_auto() {
> + local mdadm_conf="$1"
> +
> + if [ ! -w "$mdadm_conf" ]; then
> + if [ -z "$mdadm_conf" ]; then
> + verbose "mdadm_conf value is empty" ERROR
> + else
> + verbose "'$mdadm_conf' file not found writable" ERROR
> + fi
> + return 1
> + fi
> +
> + local block_md block_uuid mdadm_md mdadm_md_rc mdadm_uuid
> +
> + # Check block info for active linux_raid_members, if necessary then
> compare with mdadm, & dynamically update $mdadm_conf
> +
> + block_md=0 # counter
> + for block_uuid in $(block info 2> /dev/null | grep
[OpenWrt-Devel] [PATCH] mdadm: add new init features; documentation; bug fixes
This is a significant revision of /etc/init.d/mdadm. It adds new features, support for new configuration options, safer error handling, (configurable) verbose output, and contains multiple bug fixes. Most notably, mdadm was being started with the --config flag and that prevented it from using its built in Auto Assembly features. Users were required to put a correct uuid in /etc/config/mdadm. Configuring arrays is not for the feint of heart. The previous /etc/config/mdadm options left uncommented, that prevented mdadm from starting successfully, were corrected and the default startup mode is now to automatically assemble all RAID arrays attached to the machine using device scans rather than configuation options and a persistent state file. A new UCI section, config mdadm global, was added with new options that are supported by the accompanying /etc/init.d/mdadm. Documentation for all new (and previous) options was added as well. See the /etc/config/mdadmin or mdadm.init file itself for more details. Additionally, a new stateful 'auto' feature was added that functions similarly to the stateless Auto Assembly feature. The benefits of stateful auto assembly are to support features that mdadm 4.0 will only read from a configuration file, such as setting the MAILFROM value. The new mdadm_conf_auto() function will also greatly aid users in troubleshooting. When verbose is turned on it provides tips, better visibility for what's actually happening, and sets the stage for moderately advanced users to take more control over their RAIDs. Stateful UCI only configurations are still supported and will still work. All previously existing configurations will work in this mode. However, these users will now have to explicitly turn it on. Documentation is provided in the new mdadm.config file. I'd expect very few regressions. Anyone using the old init had to be an expert. The mdadm 4.0-5 package doesn't function out of the box. For more detail about these parts, see FS2043 here https://bugs.openwrt.org/ A new reload_service() function was added to prevent reloads from stopping mdadm. That caused issues when filesystems were mounted. Reloads will now be ignored, though the stage is set for reloads to trigger scans for new devices. Explicit restarts still work as expected. The start_service() function was enhanced to query new UCI mdadm.global options: alert_program, config, email, email_from, monitor_frequency, and verbose. Each is documented in /etc/mdadm/config (config.init) and some additional code comments were added. As well, error handling and verbose output was greatly enhanced. It's not going to do anything without users knowing what's going on (if verbose is turned on). Backward compatibility was retained. Strict reliance on a shell global ($CONF) was removed and replaced with a single global ($TMP_FILE) that's for development convenience. When/if a config file is not specified in the UCI config, it will fall back to using $TMP_FILE as the configuration file. Incremented PKG_RELEASE from 5 to 6 For more details, see: https://github.com/openwrt/openwrt/pull/1713/ Signed-off-by: Joseph Tingiris --- package/utils/mdadm/Makefile | 2 +- package/utils/mdadm/files/mdadm.config | 162 ++- package/utils/mdadm/files/mdadm.init | 491 +++-- 3 files changed, 615 insertions(+), 40 deletions(-) diff --git a/package/utils/mdadm/Makefile b/package/utils/mdadm/Makefile index ba74997..3fabc8a 100644 --- a/package/utils/mdadm/Makefile +++ b/package/utils/mdadm/Makefile @@ -9,7 +9,7 @@ include $(TOPDIR)/rules.mk PKG_NAME:=mdadm PKG_VERSION:=4.0 -PKG_RELEASE:=5 +PKG_RELEASE:=6 PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.xz PKG_SOURCE_URL:=@KERNEL/linux/utils/raid/mdadm diff --git a/package/utils/mdadm/files/mdadm.config b/package/utils/mdadm/files/mdadm.config index 50afbc2..88e0bb9 100644 --- a/package/utils/mdadm/files/mdadm.config +++ b/package/utils/mdadm/files/mdadm.config @@ -1,18 +1,154 @@ -config mdadm +# +# The mdadm 'global' section is for options that apply to all sections. +# + +config mdadm global + + # + # option 'alert_program' values may be a path to a valid, executable binary. + # + # The default 'alert_program' is not set. + # + # When mdadm detects an event it will execute this program with 3 arguments, see mdadm(8) + # $1 = will be the event + # $2 = will be the meta device + # $3 = may be a related component (if one exists) + # + # * alert_program runs independently from sendmail. + # * If both options alert_program and email are set, and both work, then an email and a + # custom alert will be generated. + # * no alert program is included in mdadm 4.0-4. + # + # Lots of possibilities exist, i.e. scripts for netdata, slack, etc. + # + #option alert_program /usr/sbin/mdadm_alerts + + + # + # option 'config'
