Package: initrd-tools Version: 0.1.74 Severity: wishlist
Hi, This is a proposal to improve mkinitrd by reusing code from the hotplug project to determine which modules are needed on the initrd image to support block devices. It is an alternative to an earlier proposal to make devices in sysfs point to the module that implements their driver. It improves on the original proposal in the following ways: - no kernel changes required, so will work with older 2.6 kernels - can cope with differences between modules in running kernel and new kernel; as an example, it understands that a USB stick in 2.6.8 needs usb-storage, but in 2.6.9 will need the ub module. The original proposal and the problems motivating it are discussed in: - http://marc.theaimsgroup.com/?l=linux-scsi&m=109570145108639&w=2 - http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=263169 The method is simple: mkinitrd runs under an old kernel on some hardware, and it needs to find the minimal set of modules needed to boot a new kernel on that hardware. Sysfs describes the hardware, complete with device identifiers, and a module map provided with the new kernel describes which modules are needed to operate a particular piece of hardware. Finally, modules.dep shows which support modules must be loaded before a module can be loaded. Hotplug uses this same information to load modules for a new device into a running kernel. In fact hotplug and mkinitrd are two sides of the same coin: hotplug finds modules for available hardware given a running kernel, and mkinitrd finds modules for some kernel given available hardware. So I copied the module table processing from hotplug and adapted it to use tables from a different kernel. After that, it's only a matter of identifying which hardware devices are used to implement block devices; essentially that's just /sys/block/*/device/. One little twist: if you're using for example a USB stick, you need not only the USB driver, but also the PCI driver for the bus that the USB host is attached to. The pathname of the device in /sys/devices identifies all such parent devices. The attached script implements this method, and seems to give more or less correct answers on a machine with SATA drive (although I wonder about the location of ide0 in sysfs): [EMAIL PROTECTED]:/dat/tmp$ uname -r 2.6.8-1-686 [EMAIL PROTECTED]:/dat/tmp$ sh joke4 =============== /sys/block/hda devno 3:0 at: /sys/devices/ide0/0.0 needs: =============== /sys/block/sda devno 8:0 at: /sys/devices/pci0000:00/0000:00:1f.2/host0/0:0:0:0 needs: ata_piix =============== /sys/block/sdb devno 8:16 at: /sys/devices/pci0000:00/0000:00:1d.7/usb5/5-4/5-4.3/5-4.3:1.0/host7/7:0:0:0 needs: ehci-hcd usbcore usbcore usb-storage recommended module load order: /lib/modules/2.6.8-1-686/kernel/drivers/scsi/libata.ko /lib/modules/2.6.8-1-686/kernel/drivers/scsi/scsi_mod.ko /lib/modules/2.6.8-1-686/kernel/drivers/scsi/ata_piix.ko /lib/modules/2.6.8-1-686/kernel/drivers/usb/core/usbcore.ko /lib/modules/2.6.8-1-686/kernel/drivers/usb/host/ehci-hcd.ko /lib/modules/2.6.8-1-686/kernel/drivers/ide/ide-core.ko /lib/modules/2.6.8-1-686/kernel/drivers/usb/storage/usb-storage.ko [EMAIL PROTECTED]:/dat/tmp$ sh joke4 2.6.10-rc2-e =============== /sys/block/hda devno 3:0 at: /sys/devices/ide0/0.0 needs: =============== /sys/block/sda devno 8:0 at: /sys/devices/pci0000:00/0000:00:1f.2/host0/0:0:0:0 needs: ata_piix =============== /sys/block/sdb devno 8:16 at: /sys/devices/pci0000:00/0000:00:1d.7/usb5/5-4/5-4.3/5-4.3:1.0/host7/7:0:0:0 needs: ehci-hcd usbcore usbcore ub recommended module load order: /lib/modules/2.6.10-rc2-e/kernel/drivers/scsi/libata.ko /lib/modules/2.6.10-rc2-e/kernel/drivers/scsi/scsi_mod.ko /lib/modules/2.6.10-rc2-e/kernel/drivers/scsi/ata_piix.ko /lib/modules/2.6.10-rc2-e/kernel/drivers/usb/core/usbcore.ko /lib/modules/2.6.10-rc2-e/kernel/drivers/usb/host/ehci-hcd.ko /lib/modules/2.6.10-rc2-e/kernel/drivers/block/ub.ko [EMAIL PROTECTED]:/dat/tmp$ I'm looking for comments: could this approach be made to work, and if so what would be the best way to go about moving from this proof of concept to something that could be used as part of a complete mkinitrd? Regards, Erik # # Given a kernel version (default the running kernel) # print an ordered minimal list of all files that must # be loaded into that kernel in order for all block # devices to be operational. # # A large part of this script is copied from the hotplug package, # http://linux-hotplug.sourceforge.net/ # # This software may be used and distributed according to the terms # of the GNU General Public License, incorporated herein by reference. # # # # Each modules.usbmap format line corresponds to one entry in a # MODULE_DEVICE_TABLE(usb,...) declaration in a kernel file. # # Think of it as a database column with up to three "match specs" # to associate kernel modules with particular devices or classes # of device. The match specs provide a reasonably good filtering # mechanism, but some driver probe() routines need to provide # extra filtering. # usb_convert_vars () { # work around 2.2.early brokenness # munges the usb_bcdDevice such that it is a integer rather # than a float: e.g. 1.0 become 0100 DEFAULT_IFS="$IFS" PRODUCT=`echo $PRODUCT | sed -e "s+\.\([0-9]\)$+.\10+" -e "s/\.$/00/" \ -e "s+/\([0-9]\)\.\([0-9][0-9]\)+/0\1\2+" \ -e "s+/\([0-9][0-9]\)\.\([0-9][0-9]\)+/\1\2+"` set $(echo $PRODUCT | sed -e 's+\([^/]*\)/\([^/]*\)/\(.*\)+\1 \2 \3+') usb_idVendor=$((0x$1)) usb_idProduct=$((0x$2)) usb_bcdDevice=$((0x$3)) if [ "$TYPE" != "" ]; then IFS=/ set $TYPE '' usb_bDeviceClass=$1 usb_bDeviceSubClass=$2 usb_bDeviceProtocol=$3 IFS="$DEFAULT_IFS" elif [ -r $SYSFS/$DEVPATH/bDeviceClass ]; then usb_bDeviceClass=$((0x$(cat $SYSFS/$DEVPATH/bDeviceClass))) usb_bDeviceSubClass=$((0x$(cat $SYSFS/$DEVPATH/bDeviceSubClass))) usb_bDeviceProtocol=$((0x$(cat $SYSFS/$DEVPATH/bDeviceProtocol))) else # out-of-range values usb_bDeviceClass=1000 usb_bDeviceSubClass=1000 usb_bDeviceProtocol=1000 fi if [ "$INTERFACE" != "" ]; then IFS=/ set $INTERFACE '' usb_bInterfaceClass=$1 usb_bInterfaceSubClass=$2 usb_bInterfaceProtocol=$3 IFS="$DEFAULT_IFS" elif [ -r $SYSFS/$DEVPATH/bInterfaceClass ]; then usb_bInterfaceClass=$((0x$(cat $SYSFS/$DEVPATH/bInterfaceClass))) usb_bInterfaceSubClass=$((0x$(cat $SYSFS/$DEVPATH/bInterfaceSubClass))) usb_bInterfaceProtocol=$((0x$(cat $SYSFS/$DEVPATH/bInterfaceProtocol))) else # out-of-range values usb_bInterfaceClass=1000 usb_bInterfaceSubClass=1000 usb_bInterfaceProtocol=1000 fi } USB_MATCH_VENDOR=$((0x0001)) USB_MATCH_PRODUCT=$((0x0002)) USB_MATCH_DEV_LO=$((0x0004)) USB_MATCH_DEV_HI=$((0x0008)) USB_MATCH_DEV_CLASS=$((0x0010)) USB_MATCH_DEV_SUBCLASS=$((0x0020)) USB_MATCH_DEV_PROTOCOL=$((0x0040)) USB_MATCH_INT_CLASS=$((0x0080)) USB_MATCH_INT_SUBCLASS=$((0x0100)) USB_MATCH_INT_PROTOCOL=$((0x0200)) # # stdin is "modules.usbmap" syntax # on return, all matching modules have been added to $DRIVERS # usb_map_modules () { local line module # look at each usb_device_id entry # collect all matches in $DRIVERS while read line do # comments are lines that start with "#" ... # be careful, they still get parsed by bash! case "$line" in \#*) continue ;; "") continue ;; esac set $line module=$1 match_flags=$(($2)) idVendor=$(($3)) idProduct=$(($4)) bcdDevice_lo=$(($5)) bcdDevice_hi=$(($6)) bDeviceClass=$(($7)) bDeviceSubClass=$(($8)) bDeviceProtocol=$(($9)) shift 9 bInterfaceClass=$(($1)) bInterfaceSubClass=$(($2)) bInterfaceProtocol=$(($3)) : checkmatch $module : idVendor $idVendor $usb_idVendor if [ $USB_MATCH_VENDOR -eq $(( $match_flags & $USB_MATCH_VENDOR )) ] && [ $idVendor -ne $usb_idVendor ]; then continue fi : idProduct $idProduct $usb_idProduct if [ $USB_MATCH_PRODUCT -eq $(( $match_flags & $USB_MATCH_PRODUCT )) ] && [ $idProduct -ne $usb_idProduct ]; then continue fi : bcdDevice range $bcdDevice_hi $bcdDevice_lo actual $usb_bcdDevice if [ $USB_MATCH_DEV_LO -eq $(( $match_flags & $USB_MATCH_DEV_LO )) ] && [ $usb_bcdDevice -lt $bcdDevice_lo ]; then continue fi # bcdDevice_lo <= bcdDevice <= bcdDevice_hi if [ $USB_MATCH_DEV_HI -eq $(( $match_flags & $USB_MATCH_DEV_HI )) ] && [ $usb_bcdDevice -gt $bcdDevice_hi ]; then continue fi : bDeviceClass $bDeviceClass $usb_bDeviceClass if [ $USB_MATCH_DEV_CLASS -eq $(( $match_flags & $USB_MATCH_DEV_CLASS )) ] && [ $bDeviceClass -ne $usb_bDeviceClass ]; then continue fi : bDeviceSubClass $bDeviceSubClass $usb_bDeviceSubClass if [ $USB_MATCH_DEV_SUBCLASS -eq $(( $match_flags & $USB_MATCH_DEV_SUBCLASS )) ] && [ $bDeviceSubClass -ne $usb_bDeviceSubClass ]; then continue fi : bDeviceProtocol $bDeviceProtocol $usb_bDeviceProtocol if [ $USB_MATCH_DEV_PROTOCOL -eq $(( $match_flags & $USB_MATCH_DEV_PROTOCOL )) ] && [ $bDeviceProtocol -ne $usb_bDeviceProtocol ]; then continue fi # NOTE: for now, this only checks the first of perhaps # several interfaces for this device. : bInterfaceClass $bInterfaceClass $usb_bInterfaceClass if [ $USB_MATCH_INT_CLASS -eq $(( $match_flags & $USB_MATCH_INT_CLASS )) ] && [ $bInterfaceClass -ne $usb_bInterfaceClass ]; then continue fi : bInterfaceSubClass $bInterfaceSubClass $usb_bInterfaceSubClass if [ $USB_MATCH_INT_SUBCLASS -eq $(( $match_flags & $USB_MATCH_INT_SUBCLASS )) ] && [ $bInterfaceSubClass -ne $usb_bInterfaceSubClass ]; then continue fi : bInterfaceProtocol $bInterfaceProtocol $usb_bInterfaceProtocol if [ $USB_MATCH_INT_PROTOCOL -eq $(( $match_flags & $USB_MATCH_INT_PROTOCOL )) ] && [ $bInterfaceProtocol -ne $usb_bInterfaceProtocol ]; then continue fi # It was a match! DRIVERS="$module $DRIVERS" : drivers $DRIVERS done } # # Given path to USB device, add any required modules # to $DRIVERS, based on modules.usbmap # usb_device() { PRODUCT=0/0/0 TYPE= INTERFACE= DEVPATH= DEVICE= export ACTION PRODUCT TYPE INTERFACE DEVPATH DEVICE DEVPATH=$1 DEVPATH=${1#$SYSFS} usb_convert_vars usb_map_modules < $MAP_USB } # # Given path to USB interface, add any required modules # to $DRIVERS, based on modules.usbmap # usb_interface() { PRODUCT=0/0/0 TYPE= INTERFACE= DEVPATH= DEVICE= export ACTION PRODUCT TYPE INTERFACE DEVPATH DEVICE DEVPATH=$1 dir=$DEVPATH/.. # XXX I don't know why this is needed, but it works PRODUCT="$(cat $dir/idVendor)/$(cat $dir/idProduct)/$(cat $dir/bcdDevice)" # echo "++++++++ $PRODUCT" DEVPATH=${DEVPATH#$SYSFS} usb_convert_vars usb_map_modules < $MAP_USB } # # Each modules.usbmap format line corresponds to one entry in a # MODULE_DEVICE_TABLE(pci,...) declaration in a kernel file. # # Think of it as a database column with up to three "match specs" # to associate kernel modules with particular devices or classes # of device. The match specs provide a reasonably good filtering # mechanism, but some driver probe() routines need to provide # extra filtering. # pci_convert_vars () { pci_class=$((0x$PCI_CLASS)) set $(echo $PCI_ID | sed -e 's/\([^:]*\):\(.*\)/\1 \2/') pci_id_vendor=$((0x$1)) pci_id_device=$((0x$2)) set $(echo $PCI_SUBSYS_ID | sed -e 's/\([^:]*\):\(.*\)/\1 \2/') pci_subid_vendor=$((0x$1)) pci_subid_device=$((0x$2)) } PCI_ANY=$((0xffffffff)) # # stdin is "modules.pcimap" syntax # on return, ONE matching module was added to $DRIVERS # pci_map_modules () { local module ignored # comment line lists (current) pci_device_id field names read ignored # look at each pci_device_id entry # collect one match in $DRIVERS while read module vendor device subvendor subdevice class class_mask ignored do # comments are lines that start with "#" ... # be careful, they still get parsed by bash! case "$module" in \#*) continue ;; esac # convert the fields from hex to dec vendor=$(($vendor)); device=$(($device)) subvendor=$(($subvendor)); subdevice=$(($subdevice)) class=$(($class)); class_mask=$(($class_mask)) : checkmatch $module : vendor $vendor $pci_id_vendor if [ $vendor -ne $PCI_ANY -a $vendor -ne $pci_id_vendor ]; then continue fi : device $device $pci_id_device if [ $device -ne $PCI_ANY -a $device -ne $pci_id_device ]; then continue fi : sub-vendor $subvendor $pci_subid_vendor if [ $subvendor -ne $PCI_ANY -a $subvendor -ne $pci_subid_vendor ]; then continue fi : sub-device $subdevice $pci_subid_device if [ $subdevice -ne $PCI_ANY -a $subdevice -ne $pci_subid_device ]; then continue fi class_temp=$(($pci_class & $class_mask)) if [ $class_temp -eq $class ]; then DRIVERS="$module $DRIVERS" : drivers $DRIVERS fi done } # # Given path to PCI device, add any required modules # to $DRIVERS, based on modules.pcimap # pci_device() { ACTION=add PCI_CLASS=0 PCI_ID=0:0 PCI_SLOT_NAME=0:0.0 PCI_SUBSYS_ID=0:0 export ACTION PCI_CLASS PCI_ID PCI_SLOT_NAME PCI_SUBSYS_ID PCI_DEVICE=$1 set `echo $PCI_DEVICE \ | sed -e 's/\([^:]*\):\(.*\):\(.*\)\.\(.*\)/\1 \2 \3 \4/'` PCI_SLOT_NAME=$2:$3.$4 PCI_CLASS="`cat $PCI_DEVICE/class`" PCI_CLASS=${PCI_CLASS#0x} vendor_id=`cat $PCI_DEVICE/vendor` device_id=`cat $PCI_DEVICE/device` PCI_ID="${vendor_id#0x}:${device_id#0x}" sub_vendor_id=`cat $PCI_DEVICE/subsystem_vendor` sub_device_id=`cat $PCI_DEVICE/subsystem_device` PCI_SUBSYS_ID="${sub_vendor_id#0x}:${sub_device_id#0x}" pci_convert_vars pci_map_modules < $MAP_PCI } # # Given path to a device, determine all devices that # connect it to the core. For each device, determine # the type (PCI, USB); add directly required modules # to $ALL_DRIVERS. Ignores possible support modules. # phys_device() { path=$1 DRIVERS="" while [ "$path" != '/' ] do # echo "in $path" if [ -f "$path/subsystem_vendor" ] then # echo "PCI device: $path" pci_device $path fi if [ -f "$path/bDeviceClass" ] then # echo "USB device: $path" usb_device $path fi if [ -f "$path/bInterfaceClass" ] then # echo "USB interface: $path" usb_interface $path fi path=`dirname $path` done echo "needs: $DRIVERS" ALL_DRIVERS="$ALL_DRIVERS $DRIVERS" } # # Iterate over all block devices, # return a list of required drivers in $ALL_DRIVERS. # A block device may require multiple drivers, # eg when an USB stick is only reachable via an USB controller on a PCI bus. # Duplicates possible # block_devs() { ALL_DRIVERS="" for blockdev in $SYSFS/block/* do if [ -e $blockdev/device ] then device=`readlink -f $blockdev/device` # ignore partitions for now; # this is a nice place to filter for root and swap. devno=$(cat $blockdev/dev) echo =============== $blockdev echo "devno $devno at: $device" phys_device $device fi done } # # Given a list of modules, make $ALL_FILES a list of files # containing those modules. # find_files() { ALL_FILES="" for DRIVER in $* do FILE=$(find $MODULE_DIR/kernel/drivers -name "$DRIVER.ko" -print) ALL_FILES="$ALL_FILES $FILE" done } # # Given a filename, make $PRE_REQ a list of all files that must # be loaded into the kernel before the file can be loaded # succesfully. # find_dep() { FILE=$1 while read KEY REST do if [ "$KEY" = "$FILE:" ] then # echo ++++++++++ BINGO: $REST PRE_REQ="$REST" break fi done < $MODULE_DIR/modules.dep } # # Given a list of filenames, add before each file X a list # of all files that need to be loaded before X; return # list in $THE_WORKS. # find_dependencies() { THE_WORKS="" for FILE in $* do find_dep $FILE THE_WORKS="$THE_WORKS $PRE_REQ $FILE" done } # # Given a list of filenames, make a list where only # the first occurence of each element is present; # return in $THE_ESSENTIALS. # prune() { THE_ESSENTIALS="" for FILE in $* do BASE=`basename $FILE` BASE=${BASE%.ko} BASE=$(echo $BASE | sed -e 's/-/_/') eval "SEEN=\$SEEN_$BASE" if [ "$SEEN" = "" ] then THE_ESSENTIALS="$THE_ESSENTIALS $FILE" fi eval "SEEN_$BASE=1" done } main() { if [ "$1" != "" ] then VERSION="$1" else VERSION=`uname -r` fi SYSFS=/sys MODULE_DIR=/lib/modules/$VERSION MAP_USB=$MODULE_DIR/modules.usbmap MAP_PCI=$MODULE_DIR/modules.pcimap block_devs find_files $ALL_DRIVERS find_dependencies $ALL_FILES prune $THE_WORKS echo "recommended module load order:" for f in $THE_ESSENTIALS do echo " $f" done } main $*
