Hi Remy, how do you ensure the ZFS kernel module is present on the Debian NFSroot? Adding
PACKAGES install zfs-dkms zfsutils-linux to /etc/fai/NFSROOT does not seem to build the ZFS kernel module. Robert Am 05.10.18 um 10:33 schrieb Rémy Dernat: > Hi, > > I am not doing this, but I think it should not be to difficult. I have > a similar script running as a post-install step; maybe you can use > something like this as a script : > > #!/bin/bash > > check_zpool() > { > zfs_value=`/sbin/zpool list|grep -ic "zfs_bigvol"` > } > > check_zpool > if [ $zfs_value -eq 0 ];then > ## Create a zfs zpool raid0 for every disks locally except > /dev/sda > /sbin/modprobe zfs > ls -l /dev/disk/by-id/|egrep -v "sr|sda|part" |awk '/scsi/ > {print $9}' > /tmp/disks-by-id.txt > while read disk; do > /sbin/parted -s /dev/disk/by-id/$disk mklabel GPT; > done < /tmp/disks-by-id.txt > all_disks_inline=`tr '\n' ' ' < /tmp/disks-by-id.txt` > # sometimes it fails to create the zpool > for i in {1..15}; do > check_zpool > if [ $zfs_value -eq 0 ];then > /sbin/zpool create -o ashift=12 -m /media/bigvol > zfs_bigvol $all_disks_inline > sleep 5 > /sbin/zfs set compression=lz4 zfs_bigvol > sleep 1 > /sbin/zfs mount -a > else > break > fi > done > rm -f /tmp/disks-by-id.txt > fi > > > This scripts creates a RAID0 volume using parted and disks id. Note > that if you have many many disks, RAID0 is obvisouly not recommanded > (perhaps you would switch to radiz3) and you may want to use ZFS with > disks by path. > > Another example using apt (this will create 3 zpool with 20 disks in > each of zpool): > > apt-get install zfsutils-linux parted > ls -l /dev/disk/by-id/|egrep -v "sr|sda|part" |awk '/scsi/ {print > $9}' |head -20 > /root/pool1.txt > ls -l /dev/disk/by-id/|egrep -v "sr|sda|part" |awk '/scsi/ {print > $9}' | awk 'NR>20&&NR<=40' > pool2.txt > ls -l /dev/disk/by-id/|egrep -v "sr|sda|part" |awk '/scsi/ {print > $9}' | awk 'NR>40&&NR<=60' > pool3.txt > mkdir -p /export/pool{1,2,3} > pool1_disks=`tr '\n' ' ' <pool1.txt` > pool2_disks=`tr '\n' ' ' <pool2.txt` > pool3_disks=`tr '\n' ' ' <pool3.txt` > apt-get install parted > while read disk; do /sbin/parted -s /dev/disk/by-id/$disk mklabel > GPT; done < pool1.txt > while read disk; do /sbin/parted -s /dev/disk/by-id/$disk mklabel > GPT; done < pool2.txt > while read disk; do /sbin/parted -s /dev/disk/by-id/$disk mklabel > GPT; done < pool3.txt > zpool create pool1 -o ashift=12 -m /export/pool1 raidz3 $pool1_disks > zpool create pool2 -o ashift=12 -m /export/pool2 raidz3 $pool2_disks > zpool create pool3 -o ashift=12 -m /export/pool3 raidz3 $pool3_disks > > > 4 kernel modules are necessary (generally, you will only see spl and > zfs) : zfs.ko, spl.ko, znvpair.ko, zcommon.ko > > If you need to build ZFS from source, you will need git (or download > the tar.gz version with wget) : > > apt-get install zlib1g-dev uuid-dev libattr1-dev libblkid-dev > libselinux-dev libudev-dev libssl-dev parted lsscsi wget ksh gdebi > apt-get install build-essential autoconf libtool gawk alien > fakeroot linux-headers-$(uname -r) > mkdir -p /opt/zfs-source > cd $_ > git clone https://github.com/zfsonlinux/spl > git clone https://github.com/zfsonlinux/zfs > cd spl > git checkout master > sh autogen.sh > ./configure > make -s -j$(nproc) > make install > cd ../zfs > git checkout master > sh autogen.sh > ./configure > make -s -j$(nproc) > make install > echo "/lib64" > /etc/ld.so.conf.d/zfs.conf > echo "/usr/local/lib" >> /etc/ld.so.conf.d/zfs.conf > ldconfig > updatedb > locate spl.ko |xargs -I{} ls -l {} > # overwritting old kernel modules with more recent ones > cp /opt/zfs-source/spl/module/spl/spl.ko /lib/modules/$(uname > -r)/extra/spl/spl/spl.ko > cp /opt/zfs-source/spl/module/spl/spl.ko /lib/modules/$(uname > -r)/kernel/zfs/spl/spl.ko > locate zfs.ko |xargs -I{} ls -l {} > cp /opt/zfs-source/zfs/module/zfs/zfs.ko /lib/modules/$(uname > -r)/kernel/zfs/zfs/zfs.ko > locate znvpair.ko |xargs -I{} ls -l {} > cp /opt/zfs-source/zfs/module/nvpair/znvpair.ko > /lib/modules/$(uname -r)/kernel/zfs/nvpair/znvpair.ko > locate zcommon.ko |xargs -I{} ls -l {} > cp /opt/zfs-source/zfs/module/zcommon/zcommon.ko > /lib/modules/$(uname -r)/kernel/zfs/zcommon/zcommon.ko > modprobe spl > modprobe znvpair > modprobe zcommon > modprobe zfs > > > So you can imagine in the `package_config` a class with all the > necessary packages and in `scripts` the same class which is running > other commands. I don't think there is anything specific in FAI to use > ZFS with the FAI `disk_config`. > > > Kind regards, > Rémy > > Le ven. 5 oct. 2018 à 09:06, Steffen Grunewald > <steffen.grunew...@aei.mpg.de <mailto:steffen.grunew...@aei.mpg.de>> a > écrit : > > Good morning, > > once again I have browsed multiple years of the list archive, > without finding > anything useful, but it's Firday and I may have gone blind over > the past week. > > What I'm looking for is FAI installation details to set up a root > filesystem > on ZFS (two mirrored disks, possibly one spare) on a number of new > machines. > (I know that FAI isn't mandatory for this, but never underestimate the > documentation effect of having the proper classes.) > > - Which additions to the nfsroot are required / recommended? > - Which packages must be added to the machine package configuration? > - Which hooks should I use to create the root filesystem? > > There must be recipes already existing - I'm perhaps just unable > to find them > (I have found the zfsonlinux wiki pages, but they don't know about > FAI; and > the only place where ZFS is mentioned on fai-project.org > <http://fai-project.org> is the roadmap.) > > Any pointer is appreciated. > > Thanks, > Steffen > > -- > Steffen Grunewald, Cluster Administrator > Max Planck Institute for Gravitational Physics (Albert Einstein > Institute) > Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany > ~~~ > Fon: +49-331-567 7274 > Mail: steffen.grunewald(at)aei.mpg.de <http://aei.mpg.de> > ~~~ >