Ubuntu-22.04
Hi,
I updated my current FAI setup to install the recently released
Ubuntu-22.04 with it.
Ubuntu-22.04 now uses the zstd(1) compression program for packages. FAI
can't install these packages out of the box because debootstrap on
Debian-11 doesn't know about this compression format. Here are the things
I changed.
1) Create the directory /srv/fai/config/debootstrap in my config space and
populate it with the content of the original /usr/share/debootstrap.
I didn't want to mess with the original files installed by Debian, so
I use a local copy that I can update for FAI.
2) Apply the attached patch "functions.patch" to
/srv/fai/config/debootstrap/functions. This will add support for zstd
compression.
3) Apply the attached patch "mk-basefile.patch" to the mk-basefile script.
Now debootstrap will a) use the patched functions and b) use "ar" to
unpack the Ubuntu-22.04 packages. Make sure you have the packages zstd
and binutils installed on the FAI server to provide the needed binaries.
These are the changes I made to create a basefile for Ubuntu-22.04.
The patches do not include other changes to support Ubuntu-22.04 with
mk-basefile to prevent to much noise. Obviously the list of packages and
the name of the OS need to be configured. Other files/scripts in the
config space also need to be updated.
Maybe this will get you started when you plan to include the new Ubuntu
release in your FAI setup.
Regards,
Stefan
--- functions.orig 2019-01-09 14:00:04.0 +0100
+++ functions 2022-04-12 14:12:51.590232162 +0200
@@ -958,6 +958,7 @@
case "$tarball" in
control.tar.gz) cat_cmd=zcat ;;
control.tar.xz) cat_cmd=xzcat ;;
+ control.tar.zst) cat_cmd=zstdcat ;;
control.tar)cat_cmd=cat ;;
*) error 1 UNKNOWNCONTROLCOMP "Unknown compression type for %s in %s" "$tarball" "$pkg" ;;
esac
@@ -980,6 +981,7 @@
data.tar.gz) cat_cmd=zcat ;;
data.tar.bz2) cat_cmd=bzcat ;;
data.tar.xz) cat_cmd=xzcat ;;
+ data.tar.zst) cat_cmd=zstdcat ;;
data.tar) cat_cmd=cat ;;
*) error 1 UNKNOWNDATACOMP "Unknown compression type for %s in %s" "$tarball" "$pkg" ;;
esac
diff --git i/basefiles/mk-basefile w/basefiles/mk-basefile
index 085b9a48..e3c2f430 100755
--- i/basefiles/mk-basefile
+++ w/basefiles/mk-basefile
@@ -15,8 +15,9 @@
# Scientific Linux Cern 5/6
#
# Packages you might want to install to use this command:
-# debootstrap, rinse, xz-utils
+# debootstrap, rinse, xz-utils, zstd
+export DEBOOTSTRAP_DIR=/srv/fai/config/debootstrap
# Define your local mirros here
# For the first stage, set the CentOS/SLC mirror in /etc/rinse/rinse.conf
@@ -195,7 +196,7 @@ debgeneric() {
else
arch=i386
fi
- debootstrap --arch $arch ${exc} $inc $dist $xtmp $mirror
+ debootstrap --extractor=ar --arch $arch ${exc} $inc $dist $xtmp $mirror
fi
cleanup-deb
tarit
--
Lisp without parentheses is like Python without whitespace.
Re: Setting up third party apt repositories
Matthew Pounsett writes: > I don't expect this to work with the above setup, because scripts are > run after packages are added. I was referring to the fcopy command in the task_repository function from FAI. So it is done very early when all the repository configuration is done. I'm actually using the described way to install the Puppet-Agent from a 3rd party repo on all of my machines. -- Stefan
Re: Setting up third party apt repositories
Matthew Pounsett writes: > Is there a "proper" way to do this? It seems like I've got a catch-22 > where in order to get indexes from https-based repositories I need to > already have some of the OS in place, but the required base packages > aren't installed yet when the "repository" and "updatebase" hooks > fire. Have you tried to install Bind after the OS is done? Add the repository file as files/etc/apt/sources.list.d/bind.list/BIND to the config space. There is a fcopy command for /etc/apt in the main scripts so this will be copied to the node if the BIND class is set. Also add an entry to package_config/DEBIAN: PACKAGES install BIND bind9 Now using the class BIND should add the required repository and install the package. I'm using this setup to install the Puppet-Agent from a 3rd party repo if requested by the node classes. -- Stefan
Re: Some troubles with grub with buster point release 10.8
Thomas Lange writes: > Ooh, it seems more complicated. Your code does not work on a machine > with one SATA disk: > $ udevadm info -r --query=symlink /dev/sda > /dev/disk/by-id/wwn-0x5001517803d00e48 > /dev/disk/by-id/ata-INTEL_SSDSC2CW120A3_CVCV249103KG120BGN > /dev/disk/by-path/pci-:00:1f.2-ata-1 > $ udevadm info -r --query=symlink /dev/sda | sed -n > 's;.*\(/dev/disk/by-id/\S\+\).*;\1;p' > /dev/disk/by-id/wwn-0x5001517803d00e48 Oh... So "sed" might not be a good idea if you want all entries. > Before changing the code, I like to know if there are more unknows > cases we should keep an eye on. This is on VMware using a paravirtualized LSI SCSI controller: # udevadm info -r --query=symlink /dev/sda /dev/disk/by-path/pci-:00:10.0-scsi-0:0:0:0 No entry below /dev/disk/by-id at all. -- Stefan
Re: Some troubles with grub with buster point release 10.8
Thomas Lange writes: > Thanks for the hint. It's now fixed. What about SCSI controllers? They are missed by the code as grep currently looks for "ata". Maybe like this: # udevadm info -r --query=symlink /dev/sda | sed -n 's;.*\(/dev/disk/by-id/\S\+\).*;\1;p' /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-0 -- Stefan
Re: Some troubles with grub with buster point release 10.8
Hi! Just starting to play with this... Too me it looks like the "grep" in mbrdevices=$($ROOTCMD udevadm info -r --query=symlink $BOOT_DEVICE | grep -Po '/dev/disk/by-id/ata\S+') will emit an exit status 1 if it does not find a match. This is propagated as the script error status (due to the trap). Maybe an additional "| cat" at the end could be used to gobble the status of "grep"... -- Stefan
Re: FAI with pre-release bullseye yielding inconsistent network names
fai1...@macrotex.net writes: > I am already using that patch. The problem seems to be that the altname > returned by "ip link show eth0" in 30-interface is "enp11s0" and hence > that is the filename what 30-network puts into /etc/network/interfaces.d, but > when the new system boots the name in /sys/class/net is "ens192". > Why is there this mismatch? Is it an issue with bullseye itself? Maybe "ip link show" can't really be trusted here... There is a documented priority list[1] and the slot based name (e.g. ens192) is preferred over the path based name (e.g. enp0s0). It seems the path based name is used when booting the NFSROOT while the slot based name is used when the machine is rebooted after installation. To me that looks like a problem with the NFSROOT like missing kernel modules or libraries. Maybe Bullseye really has changed more than previous releases in this area? [1] https://wiki.debian.org/NetworkInterfaceNames -- Stefan
Re: FAI with pre-release bullseye yielding inconsistent network names
fai1...@macrotex.net writes: > I do the initial FAI imaging using a buster NFS root and install buster in > /target. When I then run "chroot /target udevadm info /sys/class/net/eth0" > I see about 27 lines including a bunch of ID_NET_NAME_* values. > I do the above again, but this time with a bullseye NFS root and install > bullseye in /target. When I then run "chroot /target udevadm info > /sys/class/net/eth0" I see only about 6 lines and, in particular, no > ID_NET_NAME_* values. This is the reason why the script does not pick up the correct name. > I created the bullseye NFS root in the same way as for buster (I think). Why > is there this large difference in the outputs of udevadm info? Does the following show more output on Bullseye? # chroot /target udevadm info --query=all /sys/class/net/eth0 -- Stefan
Re: Building a multi-OS install ISO image?
Thomas Lange writes: > > Isn’t rinse already doing the heavy lifting in donwloading all specified > > packages? Createrepo then creates the metadata for a directory full of > > existing RPMs: > rinse does no resolve dependencies. I’ve looked into this: there is a program "repotrack" (available as part of the Debian yum-utils package). This tool takes the name of a RPM package on the commandline and will download the package and all dependencies from a configured RPM repo into a given directory. So we can do this in a loop for all packages identified by install_packages. Finally we call "createrepo" (from the Debian package of the same name) to turn the RPM files into a valid yum repo. Attached is proof of concept script "fai-mirror-rpm". It expects a valid repo configuration in /etc/yum.repos.d as used with RPM based distributions. Currently unsolved issues: * for multiple operating systems we would need different repo configs in /etc/yum.repos.d and a way to tell repotrack the correct ones. There is an option for that but the script currently does not handle this case. * Yum has a concept of package groups (in my FAI scripts/hooks I do a "yum groupinstall core" or "yum groupinstall minimal-environment" to install the base system). The repo created by "createrepo" can only create a groups-enabled repo by providing a file in XML format on the commandline. Without this the repo has no data for yum to decide which packages belong to a group. So this would have to be solved to be able to use package groups during the installation. Maybe this can be a starting point if somebody wants to do more work on the topic. fai-mirror-rpm.gz Description: fai-mirror-rpm.gz -- Stefan
Re: Building a multi-OS install ISO image?
Hi! Thomas Lange writes: > I would call fai-mirror twice. First create a Debian mirror in > /tmp/mirror/debian, then an Ubuntu mirror /tmp/mirror/ubuntu. Later > call fai-cd -m /tmp/mirror. You have to adjust the sources.list (one > for Debian, one for Ubuntu) for the installation Ah, ok. I will have a look what needs to be done for this. Thanks! > I still do not know a tool which creates a CenOS package mirror from a > list of package names. Isn’t rinse already doing the heavy lifting in donwloading all specified packages? Createrepo then creates the metadata for a directory full of existing RPMs: https://manpages.debian.org/buster/createrepo/createrepo.8.en.html > Do you know the feature of showing a menu with the list of profiles? > It looks like this: > https://fai-project.org/slideshow/page105.html Yes, I already have a customized menu. > No. The fai-cd command always includes the whole config space. It > cannot select which files/classes/profile to include. I was planning to copy my config space to a temporary directory, remove the unneeded files and then reference this temporary copy when building the CD image. I’m scripting the CD build process, so there would be possibility to do this. But it looks simpler to go for one CD including everything so there is no need to mess around with a temporary copy. Thanks for your help! -- Stefan
Building a multi-OS install ISO image?
Hi! I have a FAI server running and can successfully install VMware virtual machines with different operating systems and releases. Now I'm looking into preparing a CD to do the same with real hardware. We have a network using many different VLANs and I can use the VMware API to create a VM in the FAI VLAN for doing the install and move it to the correct VLAN later. This is not feasible for a hardware system and therefore I need to do the installation without any working network connection. I used fai-mirror and fai-cd to create one image for Debian-10 and one image for Ubuntu-18.04. Now I'm looking into two aspects to make it easier and maybe have someone less experienced do the installation: Idea 1: have only one CD image and use profiles to select the OS to install Is this doable? It seems that fai-mirror can only create a mirror for one OS. Debian and Ubuntu have multiple packages using the same name and trying to put both operating systems into the same mirror directory does not seem to work. How about a non-Debian mirror? Is there any documentation about creating a CentOS mirror usable with fai-cd? Idea 2: use one CD image for one OS but use profiles to configure options I want to use profiles to decide if the machine should have the Puppet agent and configuration added during installation. I already have this working for VMs where I can add the class to the kernel commandline for a specific node. I would like to avoid an additional CD image for this and it looks that profiles could be the way to ask the user about this. It seems that all profiles in the config space are included in the image. Is there an easy way to include one set of profiles for a Debian CD and another set of profiles for an Ubuntu CD? I'm thinking about creating a temporary copy of the config space and include that after removing the unneeded profiles. I am missing something here? -- Stefan
