Re: pxe booting qemu

2019-09-23 Thread Frantisek Rysanek
On 23 Sep 2019 at 20:25, john doe wrote:

> > I already have the preseed file working, for now, I do install Debian
> > from a usb key and at the boot prompt I fetch the preseed file from http
> > with something like:
> >
> > boot: auto console=ttyS0,115200n8 interface=auto url=
> >
> > The all purpous of this is to avoid having to type the above line
> > everytime I install Debian! :)
> > Actually, I'm stuck at this very step, How do I go from having my boot
> > media from the network?
> >
> 
> I have installed 'apt-cacher-ng' and and use that local mirror in the
> preseed file.
> 
> Now I need to understand how to add those argument:
> 
> console=ttyS0,115200n8 interface=auto url=tftp:///x.cfg
> 
> The first step is to avoid having to type the above arguments and make
> that the default If I don't select anything.
> 
> I'll also would like to thanks all the participent in this thread your
> intput/help is valuable.
> 

...well those arguments should be entered at the "kernel command 
line", which is a slightly misleading name for "someplace in your 
bootloader". Often you just append them on the same line after 
specifying an initrd file.
What bootloader do you have on your USB key?
Grub? Syslinux? Or something different?
You can even make a DOS-bootable USB stick and put syslinux or 
grub4dos on top of that.
Look for the bootloader's configuration. With Grub, this used to be 
menu.lst or grub.cfg, with syslinux it might be syslinux.cfg, or some 
such.
BTW if your goal is to kick-start a Debian installation that will 
download as much as possible from the network, and you can configure 
your own bootloader on a USB stick, you may as well download just the 
netinstall kernel and initrd (as described in my previous e-mail), 
put them on the USB stick and tell the bootloader about them...

Frank 



Re: pxe booting qemu

2019-09-23 Thread Frantisek Rysanek
On 23 Sep 2019 at 13:26, john doe wrote:
...
> > If the kernel in the netboot image gets out of sync with the kernel module 
> > packages,
> > then the modules won't load and the install will fail,
> > the usual symptoms are that messages about "missing symbols" appear
> > in the ctrl-alt-f4 console.
> >
> > To fix this, update the kernel and initrd on the netboot server.
> >
> 
> By doing 'apt-get update && apt-get upgrade'?
> In other words, I can't provide PXE booting for Stretch hosts when the
> PXE server is on Buster.
> 
No, that's not the way to do it.

When PXE-booting a Linux, e.g. a Debian Netinstall,
your pxelinux.cfg/default will contain something like:

label stretch64
  kernel stretch64/linux
  append initrd=stretch64/initrd.gz priority=low  --

Note that the two files, called "linux" (= the kernel) and initrd.gz, 
get served from some directory known to the TFTP server.
The classic name for that directory appears to be /tftpboot,
but even that is not set in stone, and the rest of the paths
(the directory structure) is definitely your own choice.
I.e. those are NOT the files living in /boot/ under your server's own 
filesystem root mount point. 

I get these two files for DHCP+TFTP Netboot / Debian Netinstall from:

http://ftp.debian.org/debian/dists/stretch/main/installer-amd64/curren
t/images/netboot/debian-installer/amd64/

If you notice, that your PXE-booting installer cannot find disk 
drives, that's because the modules package (downloaded from the 
Debian repo) is newer, not matching your installer kernel anymore 
= installer kernel has turned stale = you need to re-download the 
"linux" and "initrd.gz" files from the link above. 
Note that installer kernels are different from the "runtime, 
production" kernels that ultimately get installed on the target 
system, even speaking about the same Debian version.
Again the kernel and initrd that get served by TFTP for "PXE-booting" 
have nothing to do with the operating system on your server, you 
could just as well use some Windows-based DHCP+TFTP solution.
Not sure if the venerable jounin tftpd would work for instance:
http://tftpd32.jounin.net/tftpd32_download.html
(How would you want to "apt-get install linux-kernel" on that? :-)

This last spring in one case I needed to install the Ubuntu Disco 
Dingo. I used this method, and guess what: the initrd image in the 
Ubuntu repo was corrupt :-) So I ended up downloading the netinstall 
CD image (ISO) and booting that. That worked. I don't recall anymore
if I just booted the whole CD, or if I extracted its kernel and 
initrd and used those for direct PXE-boot. Either way, most of the 
packages got downloaded off the internet = I did not install the 
whole system from the local CD.

BTW I now have a fairly vanilla Debian 9 PXE-booting in the network
with NFSroot, mounted read-only by default (with overlayfs on top),
alternatively mounted read-write... booting into text mode or X.
That's four combinations, selectable by kernel cmdline arguments.
With autologin at the console and in X (XFCE).
Except for overlayfs, the rest went surprisingly smooth/seamless.
I'm planning to publish a howto of sorts (my notes that I collected
while driving along). Right at the moment I'm trying to run qemu 
on top of that, to facilitate some "OS deployment jobs"...

> > Oh, and by the way: If you try to pxeboot via UEFI, many 
> > tutorials etc. are plain wrong: You don't use pxelinux.0 in that 
> > case at all! You'll be loading bootnetx64.efi which then will 
> > load grubnetx64.efi. 

Michael, thanks for that teaser, and for all the links.
NetBooting in UEFI mode is working its way up my ToDo list.
The first thing I'm wondering about: is there a way for me to 
distinguish, at a DHCP server, between legacy and UEFI clients?
Because I'd need to serve a different boot file option (or boot 
server option, or both) to each of those groups of DHCP clients...
Unless that works, I'll have to stay with the "legacy PXE" booting.
Well I should probably start by RTFM and a bit of sniffing.

Frank Rysanek



Two questions on update_initramfs (in Debian 9 amd64 with systemd)

2019-09-02 Thread Frantisek Rysanek
Dear everyone,

I'm playing with NFSroot and overlayfs.
Which appears to work fine.
Based on other people's work, I've put together a script
(see it attached, including credits of my key "inspirations")
that goes into

   /etc/initramfs-tools/scripts/init-bottom/

Obviously this gets embedded in the initial ramdisk if I run

   update_initramfs -u -k my.kernel.version

and the point is that the script then runs at early stages of boot, 
where it can insert an overlayFS layer while the NFS mountpoint
stays "read only" all the time (if so desired).
There are a number of tangential notes that are off topic here, I may 
eventually leave a blog entry or something later on, when I'm happy 
with the result :-)

The NFS-booted system is a 64bit x86 Debian 9. The "source export" of 
the diskless NFS root lives in a subdirectory on a server (same 
distro). Say /var/NFSboot/stretch/. The NFS-booted Debian has been 
installed in the subdir using a pretty standard method based on 
debootstrap and some final touches in a chrooted shell.
No special sauce here.

I have two questions to ask:

Question #1:
It seems that my script, called overlay.sh, that I placed in
  /etc/initramfs-tools/scripts/init-bottom/
gets executed by update_initramfs "while building the initrd image" !
I.e., when I run update_initramfs in a chroot on my server, I end up 
with some erratic instances of overlayfs mounted on the server :-O
Does this have a rational explanation?
Apologies for being lazy to read the guts of update_initramfs.
If you've read the script, you already know my workaround for this.
(I test for the presence of an environment variable, that's only 
present "at build time" - in which case the interesting 
overlayfs-related stuff gets skipped.)

Question #2:
Anyone who's played with debootstrap or even just cloning an 
installed system to a different disk has probably reached a point
(based on numerous "Debian migration" howto's)
where you need to chroot into the target directory, but before it 
make sense to take that plunge, you first need to bind-mount a 
handful of system directories, served by the kernel via special 
filesystems. I have a tiny script to do the bind-mounts (and 
umounts!) for me, that goes like this:

   mount --bind --make-rslave /proc $DESTDIR/proc
   mount --bind --make-rslave /dev $DESTDIR/dev
   mount --bind --make-rslave /sys $DESTDIR/sys
   mount --bind --make-rslave /run $DESTDIR/run

   chroot $DESTDIR /bin/bash

   umount $DESTDIR/run
   umount $DESTDIR/sys
   umount $DESTDIR/dev
   umount $DESTDIR/proc

The older howtos use just --bind (or the equivalent "-o bind").
There's a Debian FAQ explaining that the "rslave" option is now 
needed, since the arrival of systemd.
  https://wiki.debian.org/systemd#Shared_bind_mounts
The "rslave" option serves to avoid unmounting both the "bind slave"
and the "bind source" mountpoints when just "the slave" mountpoint 
gets unmounted. And yes I have learned the hard way why the rslave 
option is needed, on my server when chrooting for update_initramfs in 
the NFSroot ;-)

So I now have the "rslave" option, and still, after I run 
update_initramfs, I get "side effects": the trailing part of my 
"chroot wrapper script" cannot unmount some of the mountpoints
on the grounds of some of them being busy. The error message from 
umount suggests fuser and one other command to reveal the culprits, 
but in my case those didn't help...

=> so the second question really is: any clues what gets launched by 
update_initramfs, that keeps some of the devices in the "chroot 
directory" open?

It's not a major problem, at least with --make-rslave my host server 
does not collapse anymore when umounting the bind-mounted
slave directories. And instead of the server-side chroot, 
alternatively I can maintain the diskless distro by mounting the 
NFSroot "rw" on a single diskless station... 
(direct RW mount of the NFS export, without an intermediate overlayfs 
layer).

Any clues to my questions would be welcome :-)

Frank Rysanek

The following section of this message contains a file attachment
prepared for transmission using the Internet MIME message format.
If you are using Pegasus Mail, or any other MIME-compliant system,
you should be able to save it or view it from within your mailer.
If you cannot, please ask your system administrator for assistance.

    File information ---
 File:  overlay.sh
 Date:  2 Sep 2019, 23:21
 Size:  2019 bytes.
 Type:  Unknown


overlay.sh
Description: Binary data