RE: specifying virtio block device as root filesystem for Debian S390X install?
Okay, looks like there are more developments to this -> -Original Message- From: Christian Borntraeger [mailto:borntrae...@de.ibm.com] Sent: Friday, November 13, 2015 7:37 AM To: kkwan@gmail.com; debian-s390@lists.debian.org Subject: Re: specifying virtio block device as root filesystem for Debian S390X install? On 11/04/2015 10:52 PM, Kevin Kwan (Personal) wrote: > Okay, I think I figured out the gist of the problem - > This being the Debian8 kernel/initrd environment, netcfg will be an > issue down the line, since it would not allow you to proceed with a > virtio NIC configure. This will need to be patched with installing > the netcfg 1.13 or > 1.134 udeb from the repo (1.13 is stable, 1.134 is testing/sid) > > Then all you conceivably have to do is to patch the dasd-postinst so > the machine will allow you to proceed So Debian 8 did support the s390-ccw-virtio machine and so should Debian 9 - which it does not. Correct? Yes. However, it looks like qemu 2.4.50/qemu-windows build 20150922 has an implementation of virtio-CCW that does not play well with Debian 9, but was okay for 8. This behavior was also observed for the qemu 2.4 package on the Debian unstable repo (so if you run qemu on stretch/sid, it's doing that as well). The newer release (2.4.90/qemu-windows build 20151105) seemed to have fixed the problem. This means that if you plan to run Debian 9 nightly it should have no problems. I am guessing that if qemu on sid is going to 2.4.1 (or wait for 2.5 to drop before Christmas),it should work on Debian itself as well. > > c) Unfortunately, that was not the technique I chose on this install - > the command line I used for the initial install was this: > > qemu-system-s390x.exe -machine type=s390-virtio -m 512 -kernel > kernel.debian9 -initrd initrd.debian9 -drive > file=linux.disk,if=none,id=disk0 -device > virtio-blk-s390,drive=disk0,id=virtio-disk0 -device > virtio-net-s390,netdev=net0 -netdev user,id=net0 -k en-us -redir > tcp:9022::22 > this is actually the old deprecated s390 machine, which uses an virtio transport that is more of a hack and not very well maintained. The default machine changed > to s390-ccw-virtio in qemu 2.4. Ah, yeah. The one that requires the s390-zipl.rom file to start up rather than s390-ccw.img on the bundle. I was wondering why it looked like such a one-off. > Yes, the old virtio transport only supports loading a disk that was prepared with a modified zipl (as provided in SLES11.2 or so). Only the s390-ccw-virtio machine has a working IPL from disks. Ah, looked like a bit of a hack. > zipl cannot clearly determine if the underlying device is a scsi disk, image file or FBA formatted DASD device. So it uses the (unlikely) FBA setup. QEMU only understands the zipl and eckd format. So solve this, you can force zipl to use the scsi layout, e.g. by adding targetbase=/dev/vda targetblocksize=512 targettype=SCSI targetoffset=0 to /etc/zipl.conf (or the command line parameters) This was changed in s390-tools 1.26, which now defaults to the scsi boot layout for virtio disks unless it finds a proper volume label (in that case it will write and ECKD boot layout) Okay, I was trying to figure out how to fix it, but then I already did a full 8 install defining a straight-up CCW-based machine instead of deprecated s390-virtio. Good to know anyways. Some other things to note is that defining the QEMU machine as SMP with CPU > 1 will result in a lockup during init on both Debian 8 and 9 (it would report the CPUs, initialize the random pool, but it won't/can't init the CPUs). I think there are some new code drops on qemu 2.5 that is supposed to address that. And frankly, I don't see the same issue in Hercules, so it looks fine to me. BTW, has anyone attempt to use Debian-s390x as a hypervisor for Hercules to launch Debian-S390X within KVM, or is that either really broken, or does that require z/VM? Any speculation as to whether it would work or not?
RE: specifying virtio block device as root filesystem for Debian S390X install?
Okay, I think I figured out the gist of the problem - a) If I boot up the machine with a debian9 (daily build) kernel using the following comamnd line: qemu-system-s390x.exe -machine type=s390-ccw-virtio-2.5 -m 512 -hda linux.disk -device virtio-net-ccw,netdev=net0 -netdev user,id=net0 -k en-us -redir tcp:9022::22 The VM will boot with issues initializing the virtio devices for the disk and the networking...looks to be a regression. b) If I boot up the machine with a debian8 (stable) kernel using the following comamnd line: qemu-system-s390x.exe -machine type=s390-ccw-virtio-2.5 -m 512 -kernel kernel.debian8 -initrd initrd.debian8 -hda linux.disk -device virtio-net-ccw,netdev=net0 -netdev user,id=net0 -k en-us -redir tcp:9022::22 The VM will boot with no problems, see the virtio hard drive and the networking. This being the Debian8 kernel/initrd environment, netcfg will be an issue down the line, since it would not allow you to proceed with a virtio NIC configure. This will need to be patched with installing the netcfg 1.13 or 1.134 udeb from the repo (1.13 is stable, 1.134 is testing/sid) Then all you conceivably have to do is to patch the dasd-postinst so the machine will allow you to proceed c) Unfortunately, that was not the technique I chose on this install - the command line I used for the initial install was this: qemu-system-s390x.exe -machine type=s390-virtio -m 512 -kernel kernel.debian9 -initrd initrd.debian9 -drive file=linux.disk,if=none,id=disk0 -device virtio-blk-s390,drive=disk0,id=virtio-disk0 -device virtio-net-s390,netdev=net0 -netdev user,id=net0 -k en-us -redir tcp:9022::22 Note that the peripherals are virtio-blk-s390 and virtio-net-s390 instead of being ccw devices. The install went seemingly well, everything went to /dev/vda1, zipl ran, and after the install was complete the QEMU session completed with no issues. I would assume that I can remove the kernel/initrd parameters on QEMU and it should IPL directly into Debian. HOWEVER, after the install when I tried to run the installed VM, I ran this: qemu-system-s390x.exe -machine type=s390-virtio -m 512 -drive file=linux.disk,if=none,id=disk0 -device virtio-blk-s390,drive=disk0,id=virtio-disk0 -device virtio-net-s390,netdev=net0 -netdev user,id=net0 -k en-us -redir tcp:9022::22 QEMU starts up, shows up with a blank console and then no discernable suggestion that it did anything. Well, so what I did was to modify the command line to this: qemu-system-s390x.exe -machine type=s390-ccw-virtio-2.5 -m 512 -hda linux.disk -device virtio-net-ccw,netdev=net0 -netdev user,id=net0 -k en-us -redir tcp:9022::22 Which remarkably enough worked. The console renders the usual Linux boot sequence, and it worked until it tried to mount the root filesystem, in which case it looks like the kernel commandline (/proc/cmdline) expected root=/dev/disk/by-path/ccw-virtio0-part1 However, since the machine booted with ccw peripherals, the disk is now: /dev/disk/by-path/ccw-0.0.0001-part1 (which is a symlink to ../../vda1) This will result in the init process bombing out to an initramfs mini-shell with a failure to find the root. This is a relatively easy fix since all I had to do is to make a symlink: cd /dev/disk/by-path/ ln -s ../../vda1 ccw-virtio0-part1 And then exit the initramfs, in which case the init process re-initializes. This time, the correct rootfs is referenced so init runs to completion and brings up Debian. That being said, I tried to correct the issue by running zipl. However, this time I run into an error: Error: Assume disk_type FBA for virtblk device, but please specify type manually: Virtblk device type not clearly determined. I am not sure how the original Debian install process ran zipl during the initial install, but how would I specify the proper type for zipl? -Original Message- From: Philipp Kern [mailto:pk...@debian.org] Sent: Wednesday, November 04, 2015 3:22 AM To: kkwan@gmail.com Cc: debian-s390@lists.debian.org Subject: RE: specifying virtio block device as root filesystem for Debian S390X install? On 2015-11-04 05:23, Kevin Kwan (Personal) wrote: > If I use the CCW version: > > qemu-system-s390x.exe -machine type=s390-ccw-virtio-2.5 -m 512 -hda > linux.disk -device virtio-net-ccw,netdev=net0 -netdev user,id=net0 -k > en-us -redir tcp:9022::22 > I just use -nographic on Linux and it does the right thing. > Then I'll see a console, followed by some complaints about missing > /sys/bus/ccw/devices/virtio/cutype (sysfs not populating info on the > CCW > bus?) Can you transcribe the actual failure you see? Kind regards Philipp Kern
RE: specifying virtio block device as root filesystem for Debian S390X install?
Okay, the installer finished, but I ran into some weirdness on the subsequent reboot - I modified the qemu command line to remove the kernel/initrd parameters (since they are no longer needed) - I would assume that ZIPL on /dev/vda1 should take care of the boot then. The following command line is used with the s390-zipl.rom file to boot the VM up - qemu-system-s390x.exe -machine type=s390-virtio -m 512 -drive file=linux.disk,if=none,id=disk0 -device virtio-blk-s390,drive=disk0,id=virtio-disk0 -device virtio-net-s390,netdev=net0 -netdev user,id=net0 -k en-us -redir tcp:9022::22 However, this time the QEMU virtual console virtcon0 shows a blank on start, I don't see any activities on the disk, sshd doesn't seem to show up so I cannot login and see what's up, and I can't tell what was happening. If I use the CCW version: qemu-system-s390x.exe -machine type=s390-ccw-virtio-2.5 -m 512 -hda linux.disk -device virtio-net-ccw,netdev=net0 -netdev user,id=net0 -k en-us -redir tcp:9022::22 Then I'll see a console, followed by some complaints about missing /sys/bus/ccw/devices/virtio/cutype (sysfs not populating info on the CCW bus?) Any pointers? -Original Message- From: Philipp Kern [mailto:pk...@debian.org] Sent: Monday, November 02, 2015 1:20 PM To: Kevin Kwan (Personal) Cc: debian-s390@lists.debian.org Subject: Re: specifying virtio block device as root filesystem for Debian S390X install? On Mon, Nov 02, 2015 at 10:46:39AM -0500, Kevin Kwan (Personal) wrote: > it turns out that since I was running the Debian 9/stretch > kernel/initrd, it was not loading the ext3/4 modules due to missing > modules. This is likely the reason why partman failed to give me an option to make the filesystem. > I had to boot the machine up on the Debian 8/jessie kernel/initrd, > shoehorn in the netcfg_1.134_s390x udeb to get rid of that virtio eth0 > error, and then mess with the dasd postint to bypass the DASD > designation. However, I was able to get partman working and got the > filesystem doing, and it's installing the base system now, so I suppose that it is okay. FWIW, there are daily builds on [1] that should have matching-up kernel/modules. But yes, that breaks quite often during testing's lifetime until it's cut to stable. Please do report bugs that you encounter. Kind regards Philipp Kern [1] http://d-i.debian.org/daily-images/s390x/daily/generic/
RE: specifying virtio block device as root filesystem for Debian S390X install?
Philipp: Thanks for the reply - it turns out that since I was running the Debian 9/stretch kernel/initrd, it was not loading the ext3/4 modules due to missing modules. This is likely the reason why partman failed to give me an option to make the filesystem. I had to boot the machine up on the Debian 8/jessie kernel/initrd, shoehorn in the netcfg_1.134_s390x udeb to get rid of that virtio eth0 error, and then mess with the dasd postint to bypass the DASD designation. However, I was able to get partman working and got the filesystem doing, and it's installing the base system now, so I suppose that it is okay. Thanks and regards, -Original Message- From: Philipp Kern [mailto:pk...@debian.org] Sent: Monday, November 02, 2015 4:38 AM To: kkwan@gmail.com Cc: debian-s390@lists.debian.org Subject: RE: specifying virtio block device as root filesystem for Debin S390X install? Hi, On 2015-11-02 00:24, Kevin Kwan (Personal) wrote: > I actually do run into fatal errors if I run this on my Debian x64 > machine, but I think it's possible issues with the S390X TCG in the > older builds of QEMU - actually have both the Windows and Linux > version running side-by-side > - the Windows version does get me further...although I do suspect > other issues down the line. qemu master actually segfaults[1] if you try to use s390x. [2] hopefully fixes that (untested). > How did you deactivate the s390 postinst on the installer shell? You can go back to the menu, go down and "Execute a shell". Then edit /var/lib/dpkg/info/s390-dasd.postinst with nano and add an "exit 0" after the shebang. I did push a fix to s390-dasd[3] yesterday night. If you use the unstable udebs, you should get it. Otherwise in five days. With that it will simply exit if there's no DASD channel found on the bus. I think that we could stuff that into stable as well. > The interesting side-bit is that I tried to define the machine as a > virtio-ccw machine using the following command, and then define the > disk and networking as channel devices: > > qemu-system-s390x.exe -machine type=s390-ccw-virtio-2.5 -m 512 -kernel > kernel.debian9 -initrd initrd.debian9 -hda linux.dsk -device > virtio-net-ccw,netdev=net0 -netdev user,id=net0 -k en-us -redir > tcp:9022::22 > > I was able to define it, but Debian cannot initialize it on boot > (virtio_ccw > 0.0.: Failed to set online: -5), so that's a bit of a dead end. Yeah, I had the same issue back in August. With current qemu and current kernel (4.1 worked, but also 4.2) it worked for me. Kind regards Philipp Kern [1] http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg7.html [2] http://lists.nongnu.org/archive/html/qemu-devel/2015-11/msg00025.html [3] https://packages.qa.debian.org/s/s390-dasd/news/20151101T222109Z.html
RE: specifying virtio block device as root filesystem for Debin S390X install?
Ah, I actually ran the Windows port of qemu (http://qemu.weilnetz.de/) - the build that I used was the latest build (2.4.50, 20150925). The command line that I used was the following: qemu-system-s390x.exe -machine type=s390-virtio -m 512 -kernel kernel.debian9 -initrd initrd.debian9 -drive file=linux.disk,if=none,id=disk0 -device virtio-blk-s390,drive=disk0,id=virtio-disk0 -device virtio-net-s390,netdev=net0 -netdev user,id=net0 -k en-us -redir tcp:9022::22 I actually do run into fatal errors if I run this on my Debian x64 machine, but I think it's possible issues with the S390X TCG in the older builds of QEMU - actually have both the Windows and Linux version running side-by-side - the Windows version does get me further...although I do suspect other issues down the line. How did you deactivate the s390 postinst on the installer shell? The interesting side-bit is that I tried to define the machine as a virtio-ccw machine using the following command, and then define the disk and networking as channel devices: qemu-system-s390x.exe -machine type=s390-ccw-virtio-2.5 -m 512 -kernel kernel.debian9 -initrd initrd.debian9 -hda linux.dsk -device virtio-net-ccw,netdev=net0 -netdev user,id=net0 -k en-us -redir tcp:9022::22 I was able to define it, but Debian cannot initialize it on boot (virtio_ccw 0.0.: Failed to set online: -5), so that's a bit of a dead end. -Original Message- From: Philipp Kern [mailto:pk...@debian.org] Sent: Sunday, November 01, 2015 5:54 AM To: Kevin Kwan (Personal) Cc: debian-s390@lists.debian.org Subject: Re: specifying virtio block device as root filesystem for Debin S390X install? On Wed, Oct 28, 2015 at 08:47:51PM -0400, Kevin Kwan (Personal) wrote: > Is it possible to specify a virtio block device as a root filesystem within > the Debian s390x installer? I specified a virtio disk for > qemu-system-s390x, and through some miracle was able to get the virtio > networking up and running via the latest debian s390x kernel/initrd, > and had no issues up until the disk partitioner. The partitioner was > able to see the disk (/dev/vda) and allows partitions to be created > (vda1/2/etc), but it does not allow it to be used. The partition > disks section only allows the option of swap space, physical volume for encryption, and "do not use". I wonder how you got into the disk partitioner in the first place. All my tries caused fatal errors in the DASD configuration part and it wouldn't let me proceed. So s390-dasd will need a fix to detect this situation. After I deactivated s390-dasd's postinst, it proceeded into the installer and offered all filesystem options. (I'm not saying that the result would work, just that the partitioner created filesystems correctly.) How did you invoke qemu? (Which seems to be incredibly fiddly, especially at HEAD.) What version? Kind regards Philipp Kern
specifying virtio block device as root filesystem for Debin S390X install?
Folks: Is it possible to specify a virtio block device as a root filesystem within the Debian s390x installer? I specified a virtio disk for qemu-system-s390x, and through some miracle was able to get the virtio networking up and running via the latest debian s390x kernel/initrd, and had no issues up until the disk partitioner. The partitioner was able to see the disk (/dev/vda) and allows partitions to be created (vda1/2/etc), but it does not allow it to be used. The partition disks section only allows the option of swap space, physical volume for encryption, and "do not use".
