Sure, 4.4.0-104 kernel gave the same message as 4.4.0-109-generic and
didn't boot as well.
````
pyGRUB version 0.6
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x Ubuntu x
x Ubuntu, with Linux 4.4.0-109-generic x
x Ubuntu, with Linux 4.4.0-109-generic (recovery mode) x
x Ubuntu, with Linux 4.4.0-108-generic x
x Ubuntu, with Linux 4.4.0-108-generic (recovery mode) x
x Ubuntu, with Linux 4.4.0-104-generic x
x Ubuntu, with Linux 4.4.0-104-generic (recovery mode) x
x Ubuntu, with Linux 4.4.0-15-generic x
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, 'e' to edit the
commands before booting, 'a' to modify the kernel arguments
before booting, or 'c' for a command line.
Traceback (most recent call last):
File "/usr/lib/xen-4.4/bin/pygrub",
line 905, in <module>
os.write(fd, ostring)
OSError: [Errno 28] No space left on device
libxl: error: libxl_bootloader.c:628:bootloader_finished: bootloader failed
- consult logfile /var/log/xen/bootloader.37.log
libxl: error:
libxl_exec.c:118:libxl_report_child_exitstatus: bootloader [-1] exited with
error status 1
libxl: error:
libxl_create.c:1024:domcreate_rebuild_done: cannot (re-)build domain: -3
````
Michael Barkdoll
On Thu, Jan 11, 2018 at 8:53 PM, Joseph Salisbury <
joseph.salisb...@canonical.com> wrote:
> Can you see if the 4.4.0-104 kernel boots? It can be downloaded from:
>
> https://launchpad.net/~canonical-kernel-
> team/+archive/ubuntu/ppa/+build/13841470
>
>
> ** Tags added: kernel-key pti
>
> ** Changed in: linux (Ubuntu)
> Importance: Undecided => High
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1742789
>
> Title:
> Xen PV Guest won't boot latest kernel- OSError: [Errno 28] No space
> left on device
>
> Status in linux package in Ubuntu:
> Confirmed
>
> Bug description:
> # lsb_release -rd
> Description: Ubuntu 16.04.3 LTS
> Release: 16.04
>
>
> This problem is related to the latest kernel for meltdown spectre
> patches for xen pv guest and pygrub.
>
> This is the kernel that was attempting to be upgraded.
> linux-image-4.4.0-109-generic 4.4.0-109.132
>
>
> I tried to report this issue with the following command but it failed:
>
> ````
> ubuntu-bug linux
>
>
>
> *** Collecting problem information
>
>
>
> The collected information can be sent to the developers to improve the
>
> application. This might take a few minutes.
>
> ...............
>
>
>
> *** Problem in linux-image-4.4.0-15-generic
>
>
>
> The problem cannot be reported:
>
>
>
> This is not an official Ubuntu package. Please remove any third party
> package and try again.
>
>
>
> Press any key to continue...
>
>
>
> No pending crash reports. Try --help for more information.
>
> ````
>
>
> I have an Ubuntu 14.04 running Xen and a VM running Ubuntu 16.04 LTS
> that I'm having issues with it booting after upgrading the kernel for
> meltdown and spectre vulnerability.
>
> ````
> sudo apt-get update
> sudo apt-get upgrade
> sudo apt-get dist-upgrade
> ````
>
> Ubuntu 14.04 xen host reboot failure log:
> ````
> $ tailf /var/log/xen/servername.log
> Domain 24 needs to be cleaned up: destroying the domain
> Done. Rebooting now
> libxl: error: libxl_bootloader.c:628:bootloader_finished: bootloader
> failed - consult logfile /var/log/xen/bootloader.27.log
> libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus:
> bootloader [-1] exited with error status 1
> libxl: error: libxl_create.c:1024:domcreate_rebuild_done: cannot
> (re-)build domain: -3
> ````
>
>
> ````
> cat /var/log/xen/bootloader.27.log
> Using <class 'grub.GrubConf.Grub2ConfigFile'> to parse /grub/grub.cfg
> pyGRUB version 0.6
> ┌───────────────────────────────────────────────────────────
> ─────────────┐
> │ Ubuntu
> │
> │ Ubuntu, with Linux 4.4.0-109-generic
> │
> │ Ubuntu, with Linux 4.4.0-109-generic (recovery mode)
> │
> │ Ubuntu, with Linux 4.4.0-108-generic
> │
> │ Ubuntu, with Linux 4.4.0-108-generic (recovery mode)
> │
> │ Ubuntu, with Linux 4.4.0-104-generic
> │
> │ Ubuntu, with Linux 4.4.0-104-generic (recovery mode)
> │
> │ Ubuntu, with Linux 4.4.0-15-generic
> │
> └───────────────────────────────────────────────────────────
> ─────────────┘
> Use the ^ and ┴ keys to select which entry is highlighted.
> Press enter to boot the selected OS, 'e' to edit the
> commands before booting, 'a' to modify the kernel arguments
> before booting, or 'c' for a command line.
>
> Traceback (most recent call last):
> File "/usr/lib/xen-4.4/bin/pygrub", line 905, in <module>
> os.write(fd, ostring)
> OSError: [Errno 28] No space left on device
> ````
>
> I was able to get the vm to boot only by rapidly attempt to access the
> menu options and boot the old kernel (4.4.0-15-generic). Disk space
> and inodes seems fine on the host Ubuntu 14.04LTS and Ubuntu 16.04LTS
> Guest.
>
> Ubuntu 14.04 Host:
> ````
> root@host:/var/log/xen# df -hFilesystem Size Used Avail Use%
> Mounted on
> udev 197M 12K 197M 1% /dev
> tmpfs 42M 1.1M 41M 3% /run
> /dev/dm-0 104G 49G 50G 50% /
> none 4.0K 0 4.0K 0% /sys/fs/cgroup
> none 5.0M 0 5.0M 0% /run/lock
> none 208M 0 208M 0% /run/shm
> none 100M 0 100M 0% /run/user
> /dev/sda1 236M 40M 184M 18% /boot
> root@host:/var/log/xen# df -hi
> Filesystem Inodes IUsed IFree IUse% Mounted on
> udev 50K 721 49K 2% /dev
> tmpfs 52K 849 52K 2% /run
> /dev/dm-0 6.6M 86K 6.5M 2% /
> none 52K 2 52K 1% /sys/fs/cgroup
> none 52K 5 52K 1% /run/lock
> none 52K 1 52K 1% /run/shm
> none 52K 2 52K 1% /run/user
> /dev/sda1 61K 298 61K 1% /boot
> root@host:/var/log/xen#
> ````
>
> Ubuntu 16.04 PV Guest:
> ````
> root@guest:~# df -hFilesystem Size Used
> Avail Use% Mounted on
> udev 950M 0 950M 0% /dev
> tmpfs 200M 3.1M 197M 2% /run
> /dev/mapper/guest--vg-root 47G 4.3G 40G 10% /
> tmpfs 997M 0 997M 0% /dev/shm
> tmpfs 5.0M 0 5.0M 0% /run/lock
> tmpfs 997M 0 997M 0%
> /sys/fs/cgroup
> /dev/xvda1 472M 192M 256M 43% /boot
> tmpfs 200M 0 200M 0%
> /run/user/1000
> root@www2:~# df -hi
> Filesystem Inodes IUsed IFree IUse% Mounted on
> udev 238K 406 237K 1% /dev
> tmpfs 250K 491 249K 1% /run
> /dev/mapper/guest--vg-root 3.0M 177K 2.8M 6% /
> tmpfs 250K 1 250K 1% /dev/shm
> tmpfs 250K 7 250K 1% /run/lock
> tmpfs 250K 16 250K 1%
> /sys/fs/cgroup
> /dev/xvda1 122K 315 122K 1% /boot
> tmpfs 250K 4 250K 1%
> /run/user/1000
> root@guest:~#
> ````
>
> Here is the guest Ubuntu 16.04 OS kernels:
> ````
> root@guest:~# dpkg -l | grep linux-image-
> ii linux-image-4.4.0-104-generic 4.4.0-104.127
> amd64 Linux kernel image for version 4.4.0 on 64 bit x86
> SMP
> ii linux-image-4.4.0-108-generic 4.4.0-108.131
> amd64 Linux kernel image for version 4.4.0 on 64 bit x86
> SMP
> ii linux-image-4.4.0-109-generic 4.4.0-109.132
> amd64 Linux kernel image for version 4.4.0 on 64 bit x86
> SMP
> ii linux-image-4.4.0-15-generic 4.4.0-15.31
> amd64 Linux kernel image for version 4.4.0 on 64 bit x86
> SMP
> ii linux-image-extra-4.4.0-104-generic 4.4.0-104.127
> amd64 Linux kernel extra modules for version 4.4.0 on 64
> bit x86 SMP
> ii linux-image-extra-4.4.0-108-generic 4.4.0-108.131
> amd64 Linux kernel extra modules for version 4.4.0 on 64
> bit x86 SMP
> ii linux-image-extra-4.4.0-109-generic 4.4.0-109.132
> amd64 Linux kernel extra modules for version 4.4.0 on 64
> bit x86 SMP
> ii linux-image-extra-4.4.0-15-generic 4.4.0-15.31
> amd64 Linux kernel extra modules for version 4.4.0 on 64
> bit x86 SMP
> ii linux-image-generic 4.4.0.109.114
> amd64 Generic Linux kernel image
> root@guest:~#
> ````
>
> I'm currently able to boot to 4.4.0-15.31
>
> Here is my xen config file that I use to start the vm:
> ````
> # cat /etc/xen/guest.cfg
> # /var/lib/xen/images/ubuntu-netboot/xenial16LTS
>
>
> name = "guest"
>
>
> #kernel = "/var/lib/xen/images/ubuntu-netboot/xenial16LTS/vmlinuz"
> #ramdisk = "/var/lib/xen/images/ubuntu-netboot/xenial16LTS/initrd.gz"
> bootloader = "/usr/lib/xen-4.4/bin/pygrub"
>
>
> memory = 2048
> vcpus = 1
>
>
>
> disk = [ '/dev/host-disk2-vg/guest-disk,raw,xvda,rw' ]
>
>
>
> #
> # Networking
> #
> vif = [ 'ip=xxx.xxx.xxx.xxx ,mac=AA:BB:CC:DD:EE:FF' ]
>
>
> #
> # Behaviour
> #
> on_poweroff = 'destroy'
> on_reboot = 'restart'
> on_crash = 'restart'
> ````
>
> I also tried launching the vm with strace output of
> ````
> strace xl create -c file.cfg
> ````
> is here: https://pastebin.com/8ctS3TQh
>
> I'm a bit lost as to the cause of this... I was hoping to just upgrade
> the kernel for meltdown and spectre, but this issue I've never faced
> before. I think it might be related to pygrub and the new kernel? I don't
> know for sure though anything I can check to figure this out better? Thanks.
> ---
> AlsaDevices:
> total 0
> crw-rw---- 1 root audio 116, 1 Jan 11 10:24 seq
> crw-rw---- 1 root audio 116, 33 Jan 11 10:24 timer
> AplayDevices: Error: [Errno 2] No such file or directory
> ApportVersion: 2.20.1-0ubuntu2.15
> Architecture: amd64
> ArecordDevices: Error: [Errno 2] No such file or directory
> AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq',
> '/dev/snd/timer'] failed with exit code 1:
> DistroRelease: Ubuntu 16.04
> HibernationDevice: RESUME=/dev/mapper/www2--vg-swap_1
> IwConfig: Error: [Errno 2] No such file or directory
> Lspci:
>
> Lsusb: Error: command ['lsusb'] failed with exit code 1:
> Package: linux (not installed)
> PciMultimedia:
>
> ProcEnviron:
> LD_LIBRARY_PATH=<set>
> TERM=xterm-256color
> PATH=(custom, no user)
> LANG=en_US.UTF-8
> SHELL=/bin/bash
> ProcFB:
>
> ProcKernelCmdLine: root=/dev/mapper/hostname--vg-root ro quiet splash
> $vt_handoff
> ProcVersionSignature: Ubuntu 4.4.0-15.31-generic 4.4.6
> RelatedPackageVersions:
> linux-restricted-modules-4.4.0-15-generic N/A
> linux-backports-modules-4.4.0-15-generic N/A
> linux-firmware 1.157.14
> RfKill: Error: [Errno 2] No such file or directory
> Tags: xenial xenial
> Uname: Linux 4.4.0-15-generic x86_64
> UnreportableReason: The report belongs to a package that is not
> installed.
> UpgradeStatus: No upgrade log present (probably fresh install)
> UserGroups:
>
> _MarkForUpload: False
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/
> 1742789/+subscriptions
>
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1742789
Title:
Xen PV Guest won't boot latest kernel- OSError: [Errno 28] No space
left on device
Status in linux package in Ubuntu:
Confirmed
Bug description:
# lsb_release -rd
Description: Ubuntu 16.04.3 LTS
Release: 16.04
This problem is related to the latest kernel for meltdown spectre patches for
xen pv guest and pygrub.
This is the kernel that was attempting to be upgraded.
linux-image-4.4.0-109-generic 4.4.0-109.132
I tried to report this issue with the following command but it failed:
````
ubuntu-bug linux
*** Collecting problem information
The collected information can be sent to the developers to improve the
application. This might take a few minutes.
...............
*** Problem in linux-image-4.4.0-15-generic
The problem cannot be reported:
This is not an official Ubuntu package. Please remove any third party
package and try again.
Press any key to continue...
No pending crash reports. Try --help for more information.
````
I have an Ubuntu 14.04 running Xen and a VM running Ubuntu 16.04 LTS
that I'm having issues with it booting after upgrading the kernel for
meltdown and spectre vulnerability.
````
sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
````
Ubuntu 14.04 xen host reboot failure log:
````
$ tailf /var/log/xen/servername.log
Domain 24 needs to be cleaned up: destroying the domain
Done. Rebooting now
libxl: error: libxl_bootloader.c:628:bootloader_finished: bootloader failed -
consult logfile /var/log/xen/bootloader.27.log
libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus: bootloader [-1]
exited with error status 1
libxl: error: libxl_create.c:1024:domcreate_rebuild_done: cannot (re-)build
domain: -3
````
````
cat /var/log/xen/bootloader.27.log
Using <class 'grub.GrubConf.Grub2ConfigFile'> to parse /grub/grub.cfg
pyGRUB version 0.6
┌────────────────────────────────────────────────────────────────────────┐
│ Ubuntu │
│ Ubuntu, with Linux 4.4.0-109-generic │
│ Ubuntu, with Linux 4.4.0-109-generic (recovery mode) │
│ Ubuntu, with Linux 4.4.0-108-generic │
│ Ubuntu, with Linux 4.4.0-108-generic (recovery mode) │
│ Ubuntu, with Linux 4.4.0-104-generic │
│ Ubuntu, with Linux 4.4.0-104-generic (recovery mode) │
│ Ubuntu, with Linux 4.4.0-15-generic │
└────────────────────────────────────────────────────────────────────────┘
Use the ^ and ┴ keys to select which entry is highlighted.
Press enter to boot the selected OS, 'e' to edit the
commands before booting, 'a' to modify the kernel arguments
before booting, or 'c' for a command line.
Traceback (most recent call last):
File "/usr/lib/xen-4.4/bin/pygrub", line 905, in <module>
os.write(fd, ostring)
OSError: [Errno 28] No space left on device
````
I was able to get the vm to boot only by rapidly attempt to access the
menu options and boot the old kernel (4.4.0-15-generic). Disk space
and inodes seems fine on the host Ubuntu 14.04LTS and Ubuntu 16.04LTS
Guest.
Ubuntu 14.04 Host:
````
root@host:/var/log/xen# df -hFilesystem Size Used Avail Use% Mounted on
udev 197M 12K 197M 1% /dev
tmpfs 42M 1.1M 41M 3% /run
/dev/dm-0 104G 49G 50G 50% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 208M 0 208M 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/sda1 236M 40M 184M 18% /boot
root@host:/var/log/xen# df -hi
Filesystem Inodes IUsed IFree IUse% Mounted on
udev 50K 721 49K 2% /dev
tmpfs 52K 849 52K 2% /run
/dev/dm-0 6.6M 86K 6.5M 2% /
none 52K 2 52K 1% /sys/fs/cgroup
none 52K 5 52K 1% /run/lock
none 52K 1 52K 1% /run/shm
none 52K 2 52K 1% /run/user
/dev/sda1 61K 298 61K 1% /boot
root@host:/var/log/xen#
````
Ubuntu 16.04 PV Guest:
````
root@guest:~# df -hFilesystem Size Used Avail
Use% Mounted on
udev 950M 0 950M 0% /dev
tmpfs 200M 3.1M 197M 2% /run
/dev/mapper/guest--vg-root 47G 4.3G 40G 10% /
tmpfs 997M 0 997M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 997M 0 997M 0% /sys/fs/cgroup
/dev/xvda1 472M 192M 256M 43% /boot
tmpfs 200M 0 200M 0% /run/user/1000
root@www2:~# df -hi
Filesystem Inodes IUsed IFree IUse% Mounted on
udev 238K 406 237K 1% /dev
tmpfs 250K 491 249K 1% /run
/dev/mapper/guest--vg-root 3.0M 177K 2.8M 6% /
tmpfs 250K 1 250K 1% /dev/shm
tmpfs 250K 7 250K 1% /run/lock
tmpfs 250K 16 250K 1% /sys/fs/cgroup
/dev/xvda1 122K 315 122K 1% /boot
tmpfs 250K 4 250K 1% /run/user/1000
root@guest:~#
````
Here is the guest Ubuntu 16.04 OS kernels:
````
root@guest:~# dpkg -l | grep linux-image-
ii linux-image-4.4.0-104-generic 4.4.0-104.127
amd64 Linux kernel image for version 4.4.0 on 64 bit x86 SMP
ii linux-image-4.4.0-108-generic 4.4.0-108.131
amd64 Linux kernel image for version 4.4.0 on 64 bit x86 SMP
ii linux-image-4.4.0-109-generic 4.4.0-109.132
amd64 Linux kernel image for version 4.4.0 on 64 bit x86 SMP
ii linux-image-4.4.0-15-generic 4.4.0-15.31
amd64 Linux kernel image for version 4.4.0 on 64 bit x86 SMP
ii linux-image-extra-4.4.0-104-generic 4.4.0-104.127
amd64 Linux kernel extra modules for version 4.4.0 on 64 bit x86 SMP
ii linux-image-extra-4.4.0-108-generic 4.4.0-108.131
amd64 Linux kernel extra modules for version 4.4.0 on 64 bit x86 SMP
ii linux-image-extra-4.4.0-109-generic 4.4.0-109.132
amd64 Linux kernel extra modules for version 4.4.0 on 64 bit x86 SMP
ii linux-image-extra-4.4.0-15-generic 4.4.0-15.31
amd64 Linux kernel extra modules for version 4.4.0 on 64 bit x86 SMP
ii linux-image-generic 4.4.0.109.114
amd64 Generic Linux kernel image
root@guest:~#
````
I'm currently able to boot to 4.4.0-15.31
Here is my xen config file that I use to start the vm:
````
# cat /etc/xen/guest.cfg
# /var/lib/xen/images/ubuntu-netboot/xenial16LTS
name = "guest"
#kernel = "/var/lib/xen/images/ubuntu-netboot/xenial16LTS/vmlinuz"
#ramdisk = "/var/lib/xen/images/ubuntu-netboot/xenial16LTS/initrd.gz"
bootloader = "/usr/lib/xen-4.4/bin/pygrub"
memory = 2048
vcpus = 1
disk = [ '/dev/host-disk2-vg/guest-disk,raw,xvda,rw' ]
#
# Networking
#
vif = [ 'ip=xxx.xxx.xxx.xxx ,mac=AA:BB:CC:DD:EE:FF' ]
#
# Behaviour
#
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
````
I also tried launching the vm with strace output of
````
strace xl create -c file.cfg
````
is here: https://pastebin.com/8ctS3TQh
I'm a bit lost as to the cause of this... I was hoping to just upgrade the
kernel for meltdown and spectre, but this issue I've never faced before. I
think it might be related to pygrub and the new kernel? I don't know for sure
though anything I can check to figure this out better? Thanks.
---
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Jan 11 10:24 seq
crw-rw---- 1 root audio 116, 33 Jan 11 10:24 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.20.1-0ubuntu2.15
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq',
'/dev/snd/timer'] failed with exit code 1:
DistroRelease: Ubuntu 16.04
HibernationDevice: RESUME=/dev/mapper/www2--vg-swap_1
IwConfig: Error: [Errno 2] No such file or directory
Lspci:
Lsusb: Error: command ['lsusb'] failed with exit code 1:
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
LD_LIBRARY_PATH=<set>
TERM=xterm-256color
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB:
ProcKernelCmdLine: root=/dev/mapper/hostname--vg-root ro quiet splash
$vt_handoff
ProcVersionSignature: Ubuntu 4.4.0-15.31-generic 4.4.6
RelatedPackageVersions:
linux-restricted-modules-4.4.0-15-generic N/A
linux-backports-modules-4.4.0-15-generic N/A
linux-firmware 1.157.14
RfKill: Error: [Errno 2] No such file or directory
Tags: xenial xenial
Uname: Linux 4.4.0-15-generic x86_64
UnreportableReason: The report belongs to a package that is not installed.
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups:
_MarkForUpload: False
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1742789/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
