** Changed in: ubuntu-power-systems
Status: Triaged => Incomplete
** Changed in: linux (Ubuntu)
Importance: Undecided => High
** Changed in: linux (Ubuntu)
Assignee: Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage) =>
Canonical Kernel Team (canonical-kernel-team)
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1810392
Title:
[Ubuntu 1810] Migrating Power9 guest back to source causes kernel Oops
inside guest
Status in The Ubuntu-power-systems project:
Incomplete
Status in linux package in Ubuntu:
New
Bug description:
---Problem Description---
Migrating back Power9 to source cause Kernel Oops inside guest
Contact Information = Balamuruhan S / balamuruh...@in.ibm.com
---uname output---
source and target host Kernel: 4.18.0-8-generic
Guest Kernel: 4.18.0-8-generic
Machine Type = Boston
---Debugger---
A debugger is not configured
---Steps to Reproduce---
Due to Bug 172310, Turn off the firewalld in source and target host
machines.
1. Define a healthy guest and placed image in NFS share.
2. Mounted the image location in destination and start the guest to
perform migration.
3. Perform migration from source to target,
08:07:54 INFO | Migrating VM avocado-vt-vm1-migration from qemu:///system to
qemu+ssh://9.40.194.13/system
08:07:54 DEBUG| Running virsh command: migrate --live --domain
avocado-vt-vm1-migration --desturi qemu+ssh://9.40.194.13/system --timeout 60
08:07:54 INFO | Running '/usr/bin/virsh -c 'qemu:///system' migrate --live
--domain avocado-vt-vm1-migration --desturi qemu+ssh://9.40.194.13/system
--timeout 60'
08:08:21 DEBUG| [stdout]
08:08:21 INFO | Command '/usr/bin/virsh -c 'qemu:///system' migrate --live
--domain avocado-vt-vm1-migration --desturi qemu+ssh://9.40.194.13/system
--timeout 60' finished with 0 after 26.9465551376s
08:08:21 DEBUG| status: 0
08:08:21 DEBUG| stdout:
08:08:21 DEBUG| stderr:
08:08:21 INFO | Migration exit status: 0
4. check VM uptime after successful migration,
08:07:44 INFO | Check VM uptime before migration: 57.66
5. Migration is successful from source to destination, now perform
migration back to source
08:08:49 INFO | Migrating VM avocado-vt-vm1-migration from
qemu+ssh://9.40.194.13/system to qemu:///system
08:08:49 DEBUG| Running virsh command: migrate --live --domain
avocado-vt-vm1-migration --desturi qemu:///system --timeout 60
08:08:49 INFO | Running '/usr/bin/virsh -c 'qemu+ssh://9.40.194.13/system'
migrate --live --domain avocado-vt-vm1-migration --desturi qemu:///system
--timeout 60'
08:09:22 DEBUG| [stdout]
08:09:22 INFO | Command '/usr/bin/virsh -c 'qemu+ssh://9.40.194.13/system'
migrate --live --domain avocado-vt-vm1-migration --desturi qemu:///system
--timeout 60' finished with 0 after 32.8640539646s
08:09:22 DEBUG| status: 0
08:09:22 DEBUG| stdout:
08:09:22 DEBUG| stderr:
08:09:22 INFO | Migration exit status: 0
6. check uptime of VM after migration and it is observed that VM have
gone for reboot
08:10:22 INFO | Check VM uptime after migrating back to source: 31.09
5. By checking the VM console Call Trace is observed as mentioned
below
The time stamp during the call trace observed inside guest matches with the
timestamp during which
migration is triggered.
Stack trace output:
no
Attachment:
1. VM sosreport
2. source and target sosreport
3. VM console log with call trace
4. Guest xml
5. Test job log
Oops output:
2018-10-26 08:08:26: root@ubuntu1810:~#
2018-10-26 08:09:11: [ 144.512549] Unable to handle kernel paging request
for data at address 0x71ff9672fa0e0989
2018-10-26 08:09:11: [ 144.512655] Faulting instruction address:
0xc0000000003af7cc
2018-10-26 08:09:11: [ 144.512741] Oops: Kernel access of bad area, sig: 11
[#1]
2018-10-26 08:09:11: [ 144.512789] LE SMP NR_CPUS=2048 NUMA pSeries
2018-10-26 08:09:11: [ 144.512859] Modules linked in: iscsi_target_mod
target_core_mod xt_CHECKSUM iptable_mangle ipt_MASQUERADE iptable_nat
nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack
ipt_REJECT nf_reject_ipv4 xt_tcpudp bridge stp llc ebtable_filter ebtables
ip6table_filter ip6_tables iptable_filter bpfilter kvm dm_multipath
scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto crct10dif_vpmsum sch_fq_codel
nfsd ib_iser auth_rpcgss rdma_cm nfs_acl iw_cm lockd ib_cm grace ib_core sunrpc
iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4
btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq
async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear
virtio_net net_failover crc32c_vpmsum virtio_blk virtio_scsi failover
2018-10-26 08:09:11: [ 144.513801] CPU: 5 PID: 1072 Comm: systemd-network
Not tainted 4.18.0-8-generic #9-Ubuntu
2018-10-26 08:09:11: [ 144.513872] NIP: c0000000003af7cc LR:
c0000000003af7b8 CTR: c000000000c9ea80
2018-10-26 08:09:11: [ 144.513943] REGS: c0000000f19ab910 TRAP: 0380 Not
tainted (4.18.0-8-generic)
2018-10-26 08:09:11: [ 144.514025] MSR: 8000000000009033
<SF,EE,ME,IR,DR,RI,LE> CR: 24002824 XER: 00000000
2018-10-26 08:09:11: [ 144.514102] CFAR: c0000000000174a0 IRQMASK: 0
2018-10-26 08:09:11: [ 144.514102] GPR00: c0000000003af670 c0000000f19abb90
c00000000178c800 0000000000000000
2018-10-26 08:09:11: [ 144.514102] GPR04: 0000000000000000 0000000000000007
0000000000000000 0000000000000006
2018-10-26 08:09:11: [ 144.514102] GPR08: b1ff967250115c09 0000000000000000
0000000000000000 c000000000d71000
2018-10-26 08:09:11: [ 144.514102] GPR12: c000000000c9ea80 c00000000fff8a00
0000000000000000 0000000000000000
2018-10-26 08:09:11: [ 144.514102] GPR16: 0000000000000000 0000000000000000
0000000000000000 0000000000000000
2018-10-26 08:09:11: [ 144.514102] GPR20: 0000000000000000 0000000000000000
0000000000000000 c0000000f19abde0
2018-10-26 08:09:11: [ 144.514102] GPR24: b1ff967250115c09 c0000000aa1f5580
71ff9672fa0e0989 c0000000fd017380
2018-10-26 08:09:11: [ 144.514102] GPR28: c000000000b5d188 00000000006000c0
c0000000aa1f5580 c0000000058a7c80
2018-10-26 08:09:11: [ 144.515013] NIP [c0000000003af7cc]
kmem_cache_alloc+0x30c/0x360
2018-10-26 08:09:11: [ 144.515102] LR [c0000000003af7b8]
kmem_cache_alloc+0x2f8/0x360
2018-10-26 08:09:11: [ 144.515186] Call Trace:
2018-10-26 08:09:11: [ 144.515224] [c0000000f19abb90] [c0000000003af670]
kmem_cache_alloc+0x1b0/0x360 (unreliable)
2018-10-26 08:09:11: [ 144.515337] [c0000000f19abc00] [c000000000b5d188]
sk_prot_alloc+0x58/0x200
2018-10-26 08:09:11: [ 144.515426] [c0000000f19abc50] [c000000000b5d3e4]
sk_alloc+0x54/0x220
2018-10-26 08:09:11: [ 144.515514] [c0000000f19abca0] [c000000000c9e8f0]
unix_create1+0xe0/0x270
2018-10-26 08:09:11: [ 144.515601] [c0000000f19abcf0] [c000000000c9eb98]
unix_create+0x118/0x150
2018-10-26 08:09:11: [ 144.515689] [c0000000f19abd10] [c000000000b58234]
__sock_create+0x134/0x2c0
2018-10-26 08:09:11: [ 144.515775] [c0000000f19abdc0] [c000000000b59f90]
__sys_socket+0x60/0x160
2018-10-26 08:09:11: [ 144.515860] [c0000000f19abe10] [c000000000b5a0bc]
sys_socket+0x2c/0x40
2018-10-26 08:09:11: [ 144.515948] [c0000000f19abe30] [c00000000000b288]
system_call+0x5c/0x70
2018-10-26 08:09:11: [ 144.516032] Instruction dump:
2018-10-26 08:09:11: [ 144.516088] 409efe1c 7c98c278 38a50001 7f3ac278
fb490000 f8aa0000 4bc67cc5 60000000
2018-10-26 08:09:11: [ 144.516195] 7fb9c040 419e0028 813f0020 e91f0138
<7d5a482a> 7d3a4a14 7d4a4278 7fa95040
2018-10-26 08:09:11: [ 144.516314] ---[ end trace ea4650f5afbe0331 ]---
Userspace tool common name: Qemu, Libvirt
Userspace rpm:
Qemu:
# dpkg -l | grep qemu
ii ipxe-qemu
1.0.0+git-20180124.fbe8c52d-0ubuntu4 all PXE boot firmware - ROM
images for qemu
ii ipxe-qemu-256k-compat-efi-roms
1.0.0+git-20150424.a25a16d-0ubuntu3 all PXE boot firmware - Compat
EFI ROM images for qemu
ii qemu-block-extra:ppc64el 1:2.12+dfsg-3ubuntu7
ppc64el extra block backend modules for qemu-system and qemu-utils
ii qemu-kvm 1:2.12+dfsg-3ubuntu7
ppc64el QEMU Full virtualization on x86 hardware
ii qemu-slof 20170724+dfsg-1ubuntu1
all Slimline Open Firmware -- QEMU PowerPC version
ii qemu-system-common 1:2.12+dfsg-3ubuntu7
ppc64el QEMU full system emulation binaries (common files)
ii qemu-system-data 1:2.12+dfsg-3ubuntu7
all QEMU full system emulation (data files)
ii qemu-system-gui 1:2.12+dfsg-3ubuntu7
ppc64el QEMU full system emulation binaries (user interface and audio
support)
ii qemu-system-ppc 1:2.12+dfsg-3ubuntu7
ppc64el QEMU full system emulation binaries (ppc)
ii qemu-utils 1:2.12+dfsg-3ubuntu7
ppc64el QEMU utilities
Libvirt:
# dpkg -l | grep libvirt
ii gir1.2-libvirt-glib-1.0:ppc64el 1.0.0-1
ppc64el GObject introspection files for the libvirt-glib library
ii gir1.2-libvirt-sandbox-1.0 0.5.1+git20160404-1
ppc64el GObject introspection files for the libvirt-sandbox library
ii libvirt-clients 4.6.0-2ubuntu3
ppc64el Programs for the libvirt library
ii libvirt-daemon 4.6.0-2ubuntu3
ppc64el Virtualization daemon
ii libvirt-daemon-driver-storage-gluster 4.6.0-2ubuntu3
ppc64el Virtualization daemon glusterfs storage driver
ii libvirt-daemon-driver-storage-rbd 4.6.0-2ubuntu3
ppc64el Virtualization daemon RBD storage driver
ii libvirt-daemon-driver-storage-sheepdog 4.6.0-2ubuntu3
ppc64el Virtualization daemon Sheedog storage driver
ii libvirt-daemon-driver-storage-zfs 4.6.0-2ubuntu3
ppc64el Virtualization daemon ZFS storage driver
ii libvirt-daemon-system 4.6.0-2ubuntu3
ppc64el Libvirt daemon configuration files
ii libvirt-dbus 1.2.0-1
ppc64el libvirt D-Bus API bindings
ii libvirt-dev:ppc64el 4.6.0-2ubuntu3
ppc64el development files for the libvirt library
ii libvirt-doc 4.6.0-2ubuntu3
all documentation for the libvirt library
ii libvirt-glib-1.0-0:ppc64el 1.0.0-1
ppc64el libvirt GLib and GObject mapping library
ii libvirt-glib-1.0-dev:ppc64el 1.0.0-1
ppc64el Development files for the libvirt-glib library
ii libvirt-ocaml 0.6.1.4-2build1
ppc64el OCaml bindings for libvirt (runtime)
ii libvirt-ocaml-dev 0.6.1.4-2build1
ppc64el OCaml bindings for libvirt (development files)
ii libvirt-sandbox-1.0-5 0.5.1+git20160404-1
ppc64el Application sandbox toolkit shared library
ii libvirt-sandbox-1.0-dev 0.5.1+git20160404-1
ppc64el Development files for libvirt-sandbox library
ii libvirt-sanlock 4.6.0-2ubuntu3
ppc64el Sanlock plugin for virtlockd
ii libvirt-wireshark 4.6.0-2ubuntu3
ppc64el Wireshark dissector for the libvirt protocol
ii libvirt0:ppc64el 4.6.0-2ubuntu3
ppc64el library for interfacing with different virtualization systems
ii libvirtodbc0 6.1.6+repack-0ubuntu9
ppc64el high-performance database - ODBC libraries
ii libvirtualpg-dev:ppc64el 2.0.0~rc0-1
ppc64el VirtualPG development files
ii libvirtualpg0:ppc64el 2.0.0~rc0-1
ppc64el VirtualPG shared library
ii python-libvirt 4.6.0-1
ppc64el libvirt Python bindings
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1810392/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
