Xen 4.2 PV fails to boot -CURRENT PV domU
Hi, I'm trying to run a recent-CURRENT XEN i386 kernel in a Xen 4.2 PV domU. I had to add the following options to the kernel config file in order to compile the XEN kernel: makeoptions WITHOUT_MODULES+="mlx4 mlx4ib mlxen mthca" The resulting kernel compiles, but fails to boot. Attached is the log. Any ideas? Thanks, Lars root@six:/home/elars# xl create -c pvm-freebsd.cfg Parsing config from pvm-freebsd.cfg Daemon running with PID 5230 WARNING: loader(8) metadata is missing! GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb APIC: Using the MPTable enumerator. SMP: Added CPU 0 (BSP) Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #22 2be0e12-dirty: Mon Sep 9 11:10:21 CEST 2013 el...@stanley.muccbc.hq.netapp.com:/usr/home/elars/obj/i386.i386/usr/home/elars/src/sys/FAS3270 i386 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 WARNING: WITNESS option enabled, expect reduced performance. Xen reported: 3000.224 MHz processor. CPU: Intel(R) Xeon(R) CPU E5240 @ 3.00GHz (3000.22-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x1067a Family = 0x6 Model = 0x17 Stepping = 10 Features=0xbfe3fbff Features2=0xc0ce3bd AMD Features=0x2010 AMD Features2=0x1 Instruction TLB: 4 KB Pages, 4-way set associative, 128 entries 1st-level instruction cache: 32 KB, 8-way set associative, 64 byte line size 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size L2 cache: 6144 kbytes, 16-way associative, 64 bytes/line real memory = 268435456 (256 MB) Physical memory chunk(s): 0x0079f000 - 0x0fb23fff, 255348736 bytes (62341 pages) avail memory = 253251584 (241 MB) random device not loaded; using insecure entropy ULE: setup cpu 0 Event-channel device installed. null: nfslock: pseudo-device io: random: initialized mem: Pentium Pro MTRR support enabled xc0: on motherboard xenstore0: on motherboard Grant table initialized xen_et0: on motherboard xen_et0: Error 2 binding VIRQ_TIMER to VCPU 0 device_attach: xen_et0 attach returned 2 Device configuration finished. procfs registered panic: No usable event timer found! cpuid = 0 KDB: enter: panic [ thread pid 0 tid 10 ] Stopped at kdb_enter+0x3d: movl$0,kdb_why db> signature.asc Description: Message signed with OpenPGP using GPGMail
Current problem reports assigned to freebsd-xen@FreeBSD.org
Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description o kern/180788 xen[xen] [panic] XEN PV kernel 9.2-BETA1 panics on boot o kern/180403 xen[xen] Problems with GENERIC and XENHVM kernels with Xe o kern/180402 xen[xen] XEN kernel does not load in XenClient 4.5.5 o kern/179814 xen[xen] mountroot fails with error=19 under Xen on 9-STA o kern/176471 xen[xen] xn driver crash on detach o kern/176053 xen[xen] [patch] i386: Correct wrong usage of vsnprintf() o kern/175954 xen[xen] XENHVM xn network driver extreme packet loss dur o kern/175822 xen[xen] FreeBSD 9.1 does not work with Xen 4.0 o kern/175757 xen[xen] [patch] xen pvhvm looses keyboard input from VNC o kern/171873 xen[xen] xn network device floods warning in dmesg o kern/171118 xen[xen] FreeBSD XENHVM guest doesn't shutdown cleanly o kern/166174 xen[xen] Problems ROOT MOUNT ERROR o kern/165418 xen[xen] Problems mounting root filesystem from XENHVM o kern/164630 xen[xen] XEN HVM kernel: run_interrupt_driven_hooks: stil o kern/164450 xen[xen] Failed to install FreeeBSD 9.0-RELEASE from CD i o kern/162677 xen[xen] FreeBSD not compatible with "Current Stable Xen" o kern/161318 xen[xen] sysinstall crashes with floating point exception o kern/155468 xen[xen] Xen PV i386 multi-kernel CPU system is not worki o kern/155353 xen[xen] [patch] put "nudging TOD" message under boot_ver o kern/154833 xen[xen]: xen 4.0 - DomU freebsd8.2RC3 i386, XEN kernel. o kern/154473 xen[xen] xen 4.0 - DomU freebsd8.1 i386, XEN kernel. Not o kern/154472 xen[xen] xen 4.0 - DomU freebsd8.1 i386 xen kernel reboot o kern/154428 xen[xen] xn0 network interface and PF - Massive performan o kern/153674 xen[xen] i386/XEN idle thread shows wrong percentages o kern/153672 xen[xen] [panic] i386/XEN panics under heavy fork load o kern/153620 xen[xen] Xen guest system clock drifts in AWS EC2 (FreeBS o kern/153477 xen[xen] XEN pmap code abuses vm page queue lock o kern/153150 xen[xen] xen/ec2: disable checksum offloading on interfac o kern/152228 xen[xen] [panic] Xen/PV panic with machdep.idle_mwait=1 o kern/144629 xen[xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143398 xen[xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor o kern/143340 xen[xen] FreeBSD 8-RELEASE XEN pvm networking doesn't wor f kern/143069 xen[xen] [panic] Xen Kernel Panic - Memory modified after f kern/135667 xenufs filesystem corruption on XEN DomU system f kern/135421 xen[xen] FreeBSD Xen PVM DomU network failure - netfronc. f kern/135178 xen[xen] Xen domU outgoing data transfer stall when TSO i p kern/135069 xen[xen] FreeBSD-current/Xen SMP doesn't function at all f i386/124516 xen[xen] FreeBSD-CURRENT Xen Kernel Segfaults when config o kern/118734 xen[xen] FreeBSD 6.3-RC1 and FreeBSD 7.0-BETA 4 fail to b 39 problems total. ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: Xen 4.2 PV fails to boot -CURRENT PV domU
On 09/09/13 11:19, Eggert, Lars wrote: > Hi, > > I'm trying to run a recent-CURRENT XEN i386 kernel in a Xen 4.2 PV domU. > > I had to add the following options to the kernel config file in order to > compile the XEN kernel: > > makeoptions WITHOUT_MODULES+="mlx4 mlx4ib mlxen mthca" > > The resulting kernel compiles, but fails to boot. Attached is the log. Any > ideas? This is due to the unification of event channel implementations. I'm attaching a patch that should fix the issue (at least when using only one vcpu). The i386 Xen PV port doesn't fill the data in struct pcpu, so we end up with uninitialised values there which are required for the new event channel implementation. Maybe someone with more knowledge of the Xen i386 PV port can provide a better solution. If you plan to use the i386 PV port you will also need to revert r244237 in order for it to work. May I ask if there's anything that prevents you from switching to the PVHVM port? I wouldn't recommend using the i386 PV port. Roger. >From 0429a1ba5bdbca844e9a798df13396e1ee11aae0 Mon Sep 17 00:00:00 2001 From: Roger Pau Monne Date: Mon, 9 Sep 2013 11:53:15 +0200 Subject: [PATCH] xen: set acpi_id for i386 PV port Set a 'fake' acpi_id for the i386 PV port, it is needed in order to use VIRQs or IPI event channels. --- sys/i386/xen/mp_machdep.c |3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/sys/i386/xen/mp_machdep.c b/sys/i386/xen/mp_machdep.c index 274598f..4e4c07d 100644 --- a/sys/i386/xen/mp_machdep.c +++ b/sys/i386/xen/mp_machdep.c @@ -251,6 +251,9 @@ cpu_add(u_int apic_id, char boot_cpu) if (bootverbose) printf("SMP: Added CPU %d (%s)\n", apic_id, boot_cpu ? "BSP" : "AP"); + + /* Set the ACPI id (it is needed by VCPU operations) */ + pcpu_find(apic_id)->pc_acpi_id = apic_id; } void -- 1.7.7.5 (Apple Git-26) ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: Xen 4.2 PV fails to boot -CURRENT PV domU
Hi, thanks, let me try your suggestion/patch. On Sep 9, 2013, at 13:53, Roger Pau Monné wrote: > May I ask if there's anything that prevents you from switching to the > PVHVM port? I wouldn't recommend using the i386 PV port. I was thinking that performance would be better, and resource requirements lower. If that's not the case, I can certainly go the PVHVM route. Lars signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Xen 4.2 PV fails to boot -CURRENT PV domU
On 09/09/13 14:06, Eggert, Lars wrote: > Hi, > > thanks, let me try your suggestion/patch. > > On Sep 9, 2013, at 13:53, Roger Pau Monné wrote: >> May I ask if there's anything that prevents you from switching to the >> PVHVM port? I wouldn't recommend using the i386 PV port. > > I was thinking that performance would be better, and resource requirements > lower. If that's not the case, I can certainly go the PVHVM route. Performance will probably be better on most workloads with PVHVM if you have HAP (Intel's Extended Page Table or AMD's Nested Page Tables). And also, i386 PV port is limited to only one vcpu, while PVHVM will allow you to use as many vcpus as you want. PVHVM certainly has a higher resource requirement because it requires you to have a Qemu instance for each running VM, but that process is going to be idle most of the time (Qemu is mainly needed for booting). Roger. ___ freebsd-xen@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-xen To unsubscribe, send any mail to "freebsd-xen-unsubscr...@freebsd.org"
Re: Xen 4.2 PV fails to boot -CURRENT PV domU
Hi, On Sep 9, 2013, at 14:16, Roger Pau Monné wrote: > Performance will probably be better on most workloads with PVHVM if you > have HAP (Intel's Extended Page Table or AMD's Nested Page Tables). And > also, i386 PV port is limited to only one vcpu, while PVHVM will allow > you to use as many vcpus as you want. > > PVHVM certainly has a higher resource requirement because it requires > you to have a Qemu instance for each running VM, but that process is > going to be idle most of the time (Qemu is mainly needed for booting). thanks for the explanation. Is there an ETA for when the PVHVM code will be merged into the main -CURRENT tree? Lars signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Xen 4.2 PV fails to boot -CURRENT PV domU
Hi, On Sep 9, 2013, at 14:06, "Eggert, Lars" wrote: > thanks, let me try your suggestion/patch. I applied the patch and tried booting. That failed (as expected) with a "bad pte" panic: ... Trying to mount root from ufs:/dev/ada0s1a []... start_init: trying /sbin/init dumpon: /dev/da1s1a: No such file or directory pid 40 (sh), uid 0: exited on signal 11 TPTE at 0xbf840258 IS ZERO @ VA 0804b000 panic: bad pte cpuid = 0 KDB: enter: panic [ thread pid 40 tid 100032 ] Stopped at kdb_enter+0x3d: movl$0,kdb_why I then reverted r244237. The domU now boots a bit further, but there still seem to be significant issues. Below is the complete log. Lars root@six:/home/elars# xl create -c pvm-freebsd.cfg Parsing config from pvm-freebsd.cfg Daemon running with PID 13661 WARNING: loader(8) metadata is missing! GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb APIC: Using the MPTable enumerator. SMP: Added CPU 0 (BSP) Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #26 bc8458f-dirty: Mon Sep 9 15:11:32 CEST 2013 el...@stanley.muccbc.hq.netapp.com:/usr/home/elars/obj/i386.i386/usr/home/elars/src/sys/FAS3270 i386 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 Xen reported: 3000.214 MHz processor. CPU: Intel(R) Xeon(R) CPU E5240 @ 3.00GHz (3000.21-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x1067a Family = 0x6 Model = 0x17 Stepping = 10 Features=0xbfe3fbff Features2=0xc0ce3bd AMD Features=0x2010 AMD Features2=0x1 Instruction TLB: 4 KB Pages, 4-way set associative, 128 entries 1st-level instruction cache: 32 KB, 8-way set associative, 64 byte line size 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size L2 cache: 6144 kbytes, 16-way associative, 64 bytes/line real memory = 268435456 (256 MB) Physical memory chunk(s): 0x00564000 - 0x0fb23fff, 257687552 bytes (62912 pages) avail memory = 255848448 (243 MB) random device not loaded; using insecure entropy ULE: setup cpu 0 Event-channel device installed. nfslock: pseudo-device io: random: initialized mem: Pentium Pro MTRR support enabled null: xc0: on motherboard xenstore0: on motherboard Grant table initialized xen_et0: on motherboard Event timer "XENTIMER" frequency 10 Hz quality 950 Timecounter "XENTIMER" frequency 10 Hz quality 950 xen_et0: registered as a time-of-day clock (resolution 1000us, adjustment 5.0s) Device configuration finished. procfs registered Timecounters tick every 10.000 msec tcp_init: net.inet.tcp.tcbhashsize auto tuned to 2048 lo0: bpf attached xctrl0: on xenstore0 xenbusb_front0: on xenstore0 xenbusb_add_device: Device device/suspend/event-channel ignored. State 6 xn0: at device/vif/0 on xenbusb_front0 xn0: bpf attached xn0: Ethernet address: 00:16:3e:13:dd:43 xenbusb_back0: on xenstore0 xn0: backend features: feature-sg feature-gso-tcp4 xbd0: 384MB at device/vbd/769 on xenbusb_front0 xbd0: attaching as ada0 xbd0: features: write_barrier xbd0: synchronize cache commands enabled. GEOM: new disk ada0 [XEN] IPI cpu=0 port=6 vector=RESCHEDULE_VECTOR (0) [XEN] IPI cpu=0 port=7 vector=CALL_FUNCTION_VECTOR (1) TSC timecounter discards lower 1 bit(s) Timecounter "TSC-low" frequency 1500107000 Hz quality 800 Trying to mount root from ufs:/dev/ada0s1a []... start_init: trying /sbin/init dumpon: /dev/da1s1a: No such file or directory pid 42 (logger), uid 0: exited on signal 8 Floating point exception /etc/rc: WARNING: unable to specify /dev/da1s1a as a dump device Entropy harvesting: interrupts ethernet point_to_pointpid 50 (dd), uid 0: exited on signal 8 pid 53 (df), uid 0: exited on signal 8 Floating point exception pid 54 (ps), uid 0: exited on signal 8 Floating point exception pid 55 (date), uid 0: exited on signal 8 Floating point exception Floating point exception pid 58 (dd), uid 0: exited on signal 8 Floating point exception kickstart. eval: cannot create /etc/hostid: Read-only file system pid 76 (logger), uid 0: exited on signal 8 Floating point exception /etc/rc: WARNING: could not store hostuuid in /etc/hostid. pid 82 (newfs), uid 0: exited on signal 8 mount_mfs: mount exited with error code 1 pid 87 (newfs), uid 0: exited on signal 8 mount_mfs: mount exited with error code 1 Mounting /etc/fstab filesystems failed, startup aborted ERROR: ABORTING BOOT (sending SIGTERM to parent)! signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Xen 4.2 PV fails to boot -CURRENT PV domU
On 09/09/13 15:13, Eggert, Lars wrote: > Hi, > > On Sep 9, 2013, at 14:06, "Eggert, Lars" > wrote: >> thanks, let me try your suggestion/patch. > > I applied the patch and tried booting. That failed (as expected) with a "bad > pte" panic: > > ... > Trying to mount root from ufs:/dev/ada0s1a []... > start_init: trying /sbin/init > dumpon: /dev/da1s1a: No such file or directory > pid 40 (sh), uid 0: exited on signal 11 > TPTE at 0xbf840258 IS ZERO @ VA 0804b000 > panic: bad pte > cpuid = 0 > KDB: enter: panic > [ thread pid 40 tid 100032 ] > Stopped at kdb_enter+0x3d: movl$0,kdb_why > > I then reverted r244237. The domU now boots a bit further, but there still > seem to be significant issues. Below is the complete log. Applying the patch I've sent and reverting r244237 seems to be enough for me, I can get the guest to boot fine. This is the cmdline that I use for booting: boot_verbose=1,vfs.root.mountfrom=ufs:/dev/xbd0p2,kern.hz=100,hw.mca.enabled=0 WARNING: loader(8) metadata is missing! GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb APIC: Using the MPTable enumerator. SMP: Added CPU 0 (BSP) Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #72 06b2ac0: Mon Sep 9 15:54:59 CEST 2013 root@:/usr/obj/usr/src/sys/XEN i386 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 WARNING: WITNESS option enabled, expect reduced performance. Xen reported: 3066.888 MHz processor. CPU: Intel(R) Xeon(R) CPU W3550 @ 3.07GHz (3066.89-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x106a5 Family = 0x6 Model = 0x1a Stepping = 5 Features=0xbfe3fbff Features2=0x9ce3bd AMD Features=0x2810 AMD Features2=0x1 Data TLB: 4 KB pages, 4-way set associative, 64 entries 1st-level data cache: 32 KB, 8-way set associative, 64 byte line size L2 cache: 256 kbytes, 8-way associative, 64 bytes/line real memory = 536870912 (512 MB) Physical memory chunk(s): 0x0085f000 - 0x1f6a0fff, 518266880 bytes (126530 pages) avail memory = 515629056 (491 MB) random device not loaded; using insecure entropy ULE: setup cpu 0 Event-channel device installed. null: nfslock: pseudo-device io: random: initialized mem: Pentium Pro MTRR support enabled xc0: on motherboard xenstore0: on motherboard Grant table initialized xen_et0: on motherboard Event timer "XENTIMER" frequency 10 Hz quality 950 Timecounter "XENTIMER" frequency 10 Hz quality 950 xen_et0: registered as a time-of-day clock (resolution 1000us, adjustment 5.0s) Device configuration finished. procfs registered Timecounters tick every 10.000 msec tcp_init: net.inet.tcp.tcbhashsize auto tuned to 4096 lo0: bpf attached xctrl0: on xenstore0 xenbusb_front0: on xenstore0 xenbusb_add_device: Device device/suspend/event-channel ignored. State 6 xn0: at device/vif/0 on xenbusb_front0 xn0: bpf attached xn0: Ethernet address: 00:16:3e:47:d4:52 xenbusb_back0: on xenstore0 xn0: backend features: feature-sg feature-gso-tcp4 xbd0: 20480MB at device/vbd/51712 on xenbusb_front0 xbd0: features: flush, write_barrier xbd0: synchronize cache commands enabled. GEOM: new disk xbd0 TSC timecounter discards lower 1 bit(s) Timecounter "TSC-low" frequency 1533444000 Hz quality 800 WARNING: WITNESS option enabled, expect reduced performance. Trying to mount root from ufs:/dev/xbd0p2 []... WARNING: / was not properly dismounted WARNING: /: mount pending error: blocks 0 files 1 start_init: trying /sbin/init Setting hostuuid: ee840805-6f57-4031-b4b8-919d4ac8021a. Setting hostid: 0xf9f637d0. No suitable dump device was found. Entropy harvesting: interrupts ethernet point_to_pointsha256: /kernel: No such file or directory kickstart. swapon: /dev/ad0p3: No such file or directory Starting file system checks: ** SU+J Recovering /dev/xbd0p2 ** Reading 33554432 byte journal from inode 4. ** Building recovery table. ** Resolving unreferenced inode list. ** Processing journal entries. ** 17 journal records in 1024 bytes for 53.12% utilization ** Freed 0 inodes (0 dirs) 0 blocks, and 0 frags. * FILE SYSTEM MARKED CLEAN * Mounting local file systems:. Writing entropy file:. xn0: link state changed to DOWN xn0: link state changed to UP Starting Network: lo0 xn0. lo0: flags=8049 metric 0 mtu 16384 options=63 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff00 nd6 options=21 xn0: flags=8843 metric 0 mtu 1500 options=503 ether 00:16:3e:47:d4:52 nd6 options=29 media: Ethernet manual status: active Starting devd. Starting dhclient. DHCPDISCOVER on xn0 to 255.255.255.255 port 67 interval 4 DHCPOFFER from 172.16.1.1 DHCPREQUEST o
Re: Xen 4.2 PV fails to boot -CURRENT PV domU
Hi, On Sep 9, 2013, at 16:02, Roger Pau Monné wrote: > Applying the patch I've sent and reverting r244237 seems to be enough > for me, I can get the guest to boot fine. same here, now that I removed the "TARGET_CPUTYPE=core2" flag that had been left over in the local setup. Thanks, Lars signature.asc Description: Message signed with OpenPGP using GPGMail