Re: FreeBSD PVHVM call for testing
Hi, On 19 Jun 2013, at 18:52, Justin T. Gibbs wrote: >> I guess the kernel-toolchain takes a long time to build…and from what I can >> see it does a clean before rebuilding also. >> >> I'm doing the kernel-toolchain step only now and will report how long it >> took. This seems to be it, that took roughly 2 hours to build. The actual kernel is probably pretty fast in building. > Oh. Without any parallelism (-j X), the build will take a really long time. > Even with only one core, you'll get a large speedup by performing a parallel > build since many steps of the build are I/O bound. Neither of the systems actually had parallelism defined, either as flag or in /etc/make.conf Jeroen. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: FreeBSD PVHVM call for testing
On Jun 19, 2013, at 10:50 AM, Jeroen van der Ham wrote: > Hi, > > On 19 Jun 2013, at 18:15, Justin T. Gibbs wrote: >> >> I've never seen a kernel build take 2 hours, much less 2 hours *longer*. >> Are you talking about buildworld? It would be interesting to know your >> results building stable/9 sources in your 10 environment to see if this is >> just due to "build bloat" or a true performance regression. >> > > I copy/pasted the command from the wiki: > >> # make kernel-toolchain && make buildkernel KERNCONF=XENHVM && make >> installkernel KERNCONF=XENHVM > > On the stable/9 I only did > >> make buildkernel KERNCONF=XENHVM && make installkernel KERNCONF=XENHVM > > > I guess the kernel-toolchain takes a long time to build…and from what I can > see it does a clean before rebuilding also. > > I'm doing the kernel-toolchain step only now and will report how long it took. > > Jeroen. Oh. Without any parallelism (-j X), the build will take a really long time. Even with only one core, you'll get a large speedup by performing a parallel build since many steps of the build are I/O bound. -- Justin ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: FreeBSD PVHVM call for testing
Hi, On 19 Jun 2013, at 18:15, Justin T. Gibbs wrote: > > I've never seen a kernel build take 2 hours, much less 2 hours *longer*. Are > you talking about buildworld? It would be interesting to know your results > building stable/9 sources in your 10 environment to see if this is just due > to "build bloat" or a true performance regression. > I copy/pasted the command from the wiki: > # make kernel-toolchain && make buildkernel KERNCONF=XENHVM && make > installkernel KERNCONF=XENHVM On the stable/9 I only did > make buildkernel KERNCONF=XENHVM && make installkernel KERNCONF=XENHVM I guess the kernel-toolchain takes a long time to build…and from what I can see it does a clean before rebuilding also. I'm doing the kernel-toolchain step only now and will report how long it took. Jeroen. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: FreeBSD PVHVM call for testing
On Jun 19, 2013, at 10:07 AM, Jeroen van der Ham wrote: > Hi, > > On 19 Jun 2013, at 14:44, Roger Pau Monné wrote: > >> On 19/06/13 14:33, Jeroen van der Ham wrote: >>> >>> On 19 Jun 2013, at 14:20, Roger Pau Monné wrote: That's because Justin recently pushed a commit that changed the ad translation to ada, you should change your /etc/fstab to ada0p2. It's commit 526f3ad11acb296481215d7c2915b3f30f1844f6. >>> >>> >>> Ah, you may want to update the wiki page also to warn for that. :) >> >> D'oh, I've completely forgot about the wiki page, it's updated now, >> thanks for the pointer. > > Okay, everything works again now. Should we encourage folks to just configure their VMs to use xbd? I hope some day that the system will just report "da" devices, so the ada name may change again. We could also suggest specifying SCSI disks in the VM config since I don't think "da" will ever change. > Additionally, I've applied the patches from FreeBSD-SA-13:06.mmap, rebuilt > the kernel and rebooted, and the system now works fine. > > I did note however that rebuilding the kernel takes an awful lot more time > than on a FreeBSD9 system. As in it took about 2 hours longer. > > Jeroen. I've never seen a kernel build take 2 hours, much less 2 hours *longer*. Are you talking about buildworld? It would be interesting to know your results building stable/9 sources in your 10 environment to see if this is just due to "build bloat" or a true performance regression. -- Justin ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: FreeBSD PVHVM call for testing
Hi, On 19 Jun 2013, at 14:44, Roger Pau Monné wrote: > On 19/06/13 14:33, Jeroen van der Ham wrote: >> >> On 19 Jun 2013, at 14:20, Roger Pau Monné wrote: >>> That's because Justin recently pushed a commit that changed the ad >>> translation to ada, you should change your /etc/fstab to ada0p2. It's >>> commit 526f3ad11acb296481215d7c2915b3f30f1844f6. >> >> >> Ah, you may want to update the wiki page also to warn for that. :) > > D'oh, I've completely forgot about the wiki page, it's updated now, > thanks for the pointer. Okay, everything works again now. Additionally, I've applied the patches from FreeBSD-SA-13:06.mmap, rebuilt the kernel and rebooted, and the system now works fine. I did note however that rebuilding the kernel takes an awful lot more time than on a FreeBSD9 system. As in it took about 2 hours longer. Jeroen. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: Proposal for better support of hypervisors and their synthetic drivers at boot-time
John, Sorry for not responding earlier to your comments, but we’re at a point where we’re trying to get our VM drivers included with the current build. Your solution for dynamic loading sounds wonderful! Is this a simple change? And how can we help? Here’s an example of the modern, preferred way (Linux has been doing this for 5+ years) to detect the presence of a hypervisor on x86/AMD chips, both 32/64 bit modes (this is pulled from our FreeBSD driver, but the formatting is off because Google's Gmail really wants to make it look "pretty"): *#define HV_X64_MSR_GUEST_OS_ID 0x4000* *#define HV_X64_CPUID_MIN 0x4005* *#define HV_X64_CPUID_MAX 0x4000* *static* *int* *hv_check_for_hyper_v*(*void*) { u_int regs[4]; *int* hyper_v_detected *=* 0; do_cpuid(1, regs); *if* (regs[2] *&* 0x8000) { */* if(a hypervisor is detected) */* */* make sure this really is Hyper-V */* */* we look at the CPUID info */* do_cpuid(HV_X64_MSR_GUEST_OS_ID, regs); hyper_v_detected *=* regs[0] *>=* HV_X64_CPUID_MIN *&&* regs[0] *<=* HV_X64_CPUID_MAX *&&* *!*memcmp("Microsoft Hv", *&*regs[1], 12); } *return* (hyper_v_detected); } Of course, a general purpose routine will simply copy the name-tag somewhere and make it available. We are also having some issues with some of the disk utilities. They need to behave differently when operating under a hypervisor. I’ll try to provide a list of all the things that should be upgraded or enhanced, then we can figure out the best way to get them done. On Mon, Apr 29, 2013 at 10:45 AM, John Baldwin wrote: > I know Alexander replied about the ATA bits already, but I wanted to reply > to > two of your other points below: > > On Tuesday, April 23, 2013 10:07:03 am Larry Melia wrote: > > (1) Move the call to init_param1() (in sys/kern/subr_parm.c), which is > used > > for hypervisor detection, to an earlier point in the boot process. > > Presently, it appears to be called after the ATA driver is selected, > which > > is too late in the boot process. (This was discovered after some testing > > with the ATA driver.) Therefore, before the bus drivers and native > > controllers are detected and selected, discovery of a host hypervisor > > should be done first. > > > > (3) Upgrade the init_param1() function (in sys/kern/subr_parm.c) to use > the > > more recent approach to hypervisor detection. This approach uses the > > CPU-identify functions to retrieve a unique signature consisting of a > fixed > > string of ASCII characters. This was done on Linux about five years. For > > backward compatibility, however, the existing logic would be retained, > but > > augmented with this new approach. It would also be conditionally added > only > > for x86/AMD64 builds. > > I definitely agree with these proposals. In addition, our current > hypervisor > detection code is completely x86-specific and does not belong in MI code. > The > only bits that should be MI are the vm_guest variable and the VM_GUEST > constants. I would argue that most of the VM_GUEST constants (for specific > VMs which we do not have currently) should be MD as well. > > Each platform that supports hypervisors would install its own SYSINIT to > set > vm_guest instead of doing it directly from init_param1(). > > Making the VM_GUEST_FOO constants be MD macros means you can use #ifdef to > test for them. Thus: > > #ifdef VM_GUEST_HYPERV > /* Include a hyper-V specific driver. */ > #endif > > The current enum approach doesn't allow for that. > > -- > John Baldwin > ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: FreeBSD PVHVM call for testing
On 19/06/13 14:33, Jeroen van der Ham wrote: > > On 19 Jun 2013, at 14:20, Roger Pau Monné wrote: >> That's because Justin recently pushed a commit that changed the ad >> translation to ada, you should change your /etc/fstab to ada0p2. It's >> commit 526f3ad11acb296481215d7c2915b3f30f1844f6. > > > Ah, you may want to update the wiki page also to warn for that. :) D'oh, I've completely forgot about the wiki page, it's updated now, thanks for the pointer. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: FreeBSD PVHVM call for testing
On 19 Jun 2013, at 14:20, Roger Pau Monné wrote: > That's because Justin recently pushed a commit that changed the ad > translation to ada, you should change your /etc/fstab to ada0p2. It's > commit 526f3ad11acb296481215d7c2915b3f30f1844f6. Ah, you may want to update the wiki page also to warn for that. :) Jeroen. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: FreeBSD PVHVM call for testing
On 19/06/13 14:16, Jeroen van der Ham wrote: > > On 19 Jun 2013, at 13:34, Roger Pau Monné wrote: >> >> Could you provide the boot log of the DomU, backtrace, Xen version and >> Dom0 kernel version? > > I did not have a console attached when it rebooted, so I did not have a log > of the initial boot. Now that I did, I see that it fails to mount its root > volume. > > It had been running previously on pvhvm_v10 for about two weeks without > problems. I updated my local git, and recompiled the kernel and rebooted. > Then this happened. > > > In order: > > Booting... > GDB: no debug ports present > KDB: debugger backends: ddb > KDB: current backend: ddb > Copyright (c) 1992-2013 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 10.0-CURRENT #2 r+6ff8d00-dirty: Tue Jun 18 12:55:16 CEST 2013 > root@image01:/usr/obj/root/freebsd/sys/XENHVM amd64 > FreeBSD clang version 3.3 (trunk 178860) 20130405 > WARNING: WITNESS option enabled, expect reduced performance. > XEN: Hypervisor version 4.0 detected. > CPU: Quad-Core AMD Opteron(tm) Processor 2374 HE (2200.07-MHz K8-class CPU) > Origin = "AuthenticAMD" Id = 0x100f42 Family = 0x10 Model = 0x4 > Stepping = 2 > > Features=0x1781fbff > Features2=0x80802001 > AMD Features=0xe2500800 > AMD Features2=0x1f3 > real memory = 536870912 (512 MB) > avail memory = 472260608 (450 MB) > Event timer "LAPIC" quality 400 > ACPI APIC Table: > FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs > FreeBSD/SMP: 1 package(s) x 2 core(s) > cpu0 (BSP): APIC ID: 0 > cpu1 (AP): APIC ID: 2 > random device not loaded; using insecure entropy > ioapic0: Changing APIC ID to 1 > MADT: Forcing active-low polarity and level trigger for SCI > ioapic0 irqs 0-47 on motherboard > kbd1 at kbdmux0 > xen_et0: on motherboard > Event timer "XENTIMER" frequency 10 Hz quality 950 > Timecounter "XENTIMER" frequency 10 Hz quality 950 > acpi0: on motherboard > acpi0: Power Button (fixed) > acpi0: Sleep Button (fixed) > acpi0: reservation of 0, a (3) failed > cpu0: on acpi0 > cpu1: on acpi0 > attimer0: port 0x40-0x43 irq 0 on acpi0 > Timecounter "i8254" frequency 1193182 Hz quality 0 > Event timer "i8254" frequency 1193182 Hz quality 100 > atrtc0: port 0x70-0x71 irq 8 on acpi0 > Event timer "RTC" frequency 32768 Hz quality 0 > Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 > acpi_timer0: <32-bit timer at 3.579545MHz> port 0x1f48-0x1f4b on acpi0 > pcib0: port 0xcf8-0xcff on acpi0 > pci0: on pcib0 > isab0: at device 1.0 on pci0 > isa0: on isab0 > atapci0: port > 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc300-0xc30f at device 1.1 on pci0 > ata0: at channel 0 on atapci0 > ata1: at channel 1 on atapci0 > pci0: at device 1.3 (no driver attached) > vgapci0: mem > 0xf000-0xf1ff,0xf300-0xf3000fff at device 2.0 on pci0 > xenpci0: port 0xc000-0xc0ff mem 0xf200-0xf2ff > irq 28 at device 3.0 on pci0 > xenstore0: on xenpci0 > atkbdc0: port 0x60,0x64 irq 1 on acpi0 > atkbd0: irq 1 on atkbdc0 > kbd0 at atkbd0 > atkbd0: [GIANT-LOCKED] > psm0: irq 12 on atkbdc0 > psm0: [GIANT-LOCKED] > psm0: model IntelliMouse Explorer, device ID 4 > fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 > fdc0: does not respond > device_attach: fdc0 attach returned 6 > uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 > uart0: console (9600,n,8,1) > ppc0: port 0x378-0x37f irq 7 on acpi0 > ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode > ppbus0: on ppc0 > lpt0: on ppbus0 > lpt0: Interrupt-driven port > ppi0: on ppbus0 > orm0: at iomem 0xc9000-0xc97ff on isa0 > sc0: at flags 0x100 on isa0 > sc0: VGA <16 virtual consoles, flags=0x300> > vga0: at port 0x3c0-0x3df iomem 0xa-0xb on isa0 > fdc0: No FDOUT register! > Timecounters tick every 10.000 msec > xenbusb_front0: on xenstore0 > cd0 at ata1 bus 0 scbus1 target 0 lun 0 > cd0: Removable CD-ROM SCSI-0 device > cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes) > cd0: cd present [360385 x 2048 byte records] > xn0: at device/vif/0 on xenbusb_front0 > xn0: Ethernet address: 00:16:3e:2f:b7:22 > xn1: at device/vif/1 on xenbusb_front0 > xn1: Ethernet address: 00:16:3e:3e:64:c5 > xenbusb_back0: on xenstore0 > xctrl0: on xenstore0 > xn0: backend features: feature-sg feature-gso-tcp4 > xn1: backend features: feature-sg feature-gso-tcp4 > xbd0: 20480MB at device/vbd/768 on xenbusb_front0 > xbd0: attaching as ada0 > xbd0: disk supports cache flush using: barriers > xbd1: 703MB at device/vbd/5632 on xenbusb_front0 > xbd1: attaching as ada2 > xbd1: disk supports cache flush using: barriers > SMP: AP CPU #1 Launched! > WARNING: WITNESS option enabled, expect reduced performance. > Trying to mount root from ufs:/dev/ad0p2 [rw]... > mountroot: waiting for device
Re: FreeBSD PVHVM call for testing
On 19 Jun 2013, at 13:34, Roger Pau Monné wrote: > > Could you provide the boot log of the DomU, backtrace, Xen version and > Dom0 kernel version? I did not have a console attached when it rebooted, so I did not have a log of the initial boot. Now that I did, I see that it fails to mount its root volume. It had been running previously on pvhvm_v10 for about two weeks without problems. I updated my local git, and recompiled the kernel and rebooted. Then this happened. In order: Booting... GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2013 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #2 r+6ff8d00-dirty: Tue Jun 18 12:55:16 CEST 2013 root@image01:/usr/obj/root/freebsd/sys/XENHVM amd64 FreeBSD clang version 3.3 (trunk 178860) 20130405 WARNING: WITNESS option enabled, expect reduced performance. XEN: Hypervisor version 4.0 detected. CPU: Quad-Core AMD Opteron(tm) Processor 2374 HE (2200.07-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x100f42 Family = 0x10 Model = 0x4 Stepping = 2 Features=0x1781fbff Features2=0x80802001 AMD Features=0xe2500800 AMD Features2=0x1f3 real memory = 536870912 (512 MB) avail memory = 472260608 (450 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 1 package(s) x 2 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 2 random device not loaded; using insecure entropy ioapic0: Changing APIC ID to 1 MADT: Forcing active-low polarity and level trigger for SCI ioapic0 irqs 0-47 on motherboard kbd1 at kbdmux0 xen_et0: on motherboard Event timer "XENTIMER" frequency 10 Hz quality 950 Timecounter "XENTIMER" frequency 10 Hz quality 950 acpi0: on motherboard acpi0: Power Button (fixed) acpi0: Sleep Button (fixed) acpi0: reservation of 0, a (3) failed cpu0: on acpi0 cpu1: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x1f48-0x1f4b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 isab0: at device 1.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc300-0xc30f at device 1.1 on pci0 ata0: at channel 0 on atapci0 ata1: at channel 1 on atapci0 pci0: at device 1.3 (no driver attached) vgapci0: mem 0xf000-0xf1ff,0xf300-0xf3000fff at device 2.0 on pci0 xenpci0: port 0xc000-0xc0ff mem 0xf200-0xf2ff irq 28 at device 3.0 on pci0 xenstore0: on xenpci0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model IntelliMouse Explorer, device ID 4 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: does not respond device_attach: fdc0 attach returned 6 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (9600,n,8,1) ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppbus0: on ppc0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 orm0: at iomem 0xc9000-0xc97ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa-0xb on isa0 fdc0: No FDOUT register! Timecounters tick every 10.000 msec xenbusb_front0: on xenstore0 cd0 at ata1 bus 0 scbus1 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: cd present [360385 x 2048 byte records] xn0: at device/vif/0 on xenbusb_front0 xn0: Ethernet address: 00:16:3e:2f:b7:22 xn1: at device/vif/1 on xenbusb_front0 xn1: Ethernet address: 00:16:3e:3e:64:c5 xenbusb_back0: on xenstore0 xctrl0: on xenstore0 xn0: backend features: feature-sg feature-gso-tcp4 xn1: backend features: feature-sg feature-gso-tcp4 xbd0: 20480MB at device/vbd/768 on xenbusb_front0 xbd0: attaching as ada0 xbd0: disk supports cache flush using: barriers xbd1: 703MB at device/vbd/5632 on xenbusb_front0 xbd1: attaching as ada2 xbd1: disk supports cache flush using: barriers SMP: AP CPU #1 Launched! WARNING: WITNESS option enabled, expect reduced performance. Trying to mount root from ufs:/dev/ad0p2 [rw]... mountroot: waiting for device /dev/ad0p2 ... Mounting from ufs:/dev/ad0p2 failed with error 19. Loader variables: vfs.root.mountfrom=ufs:/dev/ad0p2 vfs.root.mountfrom.options=rw Manual root filesystem specification: : [options] Mount using filesystem and with the specified (optional) option list. eg
Re: FreeBSD PVHVM call for testing
On 19/06/13 13:13, Jeroen van der Ham wrote: > Hi, > > I've just built a new kernel based on pvhvm_v17, but it panicked on boot. > > I still have a xen console attached, so I can provide additional information > if someone gives me the right commands. > > Jeroen. > Could you provide the boot log of the DomU, backtrace, Xen version and Dom0 kernel version? ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: FreeBSD PVHVM call for testing
Hi, I've just built a new kernel based on pvhvm_v17, but it panicked on boot. I still have a xen console attached, so I can provide additional information if someone gives me the right commands. Jeroen. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"