Re: Clock not moving in virtual machine
Rob Farmer wrote: I have a VPS from rootbsd.net which is running current, though I don't update it very often. I just built and installed a new world and kernel and now the clock will not move from the time the system was booted, ie: # date Thu Jul 15 16:15:58 PDT 2010 wait a minute # date Thu Jul 15 16:15:58 PDT 2010 I have an old kernel from May 27 which doesn't have this problem. I noticed some clock related stuff changing in current in the last couple of weeks and suspect that their VM setup doesn't play well with these changes (their site says they use Xen, but several boot messages refer to QEMU). Officially, I think they only support running 8.0 so I thought I would ask here if anyone has any ideas before putting in a support request. Here's a diff of the dmesgs - I can post full copies if needed but didn't want to start with a ridicously long message: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 0 package(s) x 16 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 0 Probably not related, but funny. :) So you have two CPUs? @@ -81,7 +81,10 @@ ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppc0: [ITHREAD] ppbus0: Parallel port bus on ppc0 -atrtc0: AT Real Time Clock at port 0x70 irq 8 on isa0 +atrtc0: AT realtime clock at port 0x70 irq 8 on isa0 +atrtc0: [FILTER] +Event timer RTC frequency 32768 Hz quality 0 +Starting kernel event timers: LAPIC @ 200Hz, RTC @ 128Hz Timecounters tick every 5.000 msec Everything seems reasonable there. Try to collect more information: sysctl kern.timecounter sysctl kern.eventtimer vmstat -ia systat -vm 1 (presence and frequencies of interrupts) It could be a bug in emulation of some timers or bug in respective timer driver, which was not triggered before last changes. You may try switch to different timecounter by setting kern.timecounter.hardware, or different eventtimers by setting kern.eventtimer.timer1 and kern.eventtimer.timer2 sysctls. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Clock not moving in virtual machine
On Thu, Jul 15, 2010 at 11:33 PM, Alexander Motin m...@freebsd.org wrote: Rob Farmer wrote: I have a VPS from rootbsd.net which is running current, though I don't update it very often. I just built and installed a new world and kernel and now the clock will not move from the time the system was booted, ie: # date Thu Jul 15 16:15:58 PDT 2010 wait a minute # date Thu Jul 15 16:15:58 PDT 2010 I have an old kernel from May 27 which doesn't have this problem. I noticed some clock related stuff changing in current in the last couple of weeks and suspect that their VM setup doesn't play well with these changes (their site says they use Xen, but several boot messages refer to QEMU). Officially, I think they only support running 8.0 so I thought I would ask here if anyone has any ideas before putting in a support request. Here's a diff of the dmesgs - I can post full copies if needed but didn't want to start with a ridicously long message: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 0 package(s) x 16 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 0 Probably not related, but funny. :) So you have two CPUs? Yes, there's 2 CPUs. It also gives the non uniform processors warning, but it has been like that forever. @@ -81,7 +81,10 @@ ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppc0: [ITHREAD] ppbus0: Parallel port bus on ppc0 -atrtc0: AT Real Time Clock at port 0x70 irq 8 on isa0 +atrtc0: AT realtime clock at port 0x70 irq 8 on isa0 +atrtc0: [FILTER] +Event timer RTC frequency 32768 Hz quality 0 +Starting kernel event timers: LAPIC @ 200Hz, RTC @ 128Hz Timecounters tick every 5.000 msec Everything seems reasonable there. Try to collect more information: sysctl kern.timecounter sysctl kern.eventtimer vmstat -ia systat -vm 1 (presence and frequencies of interrupts) It could be a bug in emulation of some timers or bug in respective timer driver, which was not triggered before last changes. You may try switch to different timecounter by setting kern.timecounter.hardware, or different eventtimers by setting kern.eventtimer.timer1 and kern.eventtimer.timer2 sysctls. -- Alexander Motin This is all on the new (not-working) kernel in single user mode: # sysctl kern.timecounter kern.timecounter.tick: 1 kern.timecounter.choice: TSC(-100) dummy(-100) kern.timecounter.hardware: dummy kern.timecounter.stepwarnings: 0 kern.timecounter.tc.TSC.mask: 4294967295 kern.timecounter.tc.TSC.counter: 205772785 kern.timecounter.tc.TSC.frequency: 2261052646 kern.timecounter.tc.TSC.quality: -100 kern.timecounter.smp_tsc: 0 kern.timecounter.invariant_tsc: 1 kern.timecounter.tc.TSC.counter changes everytime (205772785, 3200717147, 1205899870, ...) but I can't see any pattern. # sysctl kern.eventtimer kern.eventtimer.choice: LAPIC(500) RTC(0) kern.eventtimer.et.LAPIC.flags: 15 kern.eventtimer.et.LAPIC.frequency: 50001404 kern.eventtimer.et.LAPIC.quality: 500 kern.eventtimer.et.RTC.flags: 1 kern.eventtimer.et.RTC.frequency: 32768 kern.eventtimer.et.RTC.quality: 0 kern.eventtimer.timer2: RTC kern.eventtimer.timer1: LAPIC kern.eventtimer.singlemul: 4 # vmstat -ia interrupt total rate ???0 0 irq1: atkbd0 339339 stray irq1 0 0 irq0: 0 0 stray irq0 0 0 irq3: 0 0 stray irq3 0 0 irq4: uart00 0 stray irq4 0 0 irq5: re0 0 0 stray irq5 0 0 irq6: 0 0 stray irq6 0 0 irq7: ppc0 0 0 stray irq7 0 0 irq8: atrtc0 24463 24463 stray irq8 0 0 irq9: 0 0 stray irq9 0 0 irq10: re1 0 0 stray irq100 0 irq11: 0 0 stray irq110 0 irq12: psm00 0 stray irq120 0 irq13: 0 0 stray irq130 0 irq14: ata0 224224 stray irq140 0 irq15: ata10 0 stray irq150 0 irq16: 0 0 stray irq160 0 irq17: 0 0
Re: Clock not moving in virtual machine
Rob Farmer wrote: @@ -81,7 +81,10 @@ ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppc0: [ITHREAD] ppbus0: Parallel port bus on ppc0 -atrtc0: AT Real Time Clock at port 0x70 irq 8 on isa0 +atrtc0: AT realtime clock at port 0x70 irq 8 on isa0 +atrtc0: [FILTER] +Event timer RTC frequency 32768 Hz quality 0 +Starting kernel event timers: LAPIC @ 200Hz, RTC @ 128Hz Timecounters tick every 5.000 msec Everything seems reasonable there. Try to collect more information: sysctl kern.timecounter sysctl kern.eventtimer vmstat -ia systat -vm 1 (presence and frequencies of interrupts) It could be a bug in emulation of some timers or bug in respective timer driver, which was not triggered before last changes. You may try switch to different timecounter by setting kern.timecounter.hardware, or different eventtimers by setting kern.eventtimer.timer1 and kern.eventtimer.timer2 sysctls. This is all on the new (not-working) kernel in single user mode: # sysctl kern.timecounter kern.timecounter.tick: 1 kern.timecounter.choice: TSC(-100) dummy(-100) kern.timecounter.hardware: dummy kern.timecounter.stepwarnings: 0 kern.timecounter.tc.TSC.mask: 4294967295 kern.timecounter.tc.TSC.counter: 205772785 kern.timecounter.tc.TSC.frequency: 2261052646 kern.timecounter.tc.TSC.quality: -100 kern.timecounter.smp_tsc: 0 kern.timecounter.invariant_tsc: 1 kern.timecounter.tc.TSC.counter changes everytime (205772785, 3200717147, 1205899870, ...) but I can't see any pattern. It is probably hard to see pattern due to to very high clock frequency. But TSC timecounter is unreliable even on real SMP systems. What it counts on virtual SMP - even bigger question. As system seems never uses timecounters with negative quality - you've left with kern.timecounter.hardware=dummy - that's why time is not going. As last resort you may try to set sysctl kern.timecounter.hardware=TSC in run time. Previously you were using i8254 timecounter, but now you lost it as your virtual machine PnP BIOS doesn't announce it. QEMU does the same, but instead it gives HPET which your emulator seems also doesn't. To force i8254 presence add to the /boot/device.hints lines: hint.attimer.0.at=isa hint.attimer.0.port=0x40 hint.attimer.0.irq=0 -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Clock not moving in virtual machine
Bruce Cran wrote: On Fri, Jul 16, 2010 at 11:47:23PM +0300, Alexander Motin wrote: It is probably hard to see pattern due to to very high clock frequency. But TSC timecounter is unreliable even on real SMP systems. What it counts on virtual SMP - even bigger question. As system seems never uses timecounters with negative quality - you've left with kern.timecounter.hardware=dummy - that's why time is not going. As last resort you may try to set sysctl kern.timecounter.hardware=TSC in run time. I came across the same problem on rootbsd a few days ago, and set the TSC as the timecounter in /etc/sysctl.conf - I've since found it should be possible to also set kern.timecounter.smp_tsc=1 in /boot/loader.conf to let the TSC be chosen. The system's now been running for a day and I've not had any warnings about the clock going backward, and since the time has remained correct I guess Xen synchronises with the host? I have no idea about TSC in XEN, but QEMU just passes TSC from the physical CPU. So if host' TSCs are not synchronized - value may jump accidentally when QEMU process migrates between CPUs. Should I still switch back to using the i8254? I would say it depends. i8254 frequency is always known, while TSC depends on calibration. Calibration on virtual machine I think much less precise then on physical. Same time, if i8254 also used as event timer, timestamp calculation algorithm is not very trivial there and I am not sure it is absolutely reliable. So I would probably used i8254 as time counter and LAPIC+RTC as event timers. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Clock not moving in virtual machine
On Fri, Jul 16, 2010 at 2:05 PM, Bruce Cran bru...@muon.cran.org.uk wrote: On Fri, Jul 16, 2010 at 11:47:23PM +0300, Alexander Motin wrote: It is probably hard to see pattern due to to very high clock frequency. But TSC timecounter is unreliable even on real SMP systems. What it counts on virtual SMP - even bigger question. As system seems never uses timecounters with negative quality - you've left with kern.timecounter.hardware=dummy - that's why time is not going. As last resort you may try to set sysctl kern.timecounter.hardware=TSC in run time. I came across the same problem on rootbsd a few days ago, and set the TSC as the timecounter in /etc/sysctl.conf - I've since found it should be possible to also set kern.timecounter.smp_tsc=1 in /boot/loader.conf to let the TSC be chosen. The system's now been running for a day and I've not had any warnings about the clock going backward, and since the time has remained correct I guess Xen synchronises with the host? Should I still switch back to using the i8254? -- Bruce Cran Setting kern.timecounter.smp_tsc=1 works for me. I put it in /boot/loader.conf so it would automatically work for single user mode too. I don't think the host automatically synchronizes the clock - their website recommends running ntp and I saw the clock drift a fair amount before I started doing that. Thanks for the tip. -- Rob Farmer ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Clock not moving in virtual machine
On Fri, Jul 16, 2010 at 11:47:23PM +0300, Alexander Motin wrote: It is probably hard to see pattern due to to very high clock frequency. But TSC timecounter is unreliable even on real SMP systems. What it counts on virtual SMP - even bigger question. As system seems never uses timecounters with negative quality - you've left with kern.timecounter.hardware=dummy - that's why time is not going. As last resort you may try to set sysctl kern.timecounter.hardware=TSC in run time. I came across the same problem on rootbsd a few days ago, and set the TSC as the timecounter in /etc/sysctl.conf - I've since found it should be possible to also set kern.timecounter.smp_tsc=1 in /boot/loader.conf to let the TSC be chosen. The system's now been running for a day and I've not had any warnings about the clock going backward, and since the time has remained correct I guess Xen synchronises with the host? Should I still switch back to using the i8254? -- Bruce Cran ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Clock not moving in virtual machine
Hi, I have a VPS from rootbsd.net which is running current, though I don't update it very often. I just built and installed a new world and kernel and now the clock will not move from the time the system was booted, ie: # date Thu Jul 15 16:15:58 PDT 2010 wait a minute # date Thu Jul 15 16:15:58 PDT 2010 I have an old kernel from May 27 which doesn't have this problem. I noticed some clock related stuff changing in current in the last couple of weeks and suspect that their VM setup doesn't play well with these changes (their site says they use Xen, but several boot messages refer to QEMU). Officially, I think they only support running 8.0 so I thought I would ask here if anyone has any ideas before putting in a support request. Here's a diff of the dmesgs - I can post full copies if needed but didn't want to start with a ridicously long message: --- dmesg.old 2010-07-15 17:45:20.0 -0700 +++ dmesg.new 2010-07-15 17:46:45.0 -0700 @@ -2,10 +2,9 @@ Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. -FreeBSD 9.0-CURRENT #0: Thu May 27 01:26:08 PDT 2010 +FreeBSD 9.0-CURRENT #0: Thu Jul 15 16:13:28 PDT 2010 rfar...@peridot.predatorlabs.net:/usr/obj/usr/src/sys/PERIDOT i386 -Timecounter i8254 frequency 1193182 Hz quality 0 -CPU: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (2261.02-MHz 686-class CPU) +CPU: Intel(R) Xeon(R) CPU E5520 @ 2.27GHz (2261.59-MHz 686-class CPU) Origin = GenuineIntel Id = 0x106a5 Family = 6 Model = 1a Stepping = 5 Features=0x1781fbffFPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT Features2=0x80182201SSE3,SSSE3,CX16,SSE4.1,SSE4.2,b31 @@ -13,9 +12,10 @@ AMD Features2=0x1LAHF TSC: P-state invariant real memory = 268435456 (256 MB) -avail memory = 252473344 (240 MB) -ACPI Error: A valid RSDP was not found (20100428/tbxfroot-309) +avail memory = 252444672 (240 MB) +ACPI Error: A valid RSDP was not found (20100702/tbxfroot-309) MPTable: _HVMCPU_ XEN +Event timer LAPIC frequency 0 Hz quality 500 FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 0 package(s) x 16 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 0 @@ -26,7 +26,7 @@ ioapic0: Assuming intbase of 0 ioapic0 Version 1.1 irqs 0-47 on motherboard kbd1 at kbdmux0 -ACPI Error: A valid RSDP was not found (20100428/tbxfroot-309) +ACPI Error: A valid RSDP was not found (20100702/tbxfroot-309) ACPI: Table initialisation failed: AE_NOT_FOUND ACPI: Try disabling either ACPI or apic support. pcib0: Host to PCI bridge pcibus 0 on motherboard @@ -81,7 +81,10 @@ ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppc0: [ITHREAD] ppbus0: Parallel port bus on ppc0 -atrtc0: AT Real Time Clock at port 0x70 irq 8 on isa0 +atrtc0: AT realtime clock at port 0x70 irq 8 on isa0 +atrtc0: [FILTER] +Event timer RTC frequency 32768 Hz quality 0 +Starting kernel event timers: LAPIC @ 200Hz, RTC @ 128Hz Timecounters tick every 5.000 msec ad0: 10240MB QEMU HARDDISK 0.10.2 at ata0-master WDMA2 SMP: AP CPU #1 Launched! Thanks, -- Rob Farmer ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
