In 3.0.x system VM template, /proc/sys/xen/independent_wallclock is set to 0, that means system VM is using dom0 time, it is always synced with dom0 time. In 4.2 system VM template, /proc/sys/xen/independent_wallclock doesn't exist, looks like it is 1 by default by checking http://wiki.xensource.com/wiki/Xen_FAQ_DomU#How_can_i_synchronize_a_dom0_clock.3F, which might cause time drift, maybe we need to enable NTP inside system VM.
Anthony -----Original Message----- From: John Burwell [mailto:jburw...@basho.com] Sent: Wednesday, May 15, 2013 10:30 AM To: dev@cloudstack.apache.org Subject: Re: [ACS41] System VMs not syncing time - does this block the release? Chiradeep, The issue I am experiencing is that the system VMs are not syncing to dom0 on devcloud (i.e. the dom0 clock and the SSVM clock are different). As I mentioned earlier in this thread, the syncing was working previously which seems to jibe with your findings. What mechanism is used to sync the dom0 and domU clocks (e.g. NTP, kernel driver, etc)? It may be a situation where the pieces are present, but they aren't configured properly or simply not running. As an aside, we can not run VirtualBox Additions on devcloud due a conflict with the Xen kernel. Therefore, I hard execute "ntpdate pool.ntp.org" periodically on the devcloud host to keep the clock synced with the "real world". Another approach is to configure NTP with a very large drift and increase check frequency to accomodate the large clock swings that can occur. Thanks, -John On Wed, May 15, 2013 at 1:21 PM, Chiradeep Vittal < chiradeep.vit...@citrix.com> wrote: > Perhaps this is a problem with DevCloud? > http://nerdboys.com/2011/03/15/how-to-fix-virtualbox-time-synchonizati > on-pr > oblems/ > > > > On 5/15/13 10:17 AM, "Chiradeep Vittal" <chiradeep.vit...@citrix.com> > wrote: > > >According to our resident Xen expert, any PV kernel automatically > >syncs to the hardware clock on dom0. > > > >On 5/15/13 9:50 AM, "John Burwell" <jburw...@basho.com> wrote: > > > >>Marcus, > >> > >>Agreed. I think we need to add a set of hypervisor agnostic time > >>keeping guidelines to the documentation. I just wanted to make sure > >>there wasn't anything KVM specific that should be added as well. > >> > >>Thanks, > >>-John > >> > >>On May 15, 2013, at 12:48 PM, Marcus Sorensen <shadow...@gmail.com> > >>wrote: > >> > >>> Just the general one that system vms sync their time to the > >>> hypervisor, thus admins need to keep the hypervisor time correct. > >>> It sounds like that will be the case for all three, if we can manage it. > >>> > >>> On Wed, May 15, 2013 at 10:44 AM, John Burwell > >>><jburw...@basho.com> > >>>wrote: > >>>> Marcus, > >>>> > >>>> Excellent. So, it looks like we have KVM resolved. We just need > >>>>to address Xen and VMWare now. Do you think we need to any > >>>>guidance to the documentation regarding KVM time keeping (e.g. > >>>>environmental prerequisites)? > >>>> > >>>> Thanks, > >>>> -John > >>>> > >>>> > >>>> > >>>> On May 15, 2013, at 12:39 PM, Marcus Sorensen > >>>><shadow...@gmail.com> > >>>>wrote: > >>>> > >>>>> KVM LibvirtComputingResource has been patched in master. Tested > >>>>> on master, 4.1, and both the acton and current system vm > >>>>> templates. This patch makes system vms use 'kvmclock' for their > >>>>> timer, which is a vm driver that gets it's time from the > >>>>> hypervisor. No change to the system vm template itself. > >>>>> > >>>>> bfc5887a1bf6b41e88dd7a8f9987fcee8d3d9175 > >>>>> > >>>>> On Wed, May 15, 2013 at 9:08 AM, Chip Childers > >>>>> <chip.child...@sungard.com> wrote: > >>>>>> On Wed, May 15, 2013 at 11:03:16AM -0400, John Burwell wrote: > >>>>>>> Chip, > >>>>>>> > >>>>>>> One other item I neglected to mention was that clock sync, at > >>>>>>>least for Xen system VMs, wasn't an issue in the Jan-Feb timeframe. > >>>>>>>Previously when I encountered these issues, syncing the host's > >>>>>>>clock and rebuilding the system VMs addressed the issue. I > >>>>>>>assumed, but never verified, that the SSVM was syncing back > >>>>>>>against the host's clock through hypervisor. During my testing > >>>>>>>yesterday, aside from hard setting the clock, I was unable to > >>>>>>>force clock sync on the SSVM. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> -John > >>>>>> > >>>>>> I think that's our issue right now... answering the question: > >>>>>>Why is this only an issue now? Did we just get lucky up to > >>>>>>this point? > >>>>>>Since > >>>>>> the SSVMs are the same template as the timeframe you mention, I > >>>>>>tend to believe that you / we were just lucky. > >>>>>> > >>>>>> Anyone else have thoughts? > >>>>>> > >>>>>>> > >>>>>>> On May 15, 2013, at 10:18 AM, Chip Childers > >>>>>>><chip.child...@sungard.com> wrote: > >>>>>>> > >>>>>>>> Starting a thread on this specific issue. > >>>>>>>> > >>>>>>>> CLOUDSTACK-2492 was opened, which is basically the fact that > >>>>>>>>the System VMs aren't syncing time to the host or to an NTP > >>>>>>>>server. The S3 integration is broken because of this > >>>>>>>>problem, and therefore could not be considered a function > >>>>>>>>available in 4.1 if we release as is. > >>>>>>>> > >>>>>>>> We need input from people that know about the current system > >>>>>>>>VMs (the 3.x VMs), as well as the possibility of using the > >>>>>>>>newer ones that we have been considering experimental for > >>>>>>>>4.1.0. > >>>>>>>> > >>>>>>>> What should we do? > >>>>>>>> > >>>>>>>> -chip > >>>>>>> > >>>>>>> > >>>> > >> > > > >
