We appear to be at an impasse here as well. I'm going to start a VOTE on this issue, given that we have no easy answer.
On Wed, May 15, 2013 at 02:49:41PM -0700, Chiradeep Vittal wrote: > Well, I disagree, from the perspective of hundreds of production clouds. > No harm has been perceived in those clouds due to this defect. If they can > live with it for several years, then they can live with it for a few more > months. > > On 5/15/13 2:35 PM, "Wido den Hollander" <w...@widodh.nl> wrote: > > > > > > >On 05/15/2013 11:33 PM, John Burwell wrote: > >> Chiradeep, > >> > >> I disagree regarding the impact of this issue. Anyone running an SSVM > >>will be affected by this issue because clocks will eventually drift > >>(sooner rather than later) and when they do, any timestamps rendered by > >>a system VM will unreliable (e.g. files creation and modified times, log > >>entries, etc). When I put my sys admin hat on, that type behavior would > >>4.1 an unacceptable release to me. > >> > > > >Ack. Clocks should always be on time. No matter what a machine is doing. > >Have the correct time is crucial. > > > >Think about log lines, debugging, etc, etc. > > > >Nothing wrong with running NTP in KVM System VM btw. > > > >Wido > > > >> Thanks, > >> -John > >> > >> On May 15, 2013, at 5:24 PM, Chiradeep Vittal > >><chiradeep.vit...@citrix.com> wrote: > >> > >>> Sure, I agree. But I'd also point out that for the vast majority of > >>> CloudStack users (4.1 at least), this is not going to be an issue. I > >>> suggest deferring this to 4.1.1 > >>> A new template (easy or not) does require a full regression QA round. > >>> > >>> On 5/15/13 2:07 PM, "John Burwell" <jburw...@basho.com> wrote: > >>> > >>>> Chiradeep, > >>>> > >>>> As I think thought it, I don't think a documentation note will > >>>>sufficient > >>>> because the SSVM can be destroyed and respawned by CloudStack without > >>>> intervention by a human. Therefore, we can get into a situation > >>>>where an > >>>> operator installs and configures NTP, and then at some point in the > >>>> future, the SSVM gets respawned and clocks drift. The worst part > >>>>about > >>>> this scenario is that the failures for S3 sync become silent since we > >>>> have no mechanism to surface the failure to a dashboard or monitoring > >>>> system. > >>>> > >>>> How much effort (i.e. hours/days) would it be to build a new image? > >>>> > >>>> Thanks, > >>>> -John > >>>> > >>>> On May 15, 2013, at 4:59 PM, Chiradeep Vittal > >>>> <chiradeep.vit...@citrix.com> wrote: > >>>> > >>>>> Did some further digging around as to why > >>>>> /proc/sys/xen/independent_wallclock is not there on the Debian system > >>>>> vm. > >>>>> > >>>>> TLDR; the kernel is PvOps (I.e., just a regular kernel that works > >>>>>like a > >>>>> PV kernel, not a specialized paravirt kernel). To eliminate > >>>>> special-casing, the independent_wallclock feature was dropped. > >>>>>However, > >>>>> this means that the domU clock is NO LONGER synch'ed to dom0 and NTP > >>>>>is > >>>>> required on ANY domU. So the clock on the domU is only sync'ed at > >>>>>domU > >>>>> creation time. I suspect Citrix XenServer might have some workaround > >>>>> > >>>>> > >>>>> http://www.gossamer-threads.com/lists/xen/users/234750 > >>>>> > >>>>> > >>>>> What this means is that we *may* need to run ntp on system vms on > >>>>>Xen. > >>>>> This is of course a non-trivial change involving updates to > >>>>>systemvms. > >>>>> > >>>>> I suspect that it does not matter much for virtual routers or console > >>>>> proxy vms. > >>>>> > >>>>> We could have an advisory that for those users that care (e.g., those > >>>>> using S3 sync) that they need to run ntp after the SSVM has been > >>>>> created. > >>>>> I.e, login to the SSVM and run > >>>>> apt-get update > >>>>> apt-get install ntp > >>>>> > >>>>> -- > >>>>> Chiradeep > >>>>> > >>>>> > >>>>> > >>>>> On 5/15/13 1:11 PM, "Chiradeep Vittal" <chiradeep.vit...@citrix.com> > >>>>> wrote: > >>>>> > >>>>>> For VMWare, the command > >>>>>> vmware-toolbox-cmd timesync status returns 'Disabled'. I can submit > >>>>>>a > >>>>>> patch for /etc/init.d/cloud-early-config to enable it > >>>>>> > >>>>>> > >>>>>> On 5/15/13 12:23 PM, "Chiradeep Vittal" > >>>>>><chiradeep.vit...@citrix.com> > >>>>>> wrote: > >>>>>> > >>>>>>> I am not sure why it is missing, but I will refer to > >>>>>>> Citrix XenServer 6.0 Virtual Machine Installation Guide > >>>>>>> > >>>>>>> http://s.apache.org/YGn > >>>>>>> > >>>>>>> And quote > >>>>>>> "Time Handling in Linux VMs > >>>>>>> By default, the clocks in a Linux VM are synchronized to the clock > >>>>>>> running > >>>>>>> on the control domain, and cannot be > >>>>>>> independently changed. This mode is a convenient default, since > >>>>>>>only > >>>>>>> the > >>>>>>> control domain needs to be running the > >>>>>>> NTP service to keep accurate time across all VMs. Upon installation > >>>>>>> of a > >>>>>>> new Linux VM, make sure you change the > >>>>>>> time-zone from the default UTC to your local value (see the section > >>>>>>> called > >>>>>>> ³Release Notes² for specific distribution > >>>>>>> instructions). > >>>>>>> To set individual Linux VMs to maintain independent times > >>>>>>> 1. From a root prompt on the VM, run the command: echo 1 > > >>>>>>> /proc/sys/xen/independent_wallclock > >>>>>>> 2. This can be persisted across reboots by changing the > >>>>>>> /etc/sysctl.conf > >>>>>>> configuration file and adding: > >>>>>>> # Set independent wall clock time > >>>>>>> xen.independent_wallclock=1 > >>>>>>> 3. As a third alternative, independent_wallclock=1 may also be > >>>>>>>passed > >>>>>>> as > >>>>>>> a > >>>>>>> boot parameter to the VM. > >>>>>>> " > >>>>>>> > >>>>>>> > >>>>>>> On 5/15/13 12:09 PM, "Chip Childers" <chip.child...@sungard.com> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> On Wed, May 15, 2013 at 12:02:41PM -0700, Chiradeep Vittal wrote: > >>>>>>>>> /proc/sys is not a regular filesystem and cannot be added to from > >>>>>>>>> the > >>>>>>>>> shell. > >>>>>>>>> Drivers need to add nodes into this filesystem. > >>>>>>>> > >>>>>>>> Backing up a bit... > >>>>>>>> > >>>>>>>> This is the current system VM image that the 4.1 software should > >>>>>>>>be > >>>>>>>> using on Xen: > >>>>>>>> > >>>>>>>> http://download.cloud.com/templates/acton/acton- > >>>>>>>> systemvm-02062012.vhd.bz2 > >>>>>>>> > >>>>>>>> Does this system VM have the correct drives installed to set > >>>>>>>> /proc/sys/xen to use the Dom0 time for sync? > >>>>>>>> > >>>>>>>> If so, then is this a devcloud only issue for Xen? If that's the > >>>>>>>> case, > >>>>>>>> then we shouldn't block a release to simply improve things. > >>>>>>>> > >>>>>>>> We do know that a patch for kvm was needed. > >>>>>>>> > >>>>>>>> We still don't have any > >>>>>>>> answer or comments about the VMware system VM (specifically this > >>>>>>>> template: http://download.cloud.com/templates/burbank/burbank- > >>>>>>>> systemvm-08012012.ova > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> >
