Isn't there some other way we can sync the time as a triage that would
avoid a large test on the system vm template? Like adding a time flag
to one of the commonly-run system vm scripts (or even just the ones
that trigger tasks that require accurate time), which would run a
'date' command in the sy
We appear to be at an impasse here as well. I'm going to start a VOTE
on this issue, given that we have no easy answer.
On Wed, May 15, 2013 at 02:49:41PM -0700, Chiradeep Vittal wrote:
> Well, I disagree, from the perspective of hundreds of production clouds.
> No harm has been perceived in tho
On Mon, May 20, 2013 at 3:59 PM, Chip Childers
wrote:
> On Wed, May 15, 2013 at 01:11:53PM -0700, Chiradeep Vittal wrote:
>> For VMWare, the command
>> vmware-toolbox-cmd timesync status returns 'Disabled'. I can submit a
>> patch for /etc/init.d/cloud-early-config to enable it
>
> Please do!
>
>
On Wed, May 15, 2013 at 01:11:53PM -0700, Chiradeep Vittal wrote:
> For VMWare, the command
> vmware-toolbox-cmd timesync status returns 'Disabled'. I can submit a
> patch for /etc/init.d/cloud-early-config to enable it
Please do!
-chip
>
> Nothing wrong with running NTP in KVM System VM btw.
>
> Wido
>
I think they are right that it will be fighting with the hypervisor
synced clock. It might not really pose much of an issue technically,
though.
Well, I disagree, from the perspective of hundreds of production clouds.
No harm has been perceived in those clouds due to this defect. If they can
live with it for several years, then they can live with it for a few more
months.
On 5/15/13 2:35 PM, "Wido den Hollander" wrote:
>
>
>On 05/15/2013
On 05/15/2013 11:33 PM, John Burwell wrote:
> Chiradeep,
>
> I disagree regarding the impact of this issue. Anyone running an SSVM will
> be affected by this issue because clocks will eventually drift (sooner rather
> than later) and when they do, any timestamps rendered by a system VM will
Chiradeep,
I disagree regarding the impact of this issue. Anyone running an SSVM will be
affected by this issue because clocks will eventually drift (sooner rather than
later) and when they do, any timestamps rendered by a system VM will unreliable
(e.g. files creation and modified times, log
Sure, I agree. But I'd also point out that for the vast majority of
CloudStack users (4.1 at least), this is not going to be an issue. I
suggest deferring this to 4.1.1
A new template (easy or not) does require a full regression QA round.
On 5/15/13 2:07 PM, "John Burwell" wrote:
>Chiradeep,
>
>
All,
One other point to consider is that we only want NTP running for Xen system
VMs. Running NTP and VMWare Tools together causes big problems because the two
fight each other to sync the clock. I don;t know about KVM, but I wouldn't be
surprised if it doesn't have same types of problems.
T
Chiradeep,
As I think thought it, I don't think a documentation note will sufficient
because the SSVM can be destroyed and respawned by CloudStack without
intervention by a human. Therefore, we can get into a situation where an
operator installs and configures NTP, and then at some point in th
On Wed, May 15, 2013 at 01:59:13PM -0700, Chiradeep Vittal wrote:
> Did some further digging around as to why
> /proc/sys/xen/independent_wallclock is not there on the Debian system vm.
>
> TLDR; the kernel is PvOps (I.e., just a regular kernel that works like a
> PV kernel, not a specialized para
Did some further digging around as to why
/proc/sys/xen/independent_wallclock is not there on the Debian system vm.
TLDR; the kernel is PvOps (I.e., just a regular kernel that works like a
PV kernel, not a specialized paravirt kernel). To eliminate
special-casing, the independent_wallclock feature
For VMWare, the command
vmware-toolbox-cmd timesync status returns 'Disabled'. I can submit a
patch for /etc/init.d/cloud-early-config to enable it
On 5/15/13 12:23 PM, "Chiradeep Vittal"
wrote:
>I am not sure why it is missing, but I will refer to
>Citrix XenServer 6.0 Virtual Machine Installa
I am not sure why it is missing, but I will refer to
Citrix XenServer 6.0 Virtual Machine Installation Guide
http://s.apache.org/YGn
And quote
"Time Handling in Linux VMs
By default, the clocks in a Linux VM are synchronized to the clock running
on the control domain, and cannot be
independently
On Wed, May 15, 2013 at 12:02:41PM -0700, Chiradeep Vittal wrote:
> /proc/sys is not a regular filesystem and cannot be added to from the
> shell.
> Drivers need to add nodes into this filesystem.
Backing up a bit...
This is the current system VM image that the 4.1 software should be
using on Xen
/proc/sys is not a regular filesystem and cannot be added to from the
shell.
Drivers need to add nodes into this filesystem.
On 5/15/13 11:50 AM, "Chip Childers" wrote:
>On Wed, May 15, 2013 at 02:48:13PM -0400, John Burwell wrote:
>> All of these things being said, it appears that the Xen behav
On Wed, May 15, 2013 at 02:48:13PM -0400, John Burwell wrote:
> All of these things being said, it appears that the Xen behavior may be a
> regression that can be addressed with a relatively straightforward fix
> (dropping the proper file in /proc/sys/xen),
Agreed. Anyone want to submit the pa
Chiradeep,
As I mentioned earlier, this issue is larger than S3-backed Secondary Storage.
It just happens that this issue was surfaced by testing that feature.Clock
drift exceeding than a few seconds can be operational issue (e.g. file
timestamps, logging, etc). A lack of reliable clock s
The previous ones were on XS 5.6 FP2
This one's on XS 6.0.2
r-275166-VM 18:22:10 up 8 days,
domU: Wed May 15 18:22:10 UTC 2013
dom0: Wed May 15 11:22:10 PDT 2013
On 5/15/13 11:22 AM, "Chiradeep Vittal"
wrote:
>The normal S3 time sync is 15 minutes. I can't imagine a drift of 15
>minutes in a f
The normal S3 time sync is 15 minutes. I can't imagine a drift of 15
minutes in a few days of operation? I logged into 3 system vms running on
Xen and saw this drift:
r-9-VM 17:52:37 up 29 days, 10:33,
domU: Wed May 15 17:52:37 UTC 2013
dom0: Wed May 15 10:52:37 PDT 2013
r-535-VM 18:13:46 up 43 d
0_clock.3F,
> which might cause time drift, maybe we need to enable NTP inside system VM.
>
> Anthony
>
>
>
>
> -Original Message-
> From: John Burwell [mailto:jburw...@basho.com]
> Sent: Wednesday, May 15, 2013 10:30 AM
> To: dev@cloudstack.apache.org
>
e.org
Subject: Re: [ACS41] System VMs not syncing time - does this block the release?
Chiradeep,
The issue I am experiencing is that the system VMs are not syncing to dom0 on
devcloud (i.e. the dom0 clock and the SSVM clock are different). As I
mentioned earlier in this thread, the syncing was wo
Chiradeep,
The issue I am experiencing is that the system VMs are not syncing to dom0
on devcloud (i.e. the dom0 clock and the SSVM clock are different). As I
mentioned earlier in this thread, the syncing was working previously which
seems to jibe with your findings. What mechanism is used to sy
Perhaps this is a problem with DevCloud?
http://nerdboys.com/2011/03/15/how-to-fix-virtualbox-time-synchonization-pr
oblems/
On 5/15/13 10:17 AM, "Chiradeep Vittal"
wrote:
>According to our resident Xen expert, any PV kernel automatically syncs to
>the hardware clock on dom0.
>
>On 5/15/13 9:5
According to our resident Xen expert, any PV kernel automatically syncs to
the hardware clock on dom0.
On 5/15/13 9:50 AM, "John Burwell" wrote:
>Marcus,
>
>Agreed. I think we need to add a set of hypervisor agnostic time
>keeping guidelines to the documentation. I just wanted to make sure
>t
Marcus,
Agreed. I think we need to add a set of hypervisor agnostic time keeping
guidelines to the documentation. I just wanted to make sure there wasn't
anything KVM specific that should be added as well.
Thanks,
-John
On May 15, 2013, at 12:48 PM, Marcus Sorensen wrote:
> Just the gener
Just the general one that system vms sync their time to the
hypervisor, thus admins need to keep the hypervisor time correct. It
sounds like that will be the case for all three, if we can manage it.
On Wed, May 15, 2013 at 10:44 AM, John Burwell wrote:
> Marcus,
>
> Excellent. So, it looks like
Marcus,
Linux seems like reasonable assumption to me for the 4.1 and 4.2 releases
because there doesn't appear a way to easily override the images. I think we
need to reconsider how we provision system VMs post-4.2 to address a number of
issues. If/when we address those issues, it become an i
Marcus,
Excellent. So, it looks like we have KVM resolved. We just need to address
Xen and VMWare now. Do you think we need to any guidance to the documentation
regarding KVM time keeping (e.g. environmental prerequisites)?
Thanks,
-John
On May 15, 2013, at 12:39 PM, Marcus Sorensen wrot
Note that this means system vms are required to be linux, as
'kvmclock' doesn't work on other OSes. I haven't seen anyone proposing
Windows or BSD system vms, so this seemed safe, but perhaps something
to keep in mind.
On Wed, May 15, 2013 at 10:39 AM, Marcus Sorensen wrote:
> KVM LibvirtComputin
KVM LibvirtComputingResource has been patched in master. Tested on
master, 4.1, and both the acton and current system vm templates. This
patch makes system vms use 'kvmclock' for their timer, which is a vm
driver that gets it's time from the hypervisor. No change to the
system vm template itself.
On Wed, May 15, 2013 at 11:03:16AM -0400, John Burwell wrote:
> Chip,
>
> One other item I neglected to mention was that clock sync, at least for Xen
> system VMs, wasn't an issue in the Jan-Feb timeframe. Previously when I
> encountered these issues, syncing the host's clock and rebuilding the
Chip,
One other item I neglected to mention was that clock sync, at least for Xen
system VMs, wasn't an issue in the Jan-Feb timeframe. Previously when I
encountered these issues, syncing the host's clock and rebuilding the system
VMs addressed the issue. I assumed, but never verified, that t
On Wed, May 15, 2013 at 10:28:01AM -0400, John Burwell wrote:
> Chip,
>
> The issues with clock drift on the system VMs goes farther and deeper than
> S3-backed Secondary Storage. Essentially, anything the system vms do that
> involves time can not be trusted. For example, the timestamps of file
Chip,
The issues with clock drift on the system VMs goes farther and deeper than
S3-backed Secondary Storage. Essentially, anything the system vms do that
involves time can not be trusted. For example, the timestamps of files
written by the SSVM. Bear in mind that it is possible for a system vm
Starting a thread on this specific issue.
CLOUDSTACK-2492 was opened, which is basically the fact that the System
VMs aren't syncing time to the host or to an NTP server. The S3
integration is broken because of this problem, and therefore could not
be considered a function available in 4.1 if we
37 matches
Mail list logo
