On Wed, May 15, 2013 at 10:28:01AM -0400, John Burwell wrote: > Chip, > > The issues with clock drift on the system VMs goes farther and deeper than > S3-backed Secondary Storage. Essentially, anything the system vms do that > involves time can not be trusted. For example, the timestamps of files > written by the SSVM. Bear in mind that it is possible for a system vm to > have a slow clock. Therefore, in a worst case scenario, the timestamp of > the file would be in the past breaking any logic that scans for updated > files. Additionally, correlating logs on a system VM with the management > server or other parts of the infrastructure is difficult to impossible in > these types of clock drift scenarios. In summary, when time in a > distributed system gets skewed, there are a raft of subtle but significant > operational issues that emerge. > > It is also important to note that fixing this issue is larger than simply > running NTP on the system VMs. As I noted on the ticket, each hypervisor > has a recommended approach for ensuring clock synchronization (e.g. VMWare > and KVM provide daemons and/or kernel drivers to sync clocks properly). > The proper fix for the issue will be to implement those best practices in > each hypervisor-specific system VM ISO. I think the biggest challenge to > implementing the fix will be testing more than development.
Well clarified, thanks John! > > Thanks, > -John > > > > On Wed, May 15, 2013 at 10:18 AM, Chip Childers > <chip.child...@sungard.com>wrote: > > > Starting a thread on this specific issue. > > > > CLOUDSTACK-2492 was opened, which is basically the fact that the System > > VMs aren't syncing time to the host or to an NTP server. The S3 > > integration is broken because of this problem, and therefore could not > > be considered a function available in 4.1 if we release as is. > > > > We need input from people that know about the current system VMs (the > > 3.x VMs), as well as the possibility of using the newer ones that we > > have been considering experimental for 4.1.0. > > > > What should we do? > > > > -chip > >
