[
https://issues.apache.org/jira/browse/CLOUDSTACK-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13657643#comment-13657643
]
John Burwell commented on CLOUDSTACK-2492:
------------------------------------------
My first brush expectation is that each hypervisor's system vm image will need
to be modified to implement the clock synchronization strategy/best practice
for the hypervisor. The following are some links discussing clock sync per
hypervisor:
* Xen
-
http://wiki.xensource.com/wiki/Xen_FAQ_DomU#How_can_i_synchronize_a_dom0_clock.3F
* KVM
-
http://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/chap-Virtualization-KVM_guest_timing_management.html
- http://s19n.net/articles/2011/kvm_clock.html
* VMWare
-
http://www.vmware.com/files/pdf/techpaper/Timekeeping-In-VirtualMachines.pdf
> System VM Clock Drift
> ---------------------
>
> Key: CLOUDSTACK-2492
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-2492
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Components: ISO
> Affects Versions: 4.1.0
> Environment: devcloud/Xen
> Reporter: John Burwell
> Priority: Blocker
>
> Testing of S3-backed Secondary Storage has revealed that the SSVM (and likely
> all other system VMs) have no provision for clock synchronization (e.g. NTP
> to dom0 for Xen, vmware-tools for VMWare, etc). In particular, the S3
> protocol is sensitive to drift between clients and S3. As an example, the
> following is the stack trace caused by clock drift S3:
> 2013-05-14 06:51:55,400 DEBUG [cloud.utils.S3Utils] (agentRequest-Handler-3:)
> Putting directory
> /mnt/SecStorage/93fd0cb0-033b-3248-bcd0-ef6d460635ef/template/tmpl/1/5 in S3
> bucket jsb-cloudstack-templates.
> 2013-05-14 06:51:55,401 DEBUG [cloud.utils.S3Utils] (agentRequest-Handler-3:)
> Creating S3 client with configuration: [protocol: https, connectionTimeOut:
> 50000, maxErrorRetry: 3, socketTimeout: 50000]
> 2013-05-14 06:51:55,403 DEBUG [storage.resource.NfsSecondaryStorageResource]
> (agentRequest-Handler-3:) Determining key using account id 1 and template id 5
> 2013-05-14 06:51:55,403 DEBUG [cloud.utils.S3Utils] (agentRequest-Handler-3:)
> Putting file
> /mnt/SecStorage/93fd0cb0-033b-3248-bcd0-ef6d460635ef/template/tmpl/1/5/template.properties
> into bucket jsb-cloudstack-templates with key
> template/tmpl/1/5/template.properties.
> 2013-05-14 06:51:55,578 ERROR [storage.resource.NfsSecondaryStorageResource]
> (agentRequest-Handler-3:) Failed to upload template id 5
> Status Code: 403, AWS Service: Amazon S3, AWS Request ID: 970A274E132A9ACB,
> AWS Error Code: RequestTimeTooSkewed, AWS Error Message: The difference
> between the request time and the current time is too large., S3 Extended
> Request ID: 9w8a6YBxTn+WlBg96s9stxWuuP8oQ7ksZtg6++wVRHJfE2qmucrilhoEJVetJui4
> at
> com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:609)
> at
> com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:309)
> at
> com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:164)
> at
> com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:2863)
> at
> com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1100)
> at
> com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:963)
> at com.cloud.utils.S3Utils.putDirectory(S3Utils.java:282)
> at
> com.cloud.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:414)
> at
> com.cloud.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:212)
> at com.cloud.agent.Agent.processRequest(Agent.java:525)
> at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)
> at com.cloud.utils.nio.Task.run(Task.java:83)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> In addition to impacting S3, this clock drift also makes log correlation
> between the management server and system VMs very difficult, if not,
> impossible. Finally, there are suspicions that the clock drift could also
> impact operation of console proxy and virtual router VMs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
