Reviewed: https://review.openstack.org/269536 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=a40ee2ce1b887751a91cc373cdd6c3415fb3e2d7 Submitter: Jenkins Branch: master
commit a40ee2ce1b887751a91cc373cdd6c3415fb3e2d7 Author: Bob Ball <bob.b...@citrix.com> Date: Tue Jan 19 10:11:10 2016 +0000 XenAPI: Fix race on rotate_xen_guest_logs There is a race between where Xen creates the domain and starts logging and when XAPI returns to say that the domain has started. If the cronjob to rotate the guest logs runs in this period, it can delete the console logs before we set the last_dom_id which is used to preserve logs. This change adds a fixed delay before deleting any log files, allowing the VM start operation to return after the domain has been successfully created. Change-Id: I82ae3e008974f33b60256366f171abf1f0369e60 Closes-Bug: 1535616 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1535616 Title: XenAPI: Console logging race condition Status in OpenStack Compute (nova): Fix Released Bug description: Unexpected API Error in XenServer CI test_get_console_output_with_unlimited_size http://dd6b71949550285df7dc- dda4e480e005aaa13ec303551d2d8155.r49.cf1.rackcdn.com/22/267822/1/2904/index.html failed with a race condition getting the guest console logs: VM started: Jan 14 22:03:58 localhost xapi: [debug|localhost.localdomain|1158 INET 0.0.0.0:80|VM.start_on R:98346b008094|audit] VM.start_on: VM = '9bbcbe52-ca94-37df-2791-e49d8dfae4c7 (instance-00000042)'; host 'd77c538b-afe9-4e0c-b5f1-223fed0f9728 (localhost.localdomain)' Cron job kicks in, identifies that domain 88 isn't "valid" and deletes the log file Jan 14 22:04:01 localhost rotate_xen_guest_logs: Valid dom IDs: 81 87 2 85 86 83 0 82 Jan 14 22:04:01 localhost rotate_xen_guest_logs: Deleting: /var/log/xen/guest/console.88 VM start completed (XAPI returns): Jan 14 22:04:03 localhost xapi: [debug|localhost.localdomain|1158 INET 0.0.0.0:80|VM.start_on R:98346b008094|xapi] Message.create VM_STARTED 5 VM 9bbcbe52-ca94-37df-2791-e49d8dfae4c7 Attempt to preserve the log file for domain 88 made: Jan 14 22:05:01 localhost xapi: [ info|localhost.localdomain|55940 UNIX /var/xapi/xapi||cli] xe vm-param-set uuid=9bbcbe52-ca94-37df-2791-e49d8dfae4c7 other-config:last_dom_id=88 username=root password=(omitted) The console log exists _before_ XAPI has completed starting the domain, so we can't preserve the log file using the last_dom_id flag. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1535616/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp