Public bug reported:
Environment:
- Two compute nodes, running Ubuntu 12.04 LTS
- KVM Hypervisor
- Ceph (dumpling) back-end for Cinder
- Grizzly-level Openstack
Steps to reproduce:
1) Create instance and volume
2) Attach volume to instance
3) Attempt a block migration between compute nodes - eg: nova live-migration
--block-migrate 9b85b983-dced-4574-b14c-c72e4d92982a
Packages:
ii ceph 0.67.5-1precise
ii ceph-common 0.67.5-1precise
ii ceph-fs-common 0.67.5-1precise
ii ceph-fuse 0.67.5-1precise
ii ceph-mds 0.67.5-1precise
ii curl 7.29.0-1precise.ceph
ii kvm 1:84+dfsg-0ubuntu16+1.0+noroms+0ubuntu14.13
ii kvm-ipxe 1.0.0+git-3.55f6c88-0ubuntu1
ii libcephfs1 0.67.5-1precise
ii libcurl3 7.29.0-1precise.ceph
ii libcurl3-gnutls 7.29.0-1precise.ceph
ii libleveldb1 1.12.0-1precise.ceph
ii nova-common 1:2013.1.4-0ubuntu1~cloud0
ii nova-compute 1:2013.1.4-0ubuntu1~cloud0
ii nova-compute-kvm 1:2013.1.4-0ubuntu1~cloud0
ii python-ceph 0.67.5-1precise
ii python-cinderclient 1:1.0.3-0ubuntu1~cloud0
ii python-nova 1:2013.1.4-0ubuntu1~cloud0
ii python-novaclient 1:2.13.0-0ubuntu1~cloud0
ii qemu-common 1.0+noroms-0ubuntu14.13
ii qemu-kvm 1.0+noroms-0ubuntu14.13
ii qemu-utils 1.0+noroms-0ubuntu14.13
ii libvirt-bin 1.0.2-0ubuntu11.13.04.5~cloud1
ii libvirt0 1.0.2-0ubuntu11.13.04.5~cloud1
ii python-libvirt 1.0.2-0ubuntu11.13.04.5~cloud1
/var/log/nova/nova-compute on source:
2014-02-05 16:36:46.014 998 INFO nova.compute.manager [-] Lifecycle event 2 on
VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:36:46.233 998 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During sync_power_state the instance has
a pending task. Skip.
2014-02-05 16:36:46.234 998 INFO nova.compute.manager [-] Lifecycle event 2 on
VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:36:46.468 998 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During sync_power_state the instance has
a pending task. Skip.
2014-02-05 16:41:09.029 998 INFO nova.compute.manager [-] Lifecycle event 1 on
VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:41:09.265 998 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During sync_power_state the instance has
a pending task. Skip.
2014-02-05 16:41:09.640 998 ERROR nova.virt.libvirt.driver [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Live Migration failure: Unable to read
from monitor: Connection reset by peer
2014-02-05 16:41:12.165 998 WARNING nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Instance shutdown by itself. Calling the
stop API.
2014-02-05 16:41:12.398 998 INFO nova.virt.libvirt.driver [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Instance destroyed successfully.
/var/log/libvirt/libvirtd.log on source:
2014-02-05 14:41:07.607+0000: 3437: error : qemuMonitorIORead:502 : Unable to
read from monitor: Connection reset by peer
2014-02-05 14:41:09.633+0000: 3441: error :
virNetClientProgramDispatchError:175 : An error occurred, but the cause is
unknown
2014-02-05 14:41:09.634+0000: 3441: error :
qemuDomainObjEnterMonitorInternal:997 : operation failed: domain is no longer
running
2014-02-05 14:41:09.634+0000: 3441: warning : doPeer2PeerMigrate3:2872 : Guest
instance-00000315 probably left in 'paused' state on source
/var/log/nova/nova-compute.log on target:
2014-02-05 16:36:38.841 INFO nova.virt.libvirt.driver
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] Instance launched has CPU info:
{"vendor": "Intel", "model": "SandyBridge", "arch": "x86_64", "features":
["pdpe1gb", "osxsave", "dca", "pcid", "pdcm", "xtpr", "tm2", "est", "smx",
"vmx", "ds_cpl", "monitor", "dtes64", "pbe", "tm", "ht", "ss", "acpi", "ds",
"vme"], "topology": {"cores": 6, "threads": 2, "sockets": 1}}
2014-02-05 16:36:46.008 28458 INFO nova.compute.manager [-] Lifecycle event 0
on VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:36:46.244 28458 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During the sync_power process the
instance has moved from host ctpcmp003 to host ctpcmp005
2014-02-05 16:41:09.634 28458 INFO nova.compute.manager [-] Lifecycle event 1
on VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:41:09.899 28458 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During the sync_power process the
instance has moved from host ctpcmp003 to host ctpcmp005
2014-02-05 16:41:11.056 AUDIT nova.compute.manager
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Detach volume
7e2f873e-f7ba-4ee1-85ee-cee133cefdcf from mountpoint /dev/vdc
2014-02-05 16:41:11.061 WARNING nova.compute.manager
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Detaching volume from unknown instance
2014-02-05 16:41:11.066 ERROR nova.compute.manager
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Failed to detach volume
7e2f873e-f7ba-4ee1-85ee-cee133cefdcf from /dev/vdc
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Traceback (most recent call last):
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2983, in
_detach_volume
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] mp)
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1016, in
detach_volume
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] virt_dom =
self._lookup_by_name(instance_name)
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2386, in
_lookup_by_name
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] raise
exception.InstanceNotFound(instance_id=instance_name)
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] InstanceNotFound: Instance
instance-00000315 could not be found.
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a]
2014-02-05 16:41:12.261 28458 ERROR nova.virt.libvirt.driver [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During wait destroy, instance disappeared.
2014-02-05 16:41:12.872 INFO nova.virt.libvirt.driver
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Deleting instance files
/var/lib/nova/instances/9b85b983-dced-4574-b14c-c72e4d92982a
/etc/nova/nova.conf includes the directive: vncserver_listen=0.0.0.0
root@ctpcmp003:~# grep "tls\|tcp" /etc/libvirt/libvirtd.conf | grep -v "^#"
listen_tls = 0
listen_tcp = 1
auth_tcp = "none"
root@ctpcmp003:~# grep vncserver_listen /etc/nova/nova.conf
vncserver_listen=0.0.0.0
root@ctpcmp003:~# ps -ef | grep libvirtd
root 20608 1 0 16:29 ? 00:00:11 /usr/sbin/libvirtd -d -l
** Affects: nova
Importance: Undecided
Status: New
** Description changed:
Environment:
- - Two compute nodes, running Ubuntu 12.04 LTS
- - KVM Hypervisor
- - Ceph (dumpling) back-end for Cinder
- - Grizzly-level Openstack
+ - Two compute nodes, running Ubuntu 12.04 LTS
+ - KVM Hypervisor
+ - Ceph (dumpling) back-end for Cinder
+ - Grizzly-level Openstack
Steps to reproduce:
- 1) Create instance and volume
- 2) Attach volume to instance
- 3) Attempt a block migration between compute nodes
+ 1) Create instance and volume
+ 2) Attach volume to instance
+ 3) Attempt a block migration between compute nodes
Packages:
- ii ceph 0.67.5-1precise
distributed storage and file system
- ii ceph-common 0.67.5-1precise
common utilities to mount and interact with a ceph storage cluster
- ii ceph-fs-common 0.67.5-1precise
common utilities to mount and interact with a ceph file system
- ii ceph-fuse 0.67.5-1precise
FUSE-based client for the Ceph distributed file system
- ii ceph-mds 0.67.5-1precise
metadata server for the ceph distributed file system
- ii curl 7.29.0-1precise.ceph
command line tool for transferring data with URL syntax
- ii kvm
1:84+dfsg-0ubuntu16+1.0+noroms+0ubuntu14.13 dummy transitional package from kvm
to qemu-kvm
- ii kvm-ipxe 1.0.0+git-3.55f6c88-0ubuntu1
PXE ROM's for KVM
- ii libcephfs1 0.67.5-1precise
Ceph distributed file system client library
- ii libcurl3 7.29.0-1precise.ceph
easy-to-use client-side URL transfer library (OpenSSL flavour)
- ii libcurl3-gnutls 7.29.0-1precise.ceph
easy-to-use client-side URL transfer library (GnuTLS flavour)
- ii libleveldb1 1.12.0-1precise.ceph
fast key-value storage library
- ii nova-common 1:2013.1.4-0ubuntu1~cloud0
OpenStack Compute - common files
- ii nova-compute 1:2013.1.4-0ubuntu1~cloud0
OpenStack Compute - compute node
- ii nova-compute-kvm 1:2013.1.4-0ubuntu1~cloud0
OpenStack Compute - compute node (KVM)
- ii python-ceph 0.67.5-1precise
Python libraries for the Ceph distributed filesystem
- ii python-cinderclient 1:1.0.3-0ubuntu1~cloud0
python bindings to the OpenStack Volume API
- ii python-nova 1:2013.1.4-0ubuntu1~cloud0
OpenStack Compute Python libraries
- ii python-novaclient 1:2.13.0-0ubuntu1~cloud0
client library for OpenStack Compute API
- ii qemu-common 1.0+noroms-0ubuntu14.13
qemu common functionality (bios, documentation, etc)
- ii qemu-kvm 1.0+noroms-0ubuntu14.13
Full virtualization on i386 and amd64 hardware
- ii qemu-utils 1.0+noroms-0ubuntu14.13
qemu utilities
- ii libvirt-bin 1.0.2-0ubuntu11.13.04.5~cloud1
programs for the libvirt library
- ii libvirt0 1.0.2-0ubuntu11.13.04.5~cloud1
library for interfacing with different virtualization systems
- ii python-libvirt 1.0.2-0ubuntu11.13.04.5~cloud1
libvirt Python bindings
+ ii ceph 0.67.5-1precise
+ ii ceph-common 0.67.5-1precise
+ ii ceph-fs-common 0.67.5-1precise
+ ii ceph-fuse 0.67.5-1precise
+ ii ceph-mds 0.67.5-1precise
+ ii curl 7.29.0-1precise.ceph
+ ii kvm
1:84+dfsg-0ubuntu16+1.0+noroms+0ubuntu14.13
+ ii kvm-ipxe 1.0.0+git-3.55f6c88-0ubuntu1
+ ii libcephfs1 0.67.5-1precise
+ ii libcurl3 7.29.0-1precise.ceph
+ ii libcurl3-gnutls 7.29.0-1precise.ceph
+ ii libleveldb1 1.12.0-1precise.ceph
+ ii nova-common 1:2013.1.4-0ubuntu1~cloud0
+ ii nova-compute 1:2013.1.4-0ubuntu1~cloud0
+ ii nova-compute-kvm 1:2013.1.4-0ubuntu1~cloud0
+ ii python-ceph 0.67.5-1precise
+ ii python-cinderclient 1:1.0.3-0ubuntu1~cloud0
+ ii python-nova 1:2013.1.4-0ubuntu1~cloud0
+ ii python-novaclient 1:2.13.0-0ubuntu1~cloud0
+ ii qemu-common 1.0+noroms-0ubuntu14.13
+ ii qemu-kvm 1.0+noroms-0ubuntu14.13
+ ii qemu-utils 1.0+noroms-0ubuntu14.13
+ ii libvirt-bin 1.0.2-0ubuntu11.13.04.5~cloud1
+ ii libvirt0 1.0.2-0ubuntu11.13.04.5~cloud1
+ ii python-libvirt 1.0.2-0ubuntu11.13.04.5~cloud1
/var/log/nova/nova-compute on source:
2014-02-05 16:36:46.014 998 INFO nova.compute.manager [-] Lifecycle event 2
on VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:36:46.233 998 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During sync_power_state the instance has
a pending task. Skip.
2014-02-05 16:36:46.234 998 INFO nova.compute.manager [-] Lifecycle event 2
on VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:36:46.468 998 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During sync_power_state the instance has
a pending task. Skip.
2014-02-05 16:41:09.029 998 INFO nova.compute.manager [-] Lifecycle event 1
on VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:41:09.265 998 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During sync_power_state the instance has
a pending task. Skip.
2014-02-05 16:41:09.640 998 ERROR nova.virt.libvirt.driver [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Live Migration failure: Unable to read
from monitor: Connection reset by peer
2014-02-05 16:41:12.165 998 WARNING nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Instance shutdown by itself. Calling the
stop API.
2014-02-05 16:41:12.398 998 INFO nova.virt.libvirt.driver [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Instance destroyed successfully.
-
/var/log/libvirt/libvirtd.log on source:
2014-02-05 14:41:07.607+0000: 3437: error : qemuMonitorIORead:502 : Unable to
read from monitor: Connection reset by peer
2014-02-05 14:41:09.633+0000: 3441: error :
virNetClientProgramDispatchError:175 : An error occurred, but the cause is
unknown
2014-02-05 14:41:09.634+0000: 3441: error :
qemuDomainObjEnterMonitorInternal:997 : operation failed: domain is no longer
running
2014-02-05 14:41:09.634+0000: 3441: warning : doPeer2PeerMigrate3:2872 :
Guest instance-00000315 probably left in 'paused' state on source
-
/var/log/nova/nova-compute.log on target:
2014-02-05 16:36:38.841 INFO nova.virt.libvirt.driver
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] Instance launched has CPU info:
{"vendor": "Intel", "model": "SandyBridge", "arch": "x86_64", "features":
["pdpe1gb", "osxsave", "dca", "pcid", "pdcm", "xtpr", "tm2", "est", "smx",
"vmx", "ds_cpl", "monitor", "dtes64", "pbe", "tm", "ht", "ss", "acpi", "ds",
"vme"], "topology": {"cores": 6, "threads": 2, "sockets": 1}}
2014-02-05 16:36:46.008 28458 INFO nova.compute.manager [-] Lifecycle event 0
on VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:36:46.244 28458 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During the sync_power process the
instance has moved from host ctpcmp003 to host ctpcmp005
2014-02-05 16:41:09.634 28458 INFO nova.compute.manager [-] Lifecycle event 1
on VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:41:09.899 28458 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During the sync_power process the
instance has moved from host ctpcmp003 to host ctpcmp005
2014-02-05 16:41:11.056 AUDIT nova.compute.manager
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Detach volume
7e2f873e-f7ba-4ee1-85ee-cee133cefdcf from mountpoint /dev/vdc
2014-02-05 16:41:11.061 WARNING nova.compute.manager
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Detaching volume from unknown instance
2014-02-05 16:41:11.066 ERROR nova.compute.manager
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Failed to detach volume
7e2f873e-f7ba-4ee1-85ee-cee133cefdcf from /dev/vdc
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Traceback (most recent call last):
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2983, in
_detach_volume
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] mp)
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1016, in
detach_volume
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] virt_dom =
self._lookup_by_name(instance_name)
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2386, in
_lookup_by_name
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] raise
exception.InstanceNotFound(instance_id=instance_name)
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] InstanceNotFound: Instance
instance-00000315 could not be found.
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a]
2014-02-05 16:41:12.261 28458 ERROR nova.virt.libvirt.driver [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During wait destroy, instance disappeared.
2014-02-05 16:41:12.872 INFO nova.virt.libvirt.driver
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Deleting instance files
/var/lib/nova/instances/9b85b983-dced-4574-b14c-c72e4d92982a
/etc/nova/nova.conf includes the directive: vncserver_listen=0.0.0.0
root@ctpcmp003:~# grep "tls\|tcp" /etc/libvirt/libvirtd.conf | grep -v "^#"
listen_tls = 0
listen_tcp = 1
auth_tcp = "none"
root@ctpcmp003:~# grep vncserver_listen /etc/nova/nova.conf
vncserver_listen=0.0.0.0
root@ctpcmp003:~# ps -ef | grep libvirtd
root 20608 1 0 16:29 ? 00:00:11 /usr/sbin/libvirtd -d -l
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1276639
Title:
block live migration does not work when a volume is attached
Status in OpenStack Compute (Nova):
New
Bug description:
Environment:
- Two compute nodes, running Ubuntu 12.04 LTS
- KVM Hypervisor
- Ceph (dumpling) back-end for Cinder
- Grizzly-level Openstack
Steps to reproduce:
1) Create instance and volume
2) Attach volume to instance
3) Attempt a block migration between compute nodes - eg: nova live-migration
--block-migrate 9b85b983-dced-4574-b14c-c72e4d92982a
Packages:
ii ceph 0.67.5-1precise
ii ceph-common 0.67.5-1precise
ii ceph-fs-common 0.67.5-1precise
ii ceph-fuse 0.67.5-1precise
ii ceph-mds 0.67.5-1precise
ii curl 7.29.0-1precise.ceph
ii kvm
1:84+dfsg-0ubuntu16+1.0+noroms+0ubuntu14.13
ii kvm-ipxe 1.0.0+git-3.55f6c88-0ubuntu1
ii libcephfs1 0.67.5-1precise
ii libcurl3 7.29.0-1precise.ceph
ii libcurl3-gnutls 7.29.0-1precise.ceph
ii libleveldb1 1.12.0-1precise.ceph
ii nova-common 1:2013.1.4-0ubuntu1~cloud0
ii nova-compute 1:2013.1.4-0ubuntu1~cloud0
ii nova-compute-kvm 1:2013.1.4-0ubuntu1~cloud0
ii python-ceph 0.67.5-1precise
ii python-cinderclient 1:1.0.3-0ubuntu1~cloud0
ii python-nova 1:2013.1.4-0ubuntu1~cloud0
ii python-novaclient 1:2.13.0-0ubuntu1~cloud0
ii qemu-common 1.0+noroms-0ubuntu14.13
ii qemu-kvm 1.0+noroms-0ubuntu14.13
ii qemu-utils 1.0+noroms-0ubuntu14.13
ii libvirt-bin 1.0.2-0ubuntu11.13.04.5~cloud1
ii libvirt0 1.0.2-0ubuntu11.13.04.5~cloud1
ii python-libvirt 1.0.2-0ubuntu11.13.04.5~cloud1
/var/log/nova/nova-compute on source:
2014-02-05 16:36:46.014 998 INFO nova.compute.manager [-] Lifecycle event 2
on VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:36:46.233 998 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During sync_power_state the instance has
a pending task. Skip.
2014-02-05 16:36:46.234 998 INFO nova.compute.manager [-] Lifecycle event 2
on VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:36:46.468 998 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During sync_power_state the instance has
a pending task. Skip.
2014-02-05 16:41:09.029 998 INFO nova.compute.manager [-] Lifecycle event 1
on VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:41:09.265 998 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During sync_power_state the instance has
a pending task. Skip.
2014-02-05 16:41:09.640 998 ERROR nova.virt.libvirt.driver [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Live Migration failure: Unable to read
from monitor: Connection reset by peer
2014-02-05 16:41:12.165 998 WARNING nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Instance shutdown by itself. Calling the
stop API.
2014-02-05 16:41:12.398 998 INFO nova.virt.libvirt.driver [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Instance destroyed successfully.
/var/log/libvirt/libvirtd.log on source:
2014-02-05 14:41:07.607+0000: 3437: error : qemuMonitorIORead:502 : Unable to
read from monitor: Connection reset by peer
2014-02-05 14:41:09.633+0000: 3441: error :
virNetClientProgramDispatchError:175 : An error occurred, but the cause is
unknown
2014-02-05 14:41:09.634+0000: 3441: error :
qemuDomainObjEnterMonitorInternal:997 : operation failed: domain is no longer
running
2014-02-05 14:41:09.634+0000: 3441: warning : doPeer2PeerMigrate3:2872 :
Guest instance-00000315 probably left in 'paused' state on source
/var/log/nova/nova-compute.log on target:
2014-02-05 16:36:38.841 INFO nova.virt.libvirt.driver
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] Instance launched has CPU info:
{"vendor": "Intel", "model": "SandyBridge", "arch": "x86_64", "features":
["pdpe1gb", "osxsave", "dca", "pcid", "pdcm", "xtpr", "tm2", "est", "smx",
"vmx", "ds_cpl", "monitor", "dtes64", "pbe", "tm", "ht", "ss", "acpi", "ds",
"vme"], "topology": {"cores": 6, "threads": 2, "sockets": 1}}
2014-02-05 16:36:46.008 28458 INFO nova.compute.manager [-] Lifecycle event 0
on VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:36:46.244 28458 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During the sync_power process the
instance has moved from host ctpcmp003 to host ctpcmp005
2014-02-05 16:41:09.634 28458 INFO nova.compute.manager [-] Lifecycle event 1
on VM 9b85b983-dced-4574-b14c-c72e4d92982a
2014-02-05 16:41:09.899 28458 INFO nova.compute.manager [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During the sync_power process the
instance has moved from host ctpcmp003 to host ctpcmp005
2014-02-05 16:41:11.056 AUDIT nova.compute.manager
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Detach volume
7e2f873e-f7ba-4ee1-85ee-cee133cefdcf from mountpoint /dev/vdc
2014-02-05 16:41:11.061 WARNING nova.compute.manager
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Detaching volume from unknown instance
2014-02-05 16:41:11.066 ERROR nova.compute.manager
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Failed to detach volume
7e2f873e-f7ba-4ee1-85ee-cee133cefdcf from /dev/vdc
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Traceback (most recent call last):
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2983, in
_detach_volume
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] mp)
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1016, in
detach_volume
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] virt_dom =
self._lookup_by_name(instance_name)
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2386, in
_lookup_by_name
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] raise
exception.InstanceNotFound(instance_id=instance_name)
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] InstanceNotFound: Instance
instance-00000315 could not be found.
2014-02-05 16:41:11.066 28458 TRACE nova.compute.manager [instance:
9b85b983-dced-4574-b14c-c72e4d92982a]
2014-02-05 16:41:12.261 28458 ERROR nova.virt.libvirt.driver [-] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] During wait destroy, instance disappeared.
2014-02-05 16:41:12.872 INFO nova.virt.libvirt.driver
[req-0f0eaabf-9e29-4d45-88c9-20194be51d49 aaf3e92b69e04958b43348677ab7b38b
1859d80f51ff4180b591f7fe2668fd68] [instance:
9b85b983-dced-4574-b14c-c72e4d92982a] Deleting instance files
/var/lib/nova/instances/9b85b983-dced-4574-b14c-c72e4d92982a
/etc/nova/nova.conf includes the directive: vncserver_listen=0.0.0.0
root@ctpcmp003:~# grep "tls\|tcp" /etc/libvirt/libvirtd.conf | grep -v "^#"
listen_tls = 0
listen_tcp = 1
auth_tcp = "none"
root@ctpcmp003:~# grep vncserver_listen /etc/nova/nova.conf
vncserver_listen=0.0.0.0
root@ctpcmp003:~# ps -ef | grep libvirtd
root 20608 1 0 16:29 ? 00:00:11 /usr/sbin/libvirtd -d -l
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1276639/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
