[Yahoo-eng-team] [Bug 1423654] Re: Nova rescue causes LVM timeouts after moving attachments
from the last comment it seems to me that this bug is not going to be fixed in Nova but a workaround is in place in Cinder. Considering that and considering that the last comment was 1+ year ago I am going to close this ticket as "Won't fix". Please feel free to comment here if you disagree. ** Changed in: nova Status: Confirmed => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1423654 Title: Nova rescue causes LVM timeouts after moving attachments Status in OpenStack Compute (nova): Won't Fix Bug description: The Nova rescue feature powers off a running instance and, boots a rescue instance attaching the ephemeral disk of the original instance to it to allow an admin to try and recover the instance. The problem is that if a Cinder Volume is attached to that instance when we do a rescue we don't do a detach or any sort of maintenance on the block mapping that we have set up for it. We do check to see if we have it, and verify it's attached but that's it. The result is that after the rescue operation subsequent LVM calls to do things like lvs and vgs will attempt to open a device file that no longer exists which takes up to 60 seconds for each device. An example is the current tempest test: tempest.api.compute.servers.test_server_rescue_negative.ServerRescueNegativeTestJSON.test_rescued_vm_detach_volume[gate,negative,volume] Which if you look at tempest results you'll notice that test always takes in excess of 100 seconds, but it's not just because it's a long test, it's the blocking LVM calls. We should detach any cinder volumes that are attached to an instance during the rescue process. One concern with this that came from folks on the Nova team was 'what about boot from volume'? Rescue of a volume booted instance is currently an invalid case as is evident by the code that checks for it and fails here: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L2822 Probably no reason we can't automate this as part of rescue in the future but for now it's a separate enhancement independent of this bug. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1423654/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1604798] Re: Use DDT library to reduce code duplication
I agree with Markus that this is not a bug. +1 on the discussion on the ML and if there will be any interest than please submit a blueprint and/or a spec to better discuss the proposal. I am going to mark the bug as invalid. ** Changed in: nova Status: In Progress => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1604798 Title: Use DDT library to reduce code duplication Status in OpenStack Compute (nova): Invalid Bug description: Use DDT library to reduce code duplication DDT can ease up error tracing and autogenerates tests on basis of different input data. It allows to multiply one test case by running it with different test data, and make it appear as multiple test cases. This will help to reduce code duplication. Please refer example use: http://ddt.readthedocs.io/en/latest/example.html Currently DDT is implemented in openstack/cinder and openstack/rally. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1604798/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1588378] [NEW] Cancelled live migration are reported as in progress
Public bug reported: With the introduction of the API for aborting a running live-migration (https://review.openstack.org/277971) we have introduced a new status for the aborted live migration jobs. This new status called "cancelled" is not filtered by the sqlalchemy query used to return the list of migration in progress: https://github.com/openstack/nova/blob/87dc738763d6a7a10409e14b878f5cdd39ba805e/nova/db/sqlalchemy/api.py#L4851 ** Affects: nova Importance: Low Assignee: Andrea Rosa (andrea-rosa-m) Status: In Progress ** Tags: libvirt live-migration ** Changed in: nova Assignee: (unassigned) => Andrea Rosa (andrea-rosa-m) ** Tags added: libvirt ** Tags added: live-migration ** Changed in: nova Status: New => In Progress ** Changed in: nova Importance: Undecided => Low -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1588378 Title: Cancelled live migration are reported as in progress Status in OpenStack Compute (nova): In Progress Bug description: With the introduction of the API for aborting a running live-migration (https://review.openstack.org/277971) we have introduced a new status for the aborted live migration jobs. This new status called "cancelled" is not filtered by the sqlalchemy query used to return the list of migration in progress: https://github.com/openstack/nova/blob/87dc738763d6a7a10409e14b878f5cdd39ba805e/nova/db/sqlalchemy/api.py#L4851 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1588378/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1582409] Re: nova-api performance issues on "openstack server list"
based on your last comment I am going to close this bug as it is not "reproducible". Please feel free to re-open it if you have a clear way to reproduce it again. Thanks ** Changed in: nova Status: New => Incomplete ** Changed in: nova Status: Incomplete => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1582409 Title: nova-api performance issues on "openstack server list" Status in OpenStack Compute (nova): Invalid Bug description: Relevant information TL;DR: RDO stable Mitaka deployed with OpenStack puppet modules. Keystone with fernet tokens, Nova with keystone_authtoken memcached caching set (also reproduced with sql tokens and nova without memcached caching for authtoken). nova-api is handled behind apache with mod wsgi. Problem TL;DR: In an empty new deployment on bare metal (no images, no virtual machines, etc.), "openstack server list" will intermittently take >5 seconds (up to ~10) to return. The only noticeable thing that happens when "openstack server list" is slow is that the logs will be plastered with a bunch of "_create_extension_schema", "_rebuild_extension_schema", "_register_resources" and so on. All other installed projects (glance, keystone, neutron) return command output consistently under 2 seconds. Version info: # rpm -qa |grep nova openstack-nova-api-13.0.0-1.el7.noarch python-novaclient-3.3.0-1.el7.noarch openstack-nova-scheduler-13.0.0-1.el7.noarch python-nova-13.0.0-1.el7.noarch openstack-nova-conductor-13.0.0-1.el7.noarch openstack-nova-common-13.0.0-1.el7.noarch /etc/nova/nova.conf: http://paste.fedoraproject.org/367251/34295211/raw/ Example log of when things are fast: http://paste.fedoraproject.org/367272/32783146/raw/ Example log of when things are slow: http://paste.fedoraproject.org/367271/34327341/raw/ Happy to help provider further details if required. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1582409/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1582558] Re: Live-migration exception handling need improvement
I am a bit skeptical in ignoring the exception in both cases you described. For example during the roll_back if we fail calling cinder the risk is that we leave some open connections between the host and the cinder server, which is not good. The same thing could happen during the _post_live_migration. I agree with you that would be good to not stop those two actions for errors which Nova can't control but ignoring those errors doesn't seem the right approach. Probably we need a new design for implementing the rollback and the post_live_migration tasks in a more async way but with the ability to control the results and take correct actions to fix potential errors but I reckon that to do that it is a complicated task which, probably, require some help from other services (Cidner, for example). I am seeing this more than a new spec than a bug, I'd suggest to come up with a more structured proposal and put a spec where we can get a discussion involving a broader audience. I am going to mark this bug as invalid as we do not have a real bug here. if you are not happy about this decision please join the IRC meeting for the nova live migration sub-team: https://wiki.openstack.org/wiki/Meetings/NovaLiveMigration ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1582558 Title: Live-migration exception handling need improvement Status in OpenStack Compute (nova): Invalid Bug description: Description: 1.In /nova/compute/manager.py the function _rollback_live_migration, we should be catch exception of 'remove_volume_connection', let the rollback process continue to clean up other resources. Becaue,the 'remove_volume_connection' will visit cinder, the probability of an exception is relatively large. 2.In /nova/compute/manager.py the function _post_live_migration, we should be catch all exception of source host clean up resources. Because the vm has been migrated the dest host, we should try to make sure the vm run normally. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1582558/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1581977] Re: Invalid input for dns_name when spawning instance with .number at the end
I am not sure if this is a bug, we could replace the dot(s) in the host name with a different char, for example a "_" but then the user will lose the option to define a hostname as a FQDN. And according to the rfc952: ' A "name" (Net, Host, Gateway, or Domain name) is a text string up to 24 characters drawn from the alphabet (A-Z), digits (0-9), minus sign (-), and period (.). Note that periods are only allowed when they serve to delimit components of "domain style names"' So if the user uses a period it should know that it is allowed only to delimit a domain name and it has to be a valid one. I am going to mark it as invalid, please let me know if you are not happy about this decision. ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1581977 Title: Invalid input for dns_name when spawning instance with .number at the end Status in OpenStack Compute (nova): Invalid Bug description: When attempting to deploy an instance with a name which ends in dot (e.g. .123, as in an all-numeric TLD) or simply a name that, after conversion to dns_name, ends as ., nova conductor fails with the following error: 2016-05-15 13:15:04.824 ERROR nova.scheduler.utils [req-4ce865cd-e75b- 4de8-889a-ed7fc7fece18 admin demo] [instance: c4333432-f0f8-4413-82e8-7f12cdf3b5c8] Error from last host: silpixa00394065 (node silpixa00394065): [u'Traceback (most recent call last):\n', u' File "/opt/stack/nova/nova/compute/manager.py", line 1926, in _do_build_and_run_instance\nfilter_properties)\n', u' File "/opt/stack/nova/nova/compute/manager.py", line 2116, in _build_and_run_instance\ninstance_uuid=instance.uuid, reason=six.text_type(e))\n', u"RescheduledException: Build of instance c4333432-f0f8-4413-82e8-7f12cdf3b5c8 was re-scheduled: Invalid input for dns_name. Reason: 'networking-ovn-ubuntu-16.04' not a valid PQDN or FQDN. Reason: TLD '04' must not be all numeric.\nNeutron server returns request_ids: ['req-7317c3e3-2875-4073-8076-40e944845b69']\n"] This throws one instance of the infamous Horizon message: Error: No valid host was found. There are not enough hosts available. This issue was observed using stable/mitaka via DevStack (nova commit fb3f1706c68ea5b58f05ea810c6339f2449959de). In the above example, the instance name is "networking-ovn (Ubuntu 16.04)", which resulted in an attempted dns_name="networking-ovn- ubuntu-16.04", where the 04 was interpreted as a TLD and, consequently, an invalid TLD. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1581977/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1550525] Re: Abort an ongoing live migration
I added details Ito describe the new feature introduced in nova by the 2.24 version, please let me know more information are required. I think that the new feature needs to be documented in the API guide. ** Project changed: nova => openstack-api-site -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1550525 Title: Abort an ongoing live migration Status in openstack-api-site: Confirmed Bug description: https://review.openstack.org/277971 Dear bug triager. This bug was created since a commit was marked with DOCIMPACT. Your project "openstack/nova" is set up so that we directly report the documentation bugs against it. If this needs changing, the docimpact-group option needs to be added for the project. You can ask the OpenStack infra team (#openstack-infra on freenode) for help if you need to. commit fa002925460e70d988d1b4dd1ea594c680a43740 Author: Andrea Rosa <andrea.r...@hp.com> Date: Fri Feb 5 08:31:06 2016 + Abort an ongoing live migration This change adds a DELETE call on the server-migrations object to cancel a running live migration of a specific instance. TO perform the cancellation the virtualization driver needs to support it, in case that the feature is not supported we return an error. We allow a cancellation of a migration only if the migration is running at the moment of the request and if the migration type is equal to 'live-migration'. In this change we implement this feature for the libvirt driver. When the cancellation of a live migration succeeded we rollback the live migration and we set the state of the Migration object equals to 'cancelled'. The implementation of this change is based on the work done by the implementation of the feature called 'force live migration': https://review.openstack.org/245921 DocImpact ApiImpact Implements blueprint: abort-live-migration Change-Id: I1ff861e54997a069894b542bd764ac3ef1b3dbb2 To manage notifications about this bug go to: https://bugs.launchpad.net/openstack-api-site/+bug/1550525/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1499405] Re: Live block migration fails for instances with LVM ephemeral devices
I had time to did more investigation and I found that the live migration for
LVM backed instances is not supported:
https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L6665
This change https://review.openstack.org/73387 was for raising the
NotImplementedError exception when we try LVM migration.
In my case I didn't see that exception probably because I hit a different
exception before, we still need to check that the control about LVM is done at
the right time, but that will be a different bug.
I am going to mark this bug as Invalid as it is a duplicate of
https://bugs.launchpad.net/nova/+bug/1282643
** Changed in: nova
Status: New => Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1499405
Title:
Live block migration fails for instances with LVM ephemeral devices
Status in OpenStack Compute (nova):
Invalid
Bug description:
On a multinode devstack installation I have created a new lvm device:
sudo dd if=/dev/zero of=/loopy bs=1M count=5124 # 5gb
sudo losetup /dev/loop2 /loopy
sudo sfdisk /dev/loop2 << EOF
,,8e,,
EOF
sudo pvcreate /dev/loop2
sudo vgcreate encrypted-ephemeral /dev/loop2
Please note that the volume group name is "ecnrypted-ephemeral" but I am not
using any encryption.
I set up my nova compute to use lvm images for the ephemeral disks:
[libvirt]
images_type = lvm
images_volume_group = encrypted-ephemeral
Then I changed the live migration uri to use tcp instead of ssh:
live_migration_uri = qemu+tcp://%s/system
I did the same configuration on a second compute node.
At this point I booted a vm which became ACTIVE with no error then I tried to
migrate the instance:
nova live-migration devstackmn-2 --block-migrate
and the migration fails and in the libivirt log file there is this error
reported:
2015-09-24 08:23:17.674+: 26629: error :
virNetClientProgramDispatchError:175 : Failed to open file
'/dev/encrypted-ephemeral/9c1c82cb-cf11-4b2b-a994-2d94d286bfb9_disk': No such
file or directory
and in the nova-compute log file of the source node I see a similar
error:
2015-09-24 14:33:19.023 ERROR nova.virt.libvirt.driver
[req-3f358ce5-7746-4532-9bb4-bf1189181ce8 admin admin] [instance:
9c1c82cb-cf11-4b2b-a994-2d94d286bfb9] Live Migration failure: Failed to open
file '/dev/encrypted-ephemeral/9c1c82cb-cf11-4b2b-a994-2d94d286bfb9_disk': No
such file or directory
2015-09-24 14:33:19.023 DEBUG nova.virt.libvirt.driver
[req-3f358ce5-7746-4532-9bb4-bf1189181ce8 admin admin] [instance:
9c1c82cb-cf11-4b2b-a994-2d94d286bfb9] Migration operation thread notification
from (pid=21098) thread_finished
/opt/stack/nova/nova/virt/libvirt/driver.py:6041
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line
457, in fire_timers
timer()
File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line
58, in __call__
cb(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 168,
in _do_send
waiter.switch(result)
File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line
214, in main
result = function(*args, **kwargs)
File "/opt/stack/nova/nova/utils.py", line 1154, in context_wrapper
return func(*args, **kwargs)
File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5643, in
_live_migration_operation
instance=instance)
File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line
195, in __exit__
six.reraise(self.type_, self.value, self.tb)
File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5611, in
_live_migration_operation
CONF.libvirt.live_migration_bandwidth)
File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183,
in doit
result = proxy_call(self._autowrap, f, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141,
in proxy_call
rv = execute(f, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122,
in execute
six.reraise(c, e, tb)
File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80,
in tworker
rv = meth(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 1586, in
migrateToURI2
if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed',
dom=self)
libvirtError: Failed to open file
'/dev/encrypted-ephemeral/9c1c82cb-cf11-4b2b-a994-2d94d286bfb9_disk': No such
file or directory
2015-09-24 14:33:19.224 DEBUG nova.virt.libvirt.driver
[req-3f358ce5-7746-4532-9bb4-bf1189181ce8 admin admin] [instance:
9c1c82cb-cf11-4b2b-a994-2d94d286bfb9] VM running on src, migration failed from
(pid=21098) _live_migration_monitor
[Yahoo-eng-team] [Bug 1499405] [NEW] Live-migration fails for instances with LVM ephemeral devices
Public bug reported:
On a multinode devstack installation I have created a new lvm device:
sudo dd if=/dev/zero of=/loopy bs=1M count=5124 # 5gb
sudo losetup /dev/loop2 /loopy
sudo sfdisk /dev/loop2 << EOF
,,8e,,
EOF
sudo pvcreate /dev/loop2
sudo vgcreate encrypted-ephemeral /dev/loop2
Please note that the volume group name is "ecnrypted-ephemeral" but I am not
using any encryption.
I set up my nova compute to use lvm images for the ephemeral disks:
[libvirt]
images_type = lvm
images_volume_group = encrypted-ephemeral
Then I changed the live migration uri to use tcp instead of ssh:
live_migration_uri = qemu+tcp://%s/system
I did the same configuration on a second compute node.
At this point I booted a vm which became ACTIVE with no error then I tried to
migrate the instance:
nova live-migration devstackmn-2 --block-migrate
and the migration fails and in the libivirt log file there is this error
reported:
2015-09-24 08:23:17.674+: 26629: error :
virNetClientProgramDispatchError:175 : Failed to open file
'/dev/encrypted-ephemeral/9c1c82cb-cf11-4b2b-a994-2d94d286bfb9_disk': No such
file or directory
and in the nova-compute log file of the source node I see a similar
error:
2015-09-24 14:33:19.023 ERROR nova.virt.libvirt.driver
[req-3f358ce5-7746-4532-9bb4-bf1189181ce8 admin admin] [instance:
9c1c82cb-cf11-4b2b-a994-2d94d286bfb9] Live Migration failure: Failed to open
file '/dev/encrypted-ephemeral/9c1c82cb-cf11-4b2b-a994-2d94d286bfb9_disk': No
such file or directory
2015-09-24 14:33:19.023 DEBUG nova.virt.libvirt.driver
[req-3f358ce5-7746-4532-9bb4-bf1189181ce8 admin admin] [instance:
9c1c82cb-cf11-4b2b-a994-2d94d286bfb9] Migration operation thread notification
from (pid=21098) thread_finished
/opt/stack/nova/nova/virt/libvirt/driver.py:6041
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 457,
in fire_timers
timer()
File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py", line
58, in __call__
cb(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/eventlet/event.py", line 168, in
_do_send
waiter.switch(result)
File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py", line
214, in main
result = function(*args, **kwargs)
File "/opt/stack/nova/nova/utils.py", line 1154, in context_wrapper
return func(*args, **kwargs)
File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5643, in
_live_migration_operation
instance=instance)
File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line
195, in __exit__
six.reraise(self.type_, self.value, self.tb)
File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5611, in
_live_migration_operation
CONF.libvirt.live_migration_bandwidth)
File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 183, in
doit
result = proxy_call(self._autowrap, f, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in
proxy_call
rv = execute(f, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in
execute
six.reraise(c, e, tb)
File "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in
tworker
rv = meth(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 1586, in
migrateToURI2
if ret == -1: raise libvirtError ('virDomainMigrateToURI2() failed',
dom=self)
libvirtError: Failed to open file
'/dev/encrypted-ephemeral/9c1c82cb-cf11-4b2b-a994-2d94d286bfb9_disk': No such
file or directory
2015-09-24 14:33:19.224 DEBUG nova.virt.libvirt.driver
[req-3f358ce5-7746-4532-9bb4-bf1189181ce8 admin admin] [instance:
9c1c82cb-cf11-4b2b-a994-2d94d286bfb9] VM running on src, migration failed from
(pid=21098) _live_migration_monitor
/opt/stack/nova/nova/virt/libvirt/driver.py:5847
2015-09-24 14:33:19.224 DEBUG nova.virt.libvirt.driver
[req-3f358ce5-7746-4532-9bb4-bf1189181ce8 admin admin] [instance:
9c1c82cb-cf11-4b2b-a994-2d94d286bfb9] Fixed incorrect job type to be 4 from
(pid=21098) _live_migration_monitor
/opt/stack/nova/nova/virt/libvirt/driver.py:5867
2015-09-24 14:33:19.224 ERROR nova.virt.libvirt.driver
[req-3f358ce5-7746-4532-9bb4-bf1189181ce8 admin admin] [instance:
9c1c82cb-cf11-4b2b-a994-2d94d286bfb9] Migration operation has aborted
Nova version as per git log
commit 806dc025dbc9a102f7bace78be9c56f8e9af0929
Merge: 424aaf3 ae3d970
Author: Jenkins
Date: Sat Sep 19 05:03:38 2015 +
Merge "libvirt:Remove duplicated check code for config option
sysinfo_serial"
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1499405
Title:
Live-migration fails for instances with LVM ephemeral devices
Status in OpenStack
[Yahoo-eng-team] [Bug 1475353] [NEW] _get_host_sysinfo_serial_os fails if the machine-id file is empty
Public bug reported: the _get_host_sysinfo_serial_os method try to read the machine-id file to get an UUID for the host operating system. If the file is there but it is empty the code will raise an exception while it tries to parse the content of the file. To reproduce the issue just add this test to the nova/tests/unit/virt/libvirt/test_driver.py def test_get_guest_config_sysinfo_serial_os_empty_machine_id(self): self.flags(sysinfo_serial=os, group=libvirt) real_open = __builtin__.open with contextlib.nested( mock.patch.object(__builtin__, open), ) as (mock_open, ): theuuid = def fake_open(filename, *args, **kwargs): if filename == /etc/machine-id: h = mock.MagicMock() h.read.return_value = theuuid h.__enter__.return_value = h return h return real_open(filename, *args, **kwargs) mock_open.side_effect = fake_open self._test_get_guest_config_sysinfo_serial(None) ** Affects: nova Importance: Undecided Assignee: Andrea Rosa (andrea-rosa-m) Status: In Progress ** Tags: low-hanging-fruit ** Changed in: nova Assignee: (unassigned) = Andrea Rosa (andrea-rosa-m) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1475353 Title: _get_host_sysinfo_serial_os fails if the machine-id file is empty Status in OpenStack Compute (nova): In Progress Bug description: the _get_host_sysinfo_serial_os method try to read the machine-id file to get an UUID for the host operating system. If the file is there but it is empty the code will raise an exception while it tries to parse the content of the file. To reproduce the issue just add this test to the nova/tests/unit/virt/libvirt/test_driver.py def test_get_guest_config_sysinfo_serial_os_empty_machine_id(self): self.flags(sysinfo_serial=os, group=libvirt) real_open = __builtin__.open with contextlib.nested( mock.patch.object(__builtin__, open), ) as (mock_open, ): theuuid = def fake_open(filename, *args, **kwargs): if filename == /etc/machine-id: h = mock.MagicMock() h.read.return_value = theuuid h.__enter__.return_value = h return h return real_open(filename, *args, **kwargs) mock_open.side_effect = fake_open self._test_get_guest_config_sysinfo_serial(None) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1475353/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1375857] [NEW] It's not possibile to pass the cacert to the swift store
Public bug reported: The swift store device defined in the glance store doesn't allow to pass the ca cert file. When the driver creates a connection via the swift client it is not possible to pass that value. That means that if we have swift running on TLS in some cases we have to set the insecure option equals to True as the client can't correctly complete the handshake as it fails on the verification of the cert. The fix I'd like to propose is to add a new parameter to define the ca cert file and pass this value when we create the connection via the swift client. ** Affects: glance Importance: Undecided Assignee: Andrea Rosa (andrea-rosa-m) Status: New ** Tags: store swift ** Changed in: glance Assignee: (unassigned) = Andrea Rosa (andrea-rosa-m) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1375857 Title: It's not possibile to pass the cacert to the swift store Status in OpenStack Image Registry and Delivery Service (Glance): New Bug description: The swift store device defined in the glance store doesn't allow to pass the ca cert file. When the driver creates a connection via the swift client it is not possible to pass that value. That means that if we have swift running on TLS in some cases we have to set the insecure option equals to True as the client can't correctly complete the handshake as it fails on the verification of the cert. The fix I'd like to propose is to add a new parameter to define the ca cert file and pass this value when we create the connection via the swift client. To manage notifications about this bug go to: https://bugs.launchpad.net/glance/+bug/1375857/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1192749] Re: Cannot create over 1000 VM due to previous VM deletion
i think this is not a bug but just a configuration issue.
There is a parameter to check the max number of items returned, something like
that:
osapi_opts = [
cfg.IntOpt('osapi_max_limit',
default=1000,
help='the maximum number of items returned in a single '
'response from a collection resource'),
Can you check in your configuration if you have this parameter ?
** Changed in: nova
Status: New = Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1192749
Title:
Cannot create over 1000 VM due to previous VM deletion
Status in OpenStack Compute (Nova):
Invalid
Bug description:
The system currently has 1000 active VM. When I create the 1001 VM
the oldest VM 1 gets deleted without warning. I'm not sure if it is
still running on the compute/volume node but I cannot find it using
'nova show stca-00044'. I have checked the quota for enough resources
but then if the quota was exceeded for any of the resources (i.e
vcpu/memory/etc) there would have been an error. In this case the new
1001 VM becomes active and 1 VM is not longer accessible. The system
will always stay at 1000vm due to this and cannot be exceeded.
This setup was configured using Cisco Openstack Edition Folsom-multinode.
Link to configuration documentation:
http://docwiki.cisco.com/wiki/OpenStack:Folsom
The setup contains 1 control/network node and 43 compute/volume nodes.
Here is an example:
root@control-server:~# nova show stca-0044
+-+--+
| Property| Value
|
+-+--+
| status | ACTIVE
|
| updated | 2013-06-07T18:42:41Z
|
| OS-EXT-STS:task_state | None
|
| OS-EXT-SRV-ATTR:host| compute-server19
|
| net16 network | 16.0.0.52
|
| key_name| dkp
|
| image | f16_STCa_4.24
(b978b4bc-044a-4ccc-a14c-bc4ed9f7cb34) |
| hostId |
d63142d9402fb3f64934f6a5aacd8876f7df2869cecb9f653aa0acd0 |
| net15 network | 15.0.0.52
|
| OS-EXT-STS:vm_state | active
|
| OS-EXT-SRV-ATTR:instance_name | instance-0041
|
| OS-EXT-SRV-ATTR:hypervisor_hostname | compute-server19.cisco.com
|
| flavor | STCa-vm (6)
|
| id | 26825b9c-edd5-477e-acb3-c58bd1a2d775
|
| security_groups | [{u'name': u'default'}]
|
| user_id | 2562b65ac23840d483647f6deba74b0d
|
| name| stca-0044
|
| created | 2013-06-07T18:42:22Z
|
| tenant_id | 0f330a6acb4f43ac99961ba1573ced10
|
| OS-DCF:diskConfig | MANUAL
|
| accessIPv4 |
|
| accessIPv6 |
|
| progress| 0
|
| OS-EXT-STS:power_state | 1
|
| metadata| {}
|
| config_drive|
|
+-+--+
root@control-server:~# nova show stca-0044
+-+--+
| Property| Value
|
