[Yahoo-eng-team] [Bug 1437992] Re: policy file in policy.d will be reloaded every rest api call
olso.policy patch. https://review.openstack.org/#/c/168798/ ** Also affects: oslo.policy Importance: Undecided Status: New ** Changed in: oslo.policy Status: New = Confirmed ** Changed in: oslo.policy Assignee: (unassigned) = Eli Qiao (taget-9) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1437992 Title: policy file in policy.d will be reloaded every rest api call Status in OpenStack Compute (Nova): In Progress Status in Oslo Policy: Confirmed Bug description: the policy file in policy.d will be reloaded every time when do a rest api call. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1437992/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438050] [NEW] Remember used tenant
Public bug reported: I have user, which have access to several tenants. E.g. qa, persistent, scratch. I can select my current tenant in scroll box on top of page of Horizon dashboard. It is kept under current session. However when session timeouts and I have to re-login, then chosen tenants is reset to first one - in my case qa. This is quite annoying and it would be nice if Horizon can remember the chosen tenant and return to tenant I used before session timeouted or before I logoff. ** Affects: horizon Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1438050 Title: Remember used tenant Status in OpenStack Dashboard (Horizon): New Bug description: I have user, which have access to several tenants. E.g. qa, persistent, scratch. I can select my current tenant in scroll box on top of page of Horizon dashboard. It is kept under current session. However when session timeouts and I have to re-login, then chosen tenants is reset to first one - in my case qa. This is quite annoying and it would be nice if Horizon can remember the chosen tenant and return to tenant I used before session timeouted or before I logoff. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1438050/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438040] [NEW] fdb entries can't be removed when a VM is migrated
Public bug reported: this problem can be reprodeced as bellow: 1. vm A in computeA, vm B in computeB, l2 pop enable; 2. vmB continue ping vmA 3. live migrate vmA to computeB 4. when live-migrate finish, vmB ping vmA failed the reason is bellow, in l2pop driver, when vmA migrate to computeB, port status change form BUILD to ACTIVE, it add the port to self.migrated_ports when port status is ACTIVE, but 'remove_fdb_entries' in port status is BUILD : def update_port_postcommit(self, context): ... ... elif (context.host != context.original_host and context.status == const.PORT_STATUS_ACTIVE and not self.migrated_ports.get(orig['id'])): # The port has been migrated. We have to store the original # binding to send appropriate fdb once the port will be set # on the destination host self.migrated_ports[orig['id']] = ( (orig, context.original_host)) elif context.status != context.original_status: if context.status == const.PORT_STATUS_ACTIVE: self._update_port_up(context) elif context.status == const.PORT_STATUS_DOWN: fdb_entries = self._update_port_down( context, port, context.host) self.L2populationAgentNotify.remove_fdb_entries( self.rpc_ctx, fdb_entries) elif context.status == const.PORT_STATUS_BUILD: orig = self.migrated_ports.pop(port['id'], None) if orig: original_port = orig[0] original_host = orig[1] # this port has been migrated: remove its entries from fdb fdb_entries = self._update_port_down( context, original_port, original_host) self.L2populationAgentNotify.remove_fdb_entries( self.rpc_ctx, fdb_entries) ** Affects: neutron Importance: Undecided Assignee: shihanzhang (shihanzhang) Status: New ** Changed in: neutron Assignee: (unassigned) = shihanzhang (shihanzhang) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1438040 Title: fdb entries can't be removed when a VM is migrated Status in OpenStack Neutron (virtual network service): New Bug description: this problem can be reprodeced as bellow: 1. vm A in computeA, vm B in computeB, l2 pop enable; 2. vmB continue ping vmA 3. live migrate vmA to computeB 4. when live-migrate finish, vmB ping vmA failed the reason is bellow, in l2pop driver, when vmA migrate to computeB, port status change form BUILD to ACTIVE, it add the port to self.migrated_ports when port status is ACTIVE, but 'remove_fdb_entries' in port status is BUILD : def update_port_postcommit(self, context): ... ... elif (context.host != context.original_host and context.status == const.PORT_STATUS_ACTIVE and not self.migrated_ports.get(orig['id'])): # The port has been migrated. We have to store the original # binding to send appropriate fdb once the port will be set # on the destination host self.migrated_ports[orig['id']] = ( (orig, context.original_host)) elif context.status != context.original_status: if context.status == const.PORT_STATUS_ACTIVE: self._update_port_up(context) elif context.status == const.PORT_STATUS_DOWN: fdb_entries = self._update_port_down( context, port, context.host) self.L2populationAgentNotify.remove_fdb_entries( self.rpc_ctx, fdb_entries) elif context.status == const.PORT_STATUS_BUILD: orig = self.migrated_ports.pop(port['id'], None) if orig: original_port = orig[0] original_host = orig[1] # this port has been migrated: remove its entries from fdb fdb_entries = self._update_port_down( context, original_port, original_host) self.L2populationAgentNotify.remove_fdb_entries( self.rpc_ctx, fdb_entries) To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1438040/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438113] [NEW] Use plain HTTP listeners in the conductor
Public bug reported: The conductor is consuming messages form single queue which has performance limitation due to various reasons.: - per queue lock - Some broker also limiting same part of the message handling to single CPU thread/queue - Multiple broker instances needs to synchronise to queue content, which causes additional delays die to the tcp request/response times The single queue limitation is much greater than the limits getting by single mysql server, the rate is even worse when you consider slave reads. This can be workarounded by explicitly or implicit distributing the rpc calls to multiple different queue. The message broker provides additional message durability properties which is not needed just for an rpc_call, we spend resource on what we actually do not need. For TCP/HTTP traffic load balancing we have many-many tools even hardware assisted options are available providing virtually unlimited scalability. At TCP level also possible to exclude the loadbalancer node(s) form the response traffic. Why HTTP? Basically any protocol which can do request/response `thing` with arbitrary type and size of data with keep-alive connection and with ssl option, could be used. HTTP is a simple and well know protocol, with already existing many-many load balancing tool. Why not have the agents to do a regular API call? The regular API calls needs to do policy check, which in this case is not required, every authenticated user can be considered as admin. The the conductor clients needs to use at least a single shared key configured in every nova host. It has similar security as openstack used with the brokers, basically all nova node had credentials in one rabbitmq virtual host, configured in the /etc/nova/nova.conf . If any of those credentials stolen it provided access to the whole virtual host. NOTE.: HTTPs can be used with certificate or kerberos based authentication as well. I think the for `rpc_calls` which are served by the agents using AMQP is still better option, this bug is just about the situation when the conductor itself serves rpc_call(s). NOTE.: The 1 Million msq/sec rabbitmq benchmark is done 186 queues, in way which does not hits the single queue limitations. ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1438113 Title: Use plain HTTP listeners in the conductor Status in OpenStack Compute (Nova): New Bug description: The conductor is consuming messages form single queue which has performance limitation due to various reasons.: - per queue lock - Some broker also limiting same part of the message handling to single CPU thread/queue - Multiple broker instances needs to synchronise to queue content, which causes additional delays die to the tcp request/response times The single queue limitation is much greater than the limits getting by single mysql server, the rate is even worse when you consider slave reads. This can be workarounded by explicitly or implicit distributing the rpc calls to multiple different queue. The message broker provides additional message durability properties which is not needed just for an rpc_call, we spend resource on what we actually do not need. For TCP/HTTP traffic load balancing we have many-many tools even hardware assisted options are available providing virtually unlimited scalability. At TCP level also possible to exclude the loadbalancer node(s) form the response traffic. Why HTTP? Basically any protocol which can do request/response `thing` with arbitrary type and size of data with keep-alive connection and with ssl option, could be used. HTTP is a simple and well know protocol, with already existing many-many load balancing tool. Why not have the agents to do a regular API call? The regular API calls needs to do policy check, which in this case is not required, every authenticated user can be considered as admin. The the conductor clients needs to use at least a single shared key configured in every nova host. It has similar security as openstack used with the brokers, basically all nova node had credentials in one rabbitmq virtual host, configured in the /etc/nova/nova.conf . If any of those credentials stolen it provided access to the whole virtual host. NOTE.: HTTPs can be used with certificate or kerberos based authentication as well. I think the for `rpc_calls` which are served by the agents using AMQP is still better option, this bug is just about the situation when the conductor itself serves rpc_call(s). NOTE.: The 1 Million msq/sec rabbitmq benchmark is done 186 queues, in way which does not hits the single queue limitations. To manage notifications about this bug go to:
[Yahoo-eng-team] [Bug 1190559] Re: qcow2 images size should match the virtual image size
@yatin Hey, I think you might be confused with what this bug aimed to do. The bug proposed that the `size` field should match the `virtual_size` instead of the `size`. What we did to fix this was adding a new `virtual_size` field that would represent the `virtual_size` value in some of the images. I don't think nova uses the `virtual_size` field but I should double check. That said, the `virtual_size` value is an independent field that has nothing to do with the `size` of the image. Therefore, I do not believe these 2 fields should match. If anything, the virtual_size should be as big or bigger than the image size. ** Changed in: glance Status: Confirmed = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1190559 Title: qcow2 images size should match the virtual image size Status in OpenStack Image Registry and Delivery Service (Glance): Invalid Bug description: the image size of qcow2 images should match the virtual size reported by qemu-img not the actual data size # glance image-show ff8f8fb3-76a3-47c0-bfc2-79e46950 [...] | size | 699592704 | [...] # qemu-img info /var/lib/glance/images/ff8f8fb3-76a3-47c0-bfc2-79e46950 image: /var/lib/glance/images/ff8f8fb3-76a3-47c0-bfc2-79e46950 file format: qcow2 virtual size: 6.0G (6442450944 bytes) disk size: 667M cluster_size: 65536 To manage notifications about this bug go to: https://bugs.launchpad.net/glance/+bug/1190559/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438063] [NEW] Remove repeated '__init__' from api/fwaas.py
Public bug reported: https://github.com/openstack/horizon/blob/master/openstack_dashboard/api/fwaas.py#L52 Duplicated init function from what appears to be a bad merge, but unsure. ** Affects: horizon Importance: Undecided Assignee: Rob Cresswell (robcresswell) Status: In Progress -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1438063 Title: Remove repeated '__init__' from api/fwaas.py Status in OpenStack Dashboard (Horizon): In Progress Bug description: https://github.com/openstack/horizon/blob/master/openstack_dashboard/api/fwaas.py#L52 Duplicated init function from what appears to be a bad merge, but unsure. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1438063/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1437992] Re: policy file in policy.d will be reloaded every rest api call
** Also affects: oslo-incubator Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1437992 Title: policy file in policy.d will be reloaded every rest api call Status in OpenStack Compute (Nova): In Progress Status in The Oslo library incubator: New Status in Oslo Policy: Confirmed Bug description: the policy file in policy.d will be reloaded every time when do a rest api call. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1437992/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438088] [NEW] vpn row is not updating when the status changes
Public bug reported: when the status changes the corresponding row is not updating in vpn table. needs manual page reload. ** Affects: horizon Importance: Undecided Assignee: Masco Kaliyamoorthy (masco) Status: New ** Changed in: horizon Assignee: (unassigned) = Masco Kaliyamoorthy (masco) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1438088 Title: vpn row is not updating when the status changes Status in OpenStack Dashboard (Horizon): New Bug description: when the status changes the corresponding row is not updating in vpn table. needs manual page reload. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1438088/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438093] [NEW] Redundant method _set_vm_state in conductor _live_migrate
Public bug reported: In this file: https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L600 Implementing _set_vm_state() method in _live_migrate() method is unnecessary. It could be used method from line #590: def _set_vm_state_and_notify(self, context, instance_uuid, method, updates, ex, request_spec): scheduler_utils.set_vm_state_and_notify( context, instance_uuid, 'compute_task', method, updates,ex, request_spec, self.db) ** Affects: nova Importance: Undecided Status: New ** Tags: low-hanging-fruit ** Tags added: low-hanging-fruit -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1438093 Title: Redundant method _set_vm_state in conductor _live_migrate Status in OpenStack Compute (Nova): New Bug description: In this file: https://github.com/openstack/nova/blob/master/nova/conductor/manager.py#L600 Implementing _set_vm_state() method in _live_migrate() method is unnecessary. It could be used method from line #590: def _set_vm_state_and_notify(self, context, instance_uuid, method, updates, ex, request_spec): scheduler_utils.set_vm_state_and_notify( context, instance_uuid, 'compute_task', method, updates,ex, request_spec, self.db) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1438093/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1260617] Re: Provide the ability to attach volumes in the read-only mode
** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1260617 Title: Provide the ability to attach volumes in the read-only mode Status in OpenStack Dashboard (Horizon): Confirmed Status in OpenStack Compute (Nova): Opinion Status in Python client library for Nova: Triaged Bug description: Cinder now support the ability to attach volumes in the read-only mode, this should be exposed through horizon. Read-only mode could be ensured by hypervisor configuration during the attachment. Libvirt, Xen, VMware and Hyper-V support R/O volumes. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1260617/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1227653] Re: xenapi: snapshot_attached_here should wait for vhd coalesce after deleting the snapshot
Marking as Opinion as it is a bug by a core contributor without much context. ** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1227653 Title: xenapi: snapshot_attached_here should wait for vhd coalesce after deleting the snapshot Status in OpenStack Compute (Nova): Opinion Bug description: When we take a snapshot, upload it, then delete the snapshot, we should really wait to make sure the VHD chain gets restored to its original state. If the VHD chain did not collapse: * snapshots contain more and more VHDs over time * longer chains mean longer possible max size of the chain To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1227653/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1276154] Re: suspending a paused instance
** Changed in: nova
Status: Triaged = Opinion
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1276154
Title:
suspending a paused instance
Status in OpenStack Compute (Nova):
Opinion
Bug description:
Is there a compelling reason why we don't support suspending a paused
instance? At the moment we only allow 'active' and 'rescued' states to
be suspended.
In compute/api:
@check_instance_state(vm_state=[vm_states.ACTIVE, vm_states.RESCUED])
Trying to suspend a paused instance results in:
nova suspend b10f8175-1663-41b2-8533-0b0606f369ff
ERROR: Cannot 'suspend' while instance is in vm_state paused (HTTP 409)
(Request-ID: req-7349d554-ff48-4155-a62b-967f0813c59c)
I haven't tested with other hypervisors, but as far as libvirt/kvm is
concerned it suspends (virsh managedsave) both states ('running',
'paused') and resumes (virsh start) a suspended instance to the pre-
suspended state.
A use-case for this is to be able to quickly pause a bunch of
instances for security reasons and then suspend one of them
conveniently using the API rather than virsh. Taking a suspicious vm
to ‘active’ state so that it can be suspended doesn’t seem like a good
idea.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1276154/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438183] [NEW] Graceful shutdown of nova-compute service fails
Public bug reported: nova-compute doesn't shutdown gracefully on SIGTERM, e.g. booting a VM fails with: 09:29:18 AUDIT nova.compute.manager [req-9cdbba9c-af3b-4845-9deb-c68bffe63d75 None] [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286 df8] Starting instance... 09:29:18 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting ... 09:29:37 INFO nova.compute.manager [-] [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] VM Started (Lifecycle Event) 09:29:37 INFO nova.compute.manager [-] [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] VM Paused (Lifecycle Event) ... 09:34:37 WARNING nova.virt.libvirt.driver [req-9cdbba9c-af3b-4845-9deb-c68bffe63d75 None] Timeout waiting for vif plugging callback for instance 7ea3e761-6b85-49db-8dcc-79f6f2286df8 09:34:37 INFO nova.compute.manager [-] [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] VM Stopped (Lifecycle Event) 09:34:38 INFO nova.virt.libvirt.driver [req-9cdbba9c-af3b-4845-9deb-c68bffe63d75 None] [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] Deleting instance files /var/lib/nova/instances/7ea3e761-6b85-49db-8dcc-79f6f2286df8 09:34:38 ERROR nova.compute.manager [req-9cdbba9c-af3b-4845-9deb-c68bffe63d75 None] [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] Instance failed to spawn 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] Traceback (most recent call last): 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] File /usr/lib/python2.7/dist-packages/nova/compute/manager.py, line 1773, in _spawn 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] block_device_info) 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] File /usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py, line 2299, in spawn 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] block_device_info) 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] File /usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py, line 3745, in _create_domain_and_network 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] raise exception.VirtualInterfaceCreateException() 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] VirtualInterfaceCreateException: Virtual Interface creation failed 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1438183 Title: Graceful shutdown of nova-compute service fails Status in OpenStack Compute (Nova): New Bug description: nova-compute doesn't shutdown gracefully on SIGTERM, e.g. booting a VM fails with: 09:29:18 AUDIT nova.compute.manager [req-9cdbba9c-af3b-4845-9deb-c68bffe63d75 None] [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286 df8] Starting instance... 09:29:18 INFO nova.openstack.common.service [-] Caught SIGTERM, exiting ... 09:29:37 INFO nova.compute.manager [-] [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] VM Started (Lifecycle Event) 09:29:37 INFO nova.compute.manager [-] [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] VM Paused (Lifecycle Event) ... 09:34:37 WARNING nova.virt.libvirt.driver [req-9cdbba9c-af3b-4845-9deb-c68bffe63d75 None] Timeout waiting for vif plugging callback for instance 7ea3e761-6b85-49db-8dcc-79f6f2286df8 09:34:37 INFO nova.compute.manager [-] [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] VM Stopped (Lifecycle Event) 09:34:38 INFO nova.virt.libvirt.driver [req-9cdbba9c-af3b-4845-9deb-c68bffe63d75 None] [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] Deleting instance files /var/lib/nova/instances/7ea3e761-6b85-49db-8dcc-79f6f2286df8 09:34:38 ERROR nova.compute.manager [req-9cdbba9c-af3b-4845-9deb-c68bffe63d75 None] [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] Instance failed to spawn 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] Traceback (most recent call last): 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] File /usr/lib/python2.7/dist-packages/nova/compute/manager.py, line 1773, in _spawn 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] block_device_info) 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] File /usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py, line 2299, in spawn 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] block_device_info) 09:34:38 TRACE nova.compute.manager [instance: 7ea3e761-6b85-49db-8dcc-79f6f2286df8] File /usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py, line 3745,
[Yahoo-eng-team] [Bug 1206877] Re: xenapi: iso config drive not visible on windows 8 or 2012
@john has said it was a feature, marking as Opinion therefore ** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1206877 Title: xenapi: iso config drive not visible on windows 8 or 2012 Status in OpenStack Compute (Nova): Opinion Bug description: This was reported for libvirt: https://bugs.launchpad.net/nova/+bug/1155842 While Linux can mount a disk with an ISO format, windows wants the ISO added as a CD rom device. This should be fixed in the XenAPI driver. The latest cloud-init is able to read the config drive if its on Windows 2012 or later, and Windows 8 or later. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1206877/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438138] [NEW] [Heat] Initial values aren't displaying properly in Launch Stack form
Public bug reported:
Using json type default data in a template causes annoying
inconsistencies in the second step of creating a stack. For example for
this template:
heat_template_version: 2013-05-23
parameters:
key:
type: string
default: key
description: This is keypair to access into your stack.
db_password:
type: string
default: secrete
hidden: true
metadata:
type: json
default: {keyvalue: 5}
on the second step there would be metadata field with initial value
{u'keyvalue': 5}. That is not valid JSON, and pressing Launch button would
show: Expecting property name enclosed in double quotes.
** Affects: horizon
Importance: Medium
Assignee: Tatiana Ovchinnikova (tmazur)
Status: New
** Changed in: horizon
Assignee: (unassigned) = Tatiana Ovchinnikova (tmazur)
** Changed in: horizon
Importance: Undecided = Medium
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1438138
Title:
[Heat] Initial values aren't displaying properly in Launch Stack
form
Status in OpenStack Dashboard (Horizon):
New
Bug description:
Using json type default data in a template causes annoying
inconsistencies in the second step of creating a stack. For example
for this template:
heat_template_version: 2013-05-23
parameters:
key:
type: string
default: key
description: This is keypair to access into your stack.
db_password:
type: string
default: secrete
hidden: true
metadata:
type: json
default: {keyvalue: 5}
on the second step there would be metadata field with initial value
{u'keyvalue': 5}. That is not valid JSON, and pressing Launch button would
show: Expecting property name enclosed in double quotes.
To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/1438138/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1437141] Re: nova floating-ip-delete should warn or refuse if an IP is still associated
This is actually working as designed. Manually assigned floating IPs can be deleted in a single go, auto assigned ones must be disassociated first. Changing this default would probably *break* lots of users, so I don't think we should. ** Changed in: nova Status: New = Opinion ** Changed in: nova Importance: Undecided = Wishlist -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1437141 Title: nova floating-ip-delete should warn or refuse if an IP is still associated Status in OpenStack Compute (Nova): Opinion Bug description: Running 'nova floating-ip-delete IP' will happily delete IP addresses that are associated with currently-running instances. It would be safer if the user were required to either disassociate them from the instance before being permitted to delete them, or to pass a --force flag or similar. Version installed is python-novaclient 1:2.17.0-0ubuntu1~cloud0 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1437141/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438159] [NEW] Made neutron agents silent by using AMQP
Public bug reported: Problem.: Neutron agents does a lot of periodic task which leads an rpc call + database transaction, which does not even provide a new information, because nothing changed. This behaviour in scale can be called as `DDOS attack`, generally this kind of architecture is bad at scaling and can be considered as an any-pattern. Instead of periodic poll, we can leverage the AMQP brokers bind capabilities. Neutron has many situation like security group rule change or dvr related changes which needs to be communicated with multiple agents, but usually not with all agent. The agent at startup needs to synchronise the as usual, but during the sync the agent can subscribe to the interesting events to avoid the periodic tasks. (Note.: After the first subscribe loop a second one is needed to do not miss changes during the subscribe process ). The AMQP queues with 'auto-delete' can be considered as a reliable source of information which does not miss any event notification. On connection loss or full broker cluster die the agent needs to re sync everything guarded in this way, in these cases, the queue will disappear so the situation easily detectable. 1. Create a Direct exchange for all kind of resourcestype what needs to be synchronised in this way, for example.: 'neutron.securitygroups' . The exchange declaration needs to happen at q-svc start-up time or at full broker cluster die (not-found exception will tell it). The exchange SHOULD NOT be redeclared or verified at every message publish. 2. Every agent creates a dedicated per agent queue with auto-delete flag, if the agent already maintains a queue with this property he MAY reuse that one. The agents SHOULD avoid to creating multiple queues per resource type. The messages MUST contain a type information. 3. All agent creates a binding between his queue and the resource type queue with he realise he depends on the resource, for example it maintains at least one port with the given security-group. (The agents needs to remove the binding. when they stop using it) 4. The q-svc publishes just a single message when the resource related change happened. The routing key is the uuid. Alternatively a topic exchange can be used, with a single exchange. In this case the routing keys MUST contains the resource type like: neutron.resource_type.uuid , this type exchange is generally more expensive than a direct exchange (pattern matching), and only useful if you have agents which needs to listens to ALL event related to a type, but others just interested just in a few of them. ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1438159 Title: Made neutron agents silent by using AMQP Status in OpenStack Neutron (virtual network service): New Bug description: Problem.: Neutron agents does a lot of periodic task which leads an rpc call + database transaction, which does not even provide a new information, because nothing changed. This behaviour in scale can be called as `DDOS attack`, generally this kind of architecture is bad at scaling and can be considered as an any-pattern. Instead of periodic poll, we can leverage the AMQP brokers bind capabilities. Neutron has many situation like security group rule change or dvr related changes which needs to be communicated with multiple agents, but usually not with all agent. The agent at startup needs to synchronise the as usual, but during the sync the agent can subscribe to the interesting events to avoid the periodic tasks. (Note.: After the first subscribe loop a second one is needed to do not miss changes during the subscribe process ). The AMQP queues with 'auto-delete' can be considered as a reliable source of information which does not miss any event notification. On connection loss or full broker cluster die the agent needs to re sync everything guarded in this way, in these cases, the queue will disappear so the situation easily detectable. 1. Create a Direct exchange for all kind of resourcestype what needs to be synchronised in this way, for example.: 'neutron.securitygroups' . The exchange declaration needs to happen at q-svc start-up time or at full broker cluster die (not-found exception will tell it). The exchange SHOULD NOT be redeclared or verified at every message publish. 2. Every agent creates a dedicated per agent queue with auto-delete flag, if the agent already maintains a queue with this property he MAY reuse that one. The agents SHOULD avoid to creating multiple queues per resource type. The messages MUST contain a type information. 3. All agent creates a binding between his queue and the resource type queue with he realise he depends on the resource, for example it maintains at least one port with the given security-group.
[Yahoo-eng-team] [Bug 1078080] Re: libvirt rescue doesn't respect image_meta passed in
Looks fixed in code:
libvirt/driver.py
def rescue(self, context, instance, network_info, image_meta,
rescue_password):
Loads a VM using rescue images.
A rescue is normally performed when something goes wrong with the
primary images and data needs to be corrected/recovered. Rescuing
should not edit or over-ride the original image, only allow for
data recovery.
instance_dir = libvirt_utils.get_instance_path(instance)
unrescue_xml = self._get_existing_domain_xml(instance, network_info)
unrescue_xml_path = os.path.join(instance_dir, 'unrescue.xml')
libvirt_utils.write_to_file(unrescue_xml_path, unrescue_xml)
if image_meta is not None:
rescue_image_id = image_meta.get('id')
else:
rescue_image_id = None
** Changed in: nova
Status: Triaged = Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1078080
Title:
libvirt rescue doesn't respect image_meta passed in
Status in OpenStack Compute (Nova):
Fix Released
Bug description:
The libvirt driver currently ignores the `image_meta` field passed in
on the rescue method and instead uses it's driver-specific
'rescue_image_id` flag or the instance's current image.
We should probably two things here to make this code more similar
across hypervisors:
1. We should add a generic `default_rescue_image_id` flag that can
specific default rescue images for any driver. This would obsolete the
libvirt specific flag.
2. We should modify the libvirt `rescue` code to respect the
`image_meta` passed in, delegating the logic of which image to use for
rescue to the compute manager.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1078080/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1089877] Re: Extend used_limits extension to cover non-reservable resources
this looks fixed upstream, no idea when, but quota show now includes keypairs. os1:~ nova quota-show +-+---+ | Quota | Limit | +-+---+ | instances | 10| | cores | 20| | ram | 51200 | | floating_ips| 10| | fixed_ips | -1| | metadata_items | 128 | | injected_files | 5 | | injected_file_content_bytes | 10240 | | injected_file_path_bytes| 255 | | key_pairs | 100 | | security_groups | 10| | security_group_rules| 20| | server_groups | 10| | server_group_members| 10| +-+---+ ** Changed in: nova Status: Triaged = Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1089877 Title: Extend used_limits extension to cover non-reservable resources Status in OpenStack Compute (Nova): Fix Released Bug description: From bug #1062049 The used_limits extension looks in the quota_usages table to see what's used for a given resource That corresponds to the ReservableResource resources in nova.quota - i.e. instances, cores, ram, floating IPs and security groups Resources like key pairs don't record their usage info in the quota_usages table, so we don't currently expose them in the used_limits output To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1089877/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1204583] Re: xenapi: in finish_migration should clean up move_disks if not used
This is really old, owned by a core contributor. Marking as an invalid bug that was probably used for self tracking. ** Changed in: nova Status: Triaged = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1204583 Title: xenapi: in finish_migration should clean up move_disks if not used Status in OpenStack Compute (Nova): Invalid Bug description: This bug fixed the majority of the finish_migration cleanup: https://bugs.launchpad.net/nova/+bug/1073303 There is one remaining bit of cleanup that is required: * look at move_disks * if this is not run, we need to make sure the temp dir is cleaned up To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1204583/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1260274] Re: NoVNC Console not showing in Internet Explorer
This is a novnc bug, not a nova one ** Changed in: nova Status: Triaged = Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1260274 Title: NoVNC Console not showing in Internet Explorer Status in OpenStack Compute (Nova): Won't Fix Bug description: When accessing the NoVNC console through Internet Explorer (tested with IE9 and IE10) the HTML5 Canvas never renders, instead showing 'Canvas not supported'. Environment: - OS: Ubuntu 12.04 LTS - Platform: Openstack Grizzly - Packages: nova-novncproxy 1:2013.1.3-0ubuntu1~cloud0 novnc 2012.2~20120906+dfsg-0ubuntu4~cloud0 python-novnc 2012.2~20120906+dfsg-0ubuntu4~cloud0 According to https://github.com/kanaka/noVNC/wiki/Browser-support NoVNC should work with IE9 and above. [edit 6 Jan 2014] It would appear that it works properly with IE11 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1260274/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1101222] Re: Detect vncserver_proxyclient_address mis-configuration
Triage indicates the bug is basically fixable with the information provided in the bug. This is a nice to have enhancement, but not triaged. ** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1101222 Title: Detect vncserver_proxyclient_address mis-configuration Status in OpenStack Compute (Nova): Opinion Bug description: Say you have two compute nodes, compute1 and compute 2 vncserver_proxyclient_address on compute1 is set to the public address of compute1 - i.e. the address by which the proxy should connect to the VNC server on compute1 If an admin gets this wrong - e.g. copies nova.conf to compute2 and forgets to change the config option - it has pretty disastrous security implications. Without any warning or sign that there's a problem, users attempting to connect to the VNC console of their VM will be sent to the VNC console of some other VM which happens to be running on the same port on a different compute node. I've seen and debugged this in a real deployment :) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1101222/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1161557] Re: Race condition in handling of udev generated symlinks
This is unclear that it's a real bug, as it was disabled in the past. Definitely not in a triaged state. ** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1161557 Title: Race condition in handling of udev generated symlinks Status in Cinder: Triaged Status in OpenStack Compute (Nova): Opinion Bug description: In several components in both Cinder and Nova udev provided symlinks are expected to appear immediately. This is not the case as udev rules run async of device plugging. Volume drivers in Cinder and the libvirt driver in Nova seem to be the primary culprits. To solve this we should wait for udevd to finish processing events, this can be done with udevadm by calling settle. This is done by simply calling udevadm settle. Additionally it can be called with a timeout (probably a good idea). Excerpt from udevadm settle --help: Usage: udevadm settle OPTIONS --timeout=seconds maximum time to wait for events --seq-start=seqnumfirst seqnum to wait for --seq-end=seqnum last seqnum to wait for --exit-if-exists=file stop waiting if file exists --quiet do not print list after timeout --help For more intelligent use we could wrap this in a function that can use the --exit-if-exists behavior. This will ensure the symlink actually exists before we try use it. To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1161557/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1173430] Re: cleanup_running_deleted_instances should not be needed
** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1173430 Title: cleanup_running_deleted_instances should not be needed Status in OpenStack Compute (Nova): Opinion Bug description: cleanup_running_deleted_instances should likely be refactored so that said function is not needed in the first place, since in a ideal world an instance does not have the capability to be running when it is deleted (said deletion should fail in that case). To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1173430/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1195073] Re: pxe deploy timeout defaults to unset
Can't see where this code even exists in Nova at this point. The bug report also doesn't provide enough context to figure out where it might have existed. ** Changed in: nova Status: Triaged = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1195073 Title: pxe deploy timeout defaults to unset Status in OpenStack Compute (Nova): Invalid Bug description: This is not suitable for production : failed deploys never recover. We're running with 1200 in TripleO. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1195073/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1215383] Re: XenAPI: Consider removing safe_copy_vdi
Marking as Opinion as this bug discusses internal code structure changes which are partially out of date. In future bugs that have a reproduce would be more valuable. ** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1215383 Title: XenAPI: Consider removing safe_copy_vdi Status in OpenStack Compute (Nova): Opinion Bug description: safe_copy_vdi depends on ext3 and is not using supported interfaces, so should be considered for removal. It was added because the vdi.copy operation wasn't considered to be atomic. However, VDI.copy is definitely atomic and will not return until the copy has finished. It's therefore possible that the rationale for this function was based on looking at the SR during a copy operation, or use of async VDI copy. The reasons for this function need to be reviewed. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1215383/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1190076] Re: virt xenapi driver does not throw InstancePowerOffFailure exception on a failure while destroying an instance
** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1190076 Title: virt xenapi driver does not throw InstancePowerOffFailure exception on a failure while destroying an instance Status in OpenStack Compute (Nova): Opinion Bug description: _shutdown_instance in compute manager expects InstancePowerOffFailure exception and reraises it. But the xenapi driver does not raise it. On the other hand it just returns a boolean value( False ) on a failure. Its potential consequences are that terminate_instance goes ahead and marks the instance as deleted, leaving behind orphaned instances. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1190076/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1076119] Re: VM can't connect to itself on it's own floating IP
Very old bug that seems to not have a current reproduce. Putting to invalid for now, it's definitely not triaged. ** Changed in: nova Status: Triaged = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1076119 Title: VM can't connect to itself on it's own floating IP Status in OpenStack Compute (Nova): Invalid Bug description: using nova.network.manager.FlatDHCPManager: fixed pool 192.168.0.0/24 floating-ip 9.3.11.240 i create 2 VMs, both are assigned IPs from the fixed IP pool. i assign one VM the 9.3.11.240 address from the floating pool. from the host machine ((and the other VM)), i can ping and SSH to the VM on the 9.3.11.240 address. However, on the machine which is assigned the 9.3.11.240 address I can't ping or ssh back to itself on that address. this is a problem for some of the Bosh and Cloudfoundry applications who run services that may want to talk-back to themselves on the assigned floating ip. I'm running folsom on ubuntu 12.04. i've attached my nova.conf. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1076119/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1097605] Re: osapi_compute_unique_server_name_scope is a strange little option
The constraint checking kind of needs to be in the db layer because that's the only place we have access to it. Possibly a future clean up. But I think Opinion marks the current real state. ** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1097605 Title: osapi_compute_unique_server_name_scope is a strange little option Status in OpenStack Compute (Nova): Opinion Bug description: See https://review.openstack.org/19174 osapi_compute_unique_server_name_scope sounds like its a detail of the openstack API layer, but the option and its usage lives in the DB layer It would be nice if we could figure out some way of moving it into the API layer To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1097605/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1161557] Re: Race condition in handling of udev generated symlinks
** Changed in: cinder Status: Triaged = Incomplete ** Changed in: cinder Status: Incomplete = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1161557 Title: Race condition in handling of udev generated symlinks Status in Cinder: Invalid Status in OpenStack Compute (Nova): Opinion Bug description: In several components in both Cinder and Nova udev provided symlinks are expected to appear immediately. This is not the case as udev rules run async of device plugging. Volume drivers in Cinder and the libvirt driver in Nova seem to be the primary culprits. To solve this we should wait for udevd to finish processing events, this can be done with udevadm by calling settle. This is done by simply calling udevadm settle. Additionally it can be called with a timeout (probably a good idea). Excerpt from udevadm settle --help: Usage: udevadm settle OPTIONS --timeout=seconds maximum time to wait for events --seq-start=seqnumfirst seqnum to wait for --seq-end=seqnum last seqnum to wait for --exit-if-exists=file stop waiting if file exists --quiet do not print list after timeout --help For more intelligent use we could wrap this in a function that can use the --exit-if-exists behavior. This will ensure the symlink actually exists before we try use it. To manage notifications about this bug go to: https://bugs.launchpad.net/cinder/+bug/1161557/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1335554] Re: Documentation: incorrect use of weighted
doc patches welcomed, but I'm not convinced a bug is needed for this. ** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1335554 Title: Documentation: incorrect use of weighted Status in OpenStack Compute (Nova): Opinion Bug description: A recent review made me wonder about how we are using weight in the documentation. When OpenStack talks about weight, it doesn't mean mass but a statistical property. Such properties are used when weighting the members of a class, when producing a weighted average or a weighted index. But some of the OpenStack documentation talks about weighing. Here are some samples from nova docs: Correct: # If we speak about `filtering` and `weighting`, their work is quite # flexible in the Filter Scheduler. Incorrect: # During its work Filter Scheduler firstly makes dictionary of unfiltered # hosts, then filters them using filter properties and finally chooses hosts # for the requested number of instances (each time it chooses the most weighed # host and appends it to the list of selected hosts). Unfortunately, the API docs are sometimes rather unhelpful, containing explanations like this gem: # def weight_multiplier(self): # How weighted this weigher should be. Related (in openstack-manuals): bug 1335423 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1335554/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1160850] Re: ec2 (DescribeKeyPairs) does not supports filtering by fingerprint
ec2 in nova is deprecated, this should probably be addressed in the out of tree code ** Changed in: nova Status: Triaged = Opinion ** Changed in: nova Importance: Low = Wishlist -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1160850 Title: ec2 (DescribeKeyPairs) does not supports filtering by fingerprint Status in OpenStack Compute (Nova): Opinion Bug description: euca-describe-keypairs --filter fingerprint=c0:40:d9:8b:e2:09:11:2b:59:a8:86:3b:03:87:5d:e5 KEYPAIR testc0:40:d9:8b:e2:09:11:2b:59:a8:86:3b:03:87:5d:e5 KEYPAIR test2 1d:25:e8:42:1a:59:ce:59:6c:b1:2c:76:d6:4b:cd:de The listing shows me all keypairs even if I define a filter. nova version: 9a90f6ccdb88a22ff17b3f48d26378b4fa613ede To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1160850/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1312796] Re: we are able to destroy an instance while taking a snapshot
There was a reproduce ask over 6 months ago which was not done. Marking as Invalid. ** Changed in: nova Status: Triaged = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1312796 Title: we are able to destroy an instance while taking a snapshot Status in OpenStack Compute (Nova): Invalid Bug description: we are able to destroy an instance when taking a snapshot. the new image status would depend if it was already created and uploaded to /var/lib/glance/images I think that if we allow to destroy the instance when taking the snapshot we run the risk of data corruption on the new snapshot or the snapshot not being created at all. so I think that to destroy the instance while taking the snapshot we should have a --force in while the admin user knowingly destroys the instance. [root@puma31 ~(keystone_admin)]# nova list nov+--+--++--+-+--+ | ID | Name | Status | Task State | Power State | Networks | +--+--++--+-+--+ | e00ae899-e285-4f09-8cda-2c2680799bba | from | ACTIVE | image_pending_upload | Running | novanetwork=192.168.32.2 | +--+--++--+-+--+ [root@puma31 ~(keystone_admin)]# nova delete e00ae899-e285-4f09-8cda-2c2680799bba [root@puma31 ~(keystone_admin)]# nova list +--+--+++-+--+ | ID | Name | Status | Task State | Power State | Networks | +--+--+++-+--+ | e00ae899-e285-4f09-8cda-2c2680799bba | from | ACTIVE | deleting | Running | novanetwork=192.168.32.2 | +--+--+++-+--+ [root@puma31 ~(keystone_admin)]# nova image-create e00ae899-e285-4f09-8cda-2c2680799bba destroy_test --poll Server snapshotting... 50% complete Server snapshotting... 50% complete Server snapshotting... 100% complete Finished ERROR: Instance could not be found (HTTP 404) (Request-ID: req-b6b7b066-0da8-441a-8788-b6969d7b1527) [root@puma31 ~(keystone_admin)]# [root@puma31 ~(keystone_admin)]# [root@puma31 ~(keystone_admin)]# glance image-list +--+--+-+--+++ | ID | Name | Disk Format | Container Format | Size | Status | +--+--+-+--+++ | 6aa2362c-a1bb-490a-aeeb-3786ad7b9312 | destroy_test | qcow2 | bare | 3629645824 | active | | 73f92385-3080-4a4e-a100-76de38a3a569 | new_snap | qcow2 | bare | 3628728320 | active | | deddabea-475f-4c2f-88e3-0c76612e529c | poll-test1 | qcow2 | bare | 3629383680 | active | | df06e227-0d6a-4e2c-90c1-13cd32721360 | rhel | qcow2 | bare | 3628990464 | active | | 6175a441-8cb2-4d35-9b7d-241d51eaa270 | rhel1| qcow2 | bare | 3629383680 | active | +--+--+-+--+++ To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1312796/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1199475] Re: xenapi: bad error when trying to attach a volume to a non-pv guest
This really isn't addressable in current API, need new APIs to address this. ** Changed in: nova Status: Triaged = Opinion ** Changed in: nova Importance: Low = Wishlist -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1199475 Title: xenapi: bad error when trying to attach a volume to a non-pv guest Status in OpenStack Compute (Nova): Opinion Bug description: We get the following error from XenAPI when trying to do VBD.plug on a volume on a system without PV tools: VM_MISSING_PV_DRIVERS Really we need a way for these users to find out about this issue, and be able to attach the volume somehow. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1199475/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438189] [NEW] ComputeNode and Service objects out of sync with db schema
Public bug reported: A recent commit [1] removed the relationship between compute_node and service tables and in addition also made the compute_node.service_id column nullable. These changes were never replicated to the object counterparts. [1] 551be2c52a29cb2755de4825a3fcb2c8f7d7b3f1 ** Affects: nova Importance: Medium Assignee: Hans Lindgren (hanlind) Status: New ** Tags: unified-objects -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1438189 Title: ComputeNode and Service objects out of sync with db schema Status in OpenStack Compute (Nova): New Bug description: A recent commit [1] removed the relationship between compute_node and service tables and in addition also made the compute_node.service_id column nullable. These changes were never replicated to the object counterparts. [1] 551be2c52a29cb2755de4825a3fcb2c8f7d7b3f1 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1438189/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1158424] Re: Nova doesnt validate parameters while using euca-run-instances
ec2 apis in tree are deprecated, probably should be fixed out of tree instead ** Changed in: nova Status: Triaged = Opinion ** Changed in: nova Importance: Low = Wishlist -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1158424 Title: Nova doesnt validate parameters while using euca-run-instances Status in OpenStack Compute (Nova): Opinion Bug description: Using euca-run-instances -n 0 should run exactly instances instead of 1 instance according to: http://docs.aws.amazon.com/AWSEC2/latest/APIReference/api-error- codes.html To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1158424/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1435386] Re: VMs are being taken over through a VNC proxy exploit
That said, documentation recommends setting up 0.0.0.0 so that live migration can work: http://docs.openstack.org/admin-guide-cloud/content/section_configuring-compute-migrations.html You must specify vncserver_listen=0.0.0.0 or live migration will not work correctly. http://docs.openstack.org/admin-guide-cloud/content/nova-vncproxy-replaced-with-nova-novncproxy.html To connect the service to your Compute deployment, add the following configuration options to your nova.conf file: vncserver_listen=0.0.0.0 To use live migration, use the 0.0.0.0 address. ** Information type changed from Private Security to Public Security ** Also affects: openstack-manuals Importance: Undecided Status: New ** Summary changed: - VMs are being taken over through a VNC proxy exploit + Specific config setting may result in VMs being taken over through VNC -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1435386 Title: Specific config setting may result in VMs being taken over through VNC Status in OpenStack Compute (Nova): Invalid Status in OpenStack Manuals: New Status in OpenStack Security Advisories: Incomplete Bug description: Jonathan Hogg from Chargebox reports (edited): On a single-machine cloud running OpenStack Icehouse and over the last week we have seen compromises of all of the Ubuntu 14.04 VMs running on the machine. Scenario shows the attacker gaining access through VNC (via controlled reboot to reset root password). QEMU instances are running with -vnc 0.0.0.0:1, which may or may not be the issue. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1435386/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1173893] Re: Grizzly: Task instance_usage_audit is not running when it is enabled and nova-compute is running
qpid is used by nearly no one. Marking the nova bug as invalid, this should be the upstream fix ** Changed in: nova Status: Triaged = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1173893 Title: Grizzly: Task instance_usage_audit is not running when it is enabled and nova-compute is running Status in OpenStack Compute (Nova): Invalid Bug description: When using qpid as message queue backend and set instance_usage_audit = True the instance usage audit function doesn't work correct. 2013-02-26 17:34:38.727 ERROR nova.openstack.common.rpc.amqp [req-ff067345-7fba-4529-819c-ed8e5b1d6682 None None] Exception during message handling 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp Traceback (most recent call last): 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp File /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/amqp.py, line 276, in _process_data 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp rval = self.proxy.dispatch(ctxt, version, method, **args) 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp File /usr/lib/python2.6/site-packages/nova/openstack/common/rpc/dispatcher.py, line 133, in dispatch 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs) 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp File /usr/lib/python2.6/site-packages/nova/conductor/manager.py, line 348, in task_log_end_task 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp begin, end, host, errors, message) 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp File /usr/lib/python2.6/site-packages/nova/db/api.py, line 1689, in task_log_end_task 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp message) 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp File /usr/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py, line 118, in wrapper 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp return f(*args, **kwargs) 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp File /usr/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py, line 4791, in task_log_end_task 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp raise exception.TaskNotRunning(task_name=task_name, host=host) 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp TaskNotRunning: Task instance_usage_audit is not running on host WIN-CJR1366L5AH 2013-02-26 17:34:38.727 1399 TRACE nova.openstack.common.rpc.amqp 2013-02-26 17:34:38.757 ERROR nova.openstack.common.rpc.common [req-ff067345-7fba-4529-819c-ed8e5b1d6682 None None] Returning exception Task instance_usage_audit is not running on host WIN-CJR1366L5AH to caller To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1173893/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1357437] Re: nova.tests.virt.libvirt.test_driver should use constants from fakelibvirt
*** This bug is a duplicate of bug 1414708 *** https://bugs.launchpad.net/bugs/1414708 We removed the dependency on libvirt in unit tests here: https://github.com/openstack/nova/commit/50861d3221f625a5cc589d903054bb2c440b4026 ** This bug has been marked a duplicate of bug 1414708 libvirt unit tests are failing if no real libvirt installed -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1357437 Title: nova.tests.virt.libvirt.test_driver should use constants from fakelibvirt Status in OpenStack Compute (Nova): Confirmed Bug description: Commit f0883800660ab546f5667b973f339c4df4c5c458 adds some tests and new constants for _swap_volume and _live_snapshot, one of which is VIR_DOMAIN_BLOCK_REBASE_COPY. It adds the required constants to the fakelibvirt module but doesn't use them when making assertions in the test, which can fail if you're not using fakelibvirt but not using a new enough version of libvirt on your system. I realize that we now require libvirt-python = 1.2.5 for testing, but that requires libvirt = 0.9.11 and if you're on 0.9.11 it doesn't have VIR_DOMAIN_BLOCK_REBASE_COPY defined in libvirt.h.in: http://libvirt.org/git/?p=libvirt.git;a=blob;f=include/libvirt/libvirt.h.in;h=499dcd4514bf793b531e53496c56237fb055e1ba;hb=782afa98e4a5fa9a0927a9e32f9cf36082a2e8e7 It seems kind of strange to me that we use constants directly from the libvirt module at all in test_driver given the import conditional check at the top of the module: http://git.openstack.org/cgit/openstack/nova/tree/nova/tests/virt/libvirt/test_driver.py#n88 try: import libvirt except ImportError: libvirt = fakelibvirt libvirt_driver.libvirt = libvirt Given that, any change that requires testing new constants from libvirt should be in fakelibvirt, and we should use fakelibvirt when using those constants in the tests, otherwise the unit tests are really dependent on your environment. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1357437/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438238] [NEW] Several concurent scheduling requests for CPU pinning may fail due to racy host_state handling
Public bug reported: The issue happens when multiple scheduling attempts that request CPU pinning are done in parallel. 015-03-25T14:18:00.222 controller-0 nova-scheduler err Exception during message handling: Cannot pin/unpin cpus [4] from the following pinned set [3, 4, 5, 6, 7, 8, 9] 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher File /usr/lib64/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py, line 134, in _dispatch_and_reply 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher incoming.message)) 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher File /usr/lib64/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py, line 177, in _dispatch 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args) 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher File /usr/lib64/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py, line 123, in _do_dispatch 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args) 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher File /usr/lib64/python2.7/site-packages/oslo/messaging/rpc/server.py, line 139, in inner 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher return func(*args, **kwargs) 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher File ./usr/lib64/python2.7/site-packages/nova/scheduler/manager.py, line 86, in select_destinations 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher File ./usr/lib64/python2.7/site- packages/nova/scheduler/filter_scheduler.py, line 80, in select_destinations 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher File ./usr/lib64/python2.7/site- packages/nova/scheduler/filter_scheduler.py, line 241, in _schedule 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher File ./usr/lib64/python2.7/site-packages/nova/scheduler/host_manager.py, line 266, in consume_from_instance 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher File ./usr/lib64/python2.7/site-packages/nova/virt/hardware.py, line 1472, in get_host_numa_usage_from_instance 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher File ./usr/lib64/python2.7/site-packages/nova/virt/hardware.py, line 1344, in numa_usage_from_instances 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher File ./usr/lib64/python2.7/site-packages/nova/objects/numa.py, line 91, in pin_cpus 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher CPUPinningInvalid: Cannot pin/unpin cpus [4] from the following pinned set [3, 4, 5, 6, 7, 8, 9] 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher What is likely happening is: * nova-scheduler is handling several RPC calls to select_destinations at the same time, in multiple greenthreads * greenthread 1 runs the NUMATopologyFilter and selects a cpu on a particular compute node, updating host_state.instance_numa_topology * greenthread 1 then blocks for some reason * greenthread 2 runs the NUMATopologyFilter and selects the same cpu on the same compute node, updating host_state.instance_numa_topology. This also seems like an issue if a different cpu was selected, as it would be overwriting the instance_numa_topology selected by greenthread 1. * greenthread 2 then blocks for some reason * greenthread 1 gets scheduled and calls consume_from_instance, which consumes the numa resources based on what is in host_state.instance_numa_topology * greenthread 1 completes the scheduling operation * greenthread 2 gets scheduled and calls consume_from_instance, which consumes the numa resources based on what is in host_state.instance_numa_topology - since the resources were already consumed by greenthread 1, we get the exception above ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1438238 Title: Several concurent scheduling requests for CPU pinning may fail due to racy host_state handling Status in OpenStack Compute (Nova): New Bug description: The issue happens when multiple scheduling attempts that request CPU pinning are done in parallel. 015-03-25T14:18:00.222 controller-0 nova-scheduler err Exception during message handling: Cannot pin/unpin cpus [4] from the following pinned set [3, 4, 5, 6, 7, 8, 9] 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last): 2015-03-25 14:18:00.221 34127 TRACE oslo.messaging.rpc.dispatcher File /usr/lib64/python2.7/site- packages/oslo/messaging/rpc/dispatcher.py, line 134, in
[Yahoo-eng-team] [Bug 1194106] Re: db.utils.create_shadow_table() doesn't work if we have table with Enum column
This code isn't in nova any more, moving to oslo.db ** Also affects: oslo.db Importance: Undecided Status: New ** Changed in: nova Status: Triaged = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1194106 Title: db.utils.create_shadow_table() doesn't work if we have table with Enum column Status in OpenStack Compute (Nova): Invalid Status in Oslo Database library: New Bug description: In postgres we are not able to create two times Enum with the same name. So If we try to make a shadow table for table we will get exception that Enum with the same name already exists. So we should rename our Enum. I think that shadow_prefix + original_name will be good convention. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1194106/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438226] [NEW] nova libvirt driver assumes libvirt support for CPU pinning
Public bug reported:
CPU pinning support was implemented as part of this blueprint:
http://specs.openstack.org/openstack/nova-specs/specs/juno/approved
/virt-driver-cpu-pinning.html
However, CPU pinning support is broken in some libvirt versions
(summarized below), resulting in exceptions when attempting to schedule
instances with the 'hw:cpu_policy' flavor key.
We should add a libvirt version test against known broken versions and
use that to determine whether or not to support the flavor keys.
This is somewhat related to #1422775 (nova libvirt driver assumes qemu
support for NUMA pinning).
---
# Testing Configuration
Testing was conducted in a container which provided a single-node,
Fedora 21-based (3.17.8-300.fc21.x86_64) OpenStack instance (built with
devstack). The yum-provided libvirt and its dependencies were removed
and libvirt and libvirt-python were built and installed from source.
# Results
The results are as follows:
versions status
--
1.2.9 ok
1.2.9.1 ok
1.2.9.2 fail
1.2.10fail
1.2.11ok
1.2.12ok
v1.2.9.2 is broken by this (backported) patch:
https://www.redhat.com/archives/libvir-
list/2014-November/msg00275.html
This can be seen as commit
e226772 (qemu: fix domain startup failing with 'strict' mode in
numatune)
v1.2.10 inherits is broken at checkout but can be fixed by applying
these three patches (yes, one of these broke v1.2.9.2 - the irony is not
lost on me):
[0/3]
https://www.redhat.com/archives/libvir-list/2014-November/msg00274.html
- [1/3]
https://www.redhat.com/archives/libvir-list/2014-November/msg00273.html
- [2/3]
https://www.redhat.com/archives/libvir-list/2014-November/msg00276.html
- [3/3]
https://www.redhat.com/archives/libvir-list/2014-November/msg00275.html
# Error logs
v1.2.9.2 produces the following exception:
Traceback (most recent call last):
File /opt/stack/nova/nova/compute/manager.py, line 2301, in
_build_resources
yield resources
File /opt/stack/nova/nova/compute/manager.py, line 2171, in
_build_and_run_instance
flavor=flavor)
File /opt/stack/nova/nova/virt/libvirt/driver.py, line 2357, in spawn
block_device_info=block_device_info)
File /opt/stack/nova/nova/virt/libvirt/driver.py, line 4376, in
_create_domain_and_network
power_on=power_on)
File /opt/stack/nova/nova/virt/libvirt/driver.py, line 4307, in
_create_domain
LOG.error(err)
File /usr/lib/python2.7/site-packages/oslo_utils/excutils.py, line 82,
in __exit__
six.reraise(self.type_, self.value, self.tb)
File /opt/stack/nova/nova/virt/libvirt/driver.py, line 4297, in
_create_domain
domain.createWithFlags(launch_flags)
File /usr/lib/python2.7/site-packages/eventlet/tpool.py, line 183, in
doit
result = proxy_call(self._autowrap, f, *args, **kwargs)
File /usr/lib/python2.7/site-packages/eventlet/tpool.py, line 141, in
proxy_call
rv = execute(f, *args, **kwargs)
File /usr/lib/python2.7/site-packages/eventlet/tpool.py, line 122, in
execute
six.reraise(c, e, tb)
File /usr/lib/python2.7/site-packages/eventlet/tpool.py, line 80, in
tworker
rv = meth(*args, **kwargs)
File /usr/lib64/python2.7/site-packages/libvirt.py, line 1029, in
createWithFlags
if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed',
dom=self)
libvirtError: Failed to create controller cpu for group: No such file or
directory
v1.2.10 produces the following exception:
Traceback (most recent call last):
File /opt/stack/nova/nova/compute/manager.py, line 2342, in
_build_resources
yield resources
File /opt/stack/nova/nova/compute/manager.py, line 2215, in
_build_and_run_instance
block_device_info=block_device_info)
File /opt/stack/nova/nova/virt/libvirt/driver.py, line 2356, in spawn
block_device_info=block_device_info)
File /opt/stack/nova/nova/virt/libvirt/driver.py, line 4375, in
_create_domain_and_network
power_on=power_on)
File /opt/stack/nova/nova/virt/libvirt/driver.py, line 4306, in
_create_domain
LOG.error(err)
File /usr/lib/python2.7/site-packages/oslo_utils/excutils.py, line 85,
in __exit__
six.reraise(self.type_, self.value, self.tb)
File /opt/stack/nova/nova/virt/libvirt/driver.py, line 4296, in
_create_domain
domain.createWithFlags(launch_flags)
File /usr/lib/python2.7/site-packages/eventlet/tpool.py, line 183, in
doit
result = proxy_call(self._autowrap, f, *args, **kwargs)
File /usr/lib/python2.7/site-packages/eventlet/tpool.py, line 141, in
proxy_call
rv = execute(f, *args, **kwargs)
File /usr/lib/python2.7/site-packages/eventlet/tpool.py, line 122, in
execute
six.reraise(c, e, tb)
File /usr/lib/python2.7/site-packages/eventlet/tpool.py, line 80, in
tworker
[Yahoo-eng-team] [Bug 1349515] Re: Commit 0e98f5a breaks db migrations with DB2
This is being handled with a backend-specific condition in the 247 migration: https://review.openstack.org/#/c/69047/45/nova/db/sqlalchemy/migrate_repo/versions/247_nullable_mismatch.py ** Changed in: nova Status: Triaged = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1349515 Title: Commit 0e98f5a breaks db migrations with DB2 Status in OpenStack Compute (Nova): Invalid Bug description: This change: https://github.com/openstack/nova/commit/0e98f5a522c08b17c98ed108459a179d14eacd4a Sets pci_devices.deleted.nullable=True which breaks with DB2 because that column is in a UniqueConstraint: http://git.openstack.org/cgit/openstack/nova/tree/nova/db/sqlalchemy/models.py#n1374 And DB2 requires that columns in a UniqueConstraint are not nullable. We get a trace like this: 2014-07-28 03:40:33.511 7720 INFO migrate.versioning.api [-] 246 - 247... 2014-07-28 03:40:37.073 7720 CRITICAL nova [-] ProgrammingError: (ProgrammingError) ibm_db_dbi::ProgrammingError: Statement Execute Failed: [IBM][CLI Driver][DB2/LINUXX8664] SQL0542N The column named DELETED cannot be a column of a primary key or unique key constraint because it can contain null values. SQLSTATE=42831 SQLCODE=-542 '\nALTER TABLE pci_devices ALTER COLUMN deleted DROP NOT NULL' () 2014-07-28 03:40:37.073 7720 TRACE nova Traceback (most recent call last): 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/bin/nova-manage, line 10, in module 2014-07-28 03:40:37.073 7720 TRACE nova sys.exit(main()) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib/python2.6/site-packages/nova/cmd/manage.py, line 1401, in main 2014-07-28 03:40:37.073 7720 TRACE nova ret = fn(*fn_args, **fn_kwargs) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib/python2.6/site-packages/nova/cmd/manage.py, line 902, in sync 2014-07-28 03:40:37.073 7720 TRACE nova return migration.db_sync(version) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib/python2.6/site-packages/nova/db/migration.py, line 29, in db_sync 2014-07-28 03:40:37.073 7720 TRACE nova return IMPL.db_sync(version=version) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib/python2.6/site-packages/nova/db/sqlalchemy/migration.py, line 44, in db_sync 2014-07-28 03:40:37.073 7720 TRACE nova return versioning_api.upgrade(get_engine(), repository, version) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib/python2.6/site-packages/migrate/versioning/api.py, line 186, in upgrade 2014-07-28 03:40:37.073 7720 TRACE nova return _migrate(url, repository, version, upgrade=True, err=err, **opts) 2014-07-28 03:40:37.073 7720 TRACE nova File string, line 2, in _migrate 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib/python2.6/site-packages/migrate/versioning/util/__init__.py, line 160, in with_engine 2014-07-28 03:40:37.073 7720 TRACE nova return f(*a, **kw) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib/python2.6/site-packages/migrate/versioning/api.py, line 366, in _migrate 2014-07-28 03:40:37.073 7720 TRACE nova schema.runchange(ver, change, changeset.step) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib/python2.6/site-packages/migrate/versioning/schema.py, line 93, in runchange 2014-07-28 03:40:37.073 7720 TRACE nova change.run(self.engine, step) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib/python2.6/site-packages/migrate/versioning/script/py.py, line 148, in run 2014-07-28 03:40:37.073 7720 TRACE nova script_func(engine) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib/python2.6/site-packages/nova/db/sqlalchemy/migrate_repo/versions/247_nullable_mismatch.py, line 27, in upgrade 2014-07-28 03:40:37.073 7720 TRACE nova pci_devices.c.deleted.alter(nullable=True) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib/python2.6/site-packages/migrate/changeset/schema.py, line 534, in alter 2014-07-28 03:40:37.073 7720 TRACE nova return alter_column(self, *p, **k) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib/python2.6/site-packages/migrate/changeset/schema.py, line 141, in alter_column 2014-07-28 03:40:37.073 7720 TRACE nova engine._run_visitor(visitorcallable, delta) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py, line 1479, in _run_visitor 2014-07-28 03:40:37.073 7720 TRACE nova conn._run_visitor(visitorcallable, element, **kwargs) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py, line 1122, in _run_visitor 2014-07-28 03:40:37.073 7720 TRACE nova **kwargs).traverse_single(element) 2014-07-28 03:40:37.073 7720 TRACE nova File /usr/lib/python2.6/site-packages/migrate/changeset/ansisql.py, line 56, in
[Yahoo-eng-team] [Bug 1014647] Re: Tempest has no test for soft reboot
** Changed in: nova Status: Triaged = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1014647 Title: Tempest has no test for soft reboot Status in OpenStack Compute (Nova): Invalid Status in Tempest: Confirmed Bug description: 1. soft reboot requires support from the guest to operate. The current nova implementation tells the guest to reboot and then waits. If the soft reboot did not happen, it triggers a hard reboot but after a default wait of 2 minutes. Solution: Provide a new soft_reboot_image_ref, defaults to None, that is used for soft reboot tests which. If the value is None then the test is skipped. 2. Because of (1), we should only use soft reboot when we are actually testing that feature. 3. The current soft reboot test does not check that a soft reboot was done rather than hard. It should check for the server state of REBOOT. Same issue for the hard reboot test. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1014647/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1237683] Re: inconsitent virtual size in qcow base image after block-migration
Addressed in Havana ** Changed in: nova Status: Triaged = Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1237683 Title: inconsitent virtual size in qcow base image after block-migration Status in OpenStack Compute (Nova): Fix Released Bug description: We're running a Grizzly node using KVM (1.0 from cloud-archive) with local ephemeral instance storage. Since approximately the time we upgraded to Grizzly we've been receiving complaints from particular users about secondary disk corruption issues. These users in particular are noticing the issue because they are relying on the secondary drive and also because hey are using CentOS, which drops to an interactive prompt before completing boot if it cannot mount all filesystems (Ubuntu does not). We've since discovered that this is specifically linked to block- migration of such disks which were created and formatted automatically by Nova. I.e., if we launch a new instance, log in and then reformat the drive internally (even as ext3), we don't encounter corruption issues after live-migration. If we change the virt_mkfs config option to use mkfs.ext4 then we also don't have the problem. Unfortunately that's not a simple fix for an active production cloud because all existing backing files must be removed in order to force their recreation. In investigating the problem we noticed a behaviour that might be interrelated - after block-migration the instances secondary disk has a generic backing file instances/_base/ephemeral, as opposed to the backing file it was created with on the origin host, e.g., instances/_base/ephemeral_30_default. These backing files have different virtual sizes(!): $ qemu-img info _base/ephemeral image: _base/ephemeral file format: raw virtual size: 2.0G (2147483648 bytes) disk size: 778M $ qemu-img info _base/ephemeral_30_default image: _base/ephemeral_30_default file format: raw virtual size: 30G (32212254720 bytes) disk size: 614M We're no experts on qcow, but this looks like it could be problematic and may explain the corruption issues we're seeing - I can imagine there would be problems for a migrated guest that attempts to read a previously untouched sector beyond the size of the new backing file. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1237683/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1257405] Re: Not checking image format produces lots of useless messages
** Changed in: nova Status: Triaged = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1257405 Title: Not checking image format produces lots of useless messages Status in OpenStack Compute (Nova): Invalid Bug description: The code for resizing partitionless images goes with the tell, don't ask idea and attempts to run extfs / mount utilities on an image even though they may fail. This produces lots of useless messages during the instance preparation, like these: 2013-11-21 06:45:07 20902 DEBUG nova.virt.disk.mount.api [req-939d5d50-25ea-4f7a-8882-a880b9671e47 10873781609182 10816527907643] Failed to mount filesystem: Unexpected error while running command. Command: sudo nova-rootwrap /etc/nova/rootwrap.conf mount /dev/nbd8 /tmp/openstack-vfs-localfsSz6ylg Exit code: 32 Stdout: '' Stderr: 'mount: you must specify the filesystem type\n' mnt_dev /usr/lib/python2.7/dist-packages/nova/virt/disk/mount/api.py:198 2013-11-21 06:45:07 20902 DEBUG nova.virt.disk.mount.api [req-939d5d50-25ea-4f7a-8882-a880b9671e47 10873781609182 10816527907643] Fail to mount, tearing back down do_mount /usr/lib/python2.7/dist-packages/nova/virt/disk/mount/api.py:219 2013-11-21 06:45:07 20902 DEBUG nova.virt.disk.mount.api [req-939d5d50-25ea-4f7a-8882-a880b9671e47 10873781609182 10816527907643] Unmap dev /dev/nbd8 unmap_dev /usr/lib/python2.7/dist-packages/nova/virt/disk/mount/api.py:184 2013-11-21 06:45:07 20902 DEBUG nova.virt.disk.mount.nbd [req-939d5d50-25ea-4f7a-8882-a880b9671e47 10873781609182 10816527907643] Release nbd device /dev/nbd8 unget_dev /usr/lib/python2.7/dist-packages/nova/virt/disk/mount/nbd.py:128 2013-11-21 06:45:07 20902 DEBUG nova.openstack.common.processutils [req-939d5d50-25ea-4f7a-8882-a880b9671e47 10873781609182 10816527907643] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf qemu-nbd -d /dev/nbd8 execute /usr/lib/python2.7/dist-packages/nova/openstack/common/processutils.py:147 2013-11-21 06:45:07 20902 DEBUG nova.virt.disk.vfs.localfs [req-939d5d50-25ea-4f7a-8882-a880b9671e47 10873781609182 10816527907643] Failed to mount image Failed to mount filesystem: Unexpected error while running command. Command: sudo nova-rootwrap /etc/nova/rootwrap.conf mount /dev/nbd8 /tmp/openstack-vfs-localfsSz6ylg Exit code: 32 Stdout: '' Stderr: 'mount: you must specify the filesystem type\n') setup /usr/lib/python2.7/dist-packages/nova/virt/disk/vfs/localfs.py:83 2013-11-21 06:45:07 20902 DEBUG nova.virt.disk.api [req-939d5d50-25ea-4f7a-8882-a880b9671e47 10873781609182 10816527907643] Unable to mount image /var/lib/nova/instances/5159cff0-91f6-4521-a0be-d74ce4f81fad/disk with error Failed to mount filesystem: Unexpected error while running command. Command: sudo nova-rootwrap /etc/nova/rootwrap.conf mount /dev/nbd8 /tmp/openstack-vfs-localfsSz6ylg Exit code: 32 Stdout: '' Stderr: 'mount: you must specify the filesystem type\n'. Cannot resize. is_image_partitionless /usr/lib/python2.7/dist-packages/nova/virt/disk/api.py:191 This could be fixed by doing a simple check on the the image itself to pick up the magic signature. This would allow to skip e2resize on non- extfs files and simple mounting on partitioned images without the error messages. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1257405/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1437199] Re: zookeeper driver used with O(n^2) complexity by the scheduler or the api
Attilla, please don't file specs kinds of issues as bugs, this is really a specs level rearchitecture. ** Changed in: nova Status: New = Invalid ** Changed in: nova Importance: Undecided = Wishlist -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1437199 Title: zookeeper driver used with O(n^2) complexity by the scheduler or the api Status in OpenStack Compute (Nova): Invalid Bug description: (Loop1) https://github.com/openstack/nova/blob/af2d6c9576b1ac5f3b3768870bb15d9b5cf1610b/nova/scheduler/driver.py#L55 (Loop2) https://github.com/openstack/nova/blob/af2d6c9576b1ac5f3b3768870bb15d9b5cf1610b/nova/servicegroup/drivers/zk.py#L177 Iterating the hosts through the ComputeFilter also has this issue, ComputeFilter usage in a loop has other performance issues . The API loop1 is here: https://github.com/openstack/nova/blob/e5d0531d8ed4efcd612c0597557e5651c16294b5/nova/api/openstack/compute/contrib/services.py#L81 The zk driver issue can be mitigated by doing the testing `filtering` in the is_up instead of the get_all , by reorganizing the code. However better solution would be to have the scheduler to use the get_all, or redesigning the servicegroup management. A better design would be to use the DB even with the zk,mc drvier, but do update ONLY when the service actually came up or dies, in this case the sg drivers MAY need dedicated service processes. NOTE: The servicegroup driver concept was introduced to avoid doing 10_000 DB update/sec @100_000 host (10/sec update freq), if your servers are bad and every server has 1:1000 chance to die on the given day, it would lead only to 0.001 UPDATE/sec (100/day) @100_000 host. NOTE: If the up/down is knowable just form the DB, the scheduler could eliminate the dead hosts at the first DB query, without using ComputeFilter as it is used now. (The plugins SHOULD be able to extend the base hosts query) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1437199/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1359136] Re: vmware: display incorrect host_ip for hypervisor show
** Changed in: nova Status: Confirmed = Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1359136 Title: vmware: display incorrect host_ip for hypervisor show Status in OpenStack Compute (Nova): Won't Fix Bug description: From VCenter driver, with compute node configured to be cluster01. And VCenter IP is 10.9.1.43 But with hypervisor show command, it will display the host IP for the controller node. It should be pointing to the really VCenter IP. [root@dhcp-10-9-3-83 ~]# nova hypervisor-show 29 +---+--+ | Property | Value | +---+--+ | cpu_info_model| [Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz, Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz] | | cpu_info_topology_cores | 24 | | cpu_info_topology_threads | 48 | | cpu_info_vendor | [IBM, IBM] | | current_workload | 0 | | disk_available_least | - | | free_disk_gb | 763 | | free_ram_mb | 354728 | | host_ip | 10.9.3.83 | | hypervisor_hostname | domain-c17(cluster01) | | hypervisor_type | VMware vCenter Server | | hypervisor_version| 5005000 | | id| 29 | | local_gb | 794 | | local_gb_used | 31 | | memory_mb | 362336 | | memory_mb_used| 7608 | | running_vms | 3 | | service_host | dhcp-10-9-3-83.sce.cn.ibm.com | | service_id| 6 | | vcpus | 48 | | vcpus_used| 5 | +---+--+ To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1359136/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1102714] Re: nova-network related fixed ip cleanup is needed
** Changed in: nova Status: Triaged = Opinion ** Changed in: nova Importance: Low = Wishlist -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1102714 Title: nova-network related fixed ip cleanup is needed Status in OpenStack Compute (Nova): Opinion Bug description: overall description: If nova-network crashes (and then restarts) during a nova delete operation, then the ip associated with the deleted VM instance may remain in the allocated state. This may affect the subsequent network related operations. concrete example: step 1: create a fixed ip network with only 4 ips. the fixed_ips table looks like the follwoing at this stage: mysql select id,address,instance_id,allocated,leased,reserved from fixed_ips; ++---+-+---++--+ | id | address | instance_id | allocated | leased | reserved | ++---+-+---++--+ | 1 | 192.199.196.0 |NULL | 0 | 0 |1 | | 2 | 192.199.196.1 |NULL | 0 | 0 |1 | | 3 | 192.199.196.2 |NULL | 0 | 0 |0 | | 4 | 192.199.196.3 |NULL | 0 | 0 |1 | ++---+-+---++--+ step 2: create a VM and then delete it. during the execution of nova delete command, nova-compute sends a rpc call and then a rpc cast to nova-network. At the time of the rpc cast, before the rpc is sent, we stop nova-network service (to emulate a service crash). The rpc cast is lost if the rpc queue related to nova-network is auto-deleted with the stopped service. the nova delete command returns successfully. the VM is marked as deleted. but the ip associated with the VM is still marked as allocated due to the lost rpc message. the fixed_ips table looks like the following at this stage: mysql select id,address,instance_id,allocated,leased,reserved from fixed_ips; ++---+-+---++--+ | id | address | instance_id | allocated | leased | reserved | ++---+-+---++--+ | 1 | 192.199.196.0 |NULL | 0 | 0 |1 | | 2 | 192.199.196.1 |NULL | 0 | 0 |1 | | 3 | 192.199.196.2 | 1 | 1 | 0 |0 | | 4 | 192.199.196.3 |NULL | 0 | 0 |1 | ++---+-+---++--+ step 3: restart the nova-network service. try to create another VM via nova boot. this time we will get an exception due to zero fixed ips available. thought: from a user's perspective, the VM in step 3 should be created. that operation cannot succeed due to the existence of an orphan ip related to a previously deleted instance. Admittedly, the situation in the above example can be avoided by configuring the rpc queue related to nova-network to be non-auto-deleted. But a periodical orphan ip cleanup logic may help in general in such cases. Is there such a logic/module in OpenStack? If there is, should it be triggered when the fixed_ips table is about to be exhausted (as in the above example)? To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1102714/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1433805] Re: Absence of python-ironicclient in nova requirements.txt making upgrades awkward, python-ironicclient features diffucult
Can this just be handled with changing the required upgrade order? Just make ironic go first? ** Changed in: nova Status: New = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1433805 Title: Absence of python-ironicclient in nova requirements.txt making upgrades awkward, python-ironicclient features diffucult Status in OpenStack Bare Metal Provisioning Service (Ironic): Won't Fix Status in OpenStack Compute (Nova): Opinion Status in OpenStack Compute (nova) juno series: In Progress Bug description: Nova's requirements.txt does not list python-ironicclient, meaning a stable/juno nova deployment (at least in our gate) will be running with the most recent release of python-ironicclient. Many new features have been added to Ironic since juno and have been introduced incrementally via API micro-versions. The client library released at the time of stable/juno did not send any API version header. The current (kilo) server recognizes this and defaults to the lowest API version (v1.1) it supports. The desired behavior of python- ironicclient is for it to request the greatest API version it understands (presently 1.6) [3]. The nova.virt.ironic driver in juno/stable depends on node states only available in the corresponding version [1] of Ironic. These have changed since then and the new node states are exposed via new API micro-versions [2]. Using a new client library with a new server release will result in the new states being returned to Nova. In particular, the state of a node that is available for use, as returned by the v1.1 API is NOSTATE, and as returned by the current Kilo API, is AVAILABLE. The goal is to make the client transparently negotiate which version to use with the Ironic server if the latest version is not supported. This is a feature that would be introduced in a future python- ironicclient release. However, since Nova is not listing python-ironicclient in its requirements, during upgrades we can end up with a stable/juno Nova using this new client version to speak to a Kilo Ironic server via the most recent API micro versions. This would result in nova driver errors as the Ironic server would be returning node states that stable/juno driver [1] does not understand [2]. We either need to introduce python-ironicclient as a listed requirement of Nova (at least in stable), or explicitly declare that the driver use the older API version in its client interactions, or require that operators upgrade Nova (and python-ironicclient) to Kilo before upgrading Ironic. [1] https://git.openstack.org/cgit/openstack/nova/tree/nova/virt/ironic/ironic_states.py?h=stable%2Fjuno [2] https://git.openstack.org/cgit/openstack/nova/tree/nova/virt/ironic/ironic_states.py [3] http://specs.openstack.org/openstack/ironic-specs/specs/kilo/api-microversions.html To manage notifications about this bug go to: https://bugs.launchpad.net/ironic/+bug/1433805/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1414437] Re: conf file with spaces before data cause error
Working as designed. it's reporting that you have a problem with the ini file...no? ** Changed in: nova Status: Confirmed = Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1414437 Title: conf file with spaces before data cause error Status in OpenStack Compute (Nova): Won't Fix Bug description: Add a space before a conf line cause failure at restarting the service. example via nova.conf: rabbit_hosts=10.35.186.132:5672 == rabbit_hosts=10.35.186.132:5672 cause: [root@controller ~(keystone_admin)]# openstack-service restart Job for openstack-nova-cert.service failed. See 'systemctl status openstack-nova-cert.service' and 'journalctl -xn' for details. Job for openstack-nova-api.service failed. See 'systemctl status openstack-nova-api.service' and 'journalctl -xn' for details. Job for openstack-nova-consoleauth.service failed. See 'systemctl status openstack-nova-consoleauth.service' and 'journalctl -xn' for details. Job for openstack-nova-conductor.service failed. See 'systemctl status openstack-nova-conductor.service' and 'journalctl -xn' for details. Job for openstack-nova-scheduler.service failed. See 'systemctl status openstack-nova-scheduler.service' and 'journalctl -xn' for details. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1414437/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1430521] Re: nova network-create is ignoring --bridge when using the VlanManager
There are other bits of the nova-network code which assume bridge # maps to vlan #, so I'm not sure this is reasonably changable above the driver layer. I suspect a warning that vlan and bridge are incompatibile options in the client would be the near term fix. ** Changed in: nova Importance: Undecided = Low ** Also affects: python-novaclient Importance: Undecided Status: New ** Changed in: nova Status: New = Incomplete ** Changed in: python-novaclient Status: New = Confirmed ** Changed in: python-novaclient Importance: Undecided = Low -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1430521 Title: nova network-create is ignoring --bridge when using the VlanManager Status in OpenStack Compute (Nova): Incomplete Status in Python client library for Nova: Confirmed Bug description: The `nova network-create` is *silently* ignoring the --bridge when using the VlanManager. The bridge name is hardcoded to br%s % vlan. https://github.com/openstack/nova/blob/stable/juno/nova/network/manager.py#L1361 [root@jhenner-vmware ~(keystone_admin)]# nova network-create novanetwork --fixed-range-v4 192.168.32.0/22 --bridge br101 --bridge-interface br101 --uuid 874ab2f1-c57d-464a-8a3f-5dda76ac7613 --vlan 102 [root@jhenner-vmware ~(keystone_admin)]# nova network-show novanetwork +-+--+ | Property| Value| +-+--+ | bridge | br102| | bridge_interface| br101| | broadcast | 192.168.35.255 | | cidr| 192.168.32.0/22 | | cidr_v6 | -| | created_at | 2015-03-10T20:25:41.00 | | deleted | False| | deleted_at | -| | dhcp_server | 192.168.32.1 | | dhcp_start | 192.168.32.3 | | dns1| 8.8.4.4 | | dns2| -| | enable_dhcp | True | | gateway | 192.168.32.1 | | gateway_v6 | -| | host| -| | id | ca6b6f02-9f40-4578-89c6-9c8ca6d29bee | | injected| False| | label | novanetwork | | mtu | -| | multi_host | True | | netmask | 255.255.252.0| | netmask_v6 | -| | priority| -| | project_id | -| | rxtx_base | -| | share_address | False| | updated_at | -| | vlan| 102 | | vpn_private_address | 192.168.32.2 | | vpn_public_address | -| | vpn_public_port | 1000 | +-+--+ It would be good to have the bridge name configurable when using VlanManager because when the nova vcenter driver is used, nova is checking whether there is a port group in the vcenter of same name as bridge used in nova-network. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1430521/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438020] Re: replace hpy.host with hyp['host']
Not an actual bug ** Changed in: nova Status: In Progress = Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1438020 Title: replace hpy.host with hyp['host'] Status in OpenStack Compute (Nova): Won't Fix Bug description: class ComputeNode in https://github.com/openstack/nova/blob/master/nova/objects/compute_node.py#L27 is extend from NovaObjectDictCompat in https://github.com/openstack/nova/blob/master/nova/objects/base.py#L652 def __getitem__(self, name): For backwards-compatibility with dict-based objects. NOTE(danms): May be removed in the future. return getattr(self, name) as we can see above,function __getitem__ may be removed in the future.and now some codes use this function to get attribute from computenode object, hpy.host, https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/contrib/hypervisors.py#L92 but the unit test data not,data in unit test is only a pure dict. https://github.com/openstack/nova/blob/8b29068398b1217579aca07f2596ae88682d5055/nova/tests/unit/objects/test_compute_node.py In order to make it more consistent,change codes from obj.attr to obj['attr']. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1438020/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1394462] Re: nova show output should display sec-group id instead of name
** No longer affects: nova
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1394462
Title:
nova show output should display sec-group id instead of name
Status in Python client library for Nova:
Confirmed
Bug description:
As nova security-group-create allowing more than one group with the
same name. showing security-group name in nova show is not correct
I have two security groups with name default.
ram@ubuntu:~$ neutron security-group-list
+--+-+-+
| id | name| description |
+--+-+-+
| 2d3d1914-32d1-451f-b4bc-ed5eeda398ee | default | default |
| c2d19dea-0863-40d5-872c-543f97b00bd4 | default | default |
+--+-+-+
In this case, I did not specify any security group while spawning the
instance. I do not know how nova boot picks the default security group
but nova show displays default in the security group.
ram@ubuntu:~$ nova show first
+--++
| Property | Value
|
+--++
| OS-DCF:diskConfig| MANUAL
|
| OS-EXT-AZ:availability_zone | nova
|
| OS-EXT-SRV-ATTR:host | ubuntu
|
| OS-EXT-SRV-ATTR:hypervisor_hostname | ubuntu
|
| OS-EXT-SRV-ATTR:instance_name| instance-0001
|
| OS-EXT-STS:power_state | 1
|
| OS-EXT-STS:task_state| -
|
| OS-EXT-STS:vm_state | active
|
| OS-SRV-USG:launched_at | 2014-11-19T18:42:00.00
|
| OS-SRV-USG:terminated_at | -
|
| accessIPv4 |
|
| accessIPv6 |
|
| config_drive |
|
| created | 2014-11-19T18:41:15Z
|
| flavor | m1.tiny (1)
|
| hostId |
4ac39bb970bb90f0aca2efaca1f43cc2997f6550a1449f08ade677af |
| id | 8f7eb319-8f5d-46c2-bb1f-6a16838ef7b1
|
| image| cirros-0.3.2-x86_64-uec
(90aa74a4-138a-4a1a-a530-aa1cd4ee5e05) |
| key_name | -
|
| metadata | {}
|
| name | first
|
| os-extended-volumes:volumes_attached | []
|
| private network | 10.0.0.2
|
| progress | 0
|
| security_groups | default
| --- security group
| status | ACTIVE
|
| tenant_id| e747bc1a96ea4d88a0ddf7b2df8e0ad8
|
| updated | 2014-11-19T18:42:01Z
|
| user_id | 31a22dcf6b0a437294cb6c10f2996e08
|
+--++
It leads to confusion in getting the security group id used by the instance..
To manage notifications about this bug go to:
[Yahoo-eng-team] [Bug 1434172] Re: security group create errors without description
Right, optional means you are allowed to not provide the parameter. Not
that you can provide it as null. There is a python-novaclient bug here.
** Changed in: nova
Status: Incomplete = Won't Fix
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1434172
Title:
security group create errors without description
Status in OpenStack Compute (Nova):
Won't Fix
Status in Python client library for Nova:
In Progress
Status in OpenStack Command Line Client:
In Progress
Bug description:
security group create returns an error without --description supplied.
This appears to be the server rejecting the request so we should set a
default value rather than sending None.
$ openstack security group create qaz
ERROR: openstack Security group description is not a string or unicode
(HTTP 400) (Request-ID: req-dee03de3-893a-4d58-bc3d-de87d09c3fb8)
Sent body:
{security_group: {name: qaz2, description: null}}
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1434172/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1326345] Re: Legacy networking (nova-network) in OpenStack Icehouse Installation results in no internet access for instances
No longer reproducable, was fixed by the user ** Changed in: nova Status: Incomplete = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1326345 Title: Legacy networking (nova-network) in OpenStack Icehouse Installation results in no internet access for instances Status in OpenStack Compute (Nova): Invalid Bug description: I have performed installation of Openstack Icehouse on two CentOS6.5 machines from scratch (both the OS and Openstack). The problem is that the instances do not get internet access unless the following rule is added: iptables -t nat -I POSTROUTING -o eth1 -j MASQUERADE where eth1 is the public interface. This rule has to be re-inserted every time an instance is launched! A few words regarding the setup of the compute node. -It has two ethernet interfaces eth0 (internal network) and eth1(public network). -I have added a bridge interface (br100) which has been bridged with eth0. -Both eth0 and eth1 are running in promiscuous mode -In sysctl.conf ip forwarding has been enabled -In nova.conf I have declared the flat_interface to be eth0 while the public_interface is eth1 -I have followed the guide from here : http://docs.openstack.org/icehouse/install-guide/install/yum/content/ for both compute and controller nodes. -I have added the secgroup-rules for ICMP and SSH to the default group -I am launching a cirrOS instance at the default security group which is assigned the IP 10.0.0.2. -I can ping the cirrOS instance -I can ssh to the cirrOS -While logged in to the instance I can resolve external addresses (e.g. nslookup openstack.org produces a correct output) -While logged in to the instance I can ping 10.0.0.1 -While logged in to the instance I can ping the compute nodes external IP -While logged in to the instance I CANNOT ping the compute nodes gateway or anything else outside that!!! If I insert the afforementioned rule then pinging works as expected! In the case I have a second instance all the above apply but also I can ping and ssh from one instance to another. Unfortunately the problem of pinging outside the compute node from both instances remains until I re-insert the afforementioned rule. Tests have shown that every time I start an instance the rule has to be added at the top of the chain otherwise none of them works. The above are also the case when floating IPs are assigned to the instances. They cannot reach an external network unless the rule has been added. When it has instances can reach and be reached from an external network. Furthermore, comparison with an Havana installation with the same network configuration and topology (two nodes, two ethernet nics each etc.) revealed that the following chains are not populated correctly in Icehouse - Chain nova-network-OUTPUT - Chain nova-network-POSTROUTING - Chain nova-network-PREROUTING - Chain nova-network-float-snat I have been trying to find a solution to this through the mailing-lists: http://lists.openstack.org/pipermail/openstack/2014-May/007454.html http://lists.openstack.org/pipermail/openstack/2014-May/007515.html and in the IRC channel but without success. I have performed the above installations twice and every time the same problem appears leading me to think that it's a bug. Please check it! Best, G. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1326345/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438133] [NEW] django-admin.py collectstatic failing to find static/themes/default
Public bug reported: At some stage over the last 10 days tripleo has stared to to fail to run DJANGO_SETTINGS_MODULE=openstack_dashboard.settings django-admin.py collectstatic --noinput Exiting with the following error os-collect-config: dib-run-parts Fri Mar 27 03:56:44 UTC 2015 Running /usr/libexec/os-refresh-config/post-configure.d/14-horizon os-collect-config: WARNING:root:dashboards and default_dashboard in (local_)settings is DEPRECATED now and may be unsupported in some future release. The preferred way to specify the order of dashboards and the default dashboard is the pluggable dashboard mechanism (in /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/openstack_dashboard/enabled, /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/openstack_dashboard/local/enabled). os-collect-config: WARNING:py.warnings:DeprecationWarning: The oslo namespace package is deprecated. Please use oslo_config instead. os-collect-config: WARNING:py.warnings:DeprecationWarning: The oslo namespace package is deprecated. Please use oslo_serialization instead. os-collect-config: Traceback (most recent call last): os-collect-config: File /opt/stack/venvs/openstack/bin/django-admin.py, line 5, in module os-collect-config: management.execute_from_command_line() os-collect-config: File /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/django/core/management/__init__.py, line 399, in execute_from_command_line os-collect-config: utility.execute() os-collect-config: File /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/django/core/management/__init__.py, line 392, in execute os-collect-config: self.fetch_command(subcommand).run_from_argv(self.argv) os-collect-config: File /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/django/core/management/base.py, line 242, in run_from_argv os-collect-config: self.execute(*args, **options.__dict__) os-collect-config: File /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/django/core/management/base.py, line 285, in execute os-collect-config: output = self.handle(*args, **options) os-collect-config: File /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/django/core/management/base.py, line 415, in handle os-collect-config: return self.handle_noargs(**options) os-collect-config: File /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py, line 173, in handle_noargs os-collect-config: collected = self.collect() os-collect-config: File /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py, line 103, in collect os-collect-config: for path, storage in finder.list(self.ignore_patterns): os-collect-config: File /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/django/contrib/staticfiles/finders.py, line 106, in list os-collect-config: for path in utils.get_files(storage, ignore_patterns): os-collect-config: File /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/django/contrib/staticfiles/utils.py, line 25, in get_files os-collect-config: directories, files = storage.listdir(location) os-collect-config: File /opt/stack/venvs/openstack/local/lib/python2.7/site-packages/django/core/files/storage.py, line 249, in listdir os-collect-config: for entry in os.listdir(path): os-collect-config: OSError: [Errno 2] No such file or directory: '/opt/stack/venvs/openstack/local/lib/python2.7/site-packages/openstack_dashboard/local/static/themes/default' os-collect-config: [2015-03-27 03:56:45,548] (os-refresh-config) [ERROR] during post-configure phase. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/post-configure.d']' returned non-zero exit status example here http://logs.openstack.org/05/168205/1/check-tripleo/check-tripleo-ironic-undercloud-precise-nonha/3f384bd/logs/undercloud-undercloud_logs/os-collect-config.txt.gz#_Mar_27_03_56_45 ** Affects: horizon Importance: Undecided Status: New ** Affects: tripleo Importance: Critical Status: Triaged ** Tags: ci ** Also affects: horizon Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1438133 Title: django-admin.py collectstatic failing to find static/themes/default Status in OpenStack Dashboard (Horizon): New Status in tripleo - openstack on openstack: Triaged Bug description: At some stage over the last 10 days tripleo has stared to to fail to run DJANGO_SETTINGS_MODULE=openstack_dashboard.settings django- admin.py collectstatic --noinput Exiting with the following error os-collect-config: dib-run-parts Fri Mar 27 03:56:44 UTC 2015 Running
[Yahoo-eng-team] [Bug 1438113] Re: Use plain HTTP listeners in the conductor
This is definitely way beyond a bug, it should be done as a nova-spec ** Changed in: nova Status: New = Invalid ** Changed in: nova Importance: Undecided = Wishlist -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1438113 Title: Use plain HTTP listeners in the conductor Status in OpenStack Compute (Nova): Invalid Bug description: The conductor is consuming messages form single queue which has performance limitation due to various reasons.: - per queue lock - Some broker also limiting same part of the message handling to single CPU thread/queue - Multiple broker instances needs to synchronise to queue content, which causes additional delays die to the tcp request/response times The single queue limitation is much greater than the limits getting by single mysql server, the rate is even worse when you consider slave reads. This can be workarounded by explicitly or implicit distributing the rpc calls to multiple different queue. The message broker provides additional message durability properties which is not needed just for an rpc_call, we spend resource on what we actually do not need. For TCP/HTTP traffic load balancing we have many-many tools even hardware assisted options are available providing virtually unlimited scalability. At TCP level also possible to exclude the loadbalancer node(s) form the response traffic. Why HTTP? Basically any protocol which can do request/response `thing` with arbitrary type and size of data with keep-alive connection and with ssl option, could be used. HTTP is a simple and well know protocol, with already existing many-many load balancing tool. Why not have the agents to do a regular API call? The regular API calls needs to do policy check, which in this case is not required, every authenticated user can be considered as admin. The the conductor clients needs to use at least a single shared key configured in every nova host. It has similar security as openstack used with the brokers, basically all nova node had credentials in one rabbitmq virtual host, configured in the /etc/nova/nova.conf . If any of those credentials stolen it provided access to the whole virtual host. NOTE.: HTTPs can be used with certificate or kerberos based authentication as well. I think the for `rpc_calls` which are served by the agents using something like AMQP is still the better option, this bug is just about the situation when the conductor itself serves rpc_call(s). NOTE.: The 1 Million msq/sec rabbitmq benchmark is done 186 queues, in a way which does not hits the single queue limitations. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1438113/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438154] [NEW] Creating metadef object without properties raises 500 error
Public bug reported:
When you try to create a metadef object without any properties
associated with it you get an Internal Server Error 500:
glance md-object-create --name obj_name --schema '{description: some
desc}' OS::Software::DBMS
AttributeError: 'NoneType' object has no attribute 'iteritems'
The object itself is created successfully, but the notification is not
fired.
** Affects: glance
Importance: Undecided
Assignee: Kamil Rykowski (kamil-rykowski)
Status: In Progress
** Tags: metadef
** Changed in: glance
Assignee: (unassigned) = Kamil Rykowski (kamil-rykowski)
** Changed in: glance
Status: New = In Progress
** Tags added: metadef
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to Glance.
https://bugs.launchpad.net/bugs/1438154
Title:
Creating metadef object without properties raises 500 error
Status in OpenStack Image Registry and Delivery Service (Glance):
In Progress
Bug description:
When you try to create a metadef object without any properties
associated with it you get an Internal Server Error 500:
glance md-object-create --name obj_name --schema '{description:
some desc}' OS::Software::DBMS
AttributeError: 'NoneType' object has no attribute 'iteritems'
The object itself is created successfully, but the notification is not
fired.
To manage notifications about this bug go to:
https://bugs.launchpad.net/glance/+bug/1438154/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1026621] Re: nova-network gets release_fixed_ip events from someplace, but the database still keeps them associated with instances
*** This bug is a duplicate of bug 1231254 ***
https://bugs.launchpad.net/bugs/1231254
** This bug has been marked a duplicate of bug 1231254
Windows instances need timezone to be localtime
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1026621
Title:
nova-network gets release_fixed_ip events from someplace, but the
database still keeps them associated with instances
Status in OpenStack Compute (Nova):
Triaged
Status in nova package in Ubuntu:
Triaged
Bug description:
We've been seeing a lot of instances simply vanish from the network.
Usually people have been willing to work around this by simply
rebooting or re-creating their instances, but it's troubling for long-
running instances (especially those that have volumes associated).
Here's the relevant bit of nova-network.log for one of these:
2012-07-16 14:06:32 DEBUG nova.rpc.amqp [-] received {u'_context_roles':
[u'admin'], u'_context_request_id':
u'req-d0905711-c4d1-4452-a3b2-46815d1983d7', u'_context_read_deleted': u'no',
u'args': {u'address': u'10.55.60.141'}, u'_context_auth_token': 'SANITIZED',
u'_context_is_admin': True, u'_context_project_id': None,
u'_context_timestamp': u'2012-07-16T14:06:32.169100', u'_context_user_id':
None, u'method': u'release_fixed_ip', u'_context_remote_address': None} from
(pid=493) _safe_log /usr/lib/python2.7/dist-packages/nova/rpc/common.py:160
2012-07-16 14:06:32 DEBUG nova.rpc.amqp
[req-d0905711-c4d1-4452-a3b2-46815d1983d7 None None] unpacked context:
{'user_id': None, 'roles': [u'admin'], 'timestamp':
'2012-07-16T14:06:32.169100', 'auth_token': 'SANITIZED', 'remote_address':
None, 'is_admin': True, 'request_id':
u'req-d0905711-c4d1-4452-a3b2-46815d1983d7', 'project_id': None,
'read_deleted': u'no'} from (pid=493) _safe_log
/usr/lib/python2.7/dist-packages/nova/rpc/common.py:160
2012-07-16 14:06:32 DEBUG nova.network.manager
[req-d0905711-c4d1-4452-a3b2-46815d1983d7 None None] Released IP |10.55.60.141|
from (pid=493) release_fixed_ip
/usr/lib/python2.7/dist-packages/nova/network/manager.py:1260
Then the dhcpbridge shows it being revoked:
2012-07-16 14:04:29 DEBUG nova.dhcpbridge [-] Called 'old' for mac
'fa:16:3e:11:c5:37' with ip '10.55.60.141' from (pid=23699) main
/usr/bin/nova-dhcpbridge:113
2012-07-16 14:06:32 DEBUG nova.dhcpbridge [-] Called 'del' for mac
'fa:16:3e:11:c5:37' with ip '10.55.60.141' from (pid=24946) main
/usr/bin/nova-dhcpbridge:113
Is there any way we can find out what might have placed the
release_fixed_ip event on the message queue? There doesn't seeem to
be any other mention of the IP in the nova logs on any of our systems.
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1026621/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1398361] Re: Add support for EC2 API : ec2-modify-snapshot-attribute
Moving to Opinion as EC2 in tree API is deprecated. ** Changed in: nova Status: Triaged = Opinion ** Tags added: ec2 -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1398361 Title: Add support for EC2 API : ec2-modify-snapshot-attribute Status in OpenStack Compute (Nova): Opinion Bug description: The various attributes of the snapshots created from volumes can be modified for making them accessible to public or to a particular user account. The response parameters and supported attribute sets must be according the AWS specification: http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-ModifySnapshotAttribute.html To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1398361/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1398357] Re: Add support for EC2 API : ec2-reset-snapshot-attribute
Moving to Opinion as EC2 in tree API is deprecated. ** Tags added: ec2 ** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1398357 Title: Add support for EC2 API : ec2-reset-snapshot-attribute Status in OpenStack Compute (Nova): Opinion Bug description: Provide the implementation similar to the Amazon EC2 API ec2-reset-snapshot-attribute for making the volume based snapshot instances accessible only to the respective account holder. http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference /ApiReference-cmd-ResetSnapshotAttribute.html To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1398357/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1338679] Re: Some Neutron plugins might miss network-vif-* events
I don't think this is really a nova side issue. Polling from nova seems really odd here. ** Changed in: nova Status: Triaged = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1338679 Title: Some Neutron plugins might miss network-vif-* events Status in OpenStack Neutron (virtual network service): Incomplete Status in OpenStack Compute (Nova): Invalid Bug description: This has been observed with the VMware NSX plugin, but in theory this issue might occur with every Neutron plugin when the notifications to nova are enabled. In a nutshell the issue is that nova needs to receive a network-vif- plugged even from neutron in order to be able to boot an instance. On the other hand in some cases VIF unplug/plug events might be fairly quick, and thus the neutron side might not be aware at all of the status change and not send any event. As a consequence, the instance will not boot even if its VIF are correctly plugged. This bug is currently being assigned both neutron and nova because: - it might deemed a plugin problem. If the plugin is not smart enough to handle VIF plug/unplug notifications than it's not worth using the notification mechanism exposed by nova - the nova side might add an optional poll mechanism which could run alongside the mechanism waiting for an event and periodically poll neutron to get the VIF status. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1338679/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1279055] Re: performance issue on restart of nova compute when there are a large number of vms
This is kind of working as designed. The concurency mode of n-cpu kind of needs to work this way. ** Changed in: nova Status: Triaged = Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1279055 Title: performance issue on restart of nova compute when there are a large number of vms Status in OpenStack Compute (Nova): Won't Fix Bug description: When there are large number of VMs in the cloud, there is a performance issue on restarting of nova compute. Compute server tries to sync states of all the vm and does not respond until sync is complete. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1279055/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1218994] Re: file based disk images do not get scrubbed on delete
Marking as Opinion because the solution here really needs to be encryption at rest so that a crashed nova compute doesn't leave customer data out there. ** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1218994 Title: file based disk images do not get scrubbed on delete Status in OpenStack Compute (Nova): Opinion Bug description: Right now, LVM backed instances can be scrubbed (overwritten with zeros using dd) upon deletion. However, there is no such option with file backed images. While it is true that fallocate can handle some of this by returning 0s to the instance when reading any unwritten parts of the file, there are some cases where it is not desirable to enable fallocate. What would be preferred would be a similar the options cinder has implemented, so the operator can choose to shred or zero out the file, based on their organizations own internal data policies. A zero out option satisfies those that must ensure they scrub tenant data upon deletion, and shred would satisfy those beholden to DoD 5220-22. This would of course make file backed disks vulnerable to https://bugs.launchpad.net/nova/+bug/889299 but that might not be a bad thing considering its quite old. Attached an initial patch for nova/virt/libvirt/driver.py that performs the same LVM zero scrub routine to disk backed files, however it lacks any flags to enable or disable it right now. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1218994/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1330873] Re: Instance name set in horizon is not set in VMware vCenter
I think the point of using UUID for instance names is probably spot on. Marking as Opinion, because it seems like for consistency it should be done that way. ** Changed in: nova Status: In Progress = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1330873 Title: Instance name set in horizon is not set in VMware vCenter Status in OpenStack Compute (Nova): Opinion Bug description: Instance name set in horizon is not set in VMware vCenter. In vCenter the name of the instance is the id (in UUID format) of the instance. This makes it difficult for the administrator to locate the instance in vCenter using the name displayed in horizon. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1330873/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438245] [NEW] Nova accesses image for resize even when resize is disabled
Public bug reported: The resize_fs_using_block_device config option allows to configure if a filesystem within an image is resized by nova during spawn or not. When it's set to False (which is the default) nova still tries to access the image to find out if it's extendable or not, although it's not resized afterwards. On system which don't have libguestfs or nbd installed, this causes the following error messages repeatedly: 2015-03-30 16:49:40.326 ERROR nova.virt.disk.mount.nbd [req-3d3c5a50-6067-442a-9607-1f98141d795c None None] nbd module not loaded 2015-03-30 16:49:40.327 INFO nova.virt.disk.mount.api [req-3d3c5a50-6067-442a-9607-1f98141d795c None None] Device allocation failed. Will retry in 2 seconds. 2015-03-30 16:49:42.329 ERROR nova.virt.disk.mount.nbd [req-3d3c5a50-6067-442a-9607-1f98141d795c None None] nbd module not loaded 2015-03-30 16:49:42.330 INFO nova.virt.disk.mount.api [req-3d3c5a50-6067-442a-9607-1f98141d795c None None] Device allocation failed. Will retry in 2 seconds. 2015-03-30 16:49:44.332 ERROR nova.virt.disk.mount.nbd [req-3d3c5a50-6067-442a-9607-1f98141d795c None None] nbd module not loaded 2015-03-30 16:49:44.333 INFO nova.virt.disk.mount.api [req-3d3c5a50-6067-442a-9607-1f98141d795c None None] Device allocation failed. Will retry in 2 seconds. So the instance spawn takes longer than necessary (retries are done for ~30 seconds). I'll provide a patch to make sure the image is not accessed when resizing is disabled. ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1438245 Title: Nova accesses image for resize even when resize is disabled Status in OpenStack Compute (Nova): New Bug description: The resize_fs_using_block_device config option allows to configure if a filesystem within an image is resized by nova during spawn or not. When it's set to False (which is the default) nova still tries to access the image to find out if it's extendable or not, although it's not resized afterwards. On system which don't have libguestfs or nbd installed, this causes the following error messages repeatedly: 2015-03-30 16:49:40.326 ERROR nova.virt.disk.mount.nbd [req-3d3c5a50-6067-442a-9607-1f98141d795c None None] nbd module not loaded 2015-03-30 16:49:40.327 INFO nova.virt.disk.mount.api [req-3d3c5a50-6067-442a-9607-1f98141d795c None None] Device allocation failed. Will retry in 2 seconds. 2015-03-30 16:49:42.329 ERROR nova.virt.disk.mount.nbd [req-3d3c5a50-6067-442a-9607-1f98141d795c None None] nbd module not loaded 2015-03-30 16:49:42.330 INFO nova.virt.disk.mount.api [req-3d3c5a50-6067-442a-9607-1f98141d795c None None] Device allocation failed. Will retry in 2 seconds. 2015-03-30 16:49:44.332 ERROR nova.virt.disk.mount.nbd [req-3d3c5a50-6067-442a-9607-1f98141d795c None None] nbd module not loaded 2015-03-30 16:49:44.333 INFO nova.virt.disk.mount.api [req-3d3c5a50-6067-442a-9607-1f98141d795c None None] Device allocation failed. Will retry in 2 seconds. So the instance spawn takes longer than necessary (retries are done for ~30 seconds). I'll provide a patch to make sure the image is not accessed when resizing is disabled. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1438245/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1379682] Re: instance_template_name does not work for vmware driver
*** This bug is a duplicate of bug 1330873 *** https://bugs.launchpad.net/bugs/1330873 ** This bug has been marked a duplicate of bug 1330873 Instance name set in horizon is not set in VMware vCenter -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1379682 Title: instance_template_name does not work for vmware driver Status in OpenStack Compute (Nova): Confirmed Status in OpenStack Manuals: Fix Released Bug description: Currently vmware driver will adopt uuid for instance names. This will lead two problems: 1) instance name template will not apply for vmware driver. But it will display instance name with nova show command. It will be misleading. [root@cmwo cmwo]# nova show temp-vm-host1-99 +--+---+ | Property | Value | +--+---+ | OS-DCF:diskConfig| MANUAL | | OS-EXT-AZ:availability_zone | cluster01 | | OS-EXT-SRV-ATTR:host | vcenter-cluster01 | | OS-EXT-SRV-ATTR:hypervisor_hostname | domain-c129(cluster01) | | OS-EXT-SRV-ATTR:instance_name| instance-00ec | | OS-EXT-STS:power_state | 1 | | OS-EXT-STS:task_state| - | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2014-10-09T10:02:15.00 | | OS-SRV-USG:terminated_at | - | | accessIPv4 | | | accessIPv6 | | | config_drive | | | created | 2014-10-09T09:59:15Z | | demovlan network | 25.0.0.21 | | flavor | m1.tiny (1) | | hostId | 0cfba386e1e5cad832d2fbc316c33ca6a124c3d0b386127a55707070 | | id | 6d3e0f11-0a4c-46eb-a3a6-4e397d917228 | 2) This uuid is not user friendly for VM names. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1379682/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438260] [NEW] Expensive Device allocation loop when NBD is absent
Public bug reported: See log stash query: Device allocation failed. Will retry in 2 seconds. Example: http://logs.openstack.org/10/163910/6/check/gate-rally-dsvm-cinder/46d89ab/logs/screen-n-cpu.txt.gz#_2015-03-30_14_55_56_941 Logs: 2015-03-30 14:55:56.940 ERROR nova.virt.disk.mount.nbd [req-df13155c-0340-4e10-aec4-2fabea382eef None None] nbd module not loaded 2015-03-30 14:55:56.940 DEBUG oslo_concurrency.lockutils [req-df13155c-0340-4e10-aec4-2fabea382eef None None] Lock nbd-allocation-lock released by _inner_get_dev :: held 0.000s inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456 2015-03-30 14:55:56.941 INFO nova.virt.disk.mount.api [req-df13155c-0340-4e10-aec4-2fabea382eef None None] Device allocation failed. Will retry in 2 seconds. 2015-03-30 14:55:58.941 DEBUG oslo_concurrency.lockutils [req-df13155c-0340-4e10-aec4-2fabea382eef None None] Lock nbd-allocation-lock acquired by _inner_get_dev :: waited 0.000s inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:444 2015-03-30 14:55:58.941 ERROR nova.virt.disk.mount.nbd [req-df13155c-0340-4e10-aec4-2fabea382eef None None] nbd module not loaded We should not even enter the loop when /sys/block/nbd0 is clearly absent. ** Affects: nova Importance: Undecided Assignee: Davanum Srinivas (DIMS) (dims-v) Status: In Progress -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1438260 Title: Expensive Device allocation loop when NBD is absent Status in OpenStack Compute (Nova): In Progress Bug description: See log stash query: Device allocation failed. Will retry in 2 seconds. Example: http://logs.openstack.org/10/163910/6/check/gate-rally-dsvm-cinder/46d89ab/logs/screen-n-cpu.txt.gz#_2015-03-30_14_55_56_941 Logs: 2015-03-30 14:55:56.940 ERROR nova.virt.disk.mount.nbd [req-df13155c-0340-4e10-aec4-2fabea382eef None None] nbd module not loaded 2015-03-30 14:55:56.940 DEBUG oslo_concurrency.lockutils [req-df13155c-0340-4e10-aec4-2fabea382eef None None] Lock nbd-allocation-lock released by _inner_get_dev :: held 0.000s inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456 2015-03-30 14:55:56.941 INFO nova.virt.disk.mount.api [req-df13155c-0340-4e10-aec4-2fabea382eef None None] Device allocation failed. Will retry in 2 seconds. 2015-03-30 14:55:58.941 DEBUG oslo_concurrency.lockutils [req-df13155c-0340-4e10-aec4-2fabea382eef None None] Lock nbd-allocation-lock acquired by _inner_get_dev :: waited 0.000s inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:444 2015-03-30 14:55:58.941 ERROR nova.virt.disk.mount.nbd [req-df13155c-0340-4e10-aec4-2fabea382eef None None] nbd module not loaded We should not even enter the loop when /sys/block/nbd0 is clearly absent. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1438260/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1202965] Re: The name VCPUs (total) of Hypervisors is confusing
This is really a UX feature addition ** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1202965 Title: The name VCPUs (total) of Hypervisors is confusing Status in OpenStack Dashboard (Horizon): Invalid Status in OpenStack Compute (Nova): Opinion Bug description: In Hypervisors panel, VCPUs(total) and VCPUs(used) fields causes confusion as used always bigger than total. Virtual CPU to Physical CPU allocation ratio is default to 16.0. put Physical CPU total to VCPUS(total) is not correct. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1202965/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1257788] Re: nova-network doesn't handle large amount of concurrent booting instances well
This has been fixed by some of the work to move nova-net to objects. ** Changed in: nova Status: Triaged = Fix Released ** Changed in: nova Assignee: (unassigned) = Dan Smith (danms) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1257788 Title: nova-network doesn't handle large amount of concurrent booting instances well Status in OpenStack Neutron (virtual network service): Invalid Status in OpenStack Compute (Nova): Fix Released Bug description: As seen by the large-ops test nova-network has trouble booting 150 VMs concurrently. Part of the problem is that setting up the network for each instance is relatively expensive with many locks and root-wrapped calls. Both of which significantly hurt performance. One possible fix would be to do more bulk network operations when possible, so instead of 100 separate calls to a root-wrapped tool call it once. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1257788/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438259] [NEW] VPNaaS: remove dependency on Neutron in UT
Public bug reported: Some of the driver unit tests mock execute() from Neutron, which creates a (brittle) dependency (as seen by a recent breakage). The dependency can be removed, but mocking higher up, from within the *Swan process _execute() method. In addition, there are some unit tests that are not active, due to missing test base class. ** Affects: neutron Importance: Undecided Assignee: Paul Michali (pcm) Status: In Progress ** Tags: vpnaas ** Changed in: neutron Assignee: (unassigned) = Paul Michali (pcm) ** Changed in: neutron Status: New = In Progress -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1438259 Title: VPNaaS: remove dependency on Neutron in UT Status in OpenStack Neutron (virtual network service): In Progress Bug description: Some of the driver unit tests mock execute() from Neutron, which creates a (brittle) dependency (as seen by a recent breakage). The dependency can be removed, but mocking higher up, from within the *Swan process _execute() method. In addition, there are some unit tests that are not active, due to missing test base class. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1438259/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438240] [NEW] nova libvirt driver does not handle resizing with 'hw:cpu_policy' flag
*** This bug is a duplicate of bug 1417667 ***
https://bugs.launchpad.net/bugs/1417667
Public bug reported:
We conduced a series of resize checks on instances with/without the CPU
pinning parameters provided as part of this blueprint:
http://specs.openstack.org/openstack/nova-specs/specs/juno/approved
/virt-driver-cpu-pinning.html
This is related to 1417667
(https://bugs.launchpad.net/nova/+bug/1417667).
---
# Testing Configuration
Testing was conducted on a single-node, Fedora 21-based
(3.17.8-300.fc21.x86_64) OpenStack instance (built with devstack). Three
flavors were created:
* no-pinning ('hw:cpu_policy=shared')
* pinning ('hw:cpu_policy=dedicated' and 'hw:numa-nodes=1')
* pinning-no-numa ('hw:cpu_policy=dedicated')
# Results
## no pinning - pinning
Doesn't work. Attempting to do this results in little difference in the
output XML.
Before:
vcpu placement='static' cpuset='10-19,30-39'10/vcpu
After:
vcpu placement='static' cpuset='0-9,20-29'10/vcpu
STEP: Create a VM with a 'no-pinning' flavor. Dump XML with virsh.
Resize and confirm to 'pinning' flavor. Dump XML with virsh.
## no pinning - pinning (without available CPUs)
Doesn't work (due to above). Pinning does not occur so there is no
reason for failure.
STEPS: Create three 10-vcpu VMs with a 'pinning' flavor. Create two
10-vcpu VMs with a 'no-pinning' flavor. Resize and confirm one of latter
to 'pinning' flavor. Ensure failure.
## pinning w/ no numa topology - pinning with numa topology
Doesn't work. There is only one (unrelated) change in the output XML:
Before:
disk type='file' device='disk'
...
backingStore type='file' index='1'
format type='raw'/
source
file='/opt/stack/data/nova/instances/_base/598e2b66ae396bb852a504ea4a7471df6b847f8b'/
backingStore/
/backingStore
After:
disk type='file' device='disk'
...
backingStore/
STEPS: Create a VM with a 'pinning-no-numa' flavor. Dump XML with virsh.
Resize and confirm to 'pinning' flavor. Dump XML with virsh.
## pinning - no pinning
Doesn't work. Attempting to do this results in no serious change in the
output XML. Here's the important elements of the XML for a resized
(pinning - no pinning) instance:
vcpu placement='static'10/vcpu
cputune
shares10240/shares
vcpupin vcpu='0' cpuset='17'/
vcpupin vcpu='1' cpuset='37'/
vcpupin vcpu='2' cpuset='18'/
vcpupin vcpu='3' cpuset='38'/
vcpupin vcpu='4' cpuset='19'/
vcpupin vcpu='5' cpuset='39'/
vcpupin vcpu='6' cpuset='33'/
vcpupin vcpu='7' cpuset='13'/
vcpupin vcpu='8' cpuset='34'/
vcpupin vcpu='9' cpuset='14'/
emulatorpin cpuset='13-14,17-19,33-34,37-39'/
/cputune
numatune
memory mode='strict' nodeset='1'/
memnode cellid='0' mode='strict' nodeset='1'/
/numatune
cpu
topology sockets='5' cores='1' threads='2'/
numa
cell id='0' cpus='0-9' memory='4194304' unit='KiB'/
/numa
/cpu
Versus the unresized, no pinning instance:
vcpu placement='static' cpuset='10-19,30-39'10/vcpu
cputune
shares10240/shares
/cputune
cpu
topology sockets='10' cores='1' threads='1'/
/cpu
** Affects: nova
Importance: Undecided
Status: New
** Tags: libvirt
** Description changed:
We conduced a series of resize checks on instances with/without the CPU
pinning parameters provided as part of this blueprint:
- http://specs.openstack.org/openstack/nova-specs/specs/juno/approved
+ http://specs.openstack.org/openstack/nova-specs/specs/juno/approved
/virt-driver-cpu-pinning.html
---
# Testing Configuration
Testing was conducted on a single-node, Fedora 21-based
(3.17.8-300.fc21.x86_64) OpenStack instance (built with devstack). Three
flavors were created:
* no-pinning ('hw:cpu_policy=shared')
* pinning ('hw:cpu_policy=dedicated' and 'hw:numa-nodes=1')
* pinning-no-numa ('hw:cpu_policy=dedicated')
# Results
## no pinning - pinning
Doesn't work. Attempting to do this results in little difference in the
output XML.
Before:
- vcpu placement='static' cpuset='10-19,30-39'10/vcpu
+ vcpu placement='static' cpuset='10-19,30-39'10/vcpu
After:
- vcpu placement='static' cpuset='0-9,20-29'10/vcpu
+ vcpu placement='static' cpuset='0-9,20-29'10/vcpu
STEP: Create a VM with a 'no-pinning' flavor. Dump XML with virsh.
Resize and confirm to 'pinning' flavor. Dump XML with virsh.
## no pinning - pinning (without available CPUs)
Doesn't work (due to above). Pinning does not occur so there is no
reason for failure.
STEPS: Create three 10-vcpu VMs with a 'pinning' flavor. Create two
10-vcpu VMs with a 'no-pinning' flavor. Resize and confirm one of latter
to 'pinning' flavor. Ensure failure.
## pinning w/ no numa topology - pinning
[Yahoo-eng-team] [Bug 1438253] [NEW] nova libvirt pinning won't work across numa nodes
Public bug reported:
Using a CPU policy of dedicated ('hw:cpu_policy=dedicated') results in
vCPUs being pinned to pCPUs, per the original blueprint:
http://specs.openstack.org/openstack/nova-
specs/specs/kilo/implemented/virt-driver-cpu-pinning.html
When combined with the 'vcpu_pin_set' nova configuration, it should be
possible to get very good control over which pCPUs are used by
instances. This works as expected when creating a multi-vCPU instance
pinned to pCPUs on a single NUMA node. However, if these pCPUs are
spread across multiple NUMA nodes then it fails. This behavior is not
documented in the original blueprint and therefore should not function
like so.
---
# Testing Configuration
Testing was conducted on a single-node, Fedora 21-based
(3.17.8-300.fc21.x86_64) OpenStack instance (built with devstack). Two
flavors were used:
openstack flavor create --ram 4096 --disk 20 --vcpus 10 demo.no-
pinning
openstack flavor create --ram 4096 --disk 20 --vcpus 10 demo.pinning
nova flavor-key demo.pinning set hw:cpu_policy=dedicated
hw:cpu_threads_policy=separate
# Results
Passing (same NUMA socket, contiguous)
==
vcpu_pin_set = 10-19
Pinned
--
$ openstack server create --image Fedora-x86_64-20-20140618-sda
--flavor demo.pinning test1 --wait
$ ps aux | grep qemu | awk '{print }' | head -1 | xargs taskset -pc
pid 68150's current affinity list: 10-19
$ sudo -E virsh vcpucount 13
maximum config10
maximum live 10
current config10
current live 10
Unpinned
$ openstack server create --image Fedora-x86_64-20-20140618-sda
--flavor demo.no-pinning test1 --wait
$ ps aux | grep qemu | awk '{print }' | head -1 | xargs taskset -pc
pid 70249's current affinity list: 10-19
Passing (same NUMA socket, non-contiguous)
==
vcpu_pin_set = 5-9,25-29
Pinned
--
$ openstack server create --image Fedora-x86_64-20-20140618-sda
--flavor demo.pinning test1 --wait
$ ps aux | grep qemu | awk '{print }' | head -1 | xargs taskset -pc
pid 91186's current affinity list: 5-9,25-29
Unpinned
$ openstack server create --image Fedora-x86_64-20-20140618-sda
--flavor demo.no-pinning test1 --wait
$ ps aux | grep qemu | awk '{print }' | head -1 | xargs taskset -pc
pid 92212's current affinity list: 5-9,25-29
Passing (multiple NUMA sockets, non-contiguous)
===
vcpu_pin_set = 5-10,25-29
Pinned
--
$ openstack server create --image Fedora-x86_64-20-20140618-sda
--flavor demo.pinning test1 --wait
$ ps aux | grep qemu | awk '{print }' | head -1 | xargs taskset -pc
pid 93884's current affinity list: 5-9,25-29
Unpinned
$ openstack server create --image Fedora-x86_64-20-20140618-sda
--flavor demo.no-pinning test1 --wait
$ ps aux | grep qemu | awk '{print $2}' | head -1 | xargs taskset -pc
pid 94468's current affinity list: 5-9,25-29
Additional combinations
---
vcpu_pin_set = 5-14,25-29
Failing (different NUMA socket, contiguous)
===
vcpu_pin_set = 9-18
Pinned
--
$ openstack server create --image Fedora-x86_64-20-20140618-sda
--flavor demo.pinning test1 --wait
ERROR: openstackclient.compute.v2.server.CreateServer Error creating
server: test1
Unpinned
$ openstack server create --image Fedora-x86_64-20-20140618-sda
--flavor demo.no-pinning test1 --wait
$ ps aux | grep qemu | awk '{print }' | head -1 | xargs taskset -pc
pid 71194's current affinity list: 9-18
Failing (different NUMA socket, non-contiguous)
===
vcpu_pin_set = 5-9,15-19
Pinned
--
$ openstack server create --image Fedora-x86_64-20-20140618-sda
--flavor demo.no-pinning test1 --wait
ERROR: openstackclient.compute.v2.server.CreateServer Error creating
server: test1
$ openstack server show test1 | grep fault
| fault| {u'message': u'No valid host
was found. There are not enough hosts available.', u'code': 500,
Unpinned
$ openstack server create --image Fedora-x86_64-20-20140618-sda
--flavor demo.no-pinning test1 --wait
$ ps aux | grep qemu | awk '{print }' | head -1 | xargs taskset -pc
pid 88703's current affinity list: 5-9,15-19
** Affects: nova
Importance: Undecided
Status: New
** Tags: libvirt
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1438253
Title:
nova libvirt pinning won't work across numa
[Yahoo-eng-team] [Bug 1438321] [NEW] Fix process management for neutron-server
Public bug reported: The following commit to oslo-incubator [1], that was supposed to optimize waiting for children processes to exit, will break neutron-server behavior (i.e. signal handling). 1. In neutron-server eventlet monkey-patching (including patching os module) is done in parent process. That is why os.waitpid(0, 0) in _wait_child method also gets monkey-patched and eventlet goes crazy. Connecting to parent process with strace shows that os.waitpid(0, os.WNOHANG) is called, yet it is difficult to say what is really happening because the process does not react on termination signals (SIGTERM, SIGHUP, SIGINT). 2. Due to the fact that neutron-server initializes two instances of ProcessLauncher in one parent process, calling eventlet.greenthread.sleep(self.wait_interval) seems to be the only way for the process to switch contexts and allow another instance of ProcessLauncher to call _wait_child. It is important to mention that ProcessLauncher is not supposed to be used in this way (2 instances in one parent process) at all. This bug is intended to track fixing the outlined problems on Neutron side. [1] https://github.com/openstack/oslo- incubator/commit/bf92010cc9d4c2876eaf6092713aafa94dcc8b35 ** Affects: neutron Importance: Undecided Assignee: Elena Ezhova (eezhova) Status: New ** Changed in: neutron Assignee: (unassigned) = Elena Ezhova (eezhova) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1438321 Title: Fix process management for neutron-server Status in OpenStack Neutron (virtual network service): New Bug description: The following commit to oslo-incubator [1], that was supposed to optimize waiting for children processes to exit, will break neutron-server behavior (i.e. signal handling). 1. In neutron-server eventlet monkey-patching (including patching os module) is done in parent process. That is why os.waitpid(0, 0) in _wait_child method also gets monkey-patched and eventlet goes crazy. Connecting to parent process with strace shows that os.waitpid(0, os.WNOHANG) is called, yet it is difficult to say what is really happening because the process does not react on termination signals (SIGTERM, SIGHUP, SIGINT). 2. Due to the fact that neutron-server initializes two instances of ProcessLauncher in one parent process, calling eventlet.greenthread.sleep(self.wait_interval) seems to be the only way for the process to switch contexts and allow another instance of ProcessLauncher to call _wait_child. It is important to mention that ProcessLauncher is not supposed to be used in this way (2 instances in one parent process) at all. This bug is intended to track fixing the outlined problems on Neutron side. [1] https://github.com/openstack/oslo- incubator/commit/bf92010cc9d4c2876eaf6092713aafa94dcc8b35 To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1438321/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438320] [NEW] Subnet pool created should be blocked when allow_overlapping_ips=False
Public bug reported: Creation of subnet pools should be blocked when allow_overlapping_ips=False. This conflicts with the notion of subnet pools and causes allocation of overlapping prefixes to be blocked, even when allocating across different pools. The simplest solution is to declare subnet pools incompatible with allow_overlapping_ips=False and block creation of subnet pools. ** Affects: neutron Importance: Undecided Assignee: Ryan Tidwell (ryan-tidwell) Status: New ** Changed in: neutron Assignee: (unassigned) = Ryan Tidwell (ryan-tidwell) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1438320 Title: Subnet pool created should be blocked when allow_overlapping_ips=False Status in OpenStack Neutron (virtual network service): New Bug description: Creation of subnet pools should be blocked when allow_overlapping_ips=False. This conflicts with the notion of subnet pools and causes allocation of overlapping prefixes to be blocked, even when allocating across different pools. The simplest solution is to declare subnet pools incompatible with allow_overlapping_ips=False and block creation of subnet pools. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1438320/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438325] [NEW] Access and Security tab slowness
Public bug reported: Access and Security tab became very unresponsive and may time out on environments with more that 3k VMs. Deep analisys shown that it because of Floating IPs tab even if there is no floating IPs assigned :). It looks like code on this tab is suboptimal and trying to get informaton about all VMs/ports to use thoose further for assignment/associacion procedure. Anyway with more that 3k VMs Access and Security page becae very slow and horizon starts to throw timeouts very often. I've checked that behavior on Havana and it looks like code didn't changed a lot since then so filing it for master. ** Affects: horizon Importance: Undecided Status: New ** Affects: mos Importance: Undecided Status: New ** Also affects: mos Importance: Undecided Status: New ** Description changed: - Access and Security tab became very unresponcive and may time out on environments with more that 3k VMs. - Deep analisys shown that it because of Floating IPs tab even if there is no floating IPs assigned :). + Access and Security tab became very unresponsive and may time out on environments with more that 3k VMs. + Deep analisys shown that it because of Floating IPs tab even if there is no floating IPs assigned :). It looks like code on this tab is suboptimal and trying to get informaton about all VMs/ports to use thoose further for assignment/associacion procedure. Anyway with more that 3k VMs Access and Security page becae very slow and horizon starts to throw timeouts very often. I've checked that behavior on Havana and it looks like code didn't changed a lot since then so filing it for master. -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1438325 Title: Access and Security tab slowness Status in OpenStack Dashboard (Horizon): New Status in Mirantis OpenStack: New Bug description: Access and Security tab became very unresponsive and may time out on environments with more that 3k VMs. Deep analisys shown that it because of Floating IPs tab even if there is no floating IPs assigned :). It looks like code on this tab is suboptimal and trying to get informaton about all VMs/ports to use thoose further for assignment/associacion procedure. Anyway with more that 3k VMs Access and Security page becae very slow and horizon starts to throw timeouts very often. I've checked that behavior on Havana and it looks like code didn't changed a lot since then so filing it for master. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1438325/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438331] [NEW] Nova fails to delete rbd image, puts guest in to ERROR state
Public bug reported:
When removing guests that have been booted on Ceph, Nova will
occasionally put guests in to ERROR state with the following ...
Reported to the controller:
| fault| {message: error removing image,
code: 500, details: File
\/usr/lib/python2.7/site-packages/nova/compute/manager.py\, line 314, in
decorated_function |
| | return function(self, context,
*args, **kwargs)
|
| | File
\/usr/lib/python2.7/site-packages/nova/compute/manager.py\, line 2525, in
terminate_instance |
| | do_terminate_instance(instance,
bdms)
|
| | File
\/usr/lib/python2.7/site-packages/nova/openstack/common/lockutils.py\, line
272, in inner|
| | return f(*args, **kwargs)
|
| | File
\/usr/lib/python2.7/site-packages/nova/compute/manager.py\, line 2523, in
do_terminate_instance |
| |
self._set_instance_error_state(context, instance)
|
| | File
\/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py\, line
82, in __exit__ |
| | six.reraise(self.type_,
self.value, self.tb)
|
| | File
\/usr/lib/python2.7/site-packages/nova/compute/manager.py\, line 2513, in
do_terminate_instance |
| | self._delete_instance(context,
instance, bdms, quotas)
|
| | File
\/usr/lib/python2.7/site-packages/nova/hooks.py\, line 131, in inner
|
| | rv = f(*args, **kwargs)
|
| | File
\/usr/lib/python2.7/site-packages/nova/compute/manager.py\, line 2482, in
_delete_instance |
| | quotas.rollback()
|
| | File
\/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py\, line
82, in __exit__ |
| | six.reraise(self.type_,
self.value, self.tb)
|
| | File
\/usr/lib/python2.7/site-packages/nova/compute/manager.py\, line 2459, in
_delete_instance |
| | self._shutdown_instance(context,
instance, bdms)
|
| | File
\/usr/lib/python2.7/site-packages/nova/compute/manager.py\, line 2389, in
_shutdown_instance |
| | requested_networks)
|
| | File
\/usr/lib/python2.7/site-packages/nova/openstack/common/excutils.py\, line
82, in __exit__ |
| | six.reraise(self.type_,
self.value,
[Yahoo-eng-team] [Bug 1438338] [NEW] servers api should return security group ids instead of names
Public bug reported:
Creating this from https://bugs.launchpad.net/python-
novaclient/+bug/1394462
In nova-network security group names can't be duplicated, but in
neutron, they can. For this reason, it would be nice to return server
security groups as ids instead of names.
Here's is a sample request and response showing the current state:
security_groups: [{name: default}],
DEBUG (connectionpool:415) GET
/v2/038c717809174199a297f4ef774e6852/servers/d2b729b8-a626-4050-a756-d5a450c99811
HTTP/1.1 200 1757
DEBUG (session:223) RESP: [200] date: Mon, 30 Mar 2015 17:57:50 GMT
content-length: 1757 content-type: application/json x-compute-request-id:
req-d6c33e18-cf62-4848-88ff-e57b64bd55e3
RESP BODY: {server: {status: ACTIVE, updated: 2015-03-25T19:04:47Z,
hostId: cfca2250a844c76f4dd5ba369b2550ad3fb07d545e58e395b2271486,
OS-EXT-SRV-ATTR:host: vagrant-ubuntu-trusty-64.localdomain, addresses:
{private: [{OS-EXT-IPS-MAC:mac_addr: fa:16:3e:0d:0c:b3, version: 4,
addr: 10.0.0.2, OS-EXT-IPS:type: fixed}]}, links: [{href:
http://10.0.2.15:8774/v2/038c717809174199a297f4ef774e6852/servers/d2b729b8-a626-4050-a756-d5a450c99811;,
rel: self}, {href:
http://10.0.2.15:8774/038c717809174199a297f4ef774e6852/servers/d2b729b8-a626-4050-a756-d5a450c99811;,
rel: bookmark}], key_name: null, image: {id:
d0ddfda2-dbdc-48ae-b65e-27ca407d32ce, links: [{href:
http://10.0.2.15:8774/038c717809174199a297f4ef774e6852/images/d0ddfda2-dbdc-48ae-b65e-27ca407d32ce;,
rel: bookmark}]}, OS-EXT-STS:task_state: null, OS-EXT-STS:vm_state:
active, OS-EXT-SRV-ATTR:instance_name: instance-0001,
OS-SRV-USG:launched_at: 2015-03-25
T19:04:47.00, OS-EXT-SRV-ATTR:hypervisor_hostname:
vagrant-ubuntu-trusty-64.localdomain, flavor: {id: 42, links:
[{href: http://10.0.2.15:8774/038c717809174199a297f4ef774e6852/flavors/42;,
rel: bookmark}]}, id: d2b729b8-a626-4050-a756-d5a450c99811,
security_groups: [{name: default}], OS-SRV-USG:terminated_at: null,
OS-EXT-AZ:availability_zone: nova, user_id:
0771ff29994b428fa15dfb4ec1b6bc7d, name:
ServerActionsTestJSON-instance-367700261, created: 2015-03-25T19:04:35Z,
tenant_id: a0c8d64c558b42d5a7b32a229c9f9a3e, OS-DCF:diskConfig: MANUAL,
os-extended-volumes:volumes_attached: [], accessIPv4: , accessIPv6: ,
progress: 0, OS-EXT-STS:power_state: 1, config_drive: True, metadata:
{}}}
** Affects: nova
Importance: Wishlist
Status: Confirmed
** Tags: api
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1438338
Title:
servers api should return security group ids instead of names
Status in OpenStack Compute (Nova):
Confirmed
Bug description:
Creating this from https://bugs.launchpad.net/python-
novaclient/+bug/1394462
In nova-network security group names can't be duplicated, but in
neutron, they can. For this reason, it would be nice to return server
security groups as ids instead of names.
Here's is a sample request and response showing the current state:
security_groups: [{name: default}],
DEBUG (connectionpool:415) GET
/v2/038c717809174199a297f4ef774e6852/servers/d2b729b8-a626-4050-a756-d5a450c99811
HTTP/1.1 200 1757
DEBUG (session:223) RESP: [200] date: Mon, 30 Mar 2015 17:57:50 GMT
content-length: 1757 content-type: application/json x-compute-request-id:
req-d6c33e18-cf62-4848-88ff-e57b64bd55e3
RESP BODY: {server: {status: ACTIVE, updated: 2015-03-25T19:04:47Z,
hostId: cfca2250a844c76f4dd5ba369b2550ad3fb07d545e58e395b2271486,
OS-EXT-SRV-ATTR:host: vagrant-ubuntu-trusty-64.localdomain, addresses:
{private: [{OS-EXT-IPS-MAC:mac_addr: fa:16:3e:0d:0c:b3, version: 4,
addr: 10.0.0.2, OS-EXT-IPS:type: fixed}]}, links: [{href:
http://10.0.2.15:8774/v2/038c717809174199a297f4ef774e6852/servers/d2b729b8-a626-4050-a756-d5a450c99811;,
rel: self}, {href:
http://10.0.2.15:8774/038c717809174199a297f4ef774e6852/servers/d2b729b8-a626-4050-a756-d5a450c99811;,
rel: bookmark}], key_name: null, image: {id:
d0ddfda2-dbdc-48ae-b65e-27ca407d32ce, links: [{href:
http://10.0.2.15:8774/038c717809174199a297f4ef774e6852/images/d0ddfda2-dbdc-48ae-b65e-27ca407d32ce;,
rel: bookmark}]}, OS-EXT-STS:task_state: null, OS-EXT-STS:vm_state:
active, OS-EXT-SRV-ATTR:instance_name: instance-0001,
OS-SRV-USG:launched_at: 2015-03-
25T19:04:47.00, OS-EXT-SRV-ATTR:hypervisor_hostname:
vagrant-ubuntu-trusty-64.localdomain, flavor: {id: 42, links:
[{href: http://10.0.2.15:8774/038c717809174199a297f4ef774e6852/flavors/42;,
rel: bookmark}]}, id: d2b729b8-a626-4050-a756-d5a450c99811,
security_groups: [{name: default}], OS-SRV-USG:terminated_at: null,
OS-EXT-AZ:availability_zone: nova, user_id:
0771ff29994b428fa15dfb4ec1b6bc7d, name:
ServerActionsTestJSON-instance-367700261, created: 2015-03-25T19:04:35Z,
tenant_id: a0c8d64c558b42d5a7b32a229c9f9a3e, OS-DCF:diskConfig: MANUAL,
os-extended-volumes:volumes_attached: [], accessIPv4: ,
[Yahoo-eng-team] [Bug 1437516] Re: Instances with delete_on_terminate volumes do not reschedule
*** This bug is a duplicate of bug 1336127 *** https://bugs.launchpad.net/bugs/1336127 Thanks andrew, marked this as a duplicate ** This bug has been marked a duplicate of bug 1336127 The volumes will be deleted when creating a virtual machine fails with the parameter delete_on_termination being set true, which causes that the rescheduling fails -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1437516 Title: Instances with delete_on_terminate volumes do not reschedule Status in OpenStack Compute (Nova): Triaged Bug description: If you create an instance with attached delete_on_terminate volumes and it fails to launch on a host (due to a hardware or hypervisor issue) it will be rescheduled but the volumes will have already been deleted. The volumes should only be deleted if the instance is not going to be rescheduled (or perhaps not at all). To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1437516/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438159] Re: Made neutron agents silent by using AMQP
This looks like deep rework of current messaging strategy and probably should be worked on within a scope of a blueprint rather than a bug. So I would suggest to file a bp for this and provide a spec explaining these ideas so spec review could be a better place to discuss these ideas. ** Changed in: neutron Status: New = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1438159 Title: Made neutron agents silent by using AMQP Status in OpenStack Neutron (virtual network service): Opinion Bug description: Problem.: Neutron agents does a lot of periodic task which leads an rpc call + database transaction, which does not even provide a new information, because nothing changed. This behaviour in scale can be called as `DDOS attack`, generally this kind of architecture is bad at scaling and can be considered as an any-pattern. Instead of periodic poll, we can leverage the AMQP brokers bind capabilities. Neutron has many situation like security group rule change or dvr related changes which needs to be communicated with multiple agents, but usually not with all agent. The agent at startup needs to synchronise the as usual, but during the sync the agent can subscribe to the interesting events to avoid the periodic tasks. (Note.: After the first subscribe loop a second one is needed to do not miss changes during the subscribe process ). The AMQP queues with 'auto-delete' can be considered as a reliable source of information which does not miss any event notification. On connection loss or full broker cluster die the agent needs to re sync everything guarded in this way, in these cases, the queue will disappear so the situation easily detectable. 1. Create a Direct exchange for all kind of resourcestype what needs to be synchronised in this way, for example.: 'neutron.securitygroups' . The exchange declaration needs to happen at q-svc start-up time or at full broker cluster die (not-found exception will tell it). The exchange SHOULD NOT be redeclared or verified at every message publish. 2. Every agent creates a dedicated per agent queue with auto-delete flag, if the agent already maintains a queue with this property he MAY reuse that one. The agents SHOULD avoid to creating multiple queues per resource type. The messages MUST contain a type information. 3. All agent creates a binding between his queue and the resource type queue with he realise he depends on the resource, for example it maintains at least one port with the given security-group. (The agents needs to remove the binding. when they stop using it) 4. The q-svc publishes just a single message when the resource related change happened. The routing key is the uuid. Alternatively a topic exchange can be used, with a single exchange. In this case the routing keys MUST contains the resource type like: neutron.resource_type.uuid , this type exchange is generally more expensive than a direct exchange (pattern matching), and only useful if you have agents which needs to listens to ALL event related to a type, but others just interested just in a few of them. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1438159/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438329] [NEW] Example configuration files lack changes for Kilo
Public bug reported: The example configuration files in the source repository lack the changes in the list of new, updated, and deprecated options in the Kilo configuration reference guide [1]. For example, the example neutron.conf file [2] lacks the [nova] and [oslo_messaging_*] sections and options. [1] http://docs.openstack.org/trunk/config-reference/content/neutron-conf-changes-kilo.html [2] https://github.com/openstack/neutron/blob/master/etc/neutron.conf ** Affects: neutron Importance: Undecided Status: New ** Description changed: The example configuration files in the source repository lack the changes in the list of new, updated, and deprecated options in the Kilo - configuration reference guide [2]. For example, the example neutron.conf - file [1] lacks the [nova] and [oslo_messaging_*] sections and options. + configuration reference guide [1]. For example, the example neutron.conf + file [2] lacks the [nova] and [oslo_messaging_*] sections and options. - [1] https://github.com/openstack/neutron/blob/master/etc/neutron.conf - [2] http://docs.openstack.org/trunk/config-reference/content/neutron-conf-changes-kilo.html + [1] http://docs.openstack.org/trunk/config-reference/content/neutron-conf-changes-kilo.html + [2] https://github.com/openstack/neutron/blob/master/etc/neutron.conf -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1438329 Title: Example configuration files lack changes for Kilo Status in OpenStack Neutron (virtual network service): New Bug description: The example configuration files in the source repository lack the changes in the list of new, updated, and deprecated options in the Kilo configuration reference guide [1]. For example, the example neutron.conf file [2] lacks the [nova] and [oslo_messaging_*] sections and options. [1] http://docs.openstack.org/trunk/config-reference/content/neutron-conf-changes-kilo.html [2] https://github.com/openstack/neutron/blob/master/etc/neutron.conf To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1438329/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1231226] Re: nova show shows wrong flavor name
With the moving of instance_type information into the instance metadata this should be addressed moving forward. There will be some legacy instances that don't yet have this, but moving forward it's fixed. ** Changed in: nova Status: Triaged = Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1231226 Title: nova show shows wrong flavor name Status in OpenStack Compute (Nova): Fix Released Bug description: We created a flavor with flavor id 1, then deleted the flavor and created a flavor with the same id. OK, then I create a instance with flavor 1, instance's resource is ok, but nova show shows the deleted flavor name. we can fix this by 3 ways: 1. reject create a flavor with flavor id which is same as one of deleted ones. 2. flavor show should not read deleted flavors. 3. actually, system meta has flavor name, so nova can return flavor name to client, and client use it directly. I am not sure which way will be accepted. Any one has idea on this? To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1231226/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438316] [NEW] Glance does not notify systemd when the services are up.
Public bug reported: Unlike keystone, glance does not notify systemd when the api|registry|scrubber services are up. root@controller:~# time systemctl start glance-api Job for glance-api.service failed. See systemctl status glance-api.service and journalctl -xe for details. real1m30.310s user0m0.003s sys 0m0.002s The service actually starts but its status is not visible to systemd. root@controller:~# systemctl status glance-api ● glance-api.service - OpenStack Image API Service Loaded: loaded (/usr/lib/systemd/system/glance-api.service; enabled; vendor preset: disabled) Active: activating (start) since Mon 2015-03-30 17:02:33 UTC; 41s ago Main PID: 810 (glance-api) CGroup: /system.slice/glance-api.service ├─810 /usr/bin/python2 /usr/bin/glance-api ├─829 /usr/bin/python2 /usr/bin/glance-api └─830 /usr/bin/python2 /usr/bin/glance-api Mar 30 17:02:33 controller systemd[1]: Starting OpenStack Image API Service... root@controller:~# echo $? 3 root@controller:~# ps aux |grep glance glance 638 4.2 1.9 196948 78876 ?Rs 16:51 0:01 /usr/bin/python2 /usr/bin/glance-api Attached is a patch that resolves this issue, tested against 2015.1.0b3 ** Affects: glance Importance: Undecided Status: New ** Tags: systemd ** Patch added: 0001-Enable-systemd-notification.patch https://bugs.launchpad.net/bugs/1438316/+attachment/4361213/+files/0001-Enable-systemd-notification.patch -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Glance. https://bugs.launchpad.net/bugs/1438316 Title: Glance does not notify systemd when the services are up. Status in OpenStack Image Registry and Delivery Service (Glance): New Bug description: Unlike keystone, glance does not notify systemd when the api|registry|scrubber services are up. root@controller:~# time systemctl start glance-api Job for glance-api.service failed. See systemctl status glance-api.service and journalctl -xe for details. real1m30.310s user0m0.003s sys 0m0.002s The service actually starts but its status is not visible to systemd. root@controller:~# systemctl status glance-api ● glance-api.service - OpenStack Image API Service Loaded: loaded (/usr/lib/systemd/system/glance-api.service; enabled; vendor preset: disabled) Active: activating (start) since Mon 2015-03-30 17:02:33 UTC; 41s ago Main PID: 810 (glance-api) CGroup: /system.slice/glance-api.service ├─810 /usr/bin/python2 /usr/bin/glance-api ├─829 /usr/bin/python2 /usr/bin/glance-api └─830 /usr/bin/python2 /usr/bin/glance-api Mar 30 17:02:33 controller systemd[1]: Starting OpenStack Image API Service... root@controller:~# echo $? 3 root@controller:~# ps aux |grep glance glance 638 4.2 1.9 196948 78876 ?Rs 16:51 0:01 /usr/bin/python2 /usr/bin/glance-api Attached is a patch that resolves this issue, tested against 2015.1.0b3 To manage notifications about this bug go to: https://bugs.launchpad.net/glance/+bug/1438316/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438260] Re: Expensive Device allocation loop when NBD is absent
Nova is being asked to do a thing, which it can't, which is put files in the guest. This retry is actually kind of important. The bug really is exposing the fact that *no* working file injection mechanism exists in these rally jobs, but they are asking guests to do something. ** Changed in: nova Status: In Progress = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1438260 Title: Expensive Device allocation loop when NBD is absent Status in OpenStack Compute (Nova): Invalid Bug description: See log stash query: Device allocation failed. Will retry in 2 seconds. Example: http://logs.openstack.org/10/163910/6/check/gate-rally-dsvm-cinder/46d89ab/logs/screen-n-cpu.txt.gz#_2015-03-30_14_55_56_941 Logs: 2015-03-30 14:55:56.940 ERROR nova.virt.disk.mount.nbd [req-df13155c-0340-4e10-aec4-2fabea382eef None None] nbd module not loaded 2015-03-30 14:55:56.940 DEBUG oslo_concurrency.lockutils [req-df13155c-0340-4e10-aec4-2fabea382eef None None] Lock nbd-allocation-lock released by _inner_get_dev :: held 0.000s inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:456 2015-03-30 14:55:56.941 INFO nova.virt.disk.mount.api [req-df13155c-0340-4e10-aec4-2fabea382eef None None] Device allocation failed. Will retry in 2 seconds. 2015-03-30 14:55:58.941 DEBUG oslo_concurrency.lockutils [req-df13155c-0340-4e10-aec4-2fabea382eef None None] Lock nbd-allocation-lock acquired by _inner_get_dev :: waited 0.000s inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:444 2015-03-30 14:55:58.941 ERROR nova.virt.disk.mount.nbd [req-df13155c-0340-4e10-aec4-2fabea382eef None None] nbd module not loaded We should not even enter the loop when /sys/block/nbd0 is clearly absent. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1438260/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1418635] Re: Neutron API reference is inconsistent and differs from implementation
** Also affects: openstack-api-site
Importance: Undecided
Status: New
** No longer affects: neutron
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1418635
Title:
Neutron API reference is inconsistent and differs from implementation
Status in OpenStack API documentation site:
New
Bug description:
I'm implementing modules for SaltStack (not yet on GitHub) to create entities
in OpenStack Neutron and came across quite a few problems
with the API documentation:
* Link Filtering and Column Selection to
http://docs.openstack.org/api/openstack-network/2.0/content/filtering.html
404s
* Section http://developer.openstack.org/api-ref-
networking-v2.html#subnets
* POST /v2.0/subnets aka Create subnet
- 'id' is listed to be an optional parameter but the Neutron-API in
Icehouse refuses to set a user-selected ID
- parameters 'dns_nameservers' and 'host_routes' missing from
documentation (undocumented extensions?)
* GET /v2.0/subnets aka List subnets
- can't filter by allocation_pools or enable_dhcp
* PUT /v2.0/subnets/{subnet_id} aka Update subnet
- parameters allocation_pools, network_id, tenant_id, id listed
as optional request parameters but Neutron-API in Icehouse returns
Cannot update read-only attribute $PARAMETER
* Section http://developer.openstack.org/api-ref-
networking-v2.html#networks
* GET /v2.0/networks aka List networks
- parameter shared is ignored as a filter
* Section http://developer.openstack.org/api-ref-
networking-v2.html#layer3
* POST /v2.0/routers aka Create router
- Description states router:external and external_gateway_info are
valid request parameters but they're not listed in the table of
request parameters
- What's the parameter router described as A router object. supposed
to be? A router object in JSON/XML notation inside a router object in
JSON/XML notation?
I'll probably add more when implementing functions for managing
routers in Neutron.
To manage notifications about this bug go to:
https://bugs.launchpad.net/openstack-api-site/+bug/1418635/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1292999] Re: Agents APIv2 do not handle conflict errors
** Changed in: nova Status: Triaged = Incomplete ** Changed in: nova Status: Incomplete = Opinion ** Changed in: nova Importance: Low = Wishlist ** Changed in: nova Assignee: Leandro Ignacio Costantino (leandro-i-costantino) = (unassigned) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1292999 Title: Agents APIv2 do not handle conflict errors Status in OpenStack Compute (Nova): Opinion Bug description: When trying to create an agent with duplicate name, a generic exception is raised instead of HttpConflict. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1292999/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1274732] Re: Instances table doesn't get cleaned up
** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1274732 Title: Instances table doesn't get cleaned up Status in OpenStack Compute (Nova): Opinion Bug description: It seems like nova doesn't do any cleanup of the instances table. If you spin up and delete lots of VMs, this table eventually gets gigantic. Manually it can be cleaned using: `nova-manage db archive_deleted_rows 1` but an auto-cleanup of stale entries would be useful too. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1274732/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1435655] Re: Can't manually assign a distributed router to a l3 agent
case 1: I think that creating a DVR router connecting Subnets that reach VM's whose compute hosts are not DVR aware should lead to an error case 2: I think that moving agents willy nilly should lead can't be expected to work. Moving DVR routers from one agent to another has always been a bit murky, but I believe that you can only move the SNAT part; the DVR elements will have to stick to the hosts on which VM's connected through the routers are. ** Changed in: neutron Status: In Progress = Invalid ** Changed in: neutron Status: Invalid = Incomplete -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1435655 Title: Can't manually assign a distributed router to a l3 agent Status in OpenStack Neutron (virtual network service): Incomplete Bug description: Now neutron does not allow to manually assign a distributed router to a l3 agent which is in 'dvr' mode, but in bellow use case, it does not work ok: 1 case: (1)there are two computeA, B nodes which l3 agent are in legacy mode, l2 agent 'enable_distributed_routing = False' (2)create a 'dvr' router, then add subnetA to this 'dvr' router (3)create VMs with subnetA in computeA or B (4)modify 'agent_mode=dvr', 'enable_distributed_routing = True' in computeA (5)the VMs in computeA can't communicate with their gateway 2 case: (1)there is a computeA, it's 'agent_mode=dvr', 'enable_distributed_routing = True' (2)create a 'dvr' router, then add subnetA to this 'dvr' router (3)create VMs with subnetA in computeA (4)use 'l3-agent-router-remove' remove l3 agent which in computeA from 'dvr' router (5)the VMs in computeA can't communicate with their gateway, and can't manually assign it's l3 agent to dvr router To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1435655/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438325] Re: Access and Security tab slowness
** Tags added: horizon ** No longer affects: horizon -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1438325 Title: Access and Security tab slowness Status in Mirantis OpenStack: New Bug description: Access and Security tab became very unresponsive and may time out on environments with more that 3k VMs. Deep analisys shown that it because of Floating IPs tab even if there is no floating IPs assigned :). It looks like code on this tab is suboptimal and trying to get informaton about all VMs/ports to use thoose further for assignment/associacion procedure. Anyway with more that 3k VMs Access and Security page becae very slow and horizon starts to throw timeouts very often. I've checked that behavior on Havana and it looks like code didn't changed a lot since then so filing it for master. To manage notifications about this bug go to: https://bugs.launchpad.net/mos/+bug/1438325/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1089343] Re: Resize confirm should raise NotFound instead of BadRequest when confirming an invalid migration
** Changed in: nova Status: Triaged = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1089343 Title: Resize confirm should raise NotFound instead of BadRequest when confirming an invalid migration Status in OpenStack Compute (Nova): Opinion Bug description: I believe that when the user tries to confirm an invalid migration NotFound (cause the migration is not found) should be raised instead of badRequest Happens in: https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L1002 If this bug is confirmed it should be merged only in API-V3 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1089343/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1248022] Re: Nova scheduler not updated immediately when a baremetal node is added or removed
This is a much bigger architecture issue to address. ** Changed in: nova Status: Triaged = Invalid ** Changed in: nova Status: Invalid = Opinion -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1248022 Title: Nova scheduler not updated immediately when a baremetal node is added or removed Status in OpenStack Bare Metal Provisioning Service (Ironic): Triaged Status in OpenStack Compute (Nova): Opinion Bug description: With the Ironic driver, if a baremetal node is added/deleted, it is not removed from pool of available resources until the next run of update_available_resource(). During this window, the scheduler may continue to attempt to schedule instances on it (when deleted), or report NoValidHosts (when added) leading to unnecessary failures and scheduling retries. In compute manager, the update_available_resource() periodic task is responsible for updating the scheduler's knowledge of baremetal nodes: @periodic_task.periodic_task def update_available_resource(self, context): ... nodenames = set(self.driver.get_available_nodes()) for nodename in nodenames: rt = self._get_resource_tracker(nodename) rt.update_available_resource(context) update_available_resource() is also called at service startup This means that you have to wait up to 60 seconds for a node to become available/no longer available. To manage notifications about this bug go to: https://bugs.launchpad.net/ironic/+bug/1248022/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438315] [NEW] add_instance_fault_from_exc expects object but isn't always passed one
Public bug reported: 2015-03-30 15:02:58.287 28424 DEBUG oslo_messaging._drivers.amqp [-] UNIQUE_ID is 9c2e286750e7407cb78e273becca4ab0. _add_unique_id /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqp.py:224 Traceback (most recent call last): File /usr/local/lib/python2.7/dist-packages/eventlet/hubs/poll.py, line 115, in wait listener.cb(fileno) File /opt/stack/nova/nova/utils.py, line 888, in wrapper return func(*args, **kwargs) File /opt/stack/nova/nova/conductor/manager.py, line 688, in build_instances exc, request_spec) File /opt/stack/nova/nova/conductor/manager.py, line 594, in _set_vm_state_and_notify ex, request_spec, self.db) File /opt/stack/nova/nova/scheduler/utils.py, line 107, in set_vm_state_and_notify new_ref, ex, sys.exc_info()) File /opt/stack/nova/nova/compute/utils.py, line 87, in add_instance_fault_from_exc fault_obj.instance_uuid = instance.uuid AttributeError: 'dict' object has no attribute 'uuid' ** Affects: nova Importance: High Status: New ** Changed in: nova Importance: Undecided = High -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1438315 Title: add_instance_fault_from_exc expects object but isn't always passed one Status in OpenStack Compute (Nova): New Bug description: 2015-03-30 15:02:58.287 28424 DEBUG oslo_messaging._drivers.amqp [-] UNIQUE_ID is 9c2e286750e7407cb78e273becca4ab0. _add_unique_id /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqp.py:224 Traceback (most recent call last): File /usr/local/lib/python2.7/dist-packages/eventlet/hubs/poll.py, line 115, in wait listener.cb(fileno) File /opt/stack/nova/nova/utils.py, line 888, in wrapper return func(*args, **kwargs) File /opt/stack/nova/nova/conductor/manager.py, line 688, in build_instances exc, request_spec) File /opt/stack/nova/nova/conductor/manager.py, line 594, in _set_vm_state_and_notify ex, request_spec, self.db) File /opt/stack/nova/nova/scheduler/utils.py, line 107, in set_vm_state_and_notify new_ref, ex, sys.exc_info()) File /opt/stack/nova/nova/compute/utils.py, line 87, in add_instance_fault_from_exc fault_obj.instance_uuid = instance.uuid AttributeError: 'dict' object has no attribute 'uuid' To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1438315/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1425639] Re: A functional floating ip stops working sometimes if another floating ip is created and deleted.
The code structure has changed significantly and haven't heard of any other citing of this specific issue for a while so marking it as invalid for now. ** Changed in: neutron Status: In Progress = Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1425639 Title: A functional floating ip stops working sometimes if another floating ip is created and deleted. Status in OpenStack Neutron (virtual network service): Invalid Bug description: A test failure was noticed for test_cross_tenant_traffic at [1] . A floating ip ‘a’ was instantiated on router A . The floating IP was functional A floating IP ‘b’ was instantiated on router B after about 30 seconds Floating IP on router B was functional Floating IP on router B was deleted Floating IP on A was not reachable anymore Root cause appears to be: Analyzing from the logs it appears Agent gateway port was being created at instantiation of both the floating IPs and deleted on deletion of Floating IP ‘b’. [2] , [3] , [4] Expected behavior was to have Agent Gateway port created only once and not deleted on deletion of Floating IP ‘b’. Creation and deletion of Agent Gateway port is guarded by the is_first and is_last checks on the subscribers set. The subscribers set is a member of the FipNamespace Class FipNamespace objects are stored as weakreferences It appears the weakreference to the FipNamespace object created at the time of instantiation of floating IP ‘a’ was lost/garbage collected before instantiation of floating IP ‘b’ and therefore a new object of class Fipnamespace was instantiated which came along with a new subscribers set member. This in turn led to passing of the is_first and is_last checks for router B thus leading to failures. [1] http://logs.openstack.org/22/153422/8/check/check-tempest-dsvm- neutron-dvr/3db7309/logs/testr_results.html.gz [2] http://logs.openstack.org/22/153422/8/check/check-tempest-dsvm- neutron-dvr/3db7309/logs/screen-q-vpn.txt.gz#_2015-02-24_02_20_16_355 [3] http://logs.openstack.org/22/153422/8/check/check-tempest-dsvm- neutron-dvr/3db7309/logs/screen-q-vpn.txt.gz#_2015-02-24_02_22_10_735 [4] http://logs.openstack.org/22/153422/8/check/check-tempest-dsvm- neutron-dvr/3db7309/logs/screen-q-vpn.txt.gz#_2015-02-24_02_21_57_960 To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1425639/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438351] [NEW] Angular table hzSelectAll should select all visible checkboxes in table
Public bug reported: Currently, if the select-all checkbox is clicked, all rows are selected even if it's not visible. For example, when a table is paginated or filtered. The select-all checkbox should only select all the visible checkboxes in the table. ** Affects: horizon Importance: Undecided Assignee: Kelly Domico (kelly-domico) Status: New ** Changed in: horizon Assignee: (unassigned) = Kelly Domico (kelly-domico) ** Description changed: -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1438351 Title: Angular table hzSelectAll should select all visible checkboxes in table Status in OpenStack Dashboard (Horizon): New Bug description: Currently, if the select-all checkbox is clicked, all rows are selected even if it's not visible. For example, when a table is paginated or filtered. The select-all checkbox should only select all the visible checkboxes in the table. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1438351/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1334923] Re: No need to backup instance when 'rotation' equals to 0
This optimization has been awaiting someone to add unit tests for 4 months. Calling it Opinion not In Progress. ** Changed in: nova Status: In Progress = Confirmed ** Changed in: nova Status: Confirmed = Opinion ** Changed in: nova Assignee: wingwj (wingwj) = (unassigned) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1334923 Title: No need to backup instance when 'rotation' equals to 0 Status in OpenStack Compute (Nova): Opinion Bug description: Nova will delete all backup images according the 'backup_type' parameter when 'rotation' equals to 0. But according the logic now, Nova will generate one new backup, and then delete it.. So It's weird to snapshot a useless backup firstly IMO. We need to add one new branch here: if 'rotation' is equal to 0, no need to backup, just rotate it. The related discussion can be found here: http://lists.openstack.org/pipermail/openstack-dev/2014-June/038760.html To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1334923/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438426] [NEW] Functional job setup broken by devstack
Public bug reported: A recent change to devstack [1] removed lib/neutron which the functional testing setup script depends on. The script should be updated to refer to lib/neutron-legacy instead. 1: https://review.openstack.org/#/c/168591/ ** Affects: neutron Importance: Critical Assignee: Maru Newby (maru) Status: New ** Changed in: neutron Importance: Undecided = Critical ** Changed in: neutron Assignee: (unassigned) = Maru Newby (maru) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1438426 Title: Functional job setup broken by devstack Status in OpenStack Neutron (virtual network service): New Bug description: A recent change to devstack [1] removed lib/neutron which the functional testing setup script depends on. The script should be updated to refer to lib/neutron-legacy instead. 1: https://review.openstack.org/#/c/168591/ To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1438426/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438474] [NEW] [Launch Instance Fix] Adding unit tests to hz.dashboard
Public bug reported: Angular module hz.dashboard should have unit tests. ** Affects: horizon Importance: Undecided Assignee: Shaoquan Chen (sean-chen2) Status: In Progress ** Changed in: horizon Assignee: (unassigned) = Shaoquan Chen (sean-chen2) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1438474 Title: [Launch Instance Fix] Adding unit tests to hz.dashboard Status in OpenStack Dashboard (Horizon): In Progress Bug description: Angular module hz.dashboard should have unit tests. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1438474/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1438469] [NEW] auth_version config not used
Public bug reported: When auth_version is set to be v3.0 in glance-api.conf, I see this in the logs Auth Token proceeding with requested v3.0 apis but in the very next debug log I see that the authentication request actually appends v2.0 to the identity url, which seems incorrect. 2015-03-31 00:52:13.928 254 INFO keystonemiddleware.auth_token [-] Auth Token proceeding with requested v3.0 apis 2015-03-31 00:52:13.928 254 DEBUG keystoneclient.auth.identity.v2 [-] Making authentication request to https://keystone server:35357/v2.0/tokens get_auth_ref /usr/lib/python2.7/site-packages/keystoneclient/auth/identity/v2.py:76 Relevant config from glance-api.conf: [keystone_authtoken] identity_uri = https://keystone server:35357 admin_user = glance admin_password =admin_password auth_version = v3.0 ** Affects: keystone Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to Keystone. https://bugs.launchpad.net/bugs/1438469 Title: auth_version config not used Status in OpenStack Identity (Keystone): New Bug description: When auth_version is set to be v3.0 in glance-api.conf, I see this in the logs Auth Token proceeding with requested v3.0 apis but in the very next debug log I see that the authentication request actually appends v2.0 to the identity url, which seems incorrect. 2015-03-31 00:52:13.928 254 INFO keystonemiddleware.auth_token [-] Auth Token proceeding with requested v3.0 apis 2015-03-31 00:52:13.928 254 DEBUG keystoneclient.auth.identity.v2 [-] Making authentication request to https://keystone server:35357/v2.0/tokens get_auth_ref /usr/lib/python2.7/site-packages/keystoneclient/auth/identity/v2.py:76 Relevant config from glance-api.conf: [keystone_authtoken] identity_uri = https://keystone server:35357 admin_user = glance admin_password =admin_password auth_version = v3.0 To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1438469/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1362766] Re: ConnectionFailed: Connection to neutron failed: 'HTTPSConnectionPool' object has no attribute 'insecure'
** Also affects: keystonemiddleware Importance: Undecided Status: New ** Summary changed: - ConnectionFailed: Connection to neutron failed: 'HTTPSConnectionPool' object has no attribute 'insecure' + ConnectionFailed: Connection to XX failed: 'HTTPSConnectionPool' object has no attribute 'insecure' -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1362766 Title: ConnectionFailed: Connection to XX failed: 'HTTPSConnectionPool' object has no attribute 'insecure' Status in OpenStack Identity (Keystone) Middleware: New Status in OpenStack Compute (Nova): New Status in Python client library for Glance: Fix Released Status in Python client library for Neutron: Incomplete Bug description: While compute manager was trying to authenticate with neutronclient, we see the following: 2014-08-28 05:03:33.052 29982 TRACE powervc_nova.compute.manager Traceback (most recent call last): 2014-08-28 05:03:33.052 29982 TRACE powervc_nova.compute.manager File /usr/lib/python2.7/site-packages/powervc_nova/compute/manager.py, line 672, in _populate_admin_context 2014-08-28 05:03:33.052 29982 TRACE powervc_nova.compute.manager nclient.authenticate() 2014-08-28 05:03:33.052 29982 TRACE powervc_nova.compute.manager File /usr/lib/python2.7/site-packages/neutronclient/client.py, line 231, in authenticate 2014-08-28 05:03:33.052 29982 TRACE powervc_nova.compute.manager self._authenticate_keystone() 2014-08-28 05:03:33.052 29982 TRACE powervc_nova.compute.manager File /usr/lib/python2.7/site-packages/neutronclient/client.py, line 209, in _authenticate_keystone 2014-08-28 05:03:33.052 29982 TRACE powervc_nova.compute.manager allow_redirects=True) 2014-08-28 05:03:33.052 29982 TRACE powervc_nova.compute.manager File /usr/lib/python2.7/site-packages/neutronclient/client.py, line 113, in _cs_request 2014-08-28 05:03:33.052 29982 TRACE powervc_nova.compute.manager raise exceptions.ConnectionFailed(reason=e) 2014-08-28 05:03:33.052 29982 TRACE powervc_nova.compute.manager ConnectionFailed: Connection to neutron failed: 'HTTPSConnectionPool' object has no attribute 'insecure' Setting a pdb breakpoint and stepping into the code, I see that the requests library is getting a connection object from a pool. The interesting thing is that the connection object is actually from glanceclient.common.https.HTTPSConnectionPool. It seems odd to me that neutronclient is using a connection object from glanceclient pool, but I do not know this requests code. Here is the stack just before failure: /usr/lib/python2.7/site-packages/neutronclient/client.py(234)authenticate() - self._authenticate_keystone() /usr/lib/python2.7/site-packages/neutronclient/client.py(212)_authenticate_keystone() - allow_redirects=True) /usr/lib/python2.7/site-packages/neutronclient/client.py(106)_cs_request() - resp, body = self.request(*args, **kargs) /usr/lib/python2.7/site-packages/neutronclient/client.py(151)request() - **kwargs) /usr/lib/python2.7/site-packages/requests/api.py(44)request() - return session.request(method=method, url=url, **kwargs) /usr/lib/python2.7/site-packages/requests/sessions.py(335)request() - resp = self.send(prep, **send_kwargs) /usr/lib/python2.7/site-packages/requests/sessions.py(438)send() - r = adapter.send(request, **kwargs) /usr/lib/python2.7/site-packages/requests/adapters.py(292)send() - timeout=timeout /usr/lib/python2.7/site-packages/urllib3/connectionpool.py(454)urlopen() - conn = self._get_conn(timeout=pool_timeout) /usr/lib/python2.7/site-packages/urllib3/connectionpool.py(272)_get_conn() - return conn or self._new_conn() /usr/lib/python2.7/site-packages/glanceclient/common/https.py(100)_new_conn() - return VerifiedHTTPSConnection(host=self.host, The code about to run there is this: class HTTPSConnectionPool(connectionpool.HTTPSConnectionPool): HTTPSConnectionPool will be instantiated when a new connection is requested to the HTTPSAdapter.This implementation overwrites the _new_conn method and returns an instances of glanceclient's VerifiedHTTPSConnection which handles no compression. ssl_compression is hard-coded to False because this will be used just when the user sets --no-ssl-compression. scheme = 'https' def _new_conn(self): self.num_connections += 1 return VerifiedHTTPSConnection(host=self.host, port=self.port, key_file=self.key_file, cert_file=self.cert_file, cacert=self.ca_certs, insecure=self.insecure,
