Public bug reported: trying to remove compute node properly
Steps to reproduce ================== 1) remove all instances from the hypervisor: (env) vance@zs95k5:~$ nova hypervisor-servers zs93k23 +----+------+---------------+---------------------+ | ID | Name | Hypervisor ID | Hypervisor Hostname | +----+------+---------------+---------------------+ +----+------+---------------+---------------------+ 2) disable the hypervisor: (env) vance@zs95k5:~$ nova service-list +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ | 3 | nova-cert | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:13:34.000000 | - | | 4 | nova-scheduler | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:13:27.000000 | - | | 5 | nova-conductor | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:13:30.000000 | - | | 14 | nova-compute | u27-maas-machine-1 | nova | disabled | up | 2016-11-30T21:13:28.000000 | - | | 16 | nova-compute | zs95k181 | nova | enabled | up | 2016-11-30T21:13:33.000000 | - | | 17 | nova-compute | zs93k23 | nova | enabled | up | 2016-11-30T21:13:33.000000 | - | +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ (env) vance@zs95k5:~$ nova service-disable zs93k23 nova-compute +---------+--------------+----------+ | Host | Binary | Status | +---------+--------------+----------+ | zs93k23 | nova-compute | disabled | +---------+--------------+----------+ 3) delete the compute service (env) vance@zs95k5:~$ nova service-delete 17 (env) vance@zs95k5:~$ nova service-list +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ | 3 | nova-cert | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:14:54.000000 | - | | 4 | nova-scheduler | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:14:47.000000 | - | | 5 | nova-conductor | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:14:56.000000 | - | | 14 | nova-compute | u27-maas-machine-1 | nova | disabled | up | 2016-11-30T21:14:48.000000 | - | | 16 | nova-compute | zs95k181 | nova | enabled | up | 2016-11-30T21:14:53.000000 | - | +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ 4) delete the neutron agent (env) vance@zs95k5:~$ openstack network agent list +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ | 039e4b7a-3dbe-4e87-a9a5-d4b569d3113d | Open vSwitch agent | u27-maas-machine-2 | None | True | UP | neutron-openvswitch-agent | | 2aa13570-0e62-4198-96d9-dfe732d7874d | DHCP agent | u27-maas-machine-2 | nova | True | UP | neutron-dhcp-agent | | 2ab2320a-69cb-4a7f-8ae4-541d3b2bdd3b | L3 agent | u27-maas-machine-2 | nova | True | UP | neutron-l3-agent | | 48d6d83b-e459-46c8-945c-eea4197c01ec | Open vSwitch agent | zs95k181 | None | True | UP | neutron-openvswitch-agent | | a36eecd5-1fb4-436a-becd-fccc737518fd | Metering agent | u27-maas-machine-2 | None | True | UP | neutron-metering-agent | | aaee3bf0-f8bd-41b7-94ed-f9213f120016 | Open vSwitch agent | zs93k23 | None | True | UP | neutron-openvswitch-agent | | c6039c81-ad20-4258-a926-bc4a90dccc96 | Loadbalancer agent | u27-maas-machine-2 | None | True | UP | neutron-lbaas-agent | | cfecc66c-2888-4c3d-8241-1d3fcd018a1f | Metadata agent | u27-maas-machine-2 | None | True | UP | neutron-metadata-agent | | f60cbf28-f030-43ae-a598-0d0182529804 | Open vSwitch agent | u27-maas-machine-1 | None | True | UP | neutron-openvswitch-agent | +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ (env) vance@zs95k5:~$ openstack network agent delete aaee3bf0-f8bd-41b7-94ed-f9213f120016 (env) vance@zs95k5:~$ openstack network agent list +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ | 039e4b7a-3dbe-4e87-a9a5-d4b569d3113d | Open vSwitch agent | u27-maas-machine-2 | None | True | UP | neutron-openvswitch-agent | | 2aa13570-0e62-4198-96d9-dfe732d7874d | DHCP agent | u27-maas-machine-2 | nova | True | UP | neutron-dhcp-agent | | 2ab2320a-69cb-4a7f-8ae4-541d3b2bdd3b | L3 agent | u27-maas-machine-2 | nova | True | UP | neutron-l3-agent | | 48d6d83b-e459-46c8-945c-eea4197c01ec | Open vSwitch agent | zs95k181 | None | True | UP | neutron-openvswitch-agent | | a36eecd5-1fb4-436a-becd-fccc737518fd | Metering agent | u27-maas-machine-2 | None | True | UP | neutron-metering-agent | | c6039c81-ad20-4258-a926-bc4a90dccc96 | Loadbalancer agent | u27-maas-machine-2 | None | True | UP | neutron-lbaas-agent | | cfecc66c-2888-4c3d-8241-1d3fcd018a1f | Metadata agent | u27-maas-machine-2 | None | True | UP | neutron-metadata-agent | | f60cbf28-f030-43ae-a598-0d0182529804 | Open vSwitch agent | u27-maas-machine-1 | None | True | UP | neutron-openvswitch-agent | +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ 5) check hypervisor list (env) vance@zs95k5:~$ nova hypervisor-list ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'nova.exception.ComputeHostNotFound'> (HTTP 500) (Request-ID: req-2b773323-be4f-4c8c-be05-a41144ebed76) Okay, here is the error from the log: http://paste.ubuntu.com/23559892/ Expected result =============== Nova compute node is properly removed from the OS database. Environment =========== 1. OpenStack nova version 2:13.1.2-0ubuntu2 2. Hypervisor: Libvirt+KVM (KVM for IBM z Systems 1.1.3-beta4.3) 3. Networking type: OVS+GRE ** Affects: nova Importance: Undecided Status: New ** Tags: api db ** Tags added: api ** Tags added: db -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1646255 Title: removing compute node causes ComputeHostNotFound in nova-api Status in OpenStack Compute (nova): New Bug description: trying to remove compute node properly Steps to reproduce ================== 1) remove all instances from the hypervisor: (env) vance@zs95k5:~$ nova hypervisor-servers zs93k23 +----+------+---------------+---------------------+ | ID | Name | Hypervisor ID | Hypervisor Hostname | +----+------+---------------+---------------------+ +----+------+---------------+---------------------+ 2) disable the hypervisor: (env) vance@zs95k5:~$ nova service-list +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ | 3 | nova-cert | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:13:34.000000 | - | | 4 | nova-scheduler | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:13:27.000000 | - | | 5 | nova-conductor | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:13:30.000000 | - | | 14 | nova-compute | u27-maas-machine-1 | nova | disabled | up | 2016-11-30T21:13:28.000000 | - | | 16 | nova-compute | zs95k181 | nova | enabled | up | 2016-11-30T21:13:33.000000 | - | | 17 | nova-compute | zs93k23 | nova | enabled | up | 2016-11-30T21:13:33.000000 | - | +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ (env) vance@zs95k5:~$ nova service-disable zs93k23 nova-compute +---------+--------------+----------+ | Host | Binary | Status | +---------+--------------+----------+ | zs93k23 | nova-compute | disabled | +---------+--------------+----------+ 3) delete the compute service (env) vance@zs95k5:~$ nova service-delete 17 (env) vance@zs95k5:~$ nova service-list +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ | Id | Binary | Host | Zone | Status | State | Updated_at | Disabled Reason | +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ | 3 | nova-cert | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:14:54.000000 | - | | 4 | nova-scheduler | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:14:47.000000 | - | | 5 | nova-conductor | juju-605709-2-lxd-3 | internal | enabled | up | 2016-11-30T21:14:56.000000 | - | | 14 | nova-compute | u27-maas-machine-1 | nova | disabled | up | 2016-11-30T21:14:48.000000 | - | | 16 | nova-compute | zs95k181 | nova | enabled | up | 2016-11-30T21:14:53.000000 | - | +----+----------------+---------------------+----------+----------+-------+----------------------------+-----------------+ 4) delete the neutron agent (env) vance@zs95k5:~$ openstack network agent list +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ | 039e4b7a-3dbe-4e87-a9a5-d4b569d3113d | Open vSwitch agent | u27-maas-machine-2 | None | True | UP | neutron-openvswitch-agent | | 2aa13570-0e62-4198-96d9-dfe732d7874d | DHCP agent | u27-maas-machine-2 | nova | True | UP | neutron-dhcp-agent | | 2ab2320a-69cb-4a7f-8ae4-541d3b2bdd3b | L3 agent | u27-maas-machine-2 | nova | True | UP | neutron-l3-agent | | 48d6d83b-e459-46c8-945c-eea4197c01ec | Open vSwitch agent | zs95k181 | None | True | UP | neutron-openvswitch-agent | | a36eecd5-1fb4-436a-becd-fccc737518fd | Metering agent | u27-maas-machine-2 | None | True | UP | neutron-metering-agent | | aaee3bf0-f8bd-41b7-94ed-f9213f120016 | Open vSwitch agent | zs93k23 | None | True | UP | neutron-openvswitch-agent | | c6039c81-ad20-4258-a926-bc4a90dccc96 | Loadbalancer agent | u27-maas-machine-2 | None | True | UP | neutron-lbaas-agent | | cfecc66c-2888-4c3d-8241-1d3fcd018a1f | Metadata agent | u27-maas-machine-2 | None | True | UP | neutron-metadata-agent | | f60cbf28-f030-43ae-a598-0d0182529804 | Open vSwitch agent | u27-maas-machine-1 | None | True | UP | neutron-openvswitch-agent | +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ (env) vance@zs95k5:~$ openstack network agent delete aaee3bf0-f8bd-41b7-94ed-f9213f120016 (env) vance@zs95k5:~$ openstack network agent list +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ | 039e4b7a-3dbe-4e87-a9a5-d4b569d3113d | Open vSwitch agent | u27-maas-machine-2 | None | True | UP | neutron-openvswitch-agent | | 2aa13570-0e62-4198-96d9-dfe732d7874d | DHCP agent | u27-maas-machine-2 | nova | True | UP | neutron-dhcp-agent | | 2ab2320a-69cb-4a7f-8ae4-541d3b2bdd3b | L3 agent | u27-maas-machine-2 | nova | True | UP | neutron-l3-agent | | 48d6d83b-e459-46c8-945c-eea4197c01ec | Open vSwitch agent | zs95k181 | None | True | UP | neutron-openvswitch-agent | | a36eecd5-1fb4-436a-becd-fccc737518fd | Metering agent | u27-maas-machine-2 | None | True | UP | neutron-metering-agent | | c6039c81-ad20-4258-a926-bc4a90dccc96 | Loadbalancer agent | u27-maas-machine-2 | None | True | UP | neutron-lbaas-agent | | cfecc66c-2888-4c3d-8241-1d3fcd018a1f | Metadata agent | u27-maas-machine-2 | None | True | UP | neutron-metadata-agent | | f60cbf28-f030-43ae-a598-0d0182529804 | Open vSwitch agent | u27-maas-machine-1 | None | True | UP | neutron-openvswitch-agent | +--------------------------------------+--------------------+--------------------+-------------------+-------+-------+---------------------------+ 5) check hypervisor list (env) vance@zs95k5:~$ nova hypervisor-list ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'nova.exception.ComputeHostNotFound'> (HTTP 500) (Request-ID: req-2b773323-be4f-4c8c-be05-a41144ebed76) Okay, here is the error from the log: http://paste.ubuntu.com/23559892/ Expected result =============== Nova compute node is properly removed from the OS database. Environment =========== 1. OpenStack nova version 2:13.1.2-0ubuntu2 2. Hypervisor: Libvirt+KVM (KVM for IBM z Systems 1.1.3-beta4.3) 3. Networking type: OVS+GRE To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1646255/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
