Public bug reported: I am running:
Ubuntu 14.04 Contrail 2.20 root@cmpt02:~# dpkg -l | grep contrail ii contrail-lib 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail libraries ii contrail-nodemgr 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail nodemgr implementation ii contrail-nova-driver 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenStack Nova compute-node driver for OpenContrail ii contrail-utils 2.20+0~1441967460.80~1.bb1145b amd64 OpenContrail tools and utilities ii contrail-vrouter-agent 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail vrouter agent ii contrail-vrouter-dkms 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail VRouter - DKMS version ii contrail-vrouter-utils 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail VRouter - Utilities ii python-backports.ssl-match-hostname 3.4.0.2-1contrail1 all The ssl.match_hostname() function from Python 3.4 ii python-bitarray 0.8.0-2contrail1 amd64 Python module for efficient boolean array handling ii python-contrail 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail python-libs ii python-contrail-vrouter-api 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail vrouter agent api ii python-geventhttpclient 1.1.0-1contrail1 amd64 http client library for gevent ii python-opencontrail-vrouter-netns 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail vrouter network namespace package OpenStack Kilo packages root@cmpt02:~# dpkg -l | grep nova ii contrail-nova-driver 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenStack Nova compute-node driver for OpenContrail ii nova-common 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute - common files ii nova-compute 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute - compute node base ii nova-compute-kvm 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute - compute node (KVM) ii nova-compute-libvirt 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute - compute node libvirt support ii python-nova 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute Python libraries ii python-novaclient 1:2.22.0-0ubuntu1~cloud0 all client library for OpenStack Compute API Nova will not boot an instance; debug log snippet from /var/log/nova/nova-compute.log: ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s) TRACE nova.compute.manager Traceback (most recent call last): TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1782, in _allocate_network_async TRACE nova.compute.manager dhcp_options=dhcp_options) TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 406, in allocate_for_instance TRACE nova.compute.manager neutron = get_client(context) TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 221, in get_client TRACE nova.compute.manager region_name=CONF.neutron.region_name) TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1200, in __init__ TRACE nova.compute.manager self.httpclient = client.construct_http_client(**kwargs) TRACE nova.compute.manager TypeError: construct_http_client() got an unexpected keyword argument 'endpoint_override' 2015-10-12 18:40:06.334 27007 ERROR nova.compute.manager [req-58ce7b2e-bc73-40ee-a368-cfbeaed434ca 42329176f69a4cc1b7d5e6ae805080cd 7812bd244b7f4a8eba3a5cb1213210a5 - - -] [instance: 0713d74f-fe59-4992-894c- e3378fb1752d] Instance failed to spawn 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager Traceback (most recent call last): 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2461, in _build_resources 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager yield resources 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2333, in _build_and_run_insta nce 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager block_device_info=block_device_info) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2382, in spawn 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager write_to_disk=True) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4169, in _get_guest_xml 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager network_info_str = str(network_info) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 480, in __str__ 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager return self._sync_wrapper(fn, *args, **kwargs) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 463, in _sync_wrapper 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager self.wait() 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 495, in wait 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager self[:] = self._gt.wait() 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 175, in wait 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager return self._exit_event.wait() 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 125, in wait 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager current.throw(*self._exc) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager result = function(*args, **kwargs) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1782, in _allocate_network_as ync 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager dhcp_options=dhcp_options) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 406, in allocate_for_in stance 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager neutron = get_client(context) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 221, in get_client 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager region_name=CONF.neutron.region_name) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1200, in __init__ 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager self.httpclient = client.construct_http_client(**kwargs) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager TypeError: construct_http_client() got an unexpected keyword argument 'endpoint_override' 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager 2015-10-12 18:40:06.336 27007 INFO nova.compute.manager [req-58ce7b2e-bc73-40ee-a368-cfbeaed434ca 42329176f69a4cc1b7d5e6ae805080cd 7812bd244b7f4a8eba3a5cb1213210a5 - - -] [instance: 0713d74f-fe59-4992-894c-e 3378fb1752d] Terminating instance There is nothing that appears to be wrong in /var/log/neutron- server.conf or in contrail's logfiles. Has anyone come across the log "TypeError: construct_http_client() got an unexpected keyword argument 'endpoint_override'" before? ** Affects: nova Importance: Undecided Status: New ** Attachment added: "debug log for nova" https://bugs.launchpad.net/bugs/1505313/+attachment/4492592/+files/debug_kilo_nova_log.txt -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1505313 Title: nova kilo and contrail 2.2, instances fail to boot throwing an endpoint_override error Status in OpenStack Compute (nova): New Bug description: I am running: Ubuntu 14.04 Contrail 2.20 root@cmpt02:~# dpkg -l | grep contrail ii contrail-lib 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail libraries ii contrail-nodemgr 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail nodemgr implementation ii contrail-nova-driver 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenStack Nova compute-node driver for OpenContrail ii contrail-utils 2.20+0~1441967460.80~1.bb1145b amd64 OpenContrail tools and utilities ii contrail-vrouter-agent 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail vrouter agent ii contrail-vrouter-dkms 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail VRouter - DKMS version ii contrail-vrouter-utils 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail VRouter - Utilities ii python-backports.ssl-match-hostname 3.4.0.2-1contrail1 all The ssl.match_hostname() function from Python 3.4 ii python-bitarray 0.8.0-2contrail1 amd64 Python module for efficient boolean array handling ii python-contrail 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail python-libs ii python-contrail-vrouter-api 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail vrouter agent api ii python-geventhttpclient 1.1.0-1contrail1 amd64 http client library for gevent ii python-opencontrail-vrouter-netns 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenContrail vrouter network namespace package OpenStack Kilo packages root@cmpt02:~# dpkg -l | grep nova ii contrail-nova-driver 2.20+0~1443532552.81~1.c0c6c68 amd64 OpenStack Nova compute-node driver for OpenContrail ii nova-common 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute - common files ii nova-compute 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute - compute node base ii nova-compute-kvm 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute - compute node (KVM) ii nova-compute-libvirt 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute - compute node libvirt support ii python-nova 1:2015.1.1-0ubuntu1~cloud2 all OpenStack Compute Python libraries ii python-novaclient 1:2.22.0-0ubuntu1~cloud0 all client library for OpenStack Compute API Nova will not boot an instance; debug log snippet from /var/log/nova/nova-compute.log: ERROR nova.compute.manager [-] Instance failed network setup after 1 attempt(s) TRACE nova.compute.manager Traceback (most recent call last): TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1782, in _allocate_network_async TRACE nova.compute.manager dhcp_options=dhcp_options) TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 406, in allocate_for_instance TRACE nova.compute.manager neutron = get_client(context) TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 221, in get_client TRACE nova.compute.manager region_name=CONF.neutron.region_name) TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1200, in __init__ TRACE nova.compute.manager self.httpclient = client.construct_http_client(**kwargs) TRACE nova.compute.manager TypeError: construct_http_client() got an unexpected keyword argument 'endpoint_override' 2015-10-12 18:40:06.334 27007 ERROR nova.compute.manager [req-58ce7b2e-bc73-40ee-a368-cfbeaed434ca 42329176f69a4cc1b7d5e6ae805080cd 7812bd244b7f4a8eba3a5cb1213210a5 - - -] [instance: 0713d74f-fe59-4992-894c- e3378fb1752d] Instance failed to spawn 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager Traceback (most recent call last): 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2461, in _build_resources 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager yield resources 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2333, in _build_and_run_insta nce 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager block_device_info=block_device_info) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2382, in spawn 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager write_to_disk=True) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4169, in _get_guest_xml 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager network_info_str = str(network_info) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 480, in __str__ 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager return self._sync_wrapper(fn, *args, **kwargs) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 463, in _sync_wrapper 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager self.wait() 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 495, in wait 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager self[:] = self._gt.wait() 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 175, in wait 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager return self._exit_event.wait() 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 125, in wait 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager current.throw(*self._exc) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager result = function(*args, **kwargs) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1782, in _allocate_network_as ync 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager dhcp_options=dhcp_options) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 406, in allocate_for_in stance 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager neutron = get_client(context) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 221, in get_client 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager region_name=CONF.neutron.region_name) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 1200, in __init__ 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager self.httpclient = client.construct_http_client(**kwargs) 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager TypeError: construct_http_client() got an unexpected keyword argument 'endpoint_override' 2015-10-12 18:40:06.334 27007 TRACE nova.compute.manager 2015-10-12 18:40:06.336 27007 INFO nova.compute.manager [req-58ce7b2e-bc73-40ee-a368-cfbeaed434ca 42329176f69a4cc1b7d5e6ae805080cd 7812bd244b7f4a8eba3a5cb1213210a5 - - -] [instance: 0713d74f-fe59-4992-894c-e 3378fb1752d] Terminating instance There is nothing that appears to be wrong in /var/log/neutron- server.conf or in contrail's logfiles. Has anyone come across the log "TypeError: construct_http_client() got an unexpected keyword argument 'endpoint_override'" before? To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1505313/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp