Re: [openstack-dev] [puppet] [magnum] Magnum tempest fails with 400 bad request
Hi Tobias,
Thanks for looking into it.
Currently the issue i see is magnum configuration[1] is wrong:-
auth_uri=http://localhost:5000, should be https and v3 versioned as
per scenario003 deployment configuration.
Magnum relies on auth_uri param and that too versioned("v3") like below:-
auth_uri=https://[::1]:5000/v3
After fixing this config current issue would be solved. Also i think
there is more work required to fix it completely but let's clear the
current issue first.
Also would be good to try our atomic 27 image(current is too old):-
tempest::magnum::image_source
https://download.fedoraproject.org/pub/alt/atomic/stable/Fedora-Atomic-27-20180212.2/CloudImages/x86_64/images/Fedora-Atomic-27-20180212.2.x86_64.qcow2
Some other thing that would be required are below:-
The cluster vm magnum creates should be able to connect to openstack
services and to internet.
Also settings would be required to work with SSL enabled services like
either TLS_DISABLED or setting up verify_ca and cert configuration in
magnum.conf
[1]
http://logs.openstack.org/12/367012/28/check/puppet-openstack-integration-4-scenario003-tempest-centos-7/3f5252b/logs/etc/magnum/magnum.conf.txt.gz
Thanks and Regards
Yatin Karel
On Thu, May 17, 2018 at 5:37 PM, Thomas Goirand wrote:
> On 05/17/2018 09:49 AM, Tobias Urdin wrote:
>> Hello,
>>
>> I was interested in getting Magnum working in gate by getting @dms patch
>> fixed and merged [1].
>>
>> The installation goes fine on Ubuntu and CentOS however the tempest
>> testing for Magnum fails on CentOS (it not available in Ubuntu).
>>
>>
>> It seems to be related to authentication against keystone but I don't
>> understand why, please see logs [2] [3]
>>
>>
>> [1] https://review.openstack.org/#/c/367012/
>>
>> [2]
>> http://logs.openstack.org/12/367012/28/check/puppet-openstack-integration-4-scenario003-tempest-centos-7/3f5252b/logs/magnum/magnum-api.txt.gz#_2018-05-16_15_10_36_010
>>
>> [3]
>> http://logs.openstack.org/12/367012/28/check/puppet-openstack-integration-4-scenario003-tempest-centos-7/3f5252b/
>
> From that log, you're getting a 404 from nova-api.
>
> Response - Headers: {'status': '404', u'content-length': '113',
> 'content-location': 'https://[::1]:8774/v2.1/os-keypairs/default',
> u'x-compute-request-id': 'req-35ae4651-186c-4f20-9143-f68f67b7d401',
> u'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version',
> u'server': 'Apache/2.4.6 (CentOS)', u'openstack-api-version': 'compute
> 2.1', u'connection': 'close', u'x-openstack-nova-api-version': '2.1',
> u'date': 'Wed, 16 May 2018 15:10:33 GMT', u'content-type':
> 'application/json; charset=UTF-8', u'x-openstack-request-id':
> 'req-35ae4651-186c-4f20-9143-f68f67b7d401'}
>
> but that seems fine because the request right after is working, however
> just right after, you're getting a 500 error on magnum-api a bit further:
>
> Response - Headers: {'status': '500', u'content-length': '149',
> 'content-location': 'https://[::1]:9511/clustertemplates',
> u'openstack-api-maximum-version': 'container-infra 1.6', u'vary':
> 'OpenStack-API-Version', u'openstack-api-minimum-version':
> 'container-infra 1.1', u'server': 'Werkzeug/0.11.6 Python/2.7.5',
> u'openstack-api-version': 'container-infra 1.1', u'date': 'Wed, 16 May
> 2018 15:10:36 GMT', u'content-type': 'application/json',
> u'x-openstack-request-id': 'req-12c635c9-889a-48b4-91d4-ded51220ad64'}
>
> With this body:
>
> Body: {"errors": [{"status": 500, "code": "server", "links": [],
> "title": "Bad Request (HTTP 400)", "detail": "Bad Request (HTTP 400)",
> "request_id": ""}]}
> 2018-05-16 15:24:14.434432 | centos-7 | 2018-05-16 15:10:36,016
> 13619 DEBUG[tempest.lib.common.dynamic_creds] Clearing network:
> {u'provider:physical_network': None, u'ipv6_address_scope': None,
> u'revision_number': 2, u'port_security_enabled': True, u'mtu': 1400,
> u'id': u'c26c237a-0583-4f72-8300-f87051080be7', u'router:external':
> False, u'availability_zone_hints': [], u'availability_zones': [],
> u'provider:segmentation_id': 35, u'ipv4_address_scope': None, u'shared':
> False, u'project_id': u'31c5c1fbc46e4880b7e498e493700a50', u'status':
> u'ACTIVE', u'subnets': [], u'description': u'', u'tags': [],
> u'updated_at': u'2018-05-16T15:10:26Z', u'is_default': False,
> u'qos_policy_id': None, u'name': u'tempest-setUp-2113966350-network',
> u'admin_state_up': True, u'tenant_id':
> u'31c5c1fbc46e4880b7e498e493700a50', u'created_at':
> u'2018-05-16T15:10:26Z', u'provider:network_type': u'vxlan'}, subnet:
> {u'service_types': [], u'description': u'', u'enable_dhcp': True,
> u'tags': [], u'network_id': u'c26c237a-0583-4f72-8300-f87051080be7',
> u'tenant_id': u'31c5c1fbc46e4880b7e498e493700a50', u'created_at':
> u'2018-05-16T15:10:26Z', u'dns_nameservers': [], u'updated_at':
> u'2018-05-16T15:10:26Z', u'ipv6_ra_mode': None, u'allocation_pools':
> [{u'start': u'10.100.0.2', u'end': u'10.100.0.14'}], u'gateway_ip':
> u'10.100.0.1', u'revision_number': 0, u'ipv6_address_mode': None,
> u'ip_versi
Re: [openstack-dev] [puppet] [magnum] Magnum tempest fails with 400 bad request
On 05/17/2018 09:49 AM, Tobias Urdin wrote:
> Hello,
>
> I was interested in getting Magnum working in gate by getting @dms patch
> fixed and merged [1].
>
> The installation goes fine on Ubuntu and CentOS however the tempest
> testing for Magnum fails on CentOS (it not available in Ubuntu).
>
>
> It seems to be related to authentication against keystone but I don't
> understand why, please see logs [2] [3]
>
>
> [1] https://review.openstack.org/#/c/367012/
>
> [2]
> http://logs.openstack.org/12/367012/28/check/puppet-openstack-integration-4-scenario003-tempest-centos-7/3f5252b/logs/magnum/magnum-api.txt.gz#_2018-05-16_15_10_36_010
>
> [3]
> http://logs.openstack.org/12/367012/28/check/puppet-openstack-integration-4-scenario003-tempest-centos-7/3f5252b/
From that log, you're getting a 404 from nova-api.
Response - Headers: {'status': '404', u'content-length': '113',
'content-location': 'https://[::1]:8774/v2.1/os-keypairs/default',
u'x-compute-request-id': 'req-35ae4651-186c-4f20-9143-f68f67b7d401',
u'vary': 'OpenStack-API-Version,X-OpenStack-Nova-API-Version',
u'server': 'Apache/2.4.6 (CentOS)', u'openstack-api-version': 'compute
2.1', u'connection': 'close', u'x-openstack-nova-api-version': '2.1',
u'date': 'Wed, 16 May 2018 15:10:33 GMT', u'content-type':
'application/json; charset=UTF-8', u'x-openstack-request-id':
'req-35ae4651-186c-4f20-9143-f68f67b7d401'}
but that seems fine because the request right after is working, however
just right after, you're getting a 500 error on magnum-api a bit further:
Response - Headers: {'status': '500', u'content-length': '149',
'content-location': 'https://[::1]:9511/clustertemplates',
u'openstack-api-maximum-version': 'container-infra 1.6', u'vary':
'OpenStack-API-Version', u'openstack-api-minimum-version':
'container-infra 1.1', u'server': 'Werkzeug/0.11.6 Python/2.7.5',
u'openstack-api-version': 'container-infra 1.1', u'date': 'Wed, 16 May
2018 15:10:36 GMT', u'content-type': 'application/json',
u'x-openstack-request-id': 'req-12c635c9-889a-48b4-91d4-ded51220ad64'}
With this body:
Body: {"errors": [{"status": 500, "code": "server", "links": [],
"title": "Bad Request (HTTP 400)", "detail": "Bad Request (HTTP 400)",
"request_id": ""}]}
2018-05-16 15:24:14.434432 | centos-7 | 2018-05-16 15:10:36,016
13619 DEBUG[tempest.lib.common.dynamic_creds] Clearing network:
{u'provider:physical_network': None, u'ipv6_address_scope': None,
u'revision_number': 2, u'port_security_enabled': True, u'mtu': 1400,
u'id': u'c26c237a-0583-4f72-8300-f87051080be7', u'router:external':
False, u'availability_zone_hints': [], u'availability_zones': [],
u'provider:segmentation_id': 35, u'ipv4_address_scope': None, u'shared':
False, u'project_id': u'31c5c1fbc46e4880b7e498e493700a50', u'status':
u'ACTIVE', u'subnets': [], u'description': u'', u'tags': [],
u'updated_at': u'2018-05-16T15:10:26Z', u'is_default': False,
u'qos_policy_id': None, u'name': u'tempest-setUp-2113966350-network',
u'admin_state_up': True, u'tenant_id':
u'31c5c1fbc46e4880b7e498e493700a50', u'created_at':
u'2018-05-16T15:10:26Z', u'provider:network_type': u'vxlan'}, subnet:
{u'service_types': [], u'description': u'', u'enable_dhcp': True,
u'tags': [], u'network_id': u'c26c237a-0583-4f72-8300-f87051080be7',
u'tenant_id': u'31c5c1fbc46e4880b7e498e493700a50', u'created_at':
u'2018-05-16T15:10:26Z', u'dns_nameservers': [], u'updated_at':
u'2018-05-16T15:10:26Z', u'ipv6_ra_mode': None, u'allocation_pools':
[{u'start': u'10.100.0.2', u'end': u'10.100.0.14'}], u'gateway_ip':
u'10.100.0.1', u'revision_number': 0, u'ipv6_address_mode': None,
u'ip_version': 4, u'host_routes': [], u'cidr': u'10.100.0.0/28',
u'project_id': u'31c5c1fbc46e4880b7e498e493700a50', u'id':
u'a7233852-e3f1-4129-b34e-c607aef5172e', u'subnetpool_id': None,
u'name': u'tempest-setUp-2113966350-subnet'}, router: {u'status':
u'ACTIVE', u'external_gateway_info': {u'network_id':
u'c6cf6d80-fcbb-46e6-aefd-17f41b5c57b1', u'enable_snat': True,
u'external_fixed_ips': [{u'subnet_id':
u'34e589e9-86d2-4f72-a0c3-7990406561b1', u'ip_address':
u'172.24.5.13'}]}, u'availability_zone_hints': [],
u'availability_zones': [], u'description': u'', u'tags': [],
u'tenant_id': u'31c5c1fbc46e4880b7e498e493700a50', u'created_at':
u'2018-05-16T15:10:27Z', u'admin_state_up': True, u'distributed': False,
u'updated_at': u'2018-05-16T15:10:29Z', u'ha': False, u'flavor_id':
None, u'revision_number': 2, u'routes': [], u'project_id':
u'31c5c1fbc46e4880b7e498e493700a50', u'id':
u'bdf13d72-c19c-4ad1-b57d-ed6da9c569b3', u'name':
u'tempest-setUp-2113966350-router'}
And right after that, we can only see clean-up calls (removing routers,
DELETE calls, etc.).
Looking at the magnum-api log shows issues in glanceclient just right
before the 500 error.
So, something's probably going on there, with a bad glanceclient
request. Having a look into magnum.conf doesn't show anything suspicious
concerning [glance_client] though, so I went to look into tempest.conf.
And there, it shows no [magn
[openstack-dev] [puppet] [magnum] Magnum tempest fails with 400 bad request
Hello, I was interested in getting Magnum working in gate by getting @dms patch fixed and merged [1]. The installation goes fine on Ubuntu and CentOS however the tempest testing for Magnum fails on CentOS (it not available in Ubuntu). It seems to be related to authentication against keystone but I don't understand why, please see logs [2] [3] [1] https://review.openstack.org/#/c/367012/ [2] http://logs.openstack.org/12/367012/28/check/puppet-openstack-integration-4-scenario003-tempest-centos-7/3f5252b/logs/magnum/magnum-api.txt.gz#_2018-05-16_15_10_36_010 [3] http://logs.openstack.org/12/367012/28/check/puppet-openstack-integration-4-scenario003-tempest-centos-7/3f5252b/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
