** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1791478
Title: nova-api get 504 when keepalived restart Status in OpenStack Compute (nova): Invalid Bug description: With haproxy + keepalived in openstack controller service HA, it got 504 error: first keystone: DEBUG (base:176) Making authentication request to http://192.168.21.1:35357/v3/auth/tokens DEBUG (connectionpool:393) http://192.168.21.1:35357 "POST /v3/auth/tokens HTTP/1.1" 504 None DEBUG (session:868) Request returned failure status: 504 DEBUG (shell:791) Gateway Timeout (HTTP 504) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 789, in main OpenStackComputeShell().main(argv) File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 657, in main api_version = api_versions.discover_version(self.cs, api_version) File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 261, in discover_version client) File "/usr/lib/python2.7/site-packages/novaclient/api_versions.py", line 242, in _get_server_version_range version = client.versions.get_current() File "/usr/lib/python2.7/site-packages/novaclient/v2/versions.py", line 70, in get_current return self._get_current() File "/usr/lib/python2.7/site-packages/novaclient/v2/versions.py", line 52, in _get_current url = "%s" % self.api.client.get_endpoint() File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 247, in get_endpoint return self.session.get_endpoint(auth or self.auth, **kwargs) File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 1113, in get_endpoint return auth.get_endpoint(self, **kwargs) File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 380, in get_endpoint allow_version_hack=allow_version_hack, **kwargs) File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 271, in get_endpoint_data service_catalog = self.get_access(session).service_catalog File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/base.py", line 134, in get_access self.auth_ref = self.get_auth_ref(session) File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/generic/base.py", line 208, in get_auth_ref return self._plugin.get_auth_ref(session, **kwargs) File "/usr/lib/python2.7/site-packages/keystoneauth1/identity/v3/base.py", line 178, in get_auth_ref authenticated=False, log=False, **rkwargs) File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 1019, in post return self.request(url, 'POST', **kwargs) File "/usr/lib/python2.7/site-packages/keystoneauth1/session.py", line 869, in request raise exceptions.from_response(resp, method, url) GatewayTimeout: Gateway Timeout (HTTP 504) ERROR (GatewayTimeout): Gateway Timeout (HTTP 504) then after about 1 min, keystone has worked but nova-api: DEBUG (connectionpool:205) Starting new HTTP connection (1): 10.22.21.1:8774 DEBUG (connectionpool:393) http://10.22.21.1:8774 "GET /v2.1/75bd7e950f4b4792a486b6bf240c7a3d HTTP/1.1" 404 112 RESP: [404] Content-Length: 112 Content-Type: application/json; charset=UTF-8 Date: Sun, 09 Sep 2018 00:56:18 GMT X-Compute-Request-Id: req-9e16a30b-3407-4f4f-89d7-1b64f4637b25 DEBUG (session:479) RESP: [404] Content-Length: 112 Content-Type: application/json; charset=UTF-8 Date: Sun, 09 Sep 2018 00:56:18 GMT X-Compute-Request-Id: req-9e16a30b-3407-4f4f-89d7-1b64f4637b25 RESP BODY: {"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"} DEBUG (session:511) RESP BODY: {"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"} It reported service not found, and then got the 504: DEBUG (session:448) REQ: curl -g -i -X GET http://192.168.21.1:8774/v2.1/75bd7e950f4b4792a486b6bf240c7a3d/servers/detail -H "Accept: application/json" -H "OpenStack-API-Version: compute 2.42" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA1}d0dd50b3f49d2ab39fae428e4eff0050bb2088ab" -H "X-OpenStack-Nova-API-Version: 2.42" DEBUG (connectionpool:393) http://192.168.21.1:8774 "GET /v2.1/75bd7e950f4b4792a486b6bf240c7a3d/servers/detail HTTP/1.1" 504 None DEBUG (session:479) RESP: [504] Cache-Control: no-cache Connection: close Content-Type: text/html DEBUG (session:511) RESP BODY: Omitted, Content-Type is set to text/html. Only application/json responses have their bodies logged. DEBUG (shell:791) Unknown Error (HTTP 504) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 789, in main OpenStackComputeShell().main(argv) File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 711, in main args.func(self.cs, args) File "/usr/lib/python2.7/site-packages/novaclient/v2/shell.py", line 1565, in do_list limit=args.limit) File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 854, in list "servers") File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 257, in _list resp, body = self.api.client.get(url) File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 328, in get return self.request(url, 'GET', **kwargs) File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request raise exceptions.from_response(resp, body, url, method) ClientException: Unknown Error (HTTP 504) ERROR (ClientException): Unknown Error (HTTP 504) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1791478/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp