Hi, Any clue on how to fix network or port issue if that is the case?
BR, Varun On Thu, Jul 28, 2016 at 4:15 PM, varun bhatnagar <varun292...@gmail.com> wrote: > Hello Duncan, > > I can't see any logs of cinder-api service getting restarted. Although I > can see some messages getting continuously logged not sure if that will > help. > > > 2016-07-28 16:13:56.934 25399 INFO cinder.api.openstack.wsgi [-] > http://192.168.2.25:8776/ returned with HTTP 300 > 2016-07-28 16:13:56.935 25399 INFO eventlet.wsgi.server [-] 192.168.2.24 > "OPTIONS / HTTP/1.0" status: 200 len: 134 time: 0.0026720 > 2016-07-28T16:13:56.935213+02:00 cic-1 cinder-api: 2016-07-28 16:13:56.934 > 25399 INFO cinder.api.openstack.wsgi [-] http://192.168.2.25:8776/ > returned with HTTP 300 > 2016-07-28T16:13:56.935974+02:00 cic-1 cinder-api: 2016-07-28 16:13:56.935 > 25399 INFO eventlet.wsgi.server [-] 192.168.2.24 "OPTIONS / HTTP/1.0" > status: 200 len: 134 time: 0.0026720 > 2016-07-28 16:13:57.099 25399 INFO cinder.api.openstack.wsgi [-] OPTIONS > http://192.168.2.25:8776/ > 2016-07-28 16:13:57.100 25399 INFO cinder.api.openstack.wsgi [-] > http://192.168.2.25:8776/ returned with HTTP 300 > 2016-07-28 16:13:57.101 25399 INFO eventlet.wsgi.server [-] 192.168.2.26 > "OPTIONS / HTTP/1.0" status: 200 len: 134 time: 0.0020089 > 2016-07-28T16:13:57.099882+02:00 cic-1 cinder-api: 2016-07-28 16:13:57.099 > 25399 INFO cinder.api.openstack.wsgi [-] OPTIONS http://192.168.2.25:8776/ > 2016-07-28T16:13:57.100836+02:00 cic-1 cinder-api: 2016-07-28 16:13:57.100 > 25399 INFO cinder.api.openstack.wsgi [-] http://192.168.2.25:8776/ > returned with HTTP 300 > 2016-07-28T16:13:57.101321+02:00 cic-1 cinder-api: 2016-07-28 16:13:57.101 > 25399 INFO eventlet.wsgi.server [-] 192.168.2.26 "OPTIONS / HTTP/1.0" > status: 200 len: 134 time: 0.0020089 > 2016-07-28 16:14:01.144 25399 INFO cinder.api.openstack.wsgi [-] OPTIONS > http://192.168.2.25:8776/ > 2016-07-28T16:14:01.145297+02:00 cic-1 cinder-api: 2016-07-28 16:14:01.144 > 25399 INFO cinder.api.openstack.wsgi [-] OPTIONS http://192.168.2.25:8776/ > 2016-07-28 16:14:01.146 25399 INFO cinder.api.openstack.wsgi [-] > http://192.168.2.25:8776/ returned with HTTP 300 > 2016-07-28T16:14:01.146887+02:00 cic-1 cinder-api: 2016-07-28 16:14:01.146 > 25399 INFO cinder.api.openstack.wsgi [-] http://192.168.2.25:8776/ > returned with HTTP 300 > 2016-07-28 16:14:01.147 25399 INFO eventlet.wsgi.server [-] 192.168.2.26 > "OPTIONS / HTTP/1.0" status: 200 len: 134 time: 0.0032442 > 2016-07-28T16:14:01.147704+02:00 cic-1 cinder-api: 2016-07-28 16:14:01.147 > 25399 INFO eventlet.wsgi.server [-] 192.168.2.26 "OPTIONS / HTTP/1.0" > status: 200 len: 134 time: 0.0032442 > > BR, > Varun > > On Thu, Jul 28, 2016 at 3:44 PM, Duncan Thomas <duncan.tho...@gmail.com> > wrote: > >> Looks like either you've got an intermittent network problem or the >> cinder api service is restarting. Anything enlightening in the cinder-api >> log? >> >> On 28 Jul 2016 16:41, "varun bhatnagar" <varun292...@gmail.com> wrote: >> >> Hello Steve, >> >> Thanks a lot for such a quick response. >> >> Yes the IP is reachable. >> >> ping 10.33.237.104 >> PING 10.33.237.104 (10.33.237.104) 56(84) bytes of data. >> 64 bytes from 10.33.237.104: icmp_seq=1 ttl=64 time=0.587 ms >> 64 bytes from 10.33.237.104: icmp_seq=2 ttl=64 time=0.101 ms >> 64 bytes from 10.33.237.104: icmp_seq=3 ttl=64 time=0.092 ms >> 64 bytes from 10.33.237.104: icmp_seq=4 ttl=64 time=0.144 ms >> ^C >> --- 10.33.237.104 ping statistics --- >> 4 packets transmitted, 4 received, 0% packet loss, time 3003ms >> rtt min/avg/max/mdev = 0.092/0.231/0.587/0.206 ms >> root@cic-1:~# >> >> >> Endpoints for keystone and cinder are defined as below: >> >> | keystone | identity | RegionOne >> | >> | | | publicURL: >> http://10.33.237.104:5000/v2.0 | >> | | | internalURL: >> http://192.168.2.28:5000/v2.0 | >> | | | adminURL: >> http://192.168.2.28:35357/v2.0 | >> >> >> | cinderv2 | volumev2 | RegionOne >> | >> | | | publicURL: >> http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92 | >> | | | internalURL: >> http://192.168.2.28:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92 | >> | | | adminURL: >> http://192.168.2.28:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92 | >> | | | >> | >> >> >> And adding debug gives the below details: >> >> openstack volume list --debug >> START with options: ['volume', 'list', '--debug'] >> options: Namespace(access_token_endpoint='', auth_type='', auth_url=' >> http://192.168.2.28:5000/v2.0', cacert='', client_id='', >> client_secret='***', cloud='', debug=True, default_domain='Default', >> deferred_help=False, domain_id='', domain_name='', endpoint='', >> identity_provider='', identity_provider_url='', insecure=None, >> interface='', log_file=None, os_clustering_api_version='1', >> os_compute_api_version='', os_data_processing_api_version='1.1', >> os_data_processing_url='', os_dns_api_version='2', >> os_identity_api_version='', os_image_api_version='', >> os_key_manager_api_version='1', os_network_api_version='', >> os_object_api_version='', os_orchestration_api_version='1', >> os_project_id=None, os_project_name=None, os_queues_api_version='1.1', >> os_volume_api_version='', os_workflow_api_version='2', password='***', >> profile=None, project_domain_id='', project_domain_name='', project_id='', >> project_name='admin', protocol='', region_name='RegionOne', scope='', >> service_provider_endpoint='', timing=False, token='***', trust_id='', >> url='', user_domain_id='', user_domain_name='', user_id='', >> username='admin', verbose_level=3, verify=None) >> defaults: {u'auth_type': 'password', u'compute_api_version': u'2', 'key': >> None, u'database_api_version': u'1.0', 'api_timeout': None, >> u'baremetal_api_version': u'1', u'image_api_version': u'2', 'cacert': None, >> u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', >> u'orchestration_api_version': u'1', u'interface': None, >> u'network_api_version': u'2', u'image_format': u'qcow2', >> u'key_manager_api_version': u'v1', u'metering_api_version': u'2', 'verify': >> True, u'identity_api_version': u'2.0', u'volume_api_version': u'2', 'cert': >> None, u'secgroup_source': u'neutron', u'container_api_version': u'1', >> u'dns_api_version': u'2', u'object_store_api_version': u'1', >> u'disable_vendor_agent': {}} >> cloud cfg: {'auth_type': 'password', u'compute_api_version': u'2', >> u'orchestration_api_version': '1', u'database_api_version': u'1.0', >> 'data_processing_api_version': '1.1', u'network_api_version': u'2', >> u'image_format': u'qcow2', u'image_api_version': u'2', >> 'clustering_api_version': '1', 'verify': True, u'dns_api_version': '2', >> u'object_store_api_version': u'1', 'verbose_level': 3, 'region_name': >> 'RegionOne', 'api_timeout': None, u'baremetal_api_version': u'1', >> 'queues_api_version': '1.1', 'auth': {'username': 'admin', 'project_name': >> 'admin', 'password': '***', 'auth_url': 'http://192.168.2.28:5000/v2.0'}, >> 'default_domain': 'Default', u'container_api_version': u'1', >> u'image_api_use_tasks': False, u'floating_ip_source': u'neutron', 'key': >> None, 'timing': False, 'cacert': None, u'key_manager_api_version': '1', >> u'metering_api_version': u'2', 'deferred_help': False, >> u'identity_api_version': u'2.0', 'workflow_api_version': '2', >> u'volume_api_version': u'2', 'cert': None, u'secgroup_source': u'neutron', >> 'debug': True, u'interface': None, u'disable_vendor_agent': {}} >> compute API version 2, cmd group openstack.compute.v2 >> network API version 2, cmd group openstack.network.v2 >> image API version 2, cmd group openstack.image.v2 >> volume API version 2, cmd group openstack.volume.v2 >> identity API version 2.0, cmd group openstack.identity.v2 >> object_store API version 1, cmd group openstack.object_store.v1 >> orchestration API version 1, cmd group openstack.orchestration.v1 >> clustering API version 1, cmd group openstack.clustering.v1 >> workflow_engine API version 2, cmd group openstack.workflow_engine.v2 >> key_manager API version 1, cmd group openstack.key_manager.v1 >> dns API version 2, cmd group openstack.dns.v2 >> data_processing API version 1.1, cmd group openstack.data_processing.v1 >> messaging API version 1.1, cmd group openstack.messaging.v1 >> command: volume list -> openstackclient.volume.v2.volume.ListVolume >> Auth plugin password selected >> auth_type: password >> Using auth plugin: password >> Using parameters {'username': 'admin', 'password': '***', 'project_name': >> 'admin', 'auth_url': 'http://192.168.2.28:5000/v2.0'} >> Get auth_ref >> REQ: curl -g -i -X GET http://192.168.2.28:5000/v2.0 -H "Accept: >> application/json" -H "User-Agent: python-openstackclient >> keystoneauth1/2.3.0 python-requests/2.9.1 CPython/2.7.6" >> Starting new HTTP connection (1): 192.168.2.28 >> "GET /v2.0 HTTP/1.1" 200 None >> RESP: [200] Transfer-Encoding: chunked Vary: >> X-Keystone-Internal,X-Auth-Token Server: Apache Connection: close Date: >> Thu, 28 Jul 2016 13:35:11 GMT Content-Type: application/json >> x-openstack-request-id: req-27df9679-2ff5-4d6a-aecb-3499b5aa683b >> RESP BODY: {"version": {"status": "stable", "updated": >> "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", >> "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", >> "links": [{"href": "http://192.168.2.28:5000/v2.0/", "rel": "self"}, >> {"href": "http://docs.openstack.org/", "type": "text/html", "rel": >> "describedby"}]}} >> >> Making authentication request to http://192.168.2.28:5000/v2.0/tokens >> Resetting dropped connection: 192.168.2.28 >> "POST /v2.0/tokens HTTP/1.1" 200 None >> run(Namespace(all_projects=False, columns=[], formatter='table', >> long=False, max_width=0, name=None, noindent=False, project=None, >> project_domain=None, quote_mode='nonnumeric', status=None, user=None, >> user_domain=None)) >> Instantiating volume client: <class 'cinderclient.v2.client.Client'> >> Instantiating compute client for VAPI Version Major: 2, Minor: 0 >> Instantiating identity client: <class >> 'openstackclient.identity.client.IdentityClientv2'> >> Making authentication request to http://192.168.2.28:5000/v2.0/tokens >> Resetting dropped connection: 192.168.2.28 >> "POST /v2.0/tokens HTTP/1.1" 200 None >> REQ: curl -g -i -X GET http://10.33.237.104:8774/v2.1/servers/detail -H >> "User-Agent: python-novaclient" -H "Accept: application/json" -H >> "X-Auth-Token: {SHA1}c2d3c0718b45d184c4a0b99cc0858a3f65d150b3" >> Starting new HTTP connection (1): 10.33.237.104 >> "GET /v2.1/servers/detail HTTP/1.1" 200 15 >> RESP: [200] Content-Length: 15 X-Compute-Request-Id: >> req-b78348d6-750a-44e6-b303-1e4d87ddb64c Vary: X-OpenStack-Nova-API-Version >> Connection: close X-Openstack-Nova-Api-Version: 2.1 Date: Thu, 28 Jul 2016 >> 13:35:11 GMT Content-Type: application/json >> RESP BODY: {"servers": []} >> >> REQ: curl -g -i -X GET >> http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92/volumes/detail >> -H "User-Agent: python-cinderclient" -H "Accept: application/json" -H >> "X-Auth-Token: {SHA1}c2d3c0718b45d184c4a0b99cc0858a3f65d150b3" >> Starting new HTTP connection (1): 10.33.237.104 >> Unable to establish connection to >> http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92/volumes/detail >> Traceback (most recent call last): >> File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 374, in >> run_subcommand >> result = cmd.run(parsed_args) >> File >> "/usr/lib/python2.7/dist-packages/openstackclient/common/command.py", line >> 38, in run >> return super(Command, self).run(parsed_args) >> File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 92, in >> run >> column_names, data = self.take_action(parsed_args) >> File >> "/usr/lib/python2.7/dist-packages/openstackclient/volume/v2/volume.py", >> line 305, in take_action >> data = volume_client.volumes.list(search_opts=search_opts) >> File "/usr/lib/python2.7/dist-packages/cinderclient/v2/volumes.py", >> line 282, in list >> return self._list(url, resource_type, limit=limit) >> File "/usr/lib/python2.7/dist-packages/cinderclient/base.py", line 73, >> in _list >> resp, body = self.api.client.get(url) >> File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line >> 111, in get >> return self._cs_request(url, 'GET', **kwargs) >> File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line >> 108, in _cs_request >> return self.request(url, method, **kwargs) >> File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line >> 99, in request >> **kwargs) >> File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line >> 212, in request >> resp = super(LegacyJsonAdapter, self).request(*args, **kwargs) >> File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line >> 101, in request >> return self.session.request(url, method, **kwargs) >> File >> "/usr/lib/python2.7/dist-packages/openstackclient/common/session.py", line >> 40, in request >> resp = super(TimingSession, self).request(url, method, **kwargs) >> File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line >> 94, in inner >> return func(*args, **kwargs) >> File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line >> 452, in request >> resp = send(**kwargs) >> File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line >> 496, in _send_request >> raise exceptions.ConnectFailure(msg) >> ConnectFailure: Unable to establish connection to >> http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92/volumes/detail >> clean_up ListVolume: Unable to establish connection to >> http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92/volumes/detail >> Traceback (most recent call last): >> File "/usr/lib/python2.7/dist-packages/openstackclient/shell.py", line >> 118, in run >> ret_val = super(OpenStackShell, self).run(argv) >> File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 255, in run >> result = self.run_subcommand(remainder) >> File "/usr/lib/python2.7/dist-packages/openstackclient/shell.py", line >> 153, in run_subcommand >> ret_value = super(OpenStackShell, self).run_subcommand(argv) >> File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 374, in >> run_subcommand >> result = cmd.run(parsed_args) >> File >> "/usr/lib/python2.7/dist-packages/openstackclient/common/command.py", line >> 38, in run >> return super(Command, self).run(parsed_args) >> File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 92, in >> run >> column_names, data = self.take_action(parsed_args) >> File >> "/usr/lib/python2.7/dist-packages/openstackclient/volume/v2/volume.py", >> line 305, in take_action >> data = volume_client.volumes.list(search_opts=search_opts) >> File "/usr/lib/python2.7/dist-packages/cinderclient/v2/volumes.py", >> line 282, in list >> return self._list(url, resource_type, limit=limit) >> File "/usr/lib/python2.7/dist-packages/cinderclient/base.py", line 73, >> in _list >> resp, body = self.api.client.get(url) >> File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line >> 111, in get >> return self._cs_request(url, 'GET', **kwargs) >> File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line >> 108, in _cs_request >> return self.request(url, method, **kwargs) >> File "/usr/lib/python2.7/dist-packages/cinderclient/client.py", line >> 99, in request >> **kwargs) >> File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line >> 212, in request >> resp = super(LegacyJsonAdapter, self).request(*args, **kwargs) >> File "/usr/lib/python2.7/dist-packages/keystoneclient/adapter.py", line >> 101, in request >> return self.session.request(url, method, **kwargs) >> File >> "/usr/lib/python2.7/dist-packages/openstackclient/common/session.py", line >> 40, in request >> resp = super(TimingSession, self).request(url, method, **kwargs) >> File "/usr/lib/python2.7/dist-packages/positional/__init__.py", line >> 94, in inner >> return func(*args, **kwargs) >> File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line >> 452, in request >> resp = send(**kwargs) >> File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line >> 496, in _send_request >> raise exceptions.ConnectFailure(msg) >> ConnectFailure: Unable to establish connection to >> http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92/volumes/detail >> >> >> BR, >> Varun >> >> On Thu, Jul 28, 2016 at 3:17 PM, Steve Martinelli <s.martine...@gmail.com >> > wrote: >> >>> try running the command with --debug when it fails. normally that error >>> happens when the client can't reach the host, does the IP address and port >>> number look correct? >>> >>> On Thu, Jul 28, 2016 at 8:57 AM, varun bhatnagar <varun292...@gmail.com> >>> wrote: >>> >>>> Hi, >>>> >>>> I am using OpenStack Mitaka. >>>> I am trying to list volumes using openstack client, the command works >>>> sometimes but sometimes it fails: >>>> >>>> openstack volume list >>>> Unable to establish connection to >>>> http://10.33.237.104:8776/v2/3cbbffce04d9463e8cb8d3ca6480ed92/volumes/detail >>>> >>>> openstack volume list >>>> >>>> +--------------------------------------+--------------+-----------+------+-------------+ >>>> | ID | Display Name | Status | >>>> Size | Attached to | >>>> >>>> +--------------------------------------+--------------+-----------+------+-------------+ >>>> | dfab04ce-5ad3-4787-8342-3e76afd877a0 | testvol | available | >>>> 1 | | >>>> >>>> +--------------------------------------+--------------+-----------+------+-------------+ >>>> >>>> >>>> Can anyone please tell me what is wrong here. >>>> >>>> BR, >>>> Varun >>>> >>>> _______________________________________________ >>>> Mailing list: >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>> Post to : openstack@lists.openstack.org >>>> Unsubscribe : >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >>>> >>>> >>> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> >
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack