also look at the logs /var/log/nova/nova-network.log on compute node. It should tell whether it is able to connect to rabbit on controller.
-Dheerendra On Wed, Oct 30, 2013 at 6:41 PM, Dheerendra < dheerendra.madhusudh...@gmail.com> wrote: > Hi Florian > > I observed this kind of issue when there is race condition. > > 1. RabbitMQ is NOT running on controller-node > 2. You issue nova-network from compute-node. Now this step hangs. > 3. Just see what is happening in nova-api.log on controller(tail -f > /var/log/nova/nova-api.log) > 4. Now you start the RabbitMQ on the controller node. > 5. nova-network command compute returns the error. Also you will see > 'timeout' error in nova-api.log. > > From your log I suspect this because initially RabitMQ was not up. Looks > like you started it afterwards. > Did you try to run the same 'nova-network' command for the second time ? > i.e after restarting the rabbitMQ ? > > -Dheerendra > > > > On Wed, Oct 30, 2013 at 3:47 PM, Florian Lindner <mailingli...@xgm.de>wrote: > >> Am Dienstag, 29. Oktober 2013, 14:32:00 schrieb Razique Mahroua: >> > You are using Neutron? >> > Looks like the Neutron server is unable to perform the request actually >> >> Actually no, I don't think so, I think I use the nova-network. >> >> The documentation >> http://docs.openstack.org/havana/install-guide/install/apt/content/ says: >> >> Configuring Networking can be one of the most bewildering experiences you >> will >> encounter when working with OpenStack. To assist in this we have chosen >> the >> simplest production-ready configuration for this guide: the legacy >> networking >> in OpenStack Compute, with a flat network, that takes care of DHCP. >> >> This setup uses "multi-host" functionality: the networking is configured >> to be >> highly available by splitting networking functionality across multiple >> hosts. >> As a result, there is no single network controller that acts as a single >> point >> of failure. Because each compute node is configured for networking in this >> setup, no additional networking configuration is required on the >> controller. >> >> (Chapter 5: Enabling Networking) >> >> and I have installed the nova-network package. >> >> I've downloaded my entire /etc to http://xgm.de/upload/etc/ if you or >> anyone >> else wants to have a look! Yes, these "123" are the actual passwords, it's >> just from a VM on my local computer. ;-) >> >> Thanks a lot! >> >> Florian >> >> > >> > >> > On October 29, 2013 at 12:00:18, Florian Lindner (mailingli...@xgm.de) >> > wrote: >> > Am Montag, 28. Oktober 2013, 13:44:00 schrieb Craig E. Ward: >> > > On the RabbitMQ server, are vhost and user id setup correctly? >> > >> > Ok, part of the problem seemed to be that the documentation assumed I >> set >> > the password guest for the rabbit guest account. Did that now and the >> > problem changed: >> > >> > root@controller:~# nova network-create vmnet --fixed-range-v4= >> 10.0.0.0/24 -- >> > bridge-interface=br100 --multi-host=T >> > ERROR: The server has either erred or is incapable of performing the >> > requested operation. (HTTP 500) (Request-ID: >> > req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3) >> > >> > nova-api.log shows: >> > >> > http://pastebin.com/raw.php?i=13U8a4Na >> > >> > >> > 2013-10-29 19:14:18.587 1272 INFO nova.osapi_compute.wsgi.server [-] >> (1272) >> > accepted ('192.168.122.10', 53116) >> > 2013-10-29 19:14:19.207 1272 INFO nova.openstack.common.rpc.common >> > [req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3 >> a1dd41c156c64810b41d46d4062a2b17 >> > 68d6cb4e2ceb4700931b6489ad25fb6f] Connected to AMQP server on >> > controller:5672 2013-10-29 19:14:19.218 1272 INFO >> > nova.openstack.common.rpc.common >> [req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3 >> > a1dd41c156c64810b41d46d4062a2b17 68d6cb4e2ceb4700931b6489ad25fb6f] >> > Connected to AMQP server on controller:5672 2013-10-29 19:15:19.238 1272 >> > ERROR nova.api.openstack [req-900a80bd- dfc1-49fc-9599-239fd5b9b7e3 >> > a1dd41c156c64810b41d46d4062a2b17 >> > 68d6cb4e2ceb4700931b6489ad25fb6f] Caught error: Timeout while waiting >> on RPC >> > response - topic: "network", RPC method: "create_networks" info: >> > "<unknown>" 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack >> Traceback >> > (most recent call last): >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/__init__.py", line >> 119, >> > in __call__ >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return >> > req.get_response(self.application) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/webob/request.py", line 1296, in send >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack application, >> > catch_exc_info=False) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/webob/request.py", line 1260, in >> > call_application >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack app_iter = >> > application(self.environ, start_response) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__ >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return >> resp(environ, >> > start_response) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > >> "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", >> > line 539, in __call__ >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return >> self.app(env, >> > start_response) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__ >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return >> resp(environ, >> > start_response) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__ >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return >> resp(environ, >> > start_response) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in >> > __call__ 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack response >> = >> > self.app(environ, start_response) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__ >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return >> resp(environ, >> > start_response) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__ >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack resp = >> > self.call_func(req, *args, **self.kwargs) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return >> > self.func(req, *args, **kwargs) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line >> 917, in >> > __call__ >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack content_type, >> body, >> > accept) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line >> 976, in >> > _process_stack >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack action_result = >> > self.dispatch(meth, request, action_args) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line >> 1057, in >> > dispatch >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return >> > method(req=request, **action_args) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist- >> > packages/nova/api/openstack/compute/contrib/os_networks.py", line 126, >> in >> > create >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack network = >> > self.network_api.create(context, **params)[0] >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 93, in >> wrapped >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return func(self, >> > context, *args, **kwargs) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 142, in >> create >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return >> > self.network_rpcapi.create_networks(context, **kwargs) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/nova/network/rpcapi.py", line 102, in >> > create_networks >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return >> > self.client.call(ctxt, 'create_networks', **kwargs) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/nova/rpcclient.py", line 85, in call >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return >> > self._invoke(self.proxy.call, ctxt, method, **kwargs) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/nova/rpcclient.py", line 63, in >> _invoke >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return >> > cast_or_call(ctxt, msg, **self.kwargs) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File >> > "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/proxy.py", >> line >> > 130, in call >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack exc.info, >> > real_topic, msg.get('method')) >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack Timeout: Timeout >> while >> > waiting on RPC response - topic: "network", RPC method: >> "create_networks" >> > info: "<unknown>" >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack >> > 2013-10-29 19:15:19.249 1272 INFO nova.api.openstack [req-900a80bd- >> > dfc1-49fc-9599-239fd5b9b7e3 a1dd41c156c64810b41d46d4062a2b17 >> > 68d6cb4e2ceb4700931b6489ad25fb6f] >> > http://controller:8774/v2/68d6cb4e2ceb4700931b6489ad25fb6f/os-networks >> > returned with HTTP 500 >> > 2013-10-29 19:15:19.255 1272 INFO nova.osapi_compute.wsgi.server >> > [req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3 >> a1dd41c156c64810b41d46d4062a2b17 >> > 68d6cb4e2ceb4700931b6489ad25fb6f] 192.168.122.10 "POST >> > /v2/68d6cb4e2ceb4700931b6489ad25fb6f/os-networks HTTP/1.1" status: 500 >> len: >> > 335 time: 60.6673729 >> > >> > Sorry, I got not idea... >> > >> > Thanks! >> > Florian >> > >> > > % rabbitmqctl list_users >> > > >> > > % rabbitmqctl list_vhosts >> > > >> > > The user and vhost should match what is in the OpenStack >> configurations. >> > > >> > > On 10/28/13 1:14 PM, Florian Lindner wrote: >> > > > Am Montag, 28. Oktober 2013, 12:04:02 schrieb Razique Mahroua: >> > > >> hey >> > > >> is «controller» a hostame all ur servers can resolve? >> > > > >> > > > Yes, set in /etc/hosts. >> > > > >> > > > Regards, >> > > > Florian >> > > > >> > > >> - Razique >> > > >> >> > > >> On Oct 28, 2013, at 11:40, Florian Lindner <mailingli...@xgm.de> >> wrote: >> > > >>> Hello, >> > > >>> >> > > >>> I've installed OpenStack on a Ubuntu 12.04 VM according to the >> install >> > > >>> guide. It worked fine until I created the compute node on a >> second VM >> > > >>> and >> > > >>> tried to set up nova networking: >> > > >>> >> > > >>> >> http://docs.openstack.org/havana/install-guide/install/apt/content/nov >> > > >>> a-> >>> ne twork.html >> > > >>> >> > > >>> # nova network-create vmnet --fixed-range-v4=10.0.0.0/24--bridge- >> > > >>> interface=br100 --multi-host=T >> > > >>> >> > > >>> just does nothing until canceled. I tried on the controller and >> > > >>> compute1 >> > > >>> node. >> > > >>> >> > > >>> nova-api.log shows: >> > > >>> >> > > >>> ERROR nova.openstack.common.rpc.common AMQP server on >> controller:5672 >> > > >>> is >> > > >>> unreachable: Socket closed. Trying again in 1 seconds. >> > > >>> INFO nova.openstack.common.rpc.common Reconnecting to AMQP server >> on >> > > >>> controller:5672 >> > > >>> >> > > >>> controller:~# netstat -lpn | grep beam >> > > >>> tcp 0 0 0.0.0.0:36595 0.0.0.0:* >> > > >>> LISTEN >> > > >>> 1045/beam >> > > >>> tcp6 0 0 :::5672 :::* >> > > >>> LISTEN >> > > >>> 1045/beam >> > > >>> >> > > >>> It listens on this port, but IPv6. IMO on linux listing on an IPv6 >> > > >>> port >> > > >>> also means listening on the same v4 port. >> > > >>> >> > > >>> What's wrong with the installation? >> >> >> >> _______________________________________________ >> Mailing list: >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >> Post to : openstack@lists.openstack.org >> Unsubscribe : >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack >> > >
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack