Hi!
I would need some help for debugging problems on cinder volume creation
& deletion.
Quite many of operations end up to left running forever.
Our config is Openstack Kilo running in RHEL7. We are using RH RDO
packaged version.
We have 4 master nodes, running most of the services loadbalanced with
haproxy. Cinder volume is having 2 availability zones.
Cinder has been working fine, but after we had rabbitmq cluster problems
(needed reboot), cinder has not been working properly. Many volume
create operations are left running as 'creating', and deletions are left
'deleting'.
With rabbitmqctl list_queues I can see, that there are some queues
accumulating, but why? Some cinder-scheduler_fanout_LONGHEXHERE and some
reply_LONGHEXHERE.
And in cinder logs, no errors, just info messages, for example for
unsuccessfull creation:
api.log:2015-12-29 01:05:15.881 2567 INFO cinder.api.openstack.wsgi
[req-5bfdeeb8-d47d-471d-b8db-6a526c7a00cf tavasti
91ac58eb50db4384b9afeb5d8c622f8d - - -] POST http://volume-api.domain.com
/v2/91ac58eb50db4384b9afeb5d8c622f8d/volumes api.log:2015-12-29
01:05:15.882 2567 INFO cinder.api.v2.volumes
[req-5bfdeeb8-d47d-471d-b8db-6a526c7a00cf tavasti
91ac58eb50db4384b9afeb5d8c622f8d - - -] Create volume of 1 GB
After those, I cannot see anything relevant, on any of the cinder logs
on any nodes. Specially, on cinder scheduler logs, there is silence,
even with successfull volume creations.
Any ideas how to track what is broken?
--Tavasti, Openstack admin
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
