Hello all, I am testing OpenStack Newton on CentOS 7 and now I already have a OpenStack cloud infrastructure working, I would like to integrate Ceph with Cinder, Glance and Nova services.
I have found some information about how to performance this on previous releases for Openstack and Ceph: http://docs.ceph.com/docs/jewel/rbd/rbd-openstack/ And I tried to update the variables according to https://docs.openstack.org/newton/config-reference/block-storage/drivers/ceph-rbd-volume-driver.html This is my current glance-api.conf and which is working: [glance_store] stores = file,http default_store = file filesystem_store_datadir = /var/lib/glance/images/ At this is what I tried (at the moment, I have only tried it with the glance service but I did not get success): * Install python-rbd and python-rados from centos-ceph-jewel repo on the controller node. * Create a ceph user and add it to sudoers. * On ceph admin node: sudo ceph osd pool create images 150 sudo ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=images' sudo ceph auth get-or-create client.glance | ssh ceph@controller-node1 sudo tee /etc/ceph/ceph.client.glance.keyring ssh ceph@controller-node1 sudo chown glance:glance /etc/ceph/ceph.client.glance.keyring * On the controller node, I edited the glance-api.conf file with the following variables: [glance_store] stores = file,http,rbd default_store = rbd rbd_pool = images rbd_user = glance rbd_ceph_conf = /etc/ceph/ceph.conf rbd_store_chunk_size = 8 (!) I also have tried with stores = rbd but without success. * And restart the service, systemctl restart openstack-glance-api But when I try to create a new image, I get the following issue: [openstackadmin@controller-node1 ~]$ openstack image create "cirros ceph" --file /home/openstackadmin/cirros-0.3.4-x86_64-disk.raw --disk-format raw --container-format bare --public 500 Internal Server Error The server has either erred or is incapable of performing the requested operation. (HTTP 500) Is there any documentation about how to integrate Ceph jewel with OpenStack newton (Cinder, Glance and Nova services)? If it is so, could you please provide me the link? On the other hand, Is there any way to chose the store when creating an image? I mean, to choose for example if save the image on /var/lib/glance/images/ or on Ceph. I would appreciate if you could help me to set up this integration. This is the most relevant info I can see on the logs and as far as I see from it, I have a permissions error but I am not sure what I need to modify. It is my first installation with OpenStack by the way and I am trying at home to see how it works :) {{{ /var/log/glance/api.log 2017-03-01 23:43:41.419 4197 INFO eventlet.wsgi.server [req-eea55909-c963-4158-b30c-3f2779fd78c6 c41043a1ddc14ffba1b45c0a3287e0bf b2d1547f0e734f87a84feea75ccd6453 - default default] 192.168.56.2 - - [01/Mar/2017 23:43:41] "GET /v2/schemas/image HTTP/1.1" 200 4352 0.421281 2017-03-01 23:43:41.570 4197 INFO eventlet.wsgi.server [req-f5ef0b43-0ddf-4698-8459-2f75fb1822a3 c41043a1ddc14ffba1b45c0a3287e0bf b2d1547f0e734f87a84feea75ccd6453 - default default] 192.168.56.2 - - [01/Mar/2017 23:43:41] "POST /v2/images HTTP/1.1" 201 859 0.110681 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data [req-98c406fb-7bc0-47de-bace-2b0a2097b699 c41043a1ddc14ffba1b45c0a3287e0bf b2d1547f0e734f87a84feea75ccd6453 - default default] Failed to upload image data due to internal error 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data Traceback (most recent call last): 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 115, in upload 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data image.set_data(data, size) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 195, in set_data 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data self.base.set_data(data, size) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 479, in set_data 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data _send_notification(notify_error, 'image.upload', msg) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data self.force_reraise() 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data six.reraise(self.type_, self.value, self.tb) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 426, in set_data 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data self.repo.set_data(data, size) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 185, in set_data 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data return self.image.set_data(*args, **kwargs) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 304, in set_data 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data self.image.set_data(data, size=size) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance/location.py", line 440, in set_data 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data verifier=verifier) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 447, in add_to_backend 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data verifier) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 420, in store_add_to_backend 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data verifier=verifier) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance_store/capabilities.py", line 225, in op_checker 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data return store_op_fun(store, *args, **kwargs) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 447, in add 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data rados_id=self.user) as conn: 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data return self.gen.next() 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "/usr/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 270, in get_connection 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data client = rados.Rados(conffile=conffile, rados_id=rados_id) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "rados.pyx", line 525, in rados.Rados.__init__ (rados.c:5719) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "rados.pyx", line 425, in rados.requires.wrapper.validate_func (rados.c:4106) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "rados.pyx", line 568, in rados.Rados.__setup (rados.c:6448) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "rados.pyx", line 425, in rados.requires.wrapper.validate_func (rados.c:4106) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data File "rados.pyx", line 631, in rados.Rados.conf_read_file (rados.c:7384) 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data PermissionDeniedError: error calling conf_read_file 2017-03-01 23:43:41.651 4197 ERROR glance.api.v2.image_data 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi [req-98c406fb-7bc0-47de-bace-2b0a2097b699 c41043a1ddc14ffba1b45c0a3287e0bf b2d1547f0e734f87a84feea75ccd6453 - default default] Caught error: error calling conf_read_file 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi Traceback (most recent call last): 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1113, in __call__ 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi request, **action_args) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/wsgi.py", line 1152, in dispatch 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return method(*args, **kwargs) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/common/utils.py", line 363, in wrapped 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return func(self, req, *args, **kwargs) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 250, in upload 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi self._restore(image_repo, image) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi self.force_reraise() 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/v2/image_data.py", line 115, in upload 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi image.set_data(data, size) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/domain/proxy.py", line 195, in set_data 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi self.base.set_data(data, size) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 479, in set_data 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi _send_notification(notify_error, 'image.upload', msg) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi self.force_reraise() 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi six.reraise(self.type_, self.value, self.tb) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/notifier.py", line 426, in set_data 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi self.repo.set_data(data, size) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/api/policy.py", line 185, in set_data 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return self.image.set_data(*args, **kwargs) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/quota/__init__.py", line 304, in set_data 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi self.image.set_data(data, size=size) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance/location.py", line 440, in set_data 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi verifier=verifier) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 447, in add_to_backend 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi verifier) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/backend.py", line 420, in store_add_to_backend 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi verifier=verifier) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/capabilities.py", line 225, in op_checker 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return store_op_fun(store, *args, **kwargs) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 447, in add 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi rados_id=self.user) as conn: 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi return self.gen.next() 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "/usr/lib/python2.7/site-packages/glance_store/_drivers/rbd.py", line 270, in get_connection 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi client = rados.Rados(conffile=conffile, rados_id=rados_id) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", line 525, in rados.Rados.__init__ (rados.c:5719) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", line 425, in rados.requires.wrapper.validate_func (rados.c:4106) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", line 568, in rados.Rados.__setup (rados.c:6448) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", line 425, in rados.requires.wrapper.validate_func (rados.c:4106) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi File "rados.pyx", line 631, in rados.Rados.conf_read_file (rados.c:7384) 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi PermissionDeniedError: error calling conf_read_file 2017-03-01 23:43:41.709 4197 ERROR glance.common.wsgi }}} Thanks in advance, Esteban
_______________________________________________ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
