Re: [ceph-users] ceph nautilus namespaces for rbd and rbd image access problem
On Mon, May 20, 2019 at 11:14 AM Rainer Krienke wrote: > > Am 20.05.19 um 09:06 schrieb Jason Dillaman: > > >> $ rbd --namespace=testnamespace map rbd/rbdtestns --name client.rainer > >> --keyring=/etc/ceph/ceph.keyring > >> rbd: sysfs write failed > >> rbd: error opening image rbdtestns: (1) Operation not permitted > >> In some cases useful info is found in syslog - try "dmesg | tail". > >> 2019-05-20 08:18:29.187 7f42ab7fe700 -1 librbd::image::RefreshRequest: > >> failed to retrieve pool metadata: (1) Operation not permitted > >> 2019-05-20 08:18:29.187 7f42aaffd700 -1 librbd::image::OpenRequest: > >> failed to refresh image: (1) Operation not permitted > >> 2019-05-20 08:18:29.187 7f42aaffd700 -1 librbd::ImageState: > >> 0x561792408860 failed to open image: (1) Operation not permitted > >> rbd: map failed: (22) Invalid argument > > > > Hmm, it looks like we overlooked updating the 'rbd' profile when PR > > 27423 [1] was merged into v14.2.1. We'll get that fixed, but in the > > meantime, you can add a "class rbd metadata_list" cap on the base pool > > (w/o the namespace restriction) [2]. > > > > Thanks for your answer. Well I still have Kernel 4.15 so namespaces > won't work for me at the moment. > > Could you please explain what the magic behind "class rbd metadata_list" > is? Is it thought to "simply" allow access to the basepool (rbd in my > case), so I authorize access to the pool instead of a namespaces? And if > this is true then I do not understand the difference of your class cap > compared to a cap like osd 'allow rw pool=rbd'? It allows access to invoke a single OSD object class method named rbd.metadata_list, which is a read-only operation. Therefore, you are giving access to read pool-level configuration overrides but not access to read/write/execute any other things in the base pool. You could further restrict it to the "rbd_info" object when combined w/ the "object_prefix rbd_info" matcher. > -- > Rainer Krienke, Uni Koblenz, Rechenzentrum, A22, Universitaetsstrasse 1 > 56070 Koblenz, Tel: +49261287 1312 Fax +49261287 100 1312 > Web: http://userpages.uni-koblenz.de/~krienke > PGP: http://userpages.uni-koblenz.de/~krienke/mypgp.html -- Jason ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] ceph nautilus namespaces for rbd and rbd image access problem
Am 20.05.19 um 09:06 schrieb Jason Dillaman: >> $ rbd --namespace=testnamespace map rbd/rbdtestns --name client.rainer >> --keyring=/etc/ceph/ceph.keyring >> rbd: sysfs write failed >> rbd: error opening image rbdtestns: (1) Operation not permitted >> In some cases useful info is found in syslog - try "dmesg | tail". >> 2019-05-20 08:18:29.187 7f42ab7fe700 -1 librbd::image::RefreshRequest: >> failed to retrieve pool metadata: (1) Operation not permitted >> 2019-05-20 08:18:29.187 7f42aaffd700 -1 librbd::image::OpenRequest: >> failed to refresh image: (1) Operation not permitted >> 2019-05-20 08:18:29.187 7f42aaffd700 -1 librbd::ImageState: >> 0x561792408860 failed to open image: (1) Operation not permitted >> rbd: map failed: (22) Invalid argument > > Hmm, it looks like we overlooked updating the 'rbd' profile when PR > 27423 [1] was merged into v14.2.1. We'll get that fixed, but in the > meantime, you can add a "class rbd metadata_list" cap on the base pool > (w/o the namespace restriction) [2]. > Thanks for your answer. Well I still have Kernel 4.15 so namespaces won't work for me at the moment. Could you please explain what the magic behind "class rbd metadata_list" is? Is it thought to "simply" allow access to the basepool (rbd in my case), so I authorize access to the pool instead of a namespaces? And if this is true then I do not understand the difference of your class cap compared to a cap like osd 'allow rw pool=rbd'? -- Rainer Krienke, Uni Koblenz, Rechenzentrum, A22, Universitaetsstrasse 1 56070 Koblenz, Tel: +49261287 1312 Fax +49261287 100 1312 Web: http://userpages.uni-koblenz.de/~krienke PGP: http://userpages.uni-koblenz.de/~krienke/mypgp.html ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] ceph nautilus namespaces for rbd and rbd image access problem
On Mon, May 20, 2019 at 9:08 AM Rainer Krienke wrote: > > Hello, > > just saw this message on the client when trying and failing to map the > rbd image: > > May 20 08:59:42 client kernel: libceph: bad option at > '_pool_ns=testnamespace' You will need kernel v4.19 (or later) I believe to utilize RBD namespaces via krbd [1]. > Rainer > > Am 20.05.19 um 08:56 schrieb Rainer Krienke: > > Hello, > > > > on a ceph Nautilus cluster (14.2.1) running on Ubuntu 18.04 I try to set > > up rbd images with namespaces in order to allow different clients to > > access only their "own" rbd images in different namespaces in just one > > pool. The rbd image data are in an erasure encoded pool named "ecpool" > > and the metadata in the default "rbd" pool. > -- > Rainer Krienke, Uni Koblenz, Rechenzentrum, A22, Universitaetsstrasse 1 > 56070 Koblenz, Tel: +49261287 1312 Fax +49261287 100 1312 > Web: http://userpages.uni-koblenz.de/~krienke > PGP: http://userpages.uni-koblenz.de/~krienke/mypgp.html > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com [1] https://github.com/torvalds/linux/commit/b26c047b940003295d3896b7f633a66aab95bebd -- Jason ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] ceph nautilus namespaces for rbd and rbd image access problem
Hello, just saw this message on the client when trying and failing to map the rbd image: May 20 08:59:42 client kernel: libceph: bad option at '_pool_ns=testnamespace' Rainer Am 20.05.19 um 08:56 schrieb Rainer Krienke: > Hello, > > on a ceph Nautilus cluster (14.2.1) running on Ubuntu 18.04 I try to set > up rbd images with namespaces in order to allow different clients to > access only their "own" rbd images in different namespaces in just one > pool. The rbd image data are in an erasure encoded pool named "ecpool" > and the metadata in the default "rbd" pool. -- Rainer Krienke, Uni Koblenz, Rechenzentrum, A22, Universitaetsstrasse 1 56070 Koblenz, Tel: +49261287 1312 Fax +49261287 100 1312 Web: http://userpages.uni-koblenz.de/~krienke PGP: http://userpages.uni-koblenz.de/~krienke/mypgp.html ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] ceph nautilus namespaces for rbd and rbd image access problem
On Mon, May 20, 2019 at 8:56 AM Rainer Krienke wrote: > > Hello, > > on a ceph Nautilus cluster (14.2.1) running on Ubuntu 18.04 I try to set > up rbd images with namespaces in order to allow different clients to > access only their "own" rbd images in different namespaces in just one > pool. The rbd image data are in an erasure encoded pool named "ecpool" > and the metadata in the default "rbd" pool. > > With this setup I am experiencing trouble when I try to access a rbd > image in a namespace from a (OpenSuSE Leap 15.0 with Ceph 14.2.1) client > and I do not understand what I am doing wrong. Hope someone can see the > problem and give me a hint: > > # On one of the the ceph servers > > $ rbd namespace create --namespace testnamespace > $ rbd namespace ls > NAME > testnamespace > > $ ceph auth caps client.rainer mon 'profile rbd' osd 'profile rbd > pool=rbd namespace=testnamespace' > > $ ceph auth get client.rainer > [client.rainer] > key = AQCcVt5cHC+WJhBBoRPKhErEYzxGuU8U/GA0xA++ > caps mon = "profile rbd" > caps osd = "profile rbd pool=rbd namespace=testnamespace" > > $ rbd create rbd/rbdtestns --namespace testnamespace --size 50G > --data-pool=rbd-ecpool > > $ rbd --namespace testnamespace ls -l > NAME SIZE PARENT FMT PROT LOCK > rbdtestns 50 GiB 2 > > On the openSuSE Client: > > $ rbd --namespace=testnamespace map rbd/rbdtestns --name client.rainer > --keyring=/etc/ceph/ceph.keyring > rbd: sysfs write failed > rbd: error opening image rbdtestns: (1) Operation not permitted > In some cases useful info is found in syslog - try "dmesg | tail". > 2019-05-20 08:18:29.187 7f42ab7fe700 -1 librbd::image::RefreshRequest: > failed to retrieve pool metadata: (1) Operation not permitted > 2019-05-20 08:18:29.187 7f42aaffd700 -1 librbd::image::OpenRequest: > failed to refresh image: (1) Operation not permitted > 2019-05-20 08:18:29.187 7f42aaffd700 -1 librbd::ImageState: > 0x561792408860 failed to open image: (1) Operation not permitted > rbd: map failed: (22) Invalid argument Hmm, it looks like we overlooked updating the 'rbd' profile when PR 27423 [1] was merged into v14.2.1. We'll get that fixed, but in the meantime, you can add a "class rbd metadata_list" cap on the base pool (w/o the namespace restriction) [2]. > Thanks for your help > Rainer > -- > Rainer Krienke, Uni Koblenz, Rechenzentrum, A22, Universitaetsstrasse 1 > 56070 Koblenz, Tel: +49261287 1312 Fax +49261287 100 1312 > Web: http://userpages.uni-koblenz.de/~krienke > PGP: http://userpages.uni-koblenz.de/~krienke/mypgp.html > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com [1] https://github.com/ceph/ceph/pull/27423 [2] http://docs.ceph.com/docs/master/rados/operations/user-management/#authorization-capabilities -- Jason ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
