Hi Mark, Thanks for the detective work! I'll confirm whether the constraint on the client name is really intended -- I believe that it is. Once confirmed, I completely agree that ceph-deploy should not allow you to create rgw daemons with names that we know Ceph is going to reject.
- Travis On Wed, Apr 22, 2015 at 4:42 AM, Mark Kirkwood <mark.kirkw...@catalyst.net.nz> wrote: > On 13/04/15 15:33, Mark Kirkwood wrote: >> >> Hi, >> >> I've been experimenting with the new rgw creation in ceph-deploy, using >> version 1.5.23 together with ceph 0.94 (-948-gd77de49). >> >> If simply run it without any args, then it works fine. e.g: >> >> >> $ ceph-deploy rgw create ceph1 >> >> >> However if I try to set a name for the daemon: >> >> >> $ ceph-deploy rgw create ceph1:radosgw.ceph1 >> >> >> then it fails to create and start a radosgw daemon at all. Poking >> through the logs I see: >> >> $ tail ceph-client.radosgw.ceph1.log >> 2015-04-13 14:13:20.513860 7f7e36c04840 0 librados: >> client.radosgw.ceph1 authentication error (1) Operation not permitted >> >> >> Not terribly informative. Looking at the audit log: >> >> $ tail ceph.audit.log >> 015-04-13 15:10:37.880085 mon.0 192.168.122.21:6789/0 147 : audit [INF] >> from='client.? 192.168.122.22:0/1004669' entity='client.bootstrap-rgw' >> cmd=[{"prefix": "auth get-or-create", "entity": "client.radosgw.ceph2", >> "caps": ["osd", "allow rwx", "mon", "allow rw"]}]: access denied >> >> >> Looking at what ceph-deploy is doing I see that the failing command is: >> >> >> $ sudo ceph --cluster ceph --name client.bootstrap-rgw --keyring >> /var/lib/ceph/bootstrap-rgw/ceph.keyring auth get-or-create >> client.radosgw.ceph2 osd 'allow rwx' mon 'allow rw' -o >> /var/lib/ceph/radosgw/ceph-radosgw.ceph2/keyring >> >> >> This is failing whereas the equivalent command with radosgw replaced by >> rgw works. Looking at what privileges client.bootstrap-rgw has I see: >> >> $ sudo ceph auth get client.bootstrap-rgw >> [client.bootstrap-rgw] >> key = XXXXXXXXXXXXXXXXXX== >> caps mon = "allow profile bootstrap-rgw" >> >> Ah, so need to look at how the profile bootstrap-rgw is setup: >> >> $ more src/mon/MonCap.cc >> ... >> if (profile == "bootstrap-rgw") { >> profile_grants.push_back(MonCapGrant("mon", MON_CAP_R)); // read >> monmap >> profile_grants.push_back(MonCapGrant("osd", MON_CAP_R)); // read >> osdmap >> profile_grants.push_back(MonCapGrant("mon getmap")); >> profile_grants.push_back(MonCapGrant("auth get-or-create")); // >> FIXME: this can expose other mds keys >> profile_grants.back().command_args["entity"] = StringConstraint("", >> "client.rgw."); <====== >> profile_grants.back().command_args["caps_mon"] = >> StringConstraint("allow rw", ""); >> profile_grants.back().command_args["caps_osd"] = >> StringConstraint("allow rwx", ""); >> } >> ... >> >> I see - the ["entity"] check requires the name to match "client.rgw.". >> Double checking back on the deployed host (ceph1), I see that: >> >> >> $ ceph --cluster ceph --name client.bootstrap-rgw --keyring >> /var/lib/ceph/bootstrap-rgw/ceph.keyring auth get-or-create >> client.rgw.ceph3 osd 'allow rwx' mon 'allow rw' >> [client.rgw.ceph3] >> key = AQCrNytVp9bqFBAAgTJVQDVQFJqiSkFqD68XsQ== >> $ ceph --cluster ceph --name client.bootstrap-rgw --keyring >> /var/lib/ceph/bootstrap-rgw/ceph.keyring auth get-or-create >> client.radosgw.ceph3 osd 'allow rwx' mon 'allow rw' >> Error EACCES: access denied >> >> Ok so that is definitely the problem. >> >> Is this restriction intended? If so we really should make ceph-deploy >> aware of this...as it is a bit of a foot gun (and not as easy as I made >> it seem above to figure out *why* the deploy didn't work). >> >> Cheers >> >> Mark >> >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > > ping? > > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html