[ceph-users] Luminous radosgw S3/Keystone integration issues
Hi, We are operating a Luminous 12.2.2 radosgw, with the S3 Keystone authentication enabled. Some customers are uploading millions of objects per bucket at once, therefore the radosgw is doing millions of s3tokens POST requests to the Keystone. All those s3tokens requests to Keystone are the same (same customer, same EC2 credentials). But because there is no cache in radosgw for the EC2 credentials, every incoming S3 operation generates a call to the external auth Keystone. It can generate hundreds of s3tokens requests per second to Keystone. We had already this problem with Jewel, but we implemented a workaround. The EC2 credentials of the customer were added directly in the local auth engine of radosgw. So for this particular heavy user, the radosgw local authentication was checked first, and no external auth request to Keystone was necessary. But the default behavior for the S3 authentication have change in Luminous. In Luminous, if you enable the S3 Keystone authentication, every incoming S3 operation will first check for anonymous authentication, then external authentication (Keystone and/or LDAP), and only then local authentication. See https://github.com/ceph/ceph/blob/master/src/rgw/rgw_auth_s3.h#L113-L141 Is there a way to get the old authentication behavior (anonymous -> local -> external) to work again? Or is it possible to implement a caching mechanism (similar to the Token cache) for the EC2 credentials? Cheers, Valery -- SWITCH Valéry Tschopp, Software Engineer Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland email: valery.tsch...@switch.ch phone: +41 44 268 1544 30 years of pioneering the Swiss Internet. Celebrate with us at https://swit.ch/30years ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] how to use create an new radosgw user using RESTful API?
You have to check the admin Ops API documentation: http://docs.ceph.com/docs/master/radosgw/adminops/ Cheers, Valery On 18/01/18 12:32 , 13605702...@163.com wrote: hi: is there a way to create radosgw user using RESTful API ? i'm using Jewel. thanks 13605702...@163.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- SWITCH Valéry Tschopp, Software Engineer Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland email: valery.tsch...@switch.ch phone: +41 44 268 1544 30 years of pioneering the Swiss Internet. Celebrate with us at https://swit.ch/30years ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Can not delete snapshot with "ghost" children
Hi, We've a problem to delete a snapshot. There was a child image of the snapshot, but the child image was flatten. And now the snapshot still "think" it has children, and can not be deleted. Snapshot and children: $ rbd snap ls volumes/volume-49ccf5a6-4c17-434a-a087-f04acef978ef SNAPID NAME SIZE 94183 snapshot-376e23d6-e723-4dbb-b558-174b275244b5 40960 MB $ rbd children volumes/volume-49ccf5a6-4c17-434a-a087-f04acef978ef@snapshot-376e23d6-e723-4dbb-b558-174b275244b5 volumes/volume-a86350ad-2d4e-4863-bff3-67304b4b7b3c Child image (was flatten, and now is without parent): $ rbd info volumes/volume-a86350ad-2d4e-4863-bff3-67304b4b7b3c rbd image 'volume-a86350ad-2d4e-4863-bff3-67304b4b7b3c': size 40960 MB in 10240 objects order 22 (4096 kB objects) block_name_prefix: rbd_data.1d4f6a3cd198f8 format: 2 features: layering, exclusive-lock, object-map flags: When I try to delete the snapshot, ceph tells me that the snapshot is protected. And when I try to unprotect it, it fails telling me there is still a child!?!? $ rbd snap rm volumes/volume-49ccf5a6-4c17-434a-a087-f04acef978ef@snapshot-376e23d6-e723-4dbb-b558-174b275244b5 rbd: snapshot 'snapshot-376e23d6-e723-4dbb-b558-174b275244b5' is protected from removal. 2017-11-30 11:09:31.054416 7fe80e8f9100 -1 librbd::Operations: snapshot is protected $ rbd snap unprotect volumes/volume-49ccf5a6-4c17-434a-a087-f04acef978ef@snapshot-376e23d6-e723-4dbb-b558-174b275244b5 2017-11-30 11:09:56.899548 7fc6432cd700 -1 librbd::SnapshotUnprotectRequest: cannot unprotect: at least 1 child(ren) [1d4f6a3cd198f8] in pool 'volumes' 2017-11-30 11:09:56.899578 7fc6432cd700 -1 librbd::SnapshotUnprotectRequest: encountered error: (16) Device or resource busy 2017-11-30 11:09:56.899588 7fc6432cd700 -1 librbd::SnapshotUnprotectRequest: 0xab0d918540 should_complete_error: ret_val=-16 2017-11-30 11:09:56.902702 7fc6432cd700 -1 librbd::SnapshotUnprotectRequest: 0xab0d918540 should_complete_error: ret_val=-16 rbd: unprotecting snap failed: (16) Device or resource busy How can we solve this issue? Cheers, Valery -- SWITCH Valéry Tschopp, Software Engineer Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland email: valery.tsch...@switch.ch phone: +41 44 268 1544 30 years of pioneering the Swiss Internet. Celebrate with us at https://swit.ch/30years smime.p7s Description: S/MIME Cryptographic Signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] radosgw Jewel: AWS4 signature and keystone integration?
Hi, We have an Jewel radosgw, with the s3 authentication integration with keystone enabled (rgw_s3_auth_use_keystone = true) The s3 client (s3cmd) uses AWS4 signature, but the authentication to radosgw always fails: "ERROR: S3 error: 403 (InvalidAccessKeyId)" Here below is the radosgw log file and, unlike validation for AWS2 signature, there is NO callout to keystone to resolve the access_key -> tenant_id for the given user... Does radosgw really support AWS4 signature and keystone integration? Cheers, Valery 2017-02-22 15:50:09.325582 7f0ec67fc700 1 == starting new request req=0x7f0ec67f67d0 = 2017-02-22 15:50:09.325623 7f0ec67fc700 2 req 65:0.41::GET /::initializing for trans_id = tx00041-0058adb331-145bda7-default 2017-02-22 15:50:09.325643 7f0ec67fc700 10 rgw api priority: s3=5 s3website=4 2017-02-22 15:50:09.325646 7f0ec67fc700 10 host=valery-test.os.s2.scloud.switch.ch 2017-02-22 15:50:09.325655 7f0ec67fc700 20 subdomain=valery-test domain=os.s2.scloud.switch.ch in_hosted_domain=1 in_hosted_domain_s3website=0 2017-02-22 15:50:09.325663 7f0ec67fc700 20 final domain/bucket subdomain=valery-test domain=os.s2.scloud.switch.ch in_hosted_domain=1 in_hosted_domain_s3website=0 s->info.domain=os.s2.scloud.switch.ch s->info.request_uri=/valery-test/ 2017-02-22 15:50:09.325685 7f0ec67fc700 10 meta>> HTTP_X_AMZ_CONTENT_SHA256 2017-02-22 15:50:09.325696 7f0ec67fc700 10 meta>> HTTP_X_AMZ_DATE 2017-02-22 15:50:09.325702 7f0ec67fc700 10 x>> x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 2017-02-22 15:50:09.325704 7f0ec67fc700 10 x>> x-amz-date:20170222T155139Z 2017-02-22 15:50:09.325747 7f0ec67fc700 20 get_handler handler=25RGWHandler_REST_Bucket_S3 2017-02-22 15:50:09.325756 7f0ec67fc700 10 handler=25RGWHandler_REST_Bucket_S3 2017-02-22 15:50:09.325759 7f0ec67fc700 2 req 65:0.000178:s3:GET /::getting op 0 2017-02-22 15:50:09.325778 7f0ec67fc700 10 op=25RGWListBucket_ObjStore_S3 2017-02-22 15:50:09.325782 7f0ec67fc700 2 req 65:0.000200:s3:GET /:list_bucket:authorizing 2017-02-22 15:50:09.325813 7f0ec67fc700 10 v4 signature format = c535db1ceb4ed3c7eb68f2f9a35ad61849631a1bb6391dcf314f5aa7f717b3fd 2017-02-22 15:50:09.325827 7f0ec67fc700 10 v4 credential format = 0213b30621e74120b73d11a5e99240f9/20170222/US/s3/aws4_request 2017-02-22 15:50:09.325831 7f0ec67fc700 10 access key id = 0213b30621e74120b73d11a5e99240f9 2017-02-22 15:50:09.325833 7f0ec67fc700 10 credential scope = 20170222/US/s3/aws4_request 2017-02-22 15:50:09.325874 7f0ec67fc700 20 get_system_obj_state: rctx=0x7f0ec67f54c0 obj=.users:0213b30621e74120b73d11a5e99240f9 state=0x7f0e9401fc48 s->prefetch_data=0 2017-02-22 15:50:09.325896 7f0ec67fc700 10 cache get: name=.users+0213b30621e74120b73d11a5e99240f9 : type miss (requested=6, cached=0) 2017-02-22 15:50:09.327179 7f0ec67fc700 10 cache put: name=.users+0213b30621e74120b73d11a5e99240f9 info.flags=0 2017-02-22 15:50:09.327205 7f0ec67fc700 10 moving .users+0213b30621e74120b73d11a5e99240f9 to cache LRU end 2017-02-22 15:50:09.327222 7f0ec67fc700 10 error reading user info, uid=0213b30621e74120b73d11a5e99240f9 can't authenticate 2017-02-22 15:50:09.327225 7f0ec67fc700 10 failed to authorize request 2017-02-22 15:50:09.327228 7f0ec67fc700 20 handler->ERRORHANDLER: err_no=-2028 new_err_no=-2028 2017-02-22 15:50:09.327425 7f0ec67fc700 2 req 65:0.001843:s3:GET /:list_bucket:op status=0 2017-02-22 15:50:09.327440 7f0ec67fc700 2 req 65:0.001859:s3:GET /:list_bucket:http status=403 2017-02-22 15:50:09.327451 7f0ec67fc700 1 == req done req=0x7f0ec67f67d0 op status=0 http_status=403 ====== -- SWITCH -- Valery Tschopp, Software Engineer, Peta Solutions Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland email: valery.tsch...@switch.ch phone: +41 44 268 1544 smime.p7s Description: S/MIME Cryptographic Signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] radosgw-admin bucket link: empty bucket instance id
Thanks Casey,
It works using both --bucket XXX_name --bucket-id XXX_id. For both
radosgw Hammer and Jewel version.
But the documentation for the REST admin operations is completely wrong:
http://docs.ceph.com/docs/master/radosgw/adminops/#link-bucket
Cheers,
Valery
On 21/02/17 20:26 , Casey Bodley wrote:
When it complains about a missing bucket instance id, that's what it's
expecting to get from the --bucket-id argument. That's the "id" field
shown in bucket stats. Try this?
$ radosgw-admin bucket link --bucket=XXX --bucket-id=YYY --uid=ZZZ
Casey
On 02/21/2017 08:30 AM, Valery Tschopp wrote:
Hi,
I've the same problem about 'radosgw-admin bucket link --bucket XXX
--uid YYY', but with a Jewel radosgw
The admin rest API [1] do not work either :(
Any idea?
[1]: http://docs.ceph.com/docs/master/radosgw/adminops/#link-bucket
On 28/01/16 17:03 , Wido den Hollander wrote:
Hi,
I'm trying to link a bucket to a new user and this is failing for me.
The Ceph version is 0.94.5 (Hammer).
The bucket is called 'packer' and I can verify that it exists:
$ radosgw-admin bucket stats --bucket packer
{
"bucket": "packer",
"pool": ".rgw.buckets",
"index_pool": ".rgw.buckets",
"id": "ams02.5862567.3564",
"marker": "ams02.5862567.3564",
"owner": "X_beta",
"ver": "0#21975",
"master_ver": "0#0",
"mtime": "2015-08-04 12:31:06.00",
"max_marker": "0#",
"usage": {
"rgw.main": {
"size_kb": 10737764,
"size_kb_actual": 10737836,
"num_objects": 27
},
"rgw.multimeta": {
"size_kb": 0,
"size_kb_actual": 0,
"num_objects": 0
}
},
"bucket_quota": {
"enabled": false,
"max_size_kb": -1,
"max_objects": -1
}
}
Now when I try to link this bucket it fails:
$ radosgw-admin bucket link --bucket packer --uid
"failure: (22) Invalid argument: empty bucket instance id"
It seems like this is a bug in the radosgw-admin tool where it doesn't
parse the --bucket argument properly.
Any ideas?
Wido
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
SWITCH
--
Valery Tschopp, Software Engineer, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
email: valery.tsch...@switch.ch phone: +41 44 268 1544
smime.p7s
Description: S/MIME Cryptographic Signature
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] radosgw-admin bucket link: empty bucket instance id
Hi,
I've the same problem about 'radosgw-admin bucket link --bucket XXX
--uid YYY', but with a Jewel radosgw
The admin rest API [1] do not work either :(
Any idea?
[1]: http://docs.ceph.com/docs/master/radosgw/adminops/#link-bucket
On 28/01/16 17:03 , Wido den Hollander wrote:
Hi,
I'm trying to link a bucket to a new user and this is failing for me.
The Ceph version is 0.94.5 (Hammer).
The bucket is called 'packer' and I can verify that it exists:
$ radosgw-admin bucket stats --bucket packer
{
"bucket": "packer",
"pool": ".rgw.buckets",
"index_pool": ".rgw.buckets",
"id": "ams02.5862567.3564",
"marker": "ams02.5862567.3564",
"owner": "X_beta",
"ver": "0#21975",
"master_ver": "0#0",
"mtime": "2015-08-04 12:31:06.00",
"max_marker": "0#",
"usage": {
"rgw.main": {
"size_kb": 10737764,
"size_kb_actual": 10737836,
"num_objects": 27
},
"rgw.multimeta": {
"size_kb": 0,
"size_kb_actual": 0,
"num_objects": 0
}
},
"bucket_quota": {
"enabled": false,
"max_size_kb": -1,
"max_objects": -1
}
}
Now when I try to link this bucket it fails:
$ radosgw-admin bucket link --bucket packer --uid
"failure: (22) Invalid argument: empty bucket instance id"
It seems like this is a bug in the radosgw-admin tool where it doesn't
parse the --bucket argument properly.
Any ideas?
Wido
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
SWITCH
--
Valery Tschopp, Software Engineer, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
email: valery.tsch...@switch.ch phone: +41 44 268 1544
smime.p7s
Description: S/MIME Cryptographic Signature
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] bucket owner vs S3 ACL?
Hi Florent,
Yes this make sense now.
Thanks a lot
V.
On 01/07/15 20:19 , Florent MONTHEL wrote:
Hi Valery,
With the old account did you try to give FULL access to the new one user ID ?
Process should be :
From OLD account add FULL access to NEW account (S3 ACL with CloudBerry for
example)
With radosgw admin update link from OLD account to NEW account (link allow user
to see bucket with bucket list command)
From NEW account remove FULL access to old account (S3 ACL with CloudBerry for
example)
Thanks
On Jun 29, 2015, at 11:46 AM, Valery Tschopp valery.tsch...@switch.ch wrote:
Hi guys,
We use the radosgw (v0.80.9) with the Openstack Keystone integration.
One project have been deleted, so now I have to transfer the ownership of all
the buckets to another user/project.
Using radosgw-admin I have changed the owner:
radosgw-admin bucket link --uid NEW_USER_ID --bucket BUCKET_NAME
And the owner have been update:
radosgw-admin bucket stats --bucket BUCKET_NAME
{ bucket: BUCKET_NAME,
pool: .rgw.buckets,
index_pool: .rgw.buckets.index,
id: default.4063334.17,
marker: default.4063334.17,
owner: NEW_USER_ID,
ver: 66301,
master_ver: 0,
mtime: 1435583681,
max_marker: ,
usage: { rgw.main: { size_kb: 189433890,
size_kb_actual: 189473684,
num_objects: 19043},
rgw.multimeta: { size_kb: 0,
size_kb_actual: 0,
num_objects: 0}},
bucket_quota: { enabled: false,
max_size_kb: -1,
max_objects: -1}
}
But the S3 ACL of this bucket is still referencing the old user/project (from
radosgw.log) when I try to access it with the new owner:
2015-06-29 17:08:33.236265 7f40d8a76700 15 Read AccessControlPolicyAccessControlPolicy
xmlns=http://s3.amazonaws.com/doc/2006-03-01/;OwnerIDOLD_USER_ID/IDDisplayNameOLD_PROJECT_NAME/DisplayName/OwnerAccessControlListGrantGrantee
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
xsi:type=CanonicalUserIDOLD_USER_ID/IDDisplayNameOLD_PROJECT_NAME/DisplayName/GranteePermissionFULL_CONTROL/Permission/Grant/AccessControlList/AccessControlPolicy
Therefore I get a 403, because the S3 ACL still enforce the old owner, not the
new one.
How can I update these S3 ACL, and fully transfer the ownership to the new
owner/project???
Cheers,
Valery
--
SWITCH
--
Valery Tschopp, Software Engineer, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
email: valery.tsch...@switch.ch phone: +41 44 268 1544
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
SWITCH
--
Valery Tschopp, Software Engineer, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
email: valery.tsch...@switch.ch phone: +41 44 268 1544
smime.p7s
Description: S/MIME Cryptographic Signature
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] bucket owner vs S3 ACL?
Hi guys,
We use the radosgw (v0.80.9) with the Openstack Keystone integration.
One project have been deleted, so now I have to transfer the ownership
of all the buckets to another user/project.
Using radosgw-admin I have changed the owner:
radosgw-admin bucket link --uid NEW_USER_ID --bucket BUCKET_NAME
And the owner have been update:
radosgw-admin bucket stats --bucket BUCKET_NAME
{ bucket: BUCKET_NAME,
pool: .rgw.buckets,
index_pool: .rgw.buckets.index,
id: default.4063334.17,
marker: default.4063334.17,
owner: NEW_USER_ID,
ver: 66301,
master_ver: 0,
mtime: 1435583681,
max_marker: ,
usage: { rgw.main: { size_kb: 189433890,
size_kb_actual: 189473684,
num_objects: 19043},
rgw.multimeta: { size_kb: 0,
size_kb_actual: 0,
num_objects: 0}},
bucket_quota: { enabled: false,
max_size_kb: -1,
max_objects: -1}
}
But the S3 ACL of this bucket is still referencing the old user/project
(from radosgw.log) when I try to access it with the new owner:
2015-06-29 17:08:33.236265 7f40d8a76700 15 Read
AccessControlPolicyAccessControlPolicy
xmlns=http://s3.amazonaws.com/doc/2006-03-01/;OwnerIDOLD_USER_ID/IDDisplayNameOLD_PROJECT_NAME/DisplayName/OwnerAccessControlListGrantGrantee
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
xsi:type=CanonicalUserIDOLD_USER_ID/IDDisplayNameOLD_PROJECT_NAME/DisplayName/GranteePermissionFULL_CONTROL/Permission/Grant/AccessControlList/AccessControlPolicy
Therefore I get a 403, because the S3 ACL still enforce the old owner,
not the new one.
How can I update these S3 ACL, and fully transfer the ownership to the
new owner/project???
Cheers,
Valery
--
SWITCH
--
Valery Tschopp, Software Engineer, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
email: valery.tsch...@switch.ch phone: +41 44 268 1544
smime.p7s
Description: S/MIME Cryptographic Signature
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] v0.80.9 Firefly released
Hi Loic, Nope, only the versions from 0.81-trusty to 0.93-1trusty are available in http://ceph.com/debian-testing/pool/main/c/ceph/ But the firefly deb source packages for 0.80.9-1trusty is not available :( Cheers, Valery On 11/03/15 14:11 , Loic Dachary wrote: Hi Valery, They should be here http://ceph.com/debian-testing/ Cheers On 11/03/2015 10:07, Valery Tschopp wrote: Where can I find the debian trusty source package for v0.80.9? Cheers, Valery On 10/03/15 20:34 , Sage Weil wrote: This is a bugfix release for firefly. It fixes a performance regression in librbd, an important CRUSH misbehavior (see below), and several RGW bugs. We have also backported support for flock/fcntl locks to ceph-fuse and libcephfs. We recommend that all Firefly users upgrade. For more detailed information, see http://docs.ceph.com/docs/master/_downloads/v0.80.9.txt Adjusting CRUSH maps * This point release fixes several issues with CRUSH that trigger excessive data migration when adjusting OSD weights. These are most obvious when a very small weight change (e.g., a change from 0 to .01) triggers a large amount of movement, but the same set of bugs can also lead to excessive (though less noticeable) movement in other cases. However, because the bug may already have affected your cluster, fixing it may trigger movement *back* to the more correct location. For this reason, you must manually opt-in to the fixed behavior. In order to set the new tunable to correct the behavior:: ceph osd crush set-tunable straw_calc_version 1 Note that this change will have no immediate effect. However, from this point forward, any 'straw' bucket in your CRUSH map that is adjusted will get non-buggy internal weights, and that transition may trigger some rebalancing. You can estimate how much rebalancing will eventually be necessary on your cluster with:: ceph osd getcrushmap -o /tmp/cm crushtool -i /tmp/cm --num-rep 3 --test --show-mappings /tmp/a 21 crushtool -i /tmp/cm --set-straw-calc-version 1 -o /tmp/cm2 crushtool -i /tmp/cm2 --reweight -o /tmp/cm2 crushtool -i /tmp/cm2 --num-rep 3 --test --show-mappings /tmp/b 21 wc -l /tmp/a # num total mappings diff -u /tmp/a /tmp/b | grep -c ^+# num changed mappings Divide the total number of lines in /tmp/a with the number of lines changed. We've found that most clusters are under 10%. You can force all of this rebalancing to happen at once with:: ceph osd crush reweight-all Otherwise, it will happen at some unknown point in the future when CRUSH weights are next adjusted. Notable Changes --- * ceph-fuse: flock, fcntl lock support (Yan, Zheng, Greg Farnum) * crush: fix straw bucket weight calculation, add straw_calc_version tunable (#10095 Sage Weil) * crush: fix tree bucket (Rongzu Zhu) * crush: fix underflow of tree weights (Loic Dachary, Sage Weil) * crushtool: add --reweight (Sage Weil) * librbd: complete pending operations before losing image (#10299 Jason Dillaman) * librbd: fix read caching performance regression (#9854 Jason Dillaman) * librbd: gracefully handle deleted/renamed pools (#10270 Jason Dillaman) * mon: fix dump of chooseleaf_vary_r tunable (Sage Weil) * osd: fix PG ref leak in snaptrimmer on peering (#10421 Kefu Chai) * osd: handle no-op write with snapshot (#10262 Sage Weil) * radosgw-admin: create subuser when creating user (#10103 Yehuda Sadeh) * rgw: change multipart uplaod id magic (#10271 Georgio Dimitrakakis, Yehuda Sadeh) * rgw: don't overwrite bucket/object owner when setting ACLs (#10978 Yehuda Sadeh) * rgw: enable IPv6 for embedded civetweb (#10965 Yehuda Sadeh) * rgw: fix partial swift GET (#10553 Yehuda Sadeh) * rgw: fix quota disable (#9907 Dong Lei) * rgw: index swift keys appropriately (#10471 Hemant Burman, Yehuda Sadeh) * rgw: make setattrs update bucket index (#5595 Yehuda Sadeh) * rgw: pass civetweb configurables (#10907 Yehuda Sadeh) * rgw: remove swift user manifest (DLO) hash calculation (#9973 Yehuda Sadeh) * rgw: return correct len for 0-len objects (#9877 Yehuda Sadeh) * rgw: S3 object copy content-type fix (#9478 Yehuda Sadeh) * rgw: send ETag on S3 object copy (#9479 Yehuda Sadeh) * rgw: send HTTP status reason explicitly in fastcgi (Yehuda Sadeh) * rgw: set ulimit -n from sysvinit (el6) init script (#9587 Sage Weil) * rgw: update swift subuser permission masks when authenticating (#9918 Yehuda Sadeh) * rgw: URL decode query params correctly (#10271 Georgio Dimitrakakis, Yehuda Sadeh) * rgw: use attrs when reading object attrs (#10307 Yehuda Sadeh) * rgw: use \r\n for http headers (#9254 Benedikt Fraunhofer, Yehuda Sadeh) Getting Ceph * Git at git://github.com/ceph/ceph.git * Tarball at http://ceph.com/download/ceph-0.80.9.tar.gz * For packages, see http://ceph.com
Re: [ceph-users] v0.80.9 Firefly released
Where can I find the debian trusty source package for v0.80.9? Cheers, Valery On 10/03/15 20:34 , Sage Weil wrote: This is a bugfix release for firefly. It fixes a performance regression in librbd, an important CRUSH misbehavior (see below), and several RGW bugs. We have also backported support for flock/fcntl locks to ceph-fuse and libcephfs. We recommend that all Firefly users upgrade. For more detailed information, see http://docs.ceph.com/docs/master/_downloads/v0.80.9.txt Adjusting CRUSH maps * This point release fixes several issues with CRUSH that trigger excessive data migration when adjusting OSD weights. These are most obvious when a very small weight change (e.g., a change from 0 to .01) triggers a large amount of movement, but the same set of bugs can also lead to excessive (though less noticeable) movement in other cases. However, because the bug may already have affected your cluster, fixing it may trigger movement *back* to the more correct location. For this reason, you must manually opt-in to the fixed behavior. In order to set the new tunable to correct the behavior:: ceph osd crush set-tunable straw_calc_version 1 Note that this change will have no immediate effect. However, from this point forward, any 'straw' bucket in your CRUSH map that is adjusted will get non-buggy internal weights, and that transition may trigger some rebalancing. You can estimate how much rebalancing will eventually be necessary on your cluster with:: ceph osd getcrushmap -o /tmp/cm crushtool -i /tmp/cm --num-rep 3 --test --show-mappings /tmp/a 21 crushtool -i /tmp/cm --set-straw-calc-version 1 -o /tmp/cm2 crushtool -i /tmp/cm2 --reweight -o /tmp/cm2 crushtool -i /tmp/cm2 --num-rep 3 --test --show-mappings /tmp/b 21 wc -l /tmp/a # num total mappings diff -u /tmp/a /tmp/b | grep -c ^+# num changed mappings Divide the total number of lines in /tmp/a with the number of lines changed. We've found that most clusters are under 10%. You can force all of this rebalancing to happen at once with:: ceph osd crush reweight-all Otherwise, it will happen at some unknown point in the future when CRUSH weights are next adjusted. Notable Changes --- * ceph-fuse: flock, fcntl lock support (Yan, Zheng, Greg Farnum) * crush: fix straw bucket weight calculation, add straw_calc_version tunable (#10095 Sage Weil) * crush: fix tree bucket (Rongzu Zhu) * crush: fix underflow of tree weights (Loic Dachary, Sage Weil) * crushtool: add --reweight (Sage Weil) * librbd: complete pending operations before losing image (#10299 Jason Dillaman) * librbd: fix read caching performance regression (#9854 Jason Dillaman) * librbd: gracefully handle deleted/renamed pools (#10270 Jason Dillaman) * mon: fix dump of chooseleaf_vary_r tunable (Sage Weil) * osd: fix PG ref leak in snaptrimmer on peering (#10421 Kefu Chai) * osd: handle no-op write with snapshot (#10262 Sage Weil) * radosgw-admin: create subuser when creating user (#10103 Yehuda Sadeh) * rgw: change multipart uplaod id magic (#10271 Georgio Dimitrakakis, Yehuda Sadeh) * rgw: don't overwrite bucket/object owner when setting ACLs (#10978 Yehuda Sadeh) * rgw: enable IPv6 for embedded civetweb (#10965 Yehuda Sadeh) * rgw: fix partial swift GET (#10553 Yehuda Sadeh) * rgw: fix quota disable (#9907 Dong Lei) * rgw: index swift keys appropriately (#10471 Hemant Burman, Yehuda Sadeh) * rgw: make setattrs update bucket index (#5595 Yehuda Sadeh) * rgw: pass civetweb configurables (#10907 Yehuda Sadeh) * rgw: remove swift user manifest (DLO) hash calculation (#9973 Yehuda Sadeh) * rgw: return correct len for 0-len objects (#9877 Yehuda Sadeh) * rgw: S3 object copy content-type fix (#9478 Yehuda Sadeh) * rgw: send ETag on S3 object copy (#9479 Yehuda Sadeh) * rgw: send HTTP status reason explicitly in fastcgi (Yehuda Sadeh) * rgw: set ulimit -n from sysvinit (el6) init script (#9587 Sage Weil) * rgw: update swift subuser permission masks when authenticating (#9918 Yehuda Sadeh) * rgw: URL decode query params correctly (#10271 Georgio Dimitrakakis, Yehuda Sadeh) * rgw: use attrs when reading object attrs (#10307 Yehuda Sadeh) * rgw: use \r\n for http headers (#9254 Benedikt Fraunhofer, Yehuda Sadeh) Getting Ceph * Git at git://github.com/ceph/ceph.git * Tarball at http://ceph.com/download/ceph-0.80.9.tar.gz * For packages, see http://ceph.com/docs/master/install/get-packages * For ceph-deploy, see http://ceph.com/docs/master/install/install-ceph-deploy ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- SWITCH -- Valery Tschopp, Software Engineer, Peta Solutions Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland email: valery.tsch
[ceph-users] ceph v0.80.9 debian source packages???
Hi guys, The last trusty version 0.80.9 have been pushed in the deb http://ceph.com/debian-firefly/ trusty main repository yesterday. The last packages have the version 0.80.9-1trusty, but I can not find the corresponding source packages in http://gitbuilder.ceph.com/ceph-deb-trusty-x86_64-basic/ref/v0.80.9 Where can I find the correct ceph_0.80.9-1trusty.dsc and ceph_0.80.9-1trusty.tar.gz source packages? Regards, Valery -- SWITCH -- Valery Tschopp, Software Engineer, Peta Solutions Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland email: valery.tsch...@switch.ch phone: +41 44 268 1544 smime.p7s Description: S/MIME Cryptographic Signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] radosgw + s3 + keystone + Browser-Based POST problem
Hi guys, We have integrated our radosgw (v0.80.7) with our OpenStack Keystone server (icehouse) successfully. The normal S3 operations can be executed with the Keystone user's EC2 credentials (EC2_ACCESS_KEY, EC2_SECRET_KEY). The radosgw correctly handles these user credentials, ask keystone to validate them, and the resulting objects belong to the Keystone tenant/project or the user (user is member of the tenant/project). But for the Browser-based upload POST [1] it doesn't work! The user is not correctly resolved, and the radosgw returns a 403 code! It looks like the s3 keystone integration doesn't work correctly when a S3 browser-based upload POST is used. See the attached log file (radosgw.log), you can clearly see the user lookup failing, and the status being set to 403: 2015-01-29 15:11:30.151157 7f25616fa700 0 User lookup failed! 2015-01-29 15:11:30.151171 7f25616fa700 15 Read RGWCORSConfigurationCORSConfigurationCORSRuleAllowedMethodPOST/AllowedMethodAllowedOriginhttps://staging.tube.switch.ch/AllowedOriginAllowedHeader*/AllowedHeader/CORSRule/CORSConfiguration 2015-01-29 15:11:30.151184 7f25616fa700 10 Method POST is supported 2015-01-29 15:11:30.151195 7f25616fa700 2 req 1123:0.013204:s3:POST /:post_obj:http status=403 Is this a bug? Or did we miss something else? Cheers, Valery [1] http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingHTTPPOST.html -- SWITCH -- Valery Tschopp, Software Engineer, Peta Solutions Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland email: valery.tsch...@switch.ch phone: +41 44 268 1544 2015-01-29 15:11:30.130054 7f2634cef700 20 enqueued request req=0x7f26040838d0 2015-01-29 15:11:30.130084 7f2634cef700 20 RGWWQ: 2015-01-29 15:11:30.130086 7f2634cef700 20 req: 0x7f26040838d0 2015-01-29 15:11:30.130108 7f2634cef700 10 allocated request req=0x7f26040c58d0 2015-01-29 15:11:30.130200 7f2454ce1700 20 dequeued request req=0x7f26040838d0 2015-01-29 15:11:30.130208 7f2454ce1700 20 RGWWQ: empty 2015-01-29 15:11:30.130303 7f2454ce1700 20 CONTEXT_DOCUMENT_ROOT=/var/www 2015-01-29 15:11:30.130305 7f2454ce1700 20 CONTEXT_PREFIX= 2015-01-29 15:11:30.130306 7f2454ce1700 20 DOCUMENT_ROOT=/var/www 2015-01-29 15:11:30.130307 7f2454ce1700 20 FCGI_ROLE=RESPONDER 2015-01-29 15:11:30.130308 7f2454ce1700 20 GATEWAY_INTERFACE=CGI/1.1 2015-01-29 15:11:30.130308 7f2454ce1700 20 HTTP_ACCEPT=*/* 2015-01-29 15:11:30.130309 7f2454ce1700 20 HTTP_ACCEPT_ENCODING=gzip, deflate, sdch 2015-01-29 15:11:30.130310 7f2454ce1700 20 HTTP_ACCEPT_LANGUAGE=en-US,en;q=0.8,it;q=0.6 2015-01-29 15:11:30.130311 7f2454ce1700 20 HTTP_ACCESS_CONTROL_REQUEST_HEADERS=content-type 2015-01-29 15:11:30.130312 7f2454ce1700 20 HTTP_ACCESS_CONTROL_REQUEST_METHOD=POST 2015-01-29 15:11:30.130312 7f2454ce1700 20 HTTP_AUTHORIZATION= 2015-01-29 15:11:30.130313 7f2454ce1700 20 HTTP_CACHE_CONTROL=no-cache 2015-01-29 15:11:30.130314 7f2454ce1700 20 HTTP_CONNECTION=keep-alive 2015-01-29 15:11:30.130314 7f2454ce1700 20 HTTP_HOST=switch-original-staging.os.zhdk.cloud.switch.ch 2015-01-29 15:11:30.130315 7f2454ce1700 20 HTTP_ORIGIN=https://staging.tube.switch.ch 2015-01-29 15:11:30.130316 7f2454ce1700 20 HTTP_PRAGMA=no-cache 2015-01-29 15:11:30.130317 7f2454ce1700 20 HTTP_REFERER=https://staging.tube.switch.ch/channels/04238519/videos 2015-01-29 15:11:30.130318 7f2454ce1700 20 HTTP_USER_AGENT=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.93 Safari/537.36 2015-01-29 15:11:30.130320 7f2454ce1700 20 HTTPS=on 2015-01-29 15:11:30.130321 7f2454ce1700 20 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 2015-01-29 15:11:30.130322 7f2454ce1700 20 QUERY_STRING= 2015-01-29 15:11:30.130322 7f2454ce1700 20 REMOTE_ADDR=130.59.17.201 2015-01-29 15:11:30.130323 7f2454ce1700 20 REMOTE_PORT=53901 2015-01-29 15:11:30.130324 7f2454ce1700 20 REQUEST_METHOD=OPTIONS 2015-01-29 15:11:30.130325 7f2454ce1700 20 REQUEST_SCHEME=https 2015-01-29 15:11:30.130326 7f2454ce1700 20 REQUEST_URI=/ 2015-01-29 15:11:30.130327 7f2454ce1700 20 SCRIPT_FILENAME=/var/www/radosgw.fcgi 2015-01-29 15:11:30.130328 7f2454ce1700 20 SCRIPT_NAME=/ 2015-01-29 15:11:30.130329 7f2454ce1700 20 SCRIPT_URI=https://switch-original-staging.os.zhdk.cloud.switch.ch/ 2015-01-29 15:11:30.130330 7f2454ce1700 20 SCRIPT_URL=/ 2015-01-29 15:11:30.130331 7f2454ce1700 20 SERVER_ADDR=86.119.32.13 2015-01-29 15:11:30.130332 7f2454ce1700 20 SERVER_ADMIN=cl...@switch.ch 2015-01-29 15:11:30.130333 7f2454ce1700 20 SERVER_NAME=switch-original-staging.os.zhdk.cloud.switch.ch 2015-01-29 15:11:30.130334 7f2454ce1700 20 SERVER_PORT=443 2015-01-29 15:11:30.130334 7f2454ce1700 20 SERVER_PROTOCOL=HTTP/1.1 2015-01-29 15:11:30.130335 7f2454ce1700 20 SERVER_SIGNATURE= 2015-01-29 15:11:30.130350 7f2454ce1700 20 SERVER_SOFTWARE=Apache/2.4.7 (Ubuntu) 2015-01-29 15:11:30.130351 7f2454ce1700 20 SSL_TLS_SNI=switch-original-staging.os.zhdk.cloud.switch.ch 2015-01-29 15:11:30.130352
[ceph-users] RadosGW browser-based uploads using POST?
Hello, With AWS it is possible to do user browser-based uploads using POST [1]. Is it possible to do with RadosGW. Is the feature supported? Cheers, Valery [1] http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingHTTPPOST.html -- SWITCH -- Valery Tschopp, Software Engineer, Peta Solutions Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland email: valery.tsch...@switch.ch phone: +41 44 268 1544 smime.p7s Description: S/MIME Cryptographic Signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Radosgw usage
Hi Derek,
I've the same problem with the usage. How does your pools list look like
now ('radosgw-admin pools list') ?
Cheers,
Valery
On 23/10/13 22:32 , Derek Yarnell wrote:
Hi,
So the problem was that '.usage' pool was not created. I haven't
traversed the code well enough yet to know where this pool is supposed
to get created but it wasn't even though the option was on. As soon as
I hand created the pool the radosgw started logging usage.
Thanks,
derek
--
SWITCH
--
Valery Tschopp, Software Engineer, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
email: valery.tsch...@switch.ch phone: +41 44 268 1544
smime.p7s
Description: S/MIME Cryptographic Signature
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] radosgw-admin doesn't list user anymore
Hi Derek, Sorry, I added the caps, but didn't restarted the radosgw process. After a restart it works :) Cheers, Valery On 16/10/13 17:20 , Derek Yarnell wrote: On 10/16/13 4:26 AM, Valery Tschopp wrote: Hi Derek, Thanks for your example. I've added caps='metadata=*', but I still have an error and get: send: 'GET /admin/metadata/user?format=json HTTP/1.1\r\nHost: objects.bcc.switch.ch\r\nAccept-Encoding: identity\r\nDate: Wed, 16 Oct 2013 08:09:57 GMT\r\nContent-Length: 0\r\nAuthorization: AWS VC***o=\r\nUser-Agent: Boto/2.12.0 Python/2.7.5 Darwin/12.5.0\r\n\r\n' reply: 'HTTP/1.1 405 Method Not Allowed\r\n' In which version of radosgw is the /admin/metadata REST endpoint available? I currently have 0.67.4. We are using this on ceph-0.67.4. Do you have your gateways logging? -- SWITCH -- Valery Tschopp, Software Engineer, Peta Solutions Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland email: valery.tsch...@switch.ch phone: +41 44 268 1544 smime.p7s Description: S/MIME Cryptographic Signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] radosgw-admin doesn't list user anymore
Hi Derek, Thanks. Is the 'radosgw-admin metadata list user' also available as REST API? Cheers, Valery On 15/10/13 03:21 , Derek Yarnell wrote: root@ineri:~# radosgw-admin user info could not fetch user info: no user info saved Hi Valery, You need to use radosgw-admin metadata list user Thanks, derek -- SWITCH -- Valery Tschopp, Software Engineer, Peta Solutions Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland email: valery.tsch...@switch.ch phone: +41 44 268 1544 smime.p7s Description: S/MIME Cryptographic Signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] radosgw-admin doesn't list user anymore
We upgraded from 0.61.8 to 0.67.4.
The metadata commands works for the users and the buckets:
root@ineri ~$ radosgw-admin metadata list bucket
[
a4mesh,
61a75c04-34a5-11e3-9bea-8f8d15b5cf20,
6e22de72-34a5-11e3-afc4-d3f70b676c52,
...
root@ineri ~$ radosgw-admin metadata list user
[
cloudbroker,
a4mesh,
valery,
...
Cheers,
Valery
On 11/10/13 18:27 , Yehuda Sadeh wrote:
On Fri, Oct 11, 2013 at 7:46 AM, Valery Tschopp
valery.tsch...@switch.ch wrote:
Hi,
Since we upgraded ceph to 0.67.4, the radosgw-admin doesn't list all the
users anymore:
root@ineri:~# radosgw-admin user info
could not fetch user info: no user info saved
But it still work for single user:
root@ineri:~# radosgw-admin user info --uid=valery
{ user_id: valery,
display_name: Valery Tschopp,
email: valery.tsch...@switch.ch,
...
The debug log file is too big for the mailing-list, but here it is on
pastebin: http://pastebin.com/cFypJ2Qd
What version did you upgrade from?
You can try using the following:
$ radosgw-admin metadata list bucket
Thanks,
Yehuda
--
SWITCH
--
Valery Tschopp, Software Engineer, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
email: valery.tsch...@switch.ch phone: +41 44 268 1544
smime.p7s
Description: S/MIME Cryptographic Signature
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] radosgw-admin doesn't list user anymore
Hi,
Since we upgraded ceph to 0.67.4, the radosgw-admin doesn't list all the
users anymore:
root@ineri:~# radosgw-admin user info
could not fetch user info: no user info saved
But it still work for single user:
root@ineri:~# radosgw-admin user info --uid=valery
{ user_id: valery,
display_name: Valery Tschopp,
email: valery.tsch...@switch.ch,
...
The debug log file is too big for the mailing-list, but here it is on
pastebin: http://pastebin.com/cFypJ2Qd
Cheers,
Valery
--
SWITCH
--
Valery Tschopp, Software Engineer, Peta Solutions
Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland
email: valery.tsch...@switch.ch phone: +41 44 268 1544
smime.p7s
Description: S/MIME Cryptographic Signature
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] radosgw-admin users list?
Hi guys, How do I get a list of all users with the radosgw-admin command and/or REST API? # radosgw-admin --version ceph version 0.61.8 (a6fdcca3bddbc9f177e4e2bf0d9cdd85006b028b) Cheers, Valery -- SWITCH -- Valery Tschopp, Software Engineer, Peta Solutions Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland email: valery.tsch...@switch.ch phone: +41 44 268 1544 smime.p7s Description: S/MIME Cryptographic Signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] 'ceph health' Nagios plugin
Hi, For the ones using Nagios to monitor their ceph cluster, I've written a 'ceph health' Nagios plugin: https://github.com/valerytschopp/ceph-nagios-plugins The plugin is written in python, and allow to specify a client user id and keyring to execute the plugin as user 'nagios' or other... Usage - usage: check_ceph_health [-h] [-e EXE] [-c CONF] [-m MONADDRESS] [-i ID] [-k KEYRING] [-d] 'ceph health' nagios plugin. optional arguments: -h, --helpshow this help message and exit -e EXE, --exe EXE ceph executable [/usr/bin/ceph] -c CONF, --conf CONF alternative ceph conf file -m MONADDRESS, --monaddress MONADDRESS ceph monitor address[:port] -i ID, --id IDceph client id -k KEYRING, --keyring KEYRING ceph client keyring file -d, --detail exec 'ceph health detail' Example --- nagios$ ./check_ceph_health --id nagios --keyring client.nagios.keyring HEALTH WARNING: 1 pgs degraded; 1 pgs recovering; 1 pgs stuck unclean; recovery 4448/28924462 degraded (0.015%); 2/9857830 unfound (0.000%); nagios$ echo $? 1 Any comments is welcome Cheers, Valery -- SWITCH -- Valery Tschopp, Software Engineer, Peta Solutions Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland email: valery.tsch...@switch.ch phone: +41 44 268 1544 smime.p7s Description: S/MIME Cryptographic Signature ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
