[ceph-users] [radosgw] Admin REST API wrong results
Hi, I'm using the radosgw REST API (via Python's boto library and also using some radosgw-agent methods) to fetch some data from Ceph (version 0.85). When I try to get the admin log for some specific dates, the radosgw seems to give me bad results. For example when I try to get entries since 2014-09-30 08:00:00+00:00, it seems to give me ALL entries for this day instead of entries since 8 AM. Here's a sample: {u'marker': u'1_1412065259.449518_102.1', u'entries': [{u'timestamp': u'2014-09-30 07:35:08.774367Z', u'key': u'testbucket:default.4146.33', u'entity_type': u'bucket'}, {u'timestamp': u'2014-09-30 07:36:23.092266Z', u'key': u'testbucket:default.4146.33', u'entity_type': u'bucket'}, {u'timestamp': u'2014-09-30 07:36:37.068249Z', u'key': u'testbucket:default.4146.34', u'entity_type': u'bucket'}, {u'timestamp': u'2014-09-30 07:37:38.431647Z', u'key': u'testbucket:default.4146.34', u'entity_type': u'bucket'}, {u'timestamp': u'2014-09-30 07:37:45.589333Z', u'key': u'testbucket:default.4146.35', u'entity_type': u'bucket'}, [...] {u'timestamp': u'2014-09-30 08:20:59.449518Z', u'key': u'testbucket:default.4146.52', u'entity_type': u'bucket'}], u'truncated': False} The apache's access logs: ceph-rgw:80 192.168.43.1 - - [30/Sep/2014:08:21:09 +] GET /admin/log?start-time=2014-09-30+08%3A00%3A00%2B00%3A00type=dataid=82 HTTP/1.1 200 4480 - Boto/2.31.1 Python/2.7.6 Linux/3.13.0-36-generic ceph-rgw:80 192.168.43.1 - - [30/Sep/2014:08:22:13 +] GET /admin/log?start-time=2014-09-30+08%3A00%3A00%2B00%3A00type=dataid=82 HTTP/1.1 200 4480 - Boto/2.31.1 Python/2.7.6 Linux/3.13.0-36-generic ceph-rgw:80 192.168.43.1 - - [30/Sep/2014:08:23:29 +] GET /admin/log?start-time=1412064000.0type=dataid=82 HTTP/1.1 400 216 - Boto/2.31.1 Python/2.7.6 Linux/3.13.0-36-generic I've tried different dates (10AM, 7AM), etc, but nothing seems to change. Full result of admin/log on pastebin in case it is useful to someone: http://pastebin.com/KSTXdGw5 Thanks, Patrycja Szabłowska ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] [radosgw] Admin REST API wrong results
Sending this again, because it wasn't published to the mailing list (I think because I wasn't a subscriber). Hi, I'm using the radosgw REST API (via Python's boto library and also using some radosgw-agent methods) to fetch some data from Ceph (version 0.85). When I try to get the admin log for some specific dates, the radosgw seems to give me bad results. For example when I try to get entries since 2014-09-30 08:00:00+00:00, it seems to give me ALL entries for this day instead of entries since 8 AM. Here's a sample: {u'marker': u'1_1412065259.449518_102.1', u'entries': [{u'timestamp': u'2014-09-30 07:35:08.774367Z', u'key': u'testbucket:default.4146.33', u'entity_type': u'bucket'}, {u'timestamp': u'2014-09-30 07:36:23.092266Z', u'key': u'testbucket:default.4146.33', u'entity_type': u'bucket'}, {u'timestamp': u'2014-09-30 07:36:37.068249Z', u'key': u'testbucket:default.4146.34', u'entity_type': u'bucket'}, {u'timestamp': u'2014-09-30 07:37:38.431647Z', u'key': u'testbucket:default.4146.34', u'entity_type': u'bucket'}, {u'timestamp': u'2014-09-30 07:37:45.589333Z', u'key': u'testbucket:default.4146.35', u'entity_type': u'bucket'}, [...] {u'timestamp': u'2014-09-30 08:20:59.449518Z', u'key': u'testbucket:default.4146.52', u'entity_type': u'bucket'}], u'truncated': False} The apache's access logs: ceph-rgw:80 192.168.43.1 - - [30/Sep/2014:08:21:09 +] GET /admin/log?start-time=2014-09-30+08%3A00%3A00%2B00%3A00type=dataid=82 HTTP/1.1 200 4480 - Boto/2.31.1 Python/2.7.6 Linux/3.13.0-36-generic ceph-rgw:80 192.168.43.1 - - [30/Sep/2014:08:22:13 +] GET /admin/log?start-time=2014-09-30+08%3A00%3A00%2B00%3A00type=dataid=82 HTTP/1.1 200 4480 - Boto/2.31.1 Python/2.7.6 Linux/3.13.0-36-generic ceph-rgw:80 192.168.43.1 - - [30/Sep/2014:08:23:29 +] GET /admin/log?start-time=1412064000.0type=dataid=82 HTTP/1.1 400 216 - Boto/2.31.1 Python/2.7.6 Linux/3.13.0-36-generic I've tried different dates (10AM, 7AM), etc, but nothing seems to change. Full result of admin/log on pastebin in case it is useful to someone: http://pastebin.com/KSTXdGw5 Thanks, Patrycja Szabłowska ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph monitor load, low performance
Irrelevant, but I need to say this: Cephers aren't only men, you know... :-) Cheers, Patrycja 2014-08-26 12:58 GMT+02:00 pawel.orzechow...@budikom.net: Hello Gentelmen:-) Let me point one important aspect of this low performance problem: from all 4 nodes of our ceph cluster only one node shows bad metrics, that is very high latency on its osd's (from 200-600ms), while other three nodes behave normaly, thats is latency of their osds is between 1-10ms. So, the idea of putting journals on SSD is something that we are looking at, but we think that we have in general some problem with that particular node, what affects whole cluster. So can the number (4) of hosts a reason for that? Any other hints? Thanks Pawel ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] [radosgw-admin] bilog list confusion
Hi, Is there any configuration option in ceph.conf for enabling/disabling the bilog list? I mean the result of this command: radosgw-admin bilog list One ceph cluster gives me results - list of operations which were made to the bucket, and the other one gives me just an empty list. I can't see what's the reason. I can't find it anywhere here in the ceph.conf file. http://ceph.com/docs/master/rados/configuration/ceph-conf/ My guess is it's in region info, but when I've changed these values to false for the cluster with working bilog, the bilog would still show. 1. cluster with empty bilog list: zones: [ { name: default, endpoints: [], log_meta: false, log_data: false}], 2. cluster with *proper* bilog list: zones: [ { name: master-1, endpoints: [ http:\/\/[...]], log_meta: true, log_data: true}], Here are pools on both of the clusters: 1. cluster with *proper* bilog list: rbd .rgw.root .rgw.control .rgw .rgw.gc .users.uid .users.email .users .rgw.buckets .rgw.buckets.index .log '' 2. cluster with empty bilog list: data metadata rbd .rgw.root .rgw.control .rgw .rgw.gc .users.uid .users.email .users '' .rgw.buckets.index .rgw.buckets .log And here is the zone info (just the placement_pools, rest of the config is the same): 1. cluster with *proper* bilog list: placement_pools: [] 2. cluster with *empty* bilog list: placement_pools: [ { key: default-placement, val: { index_pool: .rgw.buckets.index, data_pool: .rgw.buckets, data_extra_pool: }}]} Any thoughts? I've tried to figure it out by myself, but no luck. Thanks, Patrycja Szabłowska ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] [radosgw] Creating buckets with different owner
Hi, Is it possible to set owner of a bucket or an object to someone else? I've got a user who was created with flag --system and is able to create buckets and objects. I've created a bucket using boto and I've got FULL CONTROL over it: Policy: http://acs.amazonaws.com/groups/global/AllUsers = READ, M. Tester (owner) = FULL_CONTROL but trying to set owner to someone else gives me this: boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden ?xml version=1.0 encoding=UTF-8?ErrorCodeAccessDenied/Code/Error So I wonder - is it even possible to change owner of a bucket or create a bucket for a different owner than myself? Thanks, Patrycja Szabłowska ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Multipart upload on ceph 0.8 doesn't work?
Thank you Josh, I'm not sure if this is ceph's fault or perhaps FastCGI's or Boto's. When trying to upload 10KB chunk I've had those mysterious FastCGI's errors on server side and boto.exception.BotoServerError: BotoServerError: 500 Internal Server Error on client side. I've tried now to upload 4MB part - the error on client's side was as expected EntityTooSmall - but returned at the end, when I've called the method complete_upload. Perhaps this output can be helpful to someone: $ python boto_multi.py begin upload of Bosphorus_1920x1080_30fps_420_8bit_AVC_MP4.mp4 size 7507998, 2 parts upload part 1 size 4194304 upload part 2 size 3313694 end upload Traceback (most recent call last): File boto_multi.py, line 48, in module part.complete_upload() File local/lib/python2.7/site-packages/boto/s3/multipart.py, line 319, in complete_upload self.id, xml) File local/lib/python2.7/site-packages/boto/s3/bucket.py, line 1779, in complete_multipart_upload response.status, response.reason, body) boto.exception.S3ResponseError: S3ResponseError: 400 Bad Request ?xml version=1.0 encoding=UTF-8?ErrorCodeEntityTooSmall/Code/Error Cheers, Patrycja Szabłowska 2014-07-08 0:10 GMT+02:00 Josh Durgin josh.dur...@inktank.com: On 07/07/2014 05:41 AM, Patrycja Szabłowska wrote: OK, the mystery is solved. From https://www.mail-archive.com/ceph-users@lists.ceph.com/msg10368.html During a multi part upload you can't upload parts smaller than 5M I've tried to upload smaller chunks, like 10KB. I've changed chunk size to 5MB and it works now. It's a pity that the Ceph's docs don't mention the limit (or I couldn't found it anywhere). And that the error wasn't helpful at all. Glad you figured it out. This is in the s3 docs [1], but the lack of error message is a regression. I added a couple tickets for this: http://tracker.ceph.com/issues/8764 http://tracker.ceph.com/issues/8766 Josh [1] http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPart.html ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Multipart upload on ceph 0.8 doesn't work?
I've installed Ubuntu 12.04 in order to test multiupload with the ceph's modified fastcgi (according to this: https://ceph.com/docs/master/install/install-ceph-gateway/#apache-fastcgi-w-100-continue ). The problem is still the same: I can initiate the multi part upload or upload single part, but when trying to put a part of multi part upload I get the error with Fastcgi as I've shown before. Here's a part of the error log once again: == apache.error.log == [Fri Jul 04 15:40:41.868621 2014] [fastcgi:error] [pid 14199] [client 127.0.0.1:46571] FastCGI: incomplete headers (0 bytes) received from server /home/pszablow/ceph/src/htdocs/rgw.fcgi [Fri Jul 04 15:40:42.571543 2014] [fastcgi:error] [pid 14200] (111)Connection refused: [client 127.0.0.1:46572] FastCGI: failed to connect to server /home/pszablow/ceph/src/htdocs/rgw.fcgi: connect() failed [Fri Jul 04 15:40:42.571660 2014] [fastcgi:error] [pid 14200] [client 127.0.0.1:46572] FastCGI: incomplete headers (0 bytes) received from server /home/pszablow/ceph/src/htdocs/rgw.fcgi It seems to me that it doesn't matter if I turn the rgw print continue on or off. I've got no idea what to try next to make it work... Are there any other pools needed to put parts in upload? Thanks, Patrycja Szabłowska 2014-07-04 16:26 GMT+02:00 Patrycja Szabłowska szablowska.patry...@gmail.com: Still not sure do I need the ceph's modified fastcgi or not. But I guess this explains my problem with the installation: http://tracker.ceph.com/issues/8233 It would be nice to have at least a workaround for this... Thanks, Patrycja Szabłowska 2014-07-04 16:02 GMT+02:00 Patrycja Szabłowska szablowska.patry...@gmail.com: Thank you Luis for your response. Quite unbelievable, but your solution worked! Unfortunately, I'm stuck again when trying to upload parts of the file. Apache's logs: == apache.access.log == 127.0.0.1 l - [04/Jul/2014:15:40:41 +0200] PUT /bucketbig/ HTTP/1.1 200 477 {Referer}i Boto/2.30.0 Python/2.7.6 Linux/3.13.0-30-generic 127.0.0.1 l - [04/Jul/2014:15:40:41 +0200] POST /bucketbig/Bosphorus?uploads HTTP/1.1 200 249 {Referer}i Boto/2.30.0 Python/2.7.6 Linux/3.13.0-30-generic == apache.error.log == [Fri Jul 04 15:40:41.868621 2014] [fastcgi:error] [pid 14199] [client 127.0.0.1:46571] FastCGI: incomplete headers (0 bytes) received from server /home/pszablow/ceph/src/htdocs/rgw.fcgi == apache.access.log == 127.0.0.1 l - [04/Jul/2014:15:40:41 +0200] PUT /bucketbig/Bosphorus?uploadId=2/fURJChPdpUqA3Z1oVLUjT7ROsnxIqZ9partNumber=1 HTTP/1.1 500 531 {Referer}i Boto/2.30.0 Python/2.7.6 Linux/3.13.0-30-generic == apache.error.log == [Fri Jul 04 15:40:42.571543 2014] [fastcgi:error] [pid 14200] (111)Connection refused: [client 127.0.0.1:46572] FastCGI: failed to connect to server /home/pszablow/ceph/src/htdocs/rgw.fcgi: connect() failed [Fri Jul 04 15:40:42.571660 2014] [fastcgi:error] [pid 14200] [client 127.0.0.1:46572] FastCGI: incomplete headers (0 bytes) received from server /home/pszablow/ceph/src/htdocs/rgw.fcgi I'm using the default fastcgi module, not the one provided by Ceph. I've tried installing it on my ubuntu 14.04, but unfortunately I keep getting the error: libapache2-mod-fastcgi : requires: apache2.2-common (= 2.2.4) Is the modified fastcgi module mandatory in order to use multi part upload? Thanks, Patrycja Szabłowska 2014-07-03 18:34 GMT+02:00 Luis Periquito luis.periqu...@ocado.com: I was at this issue this morning. It seems radosgw requires you to have a pool named '' to work with multipart. I just created a pool with that name rados mkpool '' either that or allow the pool be created by the radosgw... On 3 July 2014 16:27, Patrycja Szabłowska szablowska.patry...@gmail.com wrote: Hi, I'm trying to make multi part upload work. I'm using ceph 0.80-702-g9bac31b (from the ceph's github). I've tried the code provided by Mark Kirkwood here: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-October/034940.html But unfortunately, it gives me the error: (multitest)pszablow@pat-desktop:~/$ python boto_multi.py begin upload of abc.yuv size 746496, 7 parts Traceback (most recent call last): File boto_multi.py, line 36, in module part = bucket.initiate_multipart_upload(objname) File /home/pszablow/venvs/multitest/local/lib/python2.7/site-packages/boto/s3/bucket.py, line 1742, in initiate_multipart_upload response.status, response.reason, body) boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden ?xml version=1.0 encoding=UTF-8?ErrorCodeAccessDenied/Code/Error The single part upload works for me. I am able to create buckets and objects. I've tried also other similar examples, but none of them works. Any ideas what's wrong? Does the ceph's multi part upload actually work for anybody? Thanks, Patrycja Szabłowska ___ ceph-users mailing list ceph-users@lists.ceph.com
Re: [ceph-users] Multipart upload on ceph 0.8 doesn't work?
OK, the mystery is solved. From https://www.mail-archive.com/ceph-users@lists.ceph.com/msg10368.html During a multi part upload you can't upload parts smaller than 5M I've tried to upload smaller chunks, like 10KB. I've changed chunk size to 5MB and it works now. It's a pity that the Ceph's docs don't mention the limit (or I couldn't found it anywhere). And that the error wasn't helpful at all. Cheers, Patrycja Szabłowska 2014-07-07 14:05 GMT+02:00 Patrycja Szabłowska szablowska.patry...@gmail.com: I've installed Ubuntu 12.04 in order to test multiupload with the ceph's modified fastcgi (according to this: https://ceph.com/docs/master/install/install-ceph-gateway/#apache-fastcgi-w-100-continue ). The problem is still the same: I can initiate the multi part upload or upload single part, but when trying to put a part of multi part upload I get the error with Fastcgi as I've shown before. Here's a part of the error log once again: == apache.error.log == [Fri Jul 04 15:40:41.868621 2014] [fastcgi:error] [pid 14199] [client 127.0.0.1:46571] FastCGI: incomplete headers (0 bytes) received from server /home/pszablow/ceph/src/htdocs/rgw.fcgi [Fri Jul 04 15:40:42.571543 2014] [fastcgi:error] [pid 14200] (111)Connection refused: [client 127.0.0.1:46572] FastCGI: failed to connect to server /home/pszablow/ceph/src/htdocs/rgw.fcgi: connect() failed [Fri Jul 04 15:40:42.571660 2014] [fastcgi:error] [pid 14200] [client 127.0.0.1:46572] FastCGI: incomplete headers (0 bytes) received from server /home/pszablow/ceph/src/htdocs/rgw.fcgi It seems to me that it doesn't matter if I turn the rgw print continue on or off. I've got no idea what to try next to make it work... Are there any other pools needed to put parts in upload? Thanks, Patrycja Szabłowska 2014-07-04 16:26 GMT+02:00 Patrycja Szabłowska szablowska.patry...@gmail.com: Still not sure do I need the ceph's modified fastcgi or not. But I guess this explains my problem with the installation: http://tracker.ceph.com/issues/8233 It would be nice to have at least a workaround for this... Thanks, Patrycja Szabłowska 2014-07-04 16:02 GMT+02:00 Patrycja Szabłowska szablowska.patry...@gmail.com: Thank you Luis for your response. Quite unbelievable, but your solution worked! Unfortunately, I'm stuck again when trying to upload parts of the file. Apache's logs: == apache.access.log == 127.0.0.1 l - [04/Jul/2014:15:40:41 +0200] PUT /bucketbig/ HTTP/1.1 200 477 {Referer}i Boto/2.30.0 Python/2.7.6 Linux/3.13.0-30-generic 127.0.0.1 l - [04/Jul/2014:15:40:41 +0200] POST /bucketbig/Bosphorus?uploads HTTP/1.1 200 249 {Referer}i Boto/2.30.0 Python/2.7.6 Linux/3.13.0-30-generic == apache.error.log == [Fri Jul 04 15:40:41.868621 2014] [fastcgi:error] [pid 14199] [client 127.0.0.1:46571] FastCGI: incomplete headers (0 bytes) received from server /home/pszablow/ceph/src/htdocs/rgw.fcgi == apache.access.log == 127.0.0.1 l - [04/Jul/2014:15:40:41 +0200] PUT /bucketbig/Bosphorus?uploadId=2/fURJChPdpUqA3Z1oVLUjT7ROsnxIqZ9partNumber=1 HTTP/1.1 500 531 {Referer}i Boto/2.30.0 Python/2.7.6 Linux/3.13.0-30-generic == apache.error.log == [Fri Jul 04 15:40:42.571543 2014] [fastcgi:error] [pid 14200] (111)Connection refused: [client 127.0.0.1:46572] FastCGI: failed to connect to server /home/pszablow/ceph/src/htdocs/rgw.fcgi: connect() failed [Fri Jul 04 15:40:42.571660 2014] [fastcgi:error] [pid 14200] [client 127.0.0.1:46572] FastCGI: incomplete headers (0 bytes) received from server /home/pszablow/ceph/src/htdocs/rgw.fcgi I'm using the default fastcgi module, not the one provided by Ceph. I've tried installing it on my ubuntu 14.04, but unfortunately I keep getting the error: libapache2-mod-fastcgi : requires: apache2.2-common (= 2.2.4) Is the modified fastcgi module mandatory in order to use multi part upload? Thanks, Patrycja Szabłowska 2014-07-03 18:34 GMT+02:00 Luis Periquito luis.periqu...@ocado.com: I was at this issue this morning. It seems radosgw requires you to have a pool named '' to work with multipart. I just created a pool with that name rados mkpool '' either that or allow the pool be created by the radosgw... On 3 July 2014 16:27, Patrycja Szabłowska szablowska.patry...@gmail.com wrote: Hi, I'm trying to make multi part upload work. I'm using ceph 0.80-702-g9bac31b (from the ceph's github). I've tried the code provided by Mark Kirkwood here: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-October/034940.html But unfortunately, it gives me the error: (multitest)pszablow@pat-desktop:~/$ python boto_multi.py begin upload of abc.yuv size 746496, 7 parts Traceback (most recent call last): File boto_multi.py, line 36, in module part = bucket.initiate_multipart_upload(objname) File /home/pszablow/venvs/multitest/local/lib/python2.7/site-packages/boto/s3/bucket.py, line 1742, in initiate_multipart_upload
Re: [ceph-users] Multipart upload on ceph 0.8 doesn't work?
Thank you Luis for your response. Quite unbelievable, but your solution worked! Unfortunately, I'm stuck again when trying to upload parts of the file. Apache's logs: == apache.access.log == 127.0.0.1 l - [04/Jul/2014:15:40:41 +0200] PUT /bucketbig/ HTTP/1.1 200 477 {Referer}i Boto/2.30.0 Python/2.7.6 Linux/3.13.0-30-generic 127.0.0.1 l - [04/Jul/2014:15:40:41 +0200] POST /bucketbig/Bosphorus?uploads HTTP/1.1 200 249 {Referer}i Boto/2.30.0 Python/2.7.6 Linux/3.13.0-30-generic == apache.error.log == [Fri Jul 04 15:40:41.868621 2014] [fastcgi:error] [pid 14199] [client 127.0.0.1:46571] FastCGI: incomplete headers (0 bytes) received from server /home/pszablow/ceph/src/htdocs/rgw.fcgi == apache.access.log == 127.0.0.1 l - [04/Jul/2014:15:40:41 +0200] PUT /bucketbig/Bosphorus?uploadId=2/fURJChPdpUqA3Z1oVLUjT7ROsnxIqZ9partNumber=1 HTTP/1.1 500 531 {Referer}i Boto/2.30.0 Python/2.7.6 Linux/3.13.0-30-generic == apache.error.log == [Fri Jul 04 15:40:42.571543 2014] [fastcgi:error] [pid 14200] (111)Connection refused: [client 127.0.0.1:46572] FastCGI: failed to connect to server /home/pszablow/ceph/src/htdocs/rgw.fcgi: connect() failed [Fri Jul 04 15:40:42.571660 2014] [fastcgi:error] [pid 14200] [client 127.0.0.1:46572] FastCGI: incomplete headers (0 bytes) received from server /home/pszablow/ceph/src/htdocs/rgw.fcgi I'm using the default fastcgi module, not the one provided by Ceph. I've tried installing it on my ubuntu 14.04, but unfortunately I keep getting the error: libapache2-mod-fastcgi : requires: apache2.2-common (= 2.2.4) Is the modified fastcgi module mandatory in order to use multi part upload? Thanks, Patrycja Szabłowska 2014-07-03 18:34 GMT+02:00 Luis Periquito luis.periqu...@ocado.com: I was at this issue this morning. It seems radosgw requires you to have a pool named '' to work with multipart. I just created a pool with that name rados mkpool '' either that or allow the pool be created by the radosgw... On 3 July 2014 16:27, Patrycja Szabłowska szablowska.patry...@gmail.com wrote: Hi, I'm trying to make multi part upload work. I'm using ceph 0.80-702-g9bac31b (from the ceph's github). I've tried the code provided by Mark Kirkwood here: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-October/034940.html But unfortunately, it gives me the error: (multitest)pszablow@pat-desktop:~/$ python boto_multi.py begin upload of abc.yuv size 746496, 7 parts Traceback (most recent call last): File boto_multi.py, line 36, in module part = bucket.initiate_multipart_upload(objname) File /home/pszablow/venvs/multitest/local/lib/python2.7/site-packages/boto/s3/bucket.py, line 1742, in initiate_multipart_upload response.status, response.reason, body) boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden ?xml version=1.0 encoding=UTF-8?ErrorCodeAccessDenied/Code/Error The single part upload works for me. I am able to create buckets and objects. I've tried also other similar examples, but none of them works. Any ideas what's wrong? Does the ceph's multi part upload actually work for anybody? Thanks, Patrycja Szabłowska ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Luis Periquito Unix Engineer Ocado.com http://www.ocado.com/ Head Office, Titan Court, 3 Bishop Square, Hatfield Business Park, Hatfield, Herts AL10 9NE Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE. -- Pozdrawiam Patrycja Szabłowska ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Multipart upload on ceph 0.8 doesn't work?
Still not sure do I need the ceph's modified fastcgi or not. But I guess this explains my problem with the installation: http://tracker.ceph.com/issues/8233 It would be nice to have at least a workaround for this... Thanks, Patrycja Szabłowska 2014-07-04 16:02 GMT+02:00 Patrycja Szabłowska szablowska.patry...@gmail.com: Thank you Luis for your response. Quite unbelievable, but your solution worked! Unfortunately, I'm stuck again when trying to upload parts of the file. Apache's logs: == apache.access.log == 127.0.0.1 l - [04/Jul/2014:15:40:41 +0200] PUT /bucketbig/ HTTP/1.1 200 477 {Referer}i Boto/2.30.0 Python/2.7.6 Linux/3.13.0-30-generic 127.0.0.1 l - [04/Jul/2014:15:40:41 +0200] POST /bucketbig/Bosphorus?uploads HTTP/1.1 200 249 {Referer}i Boto/2.30.0 Python/2.7.6 Linux/3.13.0-30-generic == apache.error.log == [Fri Jul 04 15:40:41.868621 2014] [fastcgi:error] [pid 14199] [client 127.0.0.1:46571] FastCGI: incomplete headers (0 bytes) received from server /home/pszablow/ceph/src/htdocs/rgw.fcgi == apache.access.log == 127.0.0.1 l - [04/Jul/2014:15:40:41 +0200] PUT /bucketbig/Bosphorus?uploadId=2/fURJChPdpUqA3Z1oVLUjT7ROsnxIqZ9partNumber=1 HTTP/1.1 500 531 {Referer}i Boto/2.30.0 Python/2.7.6 Linux/3.13.0-30-generic == apache.error.log == [Fri Jul 04 15:40:42.571543 2014] [fastcgi:error] [pid 14200] (111)Connection refused: [client 127.0.0.1:46572] FastCGI: failed to connect to server /home/pszablow/ceph/src/htdocs/rgw.fcgi: connect() failed [Fri Jul 04 15:40:42.571660 2014] [fastcgi:error] [pid 14200] [client 127.0.0.1:46572] FastCGI: incomplete headers (0 bytes) received from server /home/pszablow/ceph/src/htdocs/rgw.fcgi I'm using the default fastcgi module, not the one provided by Ceph. I've tried installing it on my ubuntu 14.04, but unfortunately I keep getting the error: libapache2-mod-fastcgi : requires: apache2.2-common (= 2.2.4) Is the modified fastcgi module mandatory in order to use multi part upload? Thanks, Patrycja Szabłowska 2014-07-03 18:34 GMT+02:00 Luis Periquito luis.periqu...@ocado.com: I was at this issue this morning. It seems radosgw requires you to have a pool named '' to work with multipart. I just created a pool with that name rados mkpool '' either that or allow the pool be created by the radosgw... On 3 July 2014 16:27, Patrycja Szabłowska szablowska.patry...@gmail.com wrote: Hi, I'm trying to make multi part upload work. I'm using ceph 0.80-702-g9bac31b (from the ceph's github). I've tried the code provided by Mark Kirkwood here: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-October/034940.html But unfortunately, it gives me the error: (multitest)pszablow@pat-desktop:~/$ python boto_multi.py begin upload of abc.yuv size 746496, 7 parts Traceback (most recent call last): File boto_multi.py, line 36, in module part = bucket.initiate_multipart_upload(objname) File /home/pszablow/venvs/multitest/local/lib/python2.7/site-packages/boto/s3/bucket.py, line 1742, in initiate_multipart_upload response.status, response.reason, body) boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden ?xml version=1.0 encoding=UTF-8?ErrorCodeAccessDenied/Code/Error The single part upload works for me. I am able to create buckets and objects. I've tried also other similar examples, but none of them works. Any ideas what's wrong? Does the ceph's multi part upload actually work for anybody? Thanks, Patrycja Szabłowska ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Luis Periquito Unix Engineer Ocado.com http://www.ocado.com/ Head Office, Titan Court, 3 Bishop Square, Hatfield Business Park, Hatfield, Herts AL10 9NE Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE. -- Pozdrawiam Patrycja Szabłowska ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Multipart upload on ceph 0.8 doesn't work?
Hi, I'm trying to make multi part upload work. I'm using ceph 0.80-702-g9bac31b (from the ceph's github). I've tried the code provided by Mark Kirkwood here: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-October/034940.html But unfortunately, it gives me the error: (multitest)pszablow@pat-desktop:~/$ python boto_multi.py begin upload of abc.yuv size 746496, 7 parts Traceback (most recent call last): File boto_multi.py, line 36, in module part = bucket.initiate_multipart_upload(objname) File /home/pszablow/venvs/multitest/local/lib/python2.7/site-packages/boto/s3/bucket.py, line 1742, in initiate_multipart_upload response.status, response.reason, body) boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden ?xml version=1.0 encoding=UTF-8?ErrorCodeAccessDenied/Code/Error The single part upload works for me. I am able to create buckets and objects. I've tried also other similar examples, but none of them works. Any ideas what's wrong? Does the ceph's multi part upload actually work for anybody? Thanks, Patrycja Szabłowska ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Radosgw fastcgi problem - comm with server /var/www/s3gw.fcgi aborted: idle timeout (30 sec)
Hi, I've got a problem with uploading files into buckets I've created. I'm using the python boto client for my tests. My code creates the bucket and then uploads the file. The apache server is working, the bucket gets created, I can access it via browser. However, when trying to upload files the client hangs and says it got 500 response. Here's the apache log: [Tue Jun 17 11:28:09 2014] [error] [client 192.168.X.X] FastCGI: comm with server /var/www/s3gw.fcgi aborted: idle timeout (30 sec) [Tue Jun 17 11:28:09 2014] [error] [client 192.168.X.X] FastCGI: incomplete headers (0 bytes) received from server /var/www/s3gw.fcgi Somebody here had similar problem, but there was no solution: http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-September/004489.html And the MTU on my ceph server is 1400. My config is from ceph-ansible, but I've also tried this http://ceph.com/docs/master/radosgw/config/ The code and the cluster was previously working OK on this host, but it stopped after my experiments (I've created more zones, regions, and then deleted them without touching the default region). Perhaps somebody had similar problem and managed to fix it... Here's also the boto log (if relevant). Tue, 17 Jun 2014 09:00:57 GMT /ceph-test/ 2014-06-17 11:00:57,165 boto [DEBUG]:Signature: AWS UZWH6AMGXZIWU4QH60TC:BTleaE7rREKPph3umD9g4442/a4= 2014-06-17 11:00:57,176 boto [DEBUG]:path=/ceph-test/abc 2014-06-17 11:00:57,177 boto [DEBUG]:auth_path=/ceph-test/abc 2014-06-17 11:00:57,177 boto [DEBUG]:Method: PUT 2014-06-17 11:00:57,177 boto [DEBUG]:Path: /ceph-test/abc 2014-06-17 11:00:57,177 boto [DEBUG]:Data: 2014-06-17 11:00:57,177 boto [DEBUG]:Headers: {'Content-Length': '5', 'Content-MD5': 'j0kMILqPh1g3YDpNbEt3HA==', 'Content-Type': 'application/octet-stream', 'Expect': '100-Continue', 'User-Agent': 'Boto/2.27.0 Python/2.6.6 Linux/2.6.32-431.el6.x86_64'} 2014-06-17 11:00:57,179 boto [DEBUG]:Host: Y 2014-06-17 11:00:57,179 boto [DEBUG]:Port: 80 2014-06-17 11:00:57,179 boto [DEBUG]:Params: {} 2014-06-17 11:00:57,179 boto [DEBUG]:establishing HTTP connection: kwargs={'port': 80, 'timeout': 100} 2014-06-17 11:00:57,179 boto [DEBUG]:Token: None 2014-06-17 11:00:57,180 boto [DEBUG]:StringToSign: PUT j0kMILqPh1g3YDpNbEt3HA== application/octet-stream Tue, 17 Jun 2014 09:00:57 GMT /ceph-test/abc 2014-06-17 11:00:57,180 boto [DEBUG]:Signature: AWS UZWH6AMGXZIWU4QH60TC:rNQ+0GjYV0iMMNLrdJHFI2muL4M= 2014-06-17 11:01:27,215 boto [DEBUG]:Received 500 response. Retrying in 0.8 seconds 2014-06-17 11:01:28,001 boto [DEBUG]:Token: None 2014-06-17 11:01:28,001 boto [DEBUG]:StringToSign: PUT Thanks Patrycja Szabłowska ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] radosgw-agent - syncing two zones and regions
Hi, I'm new with ceph and I'm trying to understand how replicating data between two regions/zones work. I've read this http://ceph.com/docs/master/radosgw/federated-config/ and this http://www.sebastien-han.fr/blog/2013/01/28/ceph-geo-replication-sort-of/ and tried that http://blog.kri5.fr/?p=21 Here are my thoughts and please let me know if I'm wrong. 1. Data can be synchronized between two zones - a master and a slave. Can it be reversed? Let's say the master fails and I want to use the slave as the replacement. 2. Data can't be synchronized between two regions. Two regions can only share metadata. Why is that? I know Python and I've seen that radosgw-agent throws an exception in this case ('data sync can only occur between zones in the same region'). But I'm curious what is the reason to do so. Are there any plans to change this behaviour? 3. One cluster can live in many datacenters, but only when they are close and the latency is low. Unfortunately I haven't found any other resources about multiregion config. Perhaps someone could recommend me some other resouces to read... Thanks, Patrycja Szabłowska ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com