Re: [ceph-users] radosgw: S3 object retention: high usage of default.rgw.log pool

I was misled.In fact, this is not an automatic deletion, but the removal of one object per op by application. Reject. k ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] radosgw: S3 object retention: high usage of default.rgw.log pool

Hi. For some bucket for backup application is applied S3 retention policy, at 04:00 2+days backups will be deleted from bucket. At this time I see very high usage of default.rgw.log pool. Usage log is enabled, ops log is disabled, index pool on NVMe: - https://ibb.co/dozqPJ -

Re: [ceph-users] RADOSGW err=Input/output error

...@ifastnet.com Cc: Ceph Users Subject: Re: [ceph-users] RADOSGW err=Input/output error Hi : I use libs3 to run test . The network is IB. The error in libcurl is the following: == Info: Operation too slow. Less than 1 bytes/sec transferred the last 15 seconds == Info: Closing connection 766

Re: [ceph-users] RADOSGW err=Input/output error

t; debug ms = 1 > debug rgw = 20 > > Regards > Kev > > > - Original Message - > From: "Drew Weaver" > To: "ceph-users" > Sent: Tuesday, July 3, 2018 1:39:55 PM > Subject: [ceph-users] RADOSGW err=Input/output error > > An applicati

Re: [ceph-users] radosgw multizone not syncing large bucket completly to other zone

I changed the endpoints to bypass the loadbalancers for sync. But the Problem stil remains. Will probably resetup the bucket and recopy the data to see if that changes something. I cant make anything out of all the log messages, need to dig deeper into that On Sun, Jul 8, 2018 at 4:55 PM Enrico

[ceph-users] radosgw frontend : civetweb vs fastcgi

Hi: I see that civetweb is still using poll and multithread, compared with fastcgi, which one should I use ? Which one has better performance ? ___ ceph-users mailing list ceph-users@lists.ceph.com

Re: [ceph-users] radosgw multizone not syncing large bucket completly to other zone

Hello, yes we are using haproxy on the secondary zone, but A10 Hardware Loadbalancers on the Master zone. So i suspect there are some timeouts that may cause this issue then if having loadbalancers in front of it be the problem? Will check if changing to the ips directly will fix the issue On

Re: [ceph-users] radosgw multizone not syncing large bucket completly to other zone

Hi Enrico, On Fri, Jun 29, 2018 at 7:50 PM Enrico Kern wrote: > hmm that also pops up right away when i restart all radosgw instances. But > i will check further and see if i can find something. Maybe doing the > upgrade to mimic too. > > That bucket is basically under load on the master zone

Re: [ceph-users] RADOSGW err=Input/output error

Hi Drew, Try to increase debugging with debug ms = 1 debug rgw = 20 Regards Kev - Original Message - From: "Drew Weaver" To: "ceph-users" Sent: Tuesday, July 3, 2018 1:39:55 PM Subject: [ceph-users] RADOSGW err=Input/output error An application is having gene

[ceph-users] RADOSGW err=Input/output error

An application is having general failures writing to a test cluster we have setup. 2018-07-02 23:13:26.128282 7fe00b560700 0 WARNING: set_req_state_err err_no=5 resorting to 500 2018-07-02 23:13:26.128460 7fe00b560700 1 == req done req=0x7fe00b55a110 op status=-5 http_status=500 ==

Re: [ceph-users] radosgw multizone not syncing large bucket completly to other zone

hmm that also pops up right away when i restart all radosgw instances. But i will check further and see if i can find something. Maybe doing the upgrade to mimic too. That bucket is basically under load on the master zone all the time as we use it as historical storage for druid, so there is

Re: [ceph-users] radosgw multizone not syncing large bucket completly to other zone

On Fri, Jun 29, 2018 at 8:48 AM, Enrico Kern wrote: > also when i try to sync the bucket manual i get this error: > > ERROR: sync.run() returned ret=-16 > 2018-06-29 15:47:50.137268 7f54b7e4ecc0 0 data sync: ERROR: failed to > read sync status for bucketname:6a9448d2-bdba-4bec- >

Re: [ceph-users] radosgw multizone not syncing large bucket completly to other zone

also when i try to sync the bucket manual i get this error: ERROR: sync.run() returned ret=-16 2018-06-29 15:47:50.137268 7f54b7e4ecc0 0 data sync: ERROR: failed to read sync status for bucketname:6a9448d2-bdba-4bec-aad6-aba72cd8eac6.27150814.1 it works flawless with all other buckets. On

Re: [ceph-users] radosgw multizone not syncing large bucket completly to other zone

Hello, thanks for the reply. We have around 200k objects in the bucket. It is not automatic resharded (is that even supported in multisite?) What i see when i run a complete data sync with the debug logs after a while i see alot of informations that it is unable to perform some log and also

[ceph-users] radosgw multi file upload failure

Hello, Recently we've observed on one of our ceph clusters that uploading of a large number of small files(~2000x2k) fails. The http return code shows 200 but the file upload fails. Here is an e.g of the log 2018-06-27 07:34:40.624103 7f0dc67cc700 1 == starting new request

Re: [ceph-users] radosgw multizone not syncing large bucket completly to other zone

On Sun, Jun 24, 2018 at 12:59 AM, Enrico Kern wrote: > Hello, > > We have two ceph luminous clusters (12.2.5). > > recently one of our big buckets stopped syncing properly. We have a one > specific bucket which is around 30TB in size consisting of alot of > directories with each one having files

Re: [ceph-users] radosgw failover help

Hi, On 06/20/2018 07:20 PM, David Turner wrote: We originally used pacemaker to move a VIP between our RGWs, but ultimately decided to go with an LB in front of them. With an LB you can utilize both RGWs while they're up, but the LB will shy away from either if they're down until the check

[ceph-users] radosgw multizone not syncing large bucket completly to other zone

Hello, We have two ceph luminous clusters (12.2.5). recently one of our big buckets stopped syncing properly. We have a one specific bucket which is around 30TB in size consisting of alot of directories with each one having files of 10-20MB. The secondary zone is often completly missing

Re: [ceph-users] radosgw failover help

Has any one, done or working a way to do S3(radosgw) failover. I am trying to work out away to have 2 radosgw servers, with an VIP when one server goes down it will go over to the other. May be better failover + load balancing? For example - nginx do this + TLS. k

Re: [ceph-users] radosgw failover help

We originally used pacemaker to move a VIP between our RGWs, but ultimately decided to go with an LB in front of them. With an LB you can utilize both RGWs while they're up, but the LB will shy away from either if they're down until the check starts succeeding for that host again. We do have 2

[ceph-users] radosgw failover help

Hay All Has any one, done or working a way to do S3(radosgw) failover. I am trying to work out away to have 2 radosgw servers, with an VIP when one server goes down it will go over to the other. I am trying this with CTDB, but while testing the upload can fail and then carry on or just hand and

Re: [ceph-users] Radosgw

People have RGW and CephFS all the time. You have it for different use cases. Sometimes you need an S3 object store and other times you need a networked POSIX filesystem. If you don't use one or the other for production, perhaps he was testing it. But in any case, you can rip one or the other

Re: [ceph-users] Radosgw

Den mån 28 maj 2018 kl 15:28 skrev Marc-Antoine Desrochers < marc-antoine.desroch...@sogetel.com>: > Hi, > > > > Im new in a business and I took on the ceph project. > > Im still a newbie on that subject and I try to understand what the > previous guy was trying to do. > > > > Is there any reason

[ceph-users] Radosgw

Hi, Im new in a business and I took on the ceph project. Im still a newbie on that subject and I try to understand what the previous guy was trying to do. Is there any reason someone would install radosgw with a cephfs? If not how can I remove all radosgw configuration without

Re: [ceph-users] radosgw s3cmd --list-md5 postfix on md5sum

Sent: zaterdag 5 mei 2018 10:20 To: ceph-users Subject: [ceph-users] radosgw s3cmd --list-md5 postfix on md5sum Why do I have sometimes md5 hashes with some postfix of "-xxx" like: [@]$ s3cmd ls "s3://test2/" --list-md5 2018-05-05 13:29 12430875 d3ccb6a9d2d3bc85dbe9de519c2be

[ceph-users] radosgw s3cmd --list-md5 postfix on md5sum

Why do I have sometimes md5 hashes with some postfix of "-xxx" like: [@]$ s3cmd ls "s3://test2/" --list-md5 2018-05-05 13:29 12430875 d3ccb6a9d2d3bc85dbe9de519c2be8e1-791 s3://test2/test.img ___ ceph-users mailing list ceph-users@lists.ceph.com

Re: [ceph-users] radosgw bucket listing (s3 ls s3://$bucketname) slow with ~2 billion objects

The main problem with efficiently listing many-sharded buckets is the requirement to provide entries in sorted order. This means that each http request has to fetch ~1000 entries from every shard, combine them into a sorted order, and throw out the leftovers. The next request to continue the

Re: [ceph-users] radosgw bucket listing (s3 ls s3://$bucketname) slow with ~2 billion objects

I second the indexless bucket suggestion. The downside being that you can't use bucket policies like object expiration in that case. On Tue, May 1, 2018 at 10:02 AM, David Turner wrote: > Any time using shared storage like S3 or cephfs/nfs/gluster/etc the > absolute

Re: [ceph-users] radosgw bucket listing (s3 ls s3://$bucketname) slow with ~2 billion objects

Any time using shared storage like S3 or cephfs/nfs/gluster/etc the absolute rule that I refuse to break is to never rely on a directory listing to know where objects/files are. You should be maintaining a database of some sort or a deterministic naming scheme. The only time a full listing of a

Re: [ceph-users] radosgw bucket listing (s3 ls s3://$bucketname) slow with ~2 billion objects

Listing will always take forever when using a high shard number, AFAIK. That's the tradeoff for sharding. Are those 2B objects in one bucket? How's your read and write performance compared to a bucket with a lower number (thousands) of objects, with that shard number? On Tue, May 1, 2018 at

[ceph-users] radosgw bucket listing (s3 ls s3://$bucketname) slow with ~2 billion objects

One of our radosgw buckets has grown a lot in size, `rgw bucket stats --bucket $bucketname` reports a total of 2,110,269,538 objects with the bucket index sharded across 32768 shards, listing the root context of the bucket with `s3 ls s3://$bucketname` takes more than an hour which is the hard

Re: [ceph-users] Radosgw switch from replicated to erasure

2018-04-20 6:06 GMT+02:00 Marc Roos : > > I want to start using the radowsgw a bit. For now I am fine with the 3 > replicated setup, in the near future when I add a host. I would like to > switch to ec, is there something I should do now to make this switch > more

[ceph-users] Radosgw switch from replicated to erasure

I want to start using the radowsgw a bit. For now I am fine with the 3 replicated setup, in the near future when I add a host. I would like to switch to ec, is there something I should do now to make this switch more smoothly? ___ ceph-users

Re: [ceph-users] radosgw: can't delete bucket

Hi, I finaly managed to delete the bucket, I wrote a script that reads the omap keys from the bucket index and deletes every key without a matching object on the data pool. Not sure if this has any negative repercussions, but after the script deleted thousands of keys from the index, i was

[ceph-users] radosgw: can't delete bucket

Hi, I have a Bucket with multiple broken multipart uploads, which can't be aborted. radosgw-admin bucket check shows thousands of _multipart_ objects, unfortunately the --fix and --check-objects don't change anything. I decided to get rid of the bucket completely, but even this command:

Re: [ceph-users] Radosgw ldap info

I think I have something wrong in my ldap setup, I cannot radosgw-admin user info --uid ldap users. So I have to fix this first. -Original Message- From: Benjeman Meekhof [mailto:bmeek...@umich.edu] Sent: maandag 26 maart 2018 18:17 To: ceph-users Subject: Re: [ceph-users] Radosgw

Re: [ceph-users] Radosgw ldap info

nstantin Shalygin [mailto:k0...@k0ste.ru] > Sent: zondag 18 maart 2018 5:04 > To: ceph-users@lists.ceph.com > Cc: Marc Roos; Yehuda Sadeh-Weinraub > Subject: Re: [ceph-users] Radosgw ldap user authentication issues > > Hi Marc > > >> looks like no search is be

Re: [ceph-users] Radosgw halts writes during recovery, recovery info issues

forgot to mention - we are running jewel, 10.2.10 On 26/03/18 11:30, Josef Zelenka wrote: Hi everyone, i'm currently fighting an issue in a cluster we have for a customer. It's used for a lot of small files(113m currently) that are pulled via radosgw. We have 3 nodes, 24 OSDs in total. the

[ceph-users] Radosgw ldap info

that many things asked here could be added to the docs. -Original Message- From: Konstantin Shalygin [mailto:k0...@k0ste.ru] Sent: zondag 18 maart 2018 5:04 To: ceph-users@lists.ceph.com Cc: Marc Roos; Yehuda Sadeh-Weinraub Subject: Re: [ceph-users] Radosgw ldap user authentication

Re: [ceph-users] Radosgw ldap user authentication issues

Hi Marc, You mentioned following the instructions 'except for doing this ldap token'. Do I read that correctly that you did not generate / use an LDAP token with your client? I think that is a necessary part of triggering the LDAP authentication (Section 3.2 and 3.3 of the doc you linked). I

Re: [ceph-users] Radosgw ldap user authentication issues

Hi Marc looks like no search is being done there. rgw::auth::s3::AWSAuthStrategy denied with reason=-13 The same for me, http://tracker.ceph.com/issues/23091 But Yehuda closed this. k ___ ceph-users mailing list ceph-users@lists.ceph.com

[ceph-users] Radosgw ldap user authentication issues

I have been following this instruction, except for doing this ldap token. https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/2/html-single/ceph_object_gateway_with_ldapad_guide/ With ldapsearch I am able to query the ldap server and list the userPassword. I guess I can use a

[ceph-users] radosgw + OpenLDAP = Failed the auth strategy, reason=-13

Hi cephers. I try rgw (Luminous 12.2.2) + OpenLDAP. My settings:     "rgw_ldap_binddn": "cn=cisco,ou=people,dc=example,dc=local",     "rgw_ldap_dnattr": "uid",     "rgw_ldap_searchdn": "ou=s3_users,dc=example,dc=local",     "rgw_ldap_searchfilter": "(objectClass=inetOrgPerson)",    

[ceph-users] radosgw: Huge Performance impact during dynamic bucket index resharding

Hi, Radosgw decided to reshard a bucket with 25 million objects from 256 to 512 shards. Resharding took about 1 hour, during this time all buckets on the cluster had a huge performance drop. "GET" requests for small objects (on other buckets) took multiple seconds. Are there any

[ceph-users] Radosgw - ls not showing some files, invisible files

Hello, we run ceph 12.2.2. Now the multipart upload problem with replication to other zones were fixed but we do not see all files in a bucket with ls via API, cyberduck etc. example we have a bucket with alot of files. The file list is complete in our second datacenter and i can see all files

[ceph-users] RadosGW Admin Ops API Access Problem

Hello Everyone, I have a Ceph test setup with 3 mons, 3 RGWs, 5 OSD nodes and 22 OSDs. RadosGW instances run on the monitor nodes and they are behind a load balancer. I run RGW instances in the full debug mode (20/20 for rgw and 20/20 for civet web). I can easily access RGW via S3 API with any

Re: [ceph-users] radosgw not listening after installation

up and > running! > > Thanks > Piers. > > > From: Jean-Charles Lopez [mailto:jelo...@redhat.com] > Sent: Monday, February 05, 2018 5:23 PM > To: Piers Haken <pie...@hotmail.com> > Cc: ceph-users@lists.ceph.com > Subject: Re: [ceph-users] radosgw not listening

Re: [ceph-users] radosgw not listening after installation

ph.com Subject: Re: [ceph-users] radosgw not listening after installation Hi, first of all just in case, it looks like your script does not deploy any OSDs as you go straight from MON to RGW. then, RGW does listen by default on 7480 and what you see on 6789 is the MON listening. Investigation: - Mak

Re: [ceph-users] radosgw not listening after installation

Hi, first of all just in case, it looks like your script does not deploy any OSDs as you go straight from MON to RGW. then, RGW does listen by default on 7480 and what you see on 6789 is the MON listening. Investigation: - Make sure your ceph-radosgw process is running first. - If not

[ceph-users] radosgw not listening after installation

i'm trying to setup radosgw on a brand new clutser, but I'm running into an issue where it's not listening on the default port (7480) here's my install script: ceph-deploy new $NODE ceph-deploy install --release luminous $NODE ceph-deploy install --release luminous --rgw $NODE

Re: [ceph-users] radosgw fails with "ERROR: failed to initialize watch: (34) Numerical result out of range"

accordingly. > >> >>> >>> Best regards, >>> Nikos. >>> >>> [1] http://tracker.ceph.com/issues/22351 >>> >>>>> >>>>> i think i am going to read up on how interpret ltrace output... >>>>> &g

Re: [ceph-users] radosgw fails with "ERROR: failed to initialize watch: (34) Numerical result out of range"

'-34'. $ c++filt >> >> Best regards, >> Nikos. >> >> [1] http://tracker.ceph.com/issues/22351 >> >>>> >>>> i think i am going to read up on how interpret ltrace output... >>>> >>>> BR >>>> Alex >>>>

Re: [ceph-users] radosgw fails with "ERROR: failed to initialize watch: (34) Numerical result out of range"

- Ursprüngliche Mail - >>> Von: "Brad Hubbard" <bhubb...@redhat.com> >>> An: "Alexander Peters" <alexander.pet...@sphinx.at> >>> CC: "Ceph Users" <ceph-users@lists.ceph.com> >>> Gesendet: Montag, 15. Januar 2018 03

Re: [ceph-users] radosgw fails with "ERROR: failed to initialize watch: (34) Numerical result out of range"

ltrace output... >> >> BR >> Alex >> >> - Ursprüngliche Mail - >> Von: "Brad Hubbard" <bhubb...@redhat.com> >> An: "Alexander Peters" <alexander.pet...@sphinx.at> >> CC: "Ceph Users" <ceph-users@lists.c

Re: [ceph-users] radosgw fails with "ERROR: failed to initialize watch: (34) Numerical result out of range"

> An: "Alexander Peters" <alexander.pet...@sphinx.at> > CC: "Ceph Users" <ceph-users@lists.ceph.com> > Gesendet: Montag, 15. Januar 2018 03:09:53 > Betreff: Re: [ceph-users] radosgw fails with "ERROR: failed to initialize > watch: (34) Numerical resu

Re: [ceph-users] radosgw fails with "ERROR: failed to initialize watch: (34) Numerical result out of range"

..@redhat.com> An: "Alexander Peters" <alexander.pet...@sphinx.at> CC: "Ceph Users" <ceph-users@lists.ceph.com> Gesendet: Montag, 15. Januar 2018 03:09:53 Betreff: Re: [ceph-users] radosgw fails with "ERROR: failed to initialize watch: (34) Numerical result out of r

Re: [ceph-users] radosgw fails with "ERROR: failed to initialize watch: (34) Numerical result out of range"

On Mon, Jan 15, 2018 at 11:38 AM, Brad Hubbard wrote: > On Mon, Jan 15, 2018 at 10:38 AM, Alexander Peters > wrote: >> Thanks for the reply - unfortunatly the link you send is behind a paywall so >> at least for now i can’t read it. > > That's why

Re: [ceph-users] radosgw fails with "ERROR: failed to initialize watch: (34) Numerical result out of range"

On Mon, Jan 15, 2018 at 10:38 AM, Alexander Peters wrote: > Thanks for the reply - unfortunatly the link you send is behind a paywall so > at least for now i can’t read it. That's why I provided the cause as laid out in that article (pgp num > pg num). Do you have

Re: [ceph-users] radosgw fails with "ERROR: failed to initialize watch: (34) Numerical result out of range"

Thanks for the reply - unfortunatly the link you send is behind a paywall so at least for now i can’t read it. output of ceph osd dump shows that pgp num == pg num: [root@ctrl01 ~]# ceph osd dump epoch 142 fsid 0e2d841f-68fd-4629-9813-ab083e8c0f10 created 2017-12-20 23:04:59.781525 modified

Re: [ceph-users] radosgw fails with "ERROR: failed to initialize watch: (34) Numerical result out of range"

On Mon, Jan 15, 2018 at 8:34 AM, Alexander Peters wrote: > Hello > > I am currently experiencing a strange issue with my radosgw. It Fails to > start and all tit says is: > [root@ctrl02 ~]# /usr/bin/radosgw --cluster ceph --name client.radosgw.ctrl02 > --setuser ceph

[ceph-users] radosgw fails with "ERROR: failed to initialize watch: (34) Numerical result out of range"

Hello I am currently experiencing a strange issue with my radosgw. It Fails to start and all tit says is: [root@ctrl02 ~]# /usr/bin/radosgw --cluster ceph --name client.radosgw.ctrl02 --setuser ceph --setgroup ceph -f -d 2018-01-14 21:30:57.132007 7f44ddd18e00 0 deferred set uid:gid to 167:167

[ceph-users] RadosGW still stuck on buckets

Hello! Hope you all started the new year well... New year, same problem: Still having the issue with the frozen radosgw buckets. Some information: * Ceph 12.2.2 with bluestore * 3 OSD nodes, each housing 2 SSD OSDs for bucket index and 4 OSDs for bucket data, each having 64GB RAM and 16

[ceph-users] radosgw package for kraken missing on ubuntu

I am installing with ceph-deploy using the instructions at http://docs.ceph.com/docs/master/install/get-packages/ ceph-deploy runs fine for the first node until it dies due to not finding radosgw package. I have verified on that node (apt-cache search radosgw, apt-get install radosgw) that

Re: [ceph-users] radosgw: Couldn't init storage provider (RADOS)

Hi, try having a look at : - network connectivity issues - firewall configuration issues - missing or inaccessible keyring file for client.rgw.ceph-rgw1 - missing or inaccessible ceph.conf file Regards JC Lopez Senior Technical Instructor, Global Storage Consulting Practice Red Hat, Inc.

[ceph-users] radosgw: Couldn't init storage provider (RADOS)

Hello, I tried to install Ceph 12.2.2 (Luminous) on Ubuntu 16.04.3 LTS (kernel 4.4.0-104-generic), but I am having trouble starting radosgw service: # systemctl status ceph-rado...@rgw.ceph-rgw1 â ceph-rado...@rgw.ceph-rgw1.service - Ceph rados gateway Loaded: loaded

Re: [ceph-users] radosgw bucket rename and change owner

Is question is: how can i move a bucket in my radosgw to an swift/keystone based user. example: I'm creating a bucket via s3cmd on my ceph-radosgw and upload a file: # s3cmd mb s3://mys3bucket Bucket 's3://mys3bucket/' created # s3cmd put datei1 s3://mys3bucket/datei1 upload: 'datei1' ->

Re: [ceph-users] radosgw bucket rename and change owner

User and bucket operations have more to do with what is providing the S3 API. In this case you're using swift for that. The Ceph tools to do this would be if you're using RGW to provide the S3 API. The answers you're looking for would be in how to do this with SWIFT, if I'm not mistaken. Ceph

Re: [ceph-users] radosgw bucket rename and change owner

Or is there a way to mova the bucket contect into another bucket? Am Montag, den 20.11.2017, 17:36 +0100 schrieb Kim-Norman Sahm: > is it possible to rename a radosgw bucket and change the owner? > i'm using ceph as swift backend in openstack and want to move an old > bucket to a keystone based

[ceph-users] radosgw bucket rename and change owner

is it possible to rename a radosgw bucket and change the owner? i'm using ceph as swift backend in openstack and want to move an old bucket to a keystone based user. br Kim ___ ceph-users mailing list ceph-users@lists.ceph.com

Re: [ceph-users] radosgw multi site different period

Your period configuration is indeed consistent between zones. This "master is on a different period" error is specific to the metadata sync status. It's saying that zone b is unable to finish syncing the metadata changes from zone a that occurred during the previous period. Even though zone b

Re: [ceph-users] radosgw multi site different period

both cluster are in the same epoch and period:   root@ceph-a-1:~# radosgw-admin period get-current  { "current_period": "b7392c41-9cbe-4d92-ad03-db607dd7d569" } root@ceph-b-1:~# radosgw-admin period get-current  { "current_period": "b7392c41-9cbe-4d92-ad03-db607dd7d569" } but the sync

Re: [ceph-users] radosgw multi site different period

both cluster are in the same epoch and period: root@ceph-a-1:~# radosgw-admin period get-current  { "current_period": "b7392c41-9cbe-4d92-ad03-db607dd7d569" } root@ceph-b-1:~# radosgw-admin period get-current  { "current_period": "b7392c41-9cbe-4d92-ad03-db607dd7d569" } Am Dienstag, den

Re: [ceph-users] radosgw multi site different period

I'm assuming you've looked at the period in both places `radosgw-admin period get` and confirmed that the second site is behind the master site (based on epochs). I'm also assuming (since you linked the instructions) that you've done `radosgw-admin period pull` on the second site to get any

[ceph-users] radosgw multi site different period

hi, i've installed a ceph multi site setup with two ceph clusters and each one radosgw. the multi site setup was in sync, so i tried a failover. cluster A is going down and i've changed the zone (b) on cluster b to the new master zone. it's working fine. now i start the cluster A and try to

Re: [ceph-users] radosgw notify on creation/deletion of file in bucket

On Tue, Oct 3, 2017 at 8:59 AM, Sean Purdy wrote: > Hi, > > > Is there any way that radosgw can ping something when a file is removed or > added to a bucket? > That depends on what exactly you're looking for. You can't get that info as a user. but there is a mechanism

Re: [ceph-users] radosgw notify on creation/deletion of file in bucket

Just to make sure you're not confusing redundancy with backups. Having your data in another site does not back up your data, but makes it more redundant. For instance if an object/file is accidentally deleted from RGW and you're syncing those files to AWS, Google buckets, or a second RGW cluster

[ceph-users] radosgw notify on creation/deletion of file in bucket

Hi, Is there any way that radosgw can ping something when a file is removed or added to a bucket? Or use its sync facility to sync files to AWS/Google buckets? Just thinking about backups. What do people use for backups? Been looking at rclone. Thanks, Sean

[ceph-users] radosgw multi tenancy support with openstack newton

Hi, does swift api (radosgw on ceph jewel) multi tenancy working with openstack newton and keystone v3? http://docs.ceph.com/docs/jewel/radosgw/keystone/ I've tried to integrate ceph-radosgw as swift service in openstack newton with keystone v3 and the authentication is working but all

[ceph-users] Radosgw: object lifecycle (expiration) not working?

Greetings - I have created several test buckets in radosgw, to test different expiration durations: $ s3cmd mb s3://test2d I set a lifecycle for each of these buckets: $ s3cmd setlifecycle lifecycle2d.xml s3://test2d --signature-v2 The files look like this:

Re: [ceph-users] radosgw crashing after buffer overflows detected

oun...@lists.ceph.com> on behalf of Bryan Stillwell <bstillw...@godaddy.com> Date: Friday, September 8, 2017 at 9:26 AM To: ceph-users <ceph-users@lists.ceph.com> Subject: [ceph-users] radosgw crashing after buffer overflows detected [This sender failed our fraud detection checks

[ceph-users] radosgw crashing after buffer overflows detected

For about a week we've been seeing a decent number of buffer overflows detected across all our RGW nodes in one of our clusters. This started happening a day after we started weighing in some new OSD nodes, so we're thinking it's probably related to that. Could someone help us determine the root

[ceph-users] radosgw-admin orphans find -- Hammer

Hello, we need to reclaim a lot of wasted space by RGW orphans in our production Hammer cluster (0.94.10 on Ubuntu 14.04). According to http://tracker.ceph.com/issues/18258 there is a bug in the radosgw-admin orphans find command, that causes it to get

Re: [ceph-users] RadosGW ADMIN API

On Wed, Sep 06, 2017 at 02:08:14PM +, Engelmann Florian wrote: > we are running a luminous cluster and three radosgw to serve a s3 compatible > objectstore. As we are (currently) not using Openstack we have to use the > RadosGW Admin API to get our billing data. I tried to access the API

[ceph-users] RadosGW ADMIN API

Hi, we are running a luminous cluster and three radosgw to serve a s3 compatible objectstore. As we are (currently) not using Openstack we have to use the RadosGW Admin API to get our billing data. I tried to access the API with pathon like: [...] import rgwadmin [...] Users =

Re: [ceph-users] Radosgw returns 404 Not Found

Datum: Mittwoch, 16. August 2017 um 16:31 An: Martin Emrich <martin.emr...@empolis.com>, "ceph-us...@ceph.com" <ceph-us...@ceph.com> Betreff: Re: [ceph-users] Radosgw returns 404 Not Found You need to fix your endpoint URLs. The line that shows this is. 2017-08-16 14:02:21.7259

Re: [ceph-users] Radosgw returns 404 Not Found

You need to fix your endpoint URLs. The line that shows this is. 2017-08-16 14:02:21.725967 7fc7f5317700 10 s->object=s3testbucket-1 s->bucket=ceph-kl-mon1.de.empolis.com It thinks you're bucket is your domain name and your object is your bucket name. If you did this using an IP instead of URL

[ceph-users] Radosgw returns 404 Not Found

Hi! I have the following issue: While “radosgw bucket list” shows me my buckets, S3 API clients only get a “404 Not Found”. With debug level 20, I see the following output of the radosgw service: 2017-08-16 14:02:21.725959 7fc7f5317700 20 rgw::auth::s3::LocalEngine granted access 2017-08-16

[ceph-users] radosgw hung when OS disks went readonly, different node radosgw restart fixed it

Hi, Just had an incident in a 3-node test cluster running 12.1.1 on debian stretch Each cluster had its own mon, mgr, radosgw, and osds. Just object store. I had s3cmd looping and uploading files via S3. On one of the machines, the RAID controller barfed and dropped the OS disks. Or the

Re: [ceph-users] RadosGW Swift public links

I am trying to get radosgw to act like swift does when its provisioned with Openstack. I did end up figuring out how to make it work properly for my use case, and it would be helpful if it was documented somewhere. For anyone curious how to make radosgw function more similar to the way swift

Re: [ceph-users] RadosGW Swift public links

Are you accessing the bucket from a URL that is not configured as an endpoint in your zone? I bet if you looked at the log you would see that the bucket that doesn't exist is the URL that you are using to access it. On Thu, Jun 29, 2017, 9:07 PM Donny Davis wrote: > I

[ceph-users] RadosGW Swift public links

I have swift working well with keystone authentication, and I can upload and download files. However when I make a link public, I get nosuchbucket, and I have no idea what url to find the buckets at. When I list the buckets with radosgw-admin bucket list i get back some tenant url+ the bucket. I

Re: [ceph-users] Radosgw versioning S3 compatible?

Thanks! We'll try that and I'll report back then... Regards, Martin -Ursprüngliche Nachricht- Von: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] Im Auftrag von Daniel Gryniewicz Gesendet: Mittwoch, 28. Juni 2017 19:12 An: ceph-users@lists.ceph.com Betreff: Re: [ceph-users

Re: [ceph-users] Radosgw versioning S3 compatible?

gt; } > > ] > > > > Should that be supported? > Currently rgw bucket lifecycle rules do not versioning. Yehuda > > > Thanks > > > > Martin > > > > Von: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] Im Auftrag von > Martin

Re: [ceph-users] Radosgw versioning S3 compatible?

: [ceph-users] Radosgw versioning S3 compatible? Hi! Is the Object Gateway S3 API supposed to be compatible with Amazon S3 regarding versioning? Object Versioning is listed as supported in Ceph 12.1, but using the standard Node.js aws-sdk module (s3.putBucketVersioning()) results in "NotImplem

[ceph-users] Radosgw versioning S3 compatible?

Hi! Is the Object Gateway S3 API supposed to be compatible with Amazon S3 regarding versioning? Object Versioning is listed as supported in Ceph 12.1, but using the standard Node.js aws-sdk module (s3.putBucketVersioning()) results in "NotImplemented". Thanks Martin

Re: [ceph-users] radosgw: scrub causing slow requests in the md log

On Thu, Jun 22, 2017 at 5:31 PM, Casey Bodley wrote: > > On 06/22/2017 10:40 AM, Dan van der Ster wrote: >> >> On Thu, Jun 22, 2017 at 4:25 PM, Casey Bodley wrote: >>> >>> On 06/22/2017 04:00 AM, Dan van der Ster wrote: I'm now running the three

Re: [ceph-users] radosgw: scrub causing slow requests in the md log

On 06/22/2017 10:40 AM, Dan van der Ster wrote: On Thu, Jun 22, 2017 at 4:25 PM, Casey Bodley wrote: On 06/22/2017 04:00 AM, Dan van der Ster wrote: I'm now running the three relevant OSDs with that patch. (Recompiled, replaced /usr/lib64/rados-classes/libcls_log.so with

Re: [ceph-users] radosgw: scrub causing slow requests in the md log

On Thu, Jun 22, 2017 at 4:25 PM, Casey Bodley wrote: > > On 06/22/2017 04:00 AM, Dan van der Ster wrote: >> >> I'm now running the three relevant OSDs with that patch. (Recompiled, >> replaced /usr/lib64/rados-classes/libcls_log.so with the new version, >> then restarted the

Re: [ceph-users] radosgw: scrub causing slow requests in the md log

On 06/22/2017 04:00 AM, Dan van der Ster wrote: I'm now running the three relevant OSDs with that patch. (Recompiled, replaced /usr/lib64/rados-classes/libcls_log.so with the new version, then restarted the osds). It's working quite well, trimming 10 entries at a time instead of 1000, and no

Re: [ceph-users] radosgw: scrub causing slow requests in the md log

On Wed, Jun 21, 2017 at 4:16 PM, Peter Maloney wrote: > On 06/14/17 11:59, Dan van der Ster wrote: >> Dear ceph users, >> >> Today we had O(100) slow requests which were caused by deep-scrubbing >> of the metadata log: >> >> 2017-06-14 11:07:55.373184 osd.155

<    1   2   3   4   5   6   7   8   9   10   >