Re: [ceph-users] Trying to understand the contents of .rgw.buckets.index
On Fri, Jan 29, 2016 at 3:10 AM, Wido den Hollander wrote: > > > On 29-01-16 11:31, Micha Krause wrote: >> Hi, >> >> I'm having problems listing the contents of an s3 bucket with ~2M objects. >> >> I already found the new bucket index sharding feature, but I'm >> interested how these Indexes are stored. >> >> My index pool shows no space used, and all objects have 0B. >> >> root@mon01:~ # rados df -p .rgw.buckets.index >> pool name KB objects clones degraded >> unfound rdrd KB wrwr KB >> .rgw.buckets.index0 620 >> 0 0 28177514336051356 228972310 >> >> Why would sharing a 0B object make any difference? >> > The index is stored in the omap of the object which you can list with > the 'rados' command. > > So it's not data inside the RADOS object, but in the omap key/value store. ...and this is an unfortunate accounting problem in terms of RADOS pools, but a solution is very difficult technically so nobody's come up with a good one. :( ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Trying to understand the contents of .rgw.buckets.index
Hi, > The index is stored in the omap of the object which you can list with> the 'rados' command. > > So it's not data inside the RADOS object, but in the omap key/value store. Thank you very much: rados -p .rgw.buckets.index listomapkeys .dir.default.55059808.22 | wc -l 2228777 So this data is then stored in the omap directory on my osd as .sst files? is there a way to correlate a rados object with a specific sst (leveldb?) file? Micha Krause ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Trying to understand the contents of .rgw.buckets.index
On 29-01-16 11:31, Micha Krause wrote: > Hi, > > I'm having problems listing the contents of an s3 bucket with ~2M objects. > > I already found the new bucket index sharding feature, but I'm > interested how these Indexes are stored. > > My index pool shows no space used, and all objects have 0B. > > root@mon01:~ # rados df -p .rgw.buckets.index > pool name KB objects clones degraded > unfound rdrd KB wrwr KB > .rgw.buckets.index0 620 > 0 0 28177514336051356 228972310 > > Why would sharing a 0B object make any difference? > The index is stored in the omap of the object which you can list with the 'rados' command. So it's not data inside the RADOS object, but in the omap key/value store. Wido > > > Micha Krause > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Trying to understand the contents of .rgw.buckets.index
Hi, I'm having problems listing the contents of an s3 bucket with ~2M objects. I already found the new bucket index sharding feature, but I'm interested how these Indexes are stored. My index pool shows no space used, and all objects have 0B. root@mon01:~ # rados df -p .rgw.buckets.index pool name KB objects clones degraded unfound rdrd KB wrwr KB .rgw.buckets.index0 6200 0 28177514336051356 228972310 Why would sharing a 0B object make any difference? Micha Krause ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
