Hmm, It's useful to rebuild the index by rewriting a object. But at first, I need know the all keys of objects. If I want to know all keys, I need list_objects ... Maybe I can make an union set of instances, then copy all of them into themselves.
Anyway, I want to find out more about why it happens and how to avoid it. Yehuda Sadeh-Weinraub <yeh...@redhat.com>于2018年10月19日周五 上午2:25写道: > On Wed, Oct 17, 2018 at 1:14 AM Yang Yang <inksin...@gmail.com> wrote: > > > > Hi, > > A few weeks ago I found radosgw index has been inconsistent with > reality. Some object I can not list, but I can get them by key. Please see > the details below: > > > > BACKGROUND: > > Ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) > luminous (stable) > > Index pool is on ssd. > > There is a very big bucket with more than 10 million object and > 500TB data. > > Ceph health is OK. > > I use s3 api on radosgw. > > > > DESCRIBE: > > When use s3 list_object() to list, some uploaded object can not be > listed and some uploaded object have an old lastModified time. > > But at the same time, we can get this object by an exact key. And if > I put a new object into this bucket, it can be listed. > > It seems that some indexes during a period of time have been lost. > > > > I try to run "radosgw-admin bucket check --bucket <bucketname> --fix > --check-objects" and I get nothing at all. > > > > SOME ELSE: > > I found that one bucket will have many indexes, and we can use > "radosgw-admin metadata list bucket.instance | grep "{bucket name}" to show > them. But I can not found a doc to describe this feature. And we can use > "radosgw-admin bucket stats --bucket {bucket_name}" to get id as the active > instance id. > > I use "rados listomapkeys" at active(or latest) index to get all > object in a index, it is really lost. But when I use "rados listomapkeys" > at another index which is not active as mentioned above, I found the lost > object index. > > > > Resharding is within my consideration. Listomapkeys means do this > action on all shards(more than 300). > > In my understanding, a big bucket has one latest index and many old > indexes. Every index has many shards. So listomapkeys on a index means > listomapkeys on many shards. > > > > QUESTION: > > Why my index lost? > > How to recover? > > I don't really know what happened, haven't seen this exact issue > before. You can try copying objects into themselves. That should > recreate their bucket index entry. > > > Why radosgw has many index instances, how do radosgw use them and > how to change active index? > > Could be related to an existing bug. You can unlink the bucket and > then link a specific bucket instance version (to the user), however, > I'm not sure I recommend going this path if it isn't necessary. > > Regards, > Yehuda > > > > > > Thanks, > > > > Inksink > > > > _______________________________________________ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com