[ceph-users] Re: RGW dynamic resharding blocks write ops

Hi again, I got the log excerpt with the rgw error message: s3:put_obj block_while_resharding ERROR: bucket is still resharding, please retry Below is the message in context, I don't see a return code though, only 206 for the get requests. Unfortunately, we only have a recorded putty sess

[ceph-users] Re: RGW dynamic resharding blocks write ops

We had a quite small window yesterday to debug, I found the error messages but we didn't collect the logs yet, I will ask them to do that on Monday. I *think* the error was something like this: resharding operation on bucket index detected, blocking block_while_resharding ERROR: bucket is s

[ceph-users] Re: RGW dynamic resharding blocks write ops

while a bucket is resharding, rgw will retry several times internally to apply the write before returning an error to the client. while most buckets can be resharded within seconds, very large buckets may hit these timeouts. any other cause of slow osd ops could also have that effect. it can be hel

[ceph-users] Re: RGW dynamic resharding blocks write ops

That sounds reasonable, I will keep that in mind and discuss it with them, maybe they can establish something similar with their clients. Thanks! Zitat von "Szabo, Istvan (Agoda)" : I do manual reshard if needed but try to do pre-shard in advance. I try to deal with the user and ask them b

[ceph-users] Re: RGW dynamic resharding blocks write ops

I do manual reshard if needed but try to do pre-shard in advance. I try to deal with the user and ask them before onboard them, do they need bucket with more than a million objects (default 11 shard) or it’s enough. If they need I preshard (to a prime numbered shard number), if not then stay wit

[ceph-users] Re: RGW dynamic resharding blocks write ops

Okay, thanks for the comment. But does that mean that you never reshard or do you manually reshard? Do you experience performance degradation? Maybe I should also add that they have their index pool on HDDs (with rocksdb on SSD), not sure how big the impact is during resharding though. Zi

[ceph-users] Re: RGW dynamic resharding blocks write ops

I turned off :) Istvan Szabo Staff Infrastructure Engineer --- Agoda Services Co., Ltd. e: istvan.sz...@agoda.com --- On 2023. Jul 7., at 17:35, Eugen Block wrote: Ema