Hi there, We have an old cluster that was built on Giant that we have maintained and upgraded over time and are now running Mimic 13.2.5. The other day we received a HEALTH_WARN about 1 large omap object in the pool '.usage' which is our usage_log_pool defined in our radosgw zone.
I am trying to understand the purpose of the usage_log_pool and whether or not we have appropriate settings (shards, replicas, etc) in place. We were able to identify the 1 large omap object as 'usage.22' in the .usage pool. This particular "bucket" had over 2 million "omapkeys" ```for i in `rados -p .usage ls`; do echo $i; rados -p .usage listomapkeys $i | wc -l; done``` -snip- usage.13 20 usage.22 2023790 usage.25 14 -snip- These keys all seem to be metadata/pointers of valid data from our OpenStack's object storage where we hold about 1PB of unique data. To resolve the HEALTH_WARN we changeg the 'osd_deep_scrub_large_omap_object_key_threshold' from '2000000' to '2500000' using 'ceph config set osd ...' on our Mon's. I'd like to know the importance of this pool as I also noticed that this pool's replication is only set to 2, instead of 3 like all our other pools with the exception of .users.email (also 2). If important, I'd like to set the replication to 3 and curious to know if there would be any negative impact to the cluster. The .usage pool says 0 bytes used in 'ceph df' but it contains 30 objects for which there are many omapkeys. I am also wondering about bucket index max shards for which we have '8' set in the config. ``` "rgw_override_bucket_index_max_shards": "8",```. Should this be increased? Thanks in advance for any responses, I have found this mailing list to be an excellent source of information! Jared Baker Ontario Institute for Cancer Research
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
