Average size is good to know, but doesn’t give much information about 
distribution.  In a multitenant deployment, most likely there will be buckets 
with a variety of internal distributions too, all small, all large, mix etc.

A cluster with 1M 1kB objects and 1M 10MB objects will have an average 
effectively the same as a cluster with 2M 5MB objects.

> Do keep in mind that small objects are much more expensive then large 
> objects. The metadata overhead becomes costly and can even become problematic 
> if you have millions of tiny (few kb) objects.

Absolutely, which is one reason why it’s good to have more detailed 
information, to inform architecture to consider media types, multiple storage 
classes, Varnish caches etc. 

> Hope this is still relevant... but you may want to have a look at this
> example:
> 
> https://github.com/ceph/ceph/blob/master/examples/lua/prometheus_adapter.lua
> https://github.com/ceph/ceph/blob/master/examples/lua/prometheus_adapter.md
> 
> where we log RGW object sizes to Prometheus.
> would be easy to change it so it is per bucket and not per operation type.


That’s really terrific, SO wish I would have had that a couple of years ago!  
Lua scripting is shaping up to have all kinds of great applications, especially 
since it’s fast.
Makes me want to go write up a Grafana histogram pane, though it might be 
really slow over a long time region.

> Just got the chance to have a look, but I see lua scripting is new in version 
> pacific ☹
> I have octopus 15.2.14, will it be backported or no chance?


I can’t speak to the potential for backporting, but running a Pacific RGW as a 
client of an Octopus cluster *might* work fine, but there also might be subtle 
incompatibilities.


> 
> 
> Actually it's a good idea, metrics are already there so easy to make the 
> Grafana dash, thank you 😊
> 
> Istvan Szabo
> Senior Infrastructure Engineer
> ---------------------------------------------------
> Agoda Services Co., Ltd.
> e: istvan.sz...@agoda.com
> ---------------------------------------------------
> 
> -----Original Message-----
> From: Christian Rohmann <christian.rohm...@inovex.de> 
> Sent: Thursday, October 14, 2021 3:08 PM
> To: ceph-users@ceph.io
> Subject: [ceph-users] Re: Metrics for object sizes
> 
> Email received from the internet. If in doubt, don't click any link nor open 
> any attachment !
> ________________________________
> 
> On 23/04/2021 03:53, Szabo, Istvan (Agoda) wrote:
>> Objects inside RGW buckets like in couch base software they have their own 
>> metrics and has this information.
> 
> Not as detailed as you would like, but how about using the bucket stats on 
> bucket size and number of objects?
>  $ radosgw-admin bucket stats --bucket mybucket
> 
> 
> Doing a bucket_size / number_of_objects gives you an average object size per 
> bucket and that certainly is an indication on buckets with rather small 
> objects.
> 
> 
> 
> Regards
> 
> 
> Christian
> 
> _______________________________________________
> ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to 
> ceph-users-le...@ceph.io
> _______________________________________________
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io

_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

Reply via email to