[ceph-users] Re: Storage class usage stats
Is there any PR ongoing to add such counters to bucket stats? rados-level is not an option if those counters are needd to do, eg. rating/billing. From: Casey Bodley Sent: Wednesday, September 9, 2020 7:50:12 PM To: Tobias Urdin Cc: ceph-users@ceph.io Subject: [ceph-users] Re: Storage class usage stats That's right, radosgw doesn't do accounting per storage class. All you have to go on is the rados-level pool stats for those storage classes. On Mon, Sep 7, 2020 at 7:05 AM Tobias Urdin wrote: > > Hello, > > Anybody have any feedback or ways they have resolved this issue? > > Best regards > > From: Tobias Urdin > Sent: Wednesday, August 26, 2020 3:01:49 PM > To: ceph-users@ceph.io > Subject: [ceph-users] Storage class usage stats > > Hello, > > I've been trying to understand if there is any way to get usage information > based on storage classes for buckets. > > Since there is no information available from the "radosgw-admin bucket stats" > commands nor any other endpoint I > tried to browse the source code but couldn't find any references where the > storage class would be exposed in such a way. > > It also seems that RadosGW today is not saving any counters on amount of > objects stored in storage classes when it's > collecting usage stats, which means there is no such metadata saved for a > bucket. > > > I was hoping it was atleast saved but not exposed because then it would have > been a easier fix than adding support to count number of objects in storage > classes based on operations which would involve a lot of places and mean > writing to the bucket metadata on each op :( > > > Is my assumptions correct that there is no way to retrieve such information, > meaning there is no way to measure such usage? > > If the answer is yes, I assume the only way to get something that could be > measured would be to instead have multiple placement > targets since that is exposed from in bucket info. The bad things would be > though that you lose a lot of functionality related to lifecycle > and moving a single object to another storage class. > > Best regards > Tobias > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Storage class usage stats
On Thu, Oct 28, 2021 at 3:46 AM Engelmann Florian wrote: > > Is there any PR ongoing to add such counters to bucket stats? rados-level is > not an option if those counters are needd to do, eg. rating/billing. i'm not aware of any work in progress here. a feature request at https://tracker.ceph.com/projects/rgw/issues/new (set Tracker=Feature) is probably a good place to start there have been some requests to enable user/bucket quotas on a per-storage-class basis, and that would probably require that buckets track these per-storage-class stats as well > > From: Casey Bodley > Sent: Wednesday, September 9, 2020 7:50:12 PM > To: Tobias Urdin > Cc: ceph-users@ceph.io > Subject: [ceph-users] Re: Storage class usage stats > > That's right, radosgw doesn't do accounting per storage class. All you > have to go on is the rados-level pool stats for those storage classes. > > On Mon, Sep 7, 2020 at 7:05 AM Tobias Urdin wrote: > > > > Hello, > > > > Anybody have any feedback or ways they have resolved this issue? > > > > Best regards > > > > From: Tobias Urdin > > Sent: Wednesday, August 26, 2020 3:01:49 PM > > To: ceph-users@ceph.io > > Subject: [ceph-users] Storage class usage stats > > > > Hello, > > > > I've been trying to understand if there is any way to get usage information > > based on storage classes for buckets. > > > > Since there is no information available from the "radosgw-admin bucket > > stats" commands nor any other endpoint I > > tried to browse the source code but couldn't find any references where the > > storage class would be exposed in such a way. > > > > It also seems that RadosGW today is not saving any counters on amount of > > objects stored in storage classes when it's > > collecting usage stats, which means there is no such metadata saved for a > > bucket. > > > > > > I was hoping it was atleast saved but not exposed because then it would > > have been a easier fix than adding support to count number of objects in > > storage classes based on operations which would involve a lot of places and > > mean writing to the bucket metadata on each op :( > > > > > > Is my assumptions correct that there is no way to retrieve such > > information, meaning there is no way to measure such usage? > > > > If the answer is yes, I assume the only way to get something that could be > > measured would be to instead have multiple placement > > targets since that is exposed from in bucket info. The bad things would be > > though that you lose a lot of functionality related to lifecycle > > and moving a single object to another storage class. > > > > Best regards > > Tobias > > ___ > > ceph-users mailing list -- ceph-users@ceph.io > > To unsubscribe send an email to ceph-users-le...@ceph.io > > ___ > > ceph-users mailing list -- ceph-users@ceph.io > > To unsubscribe send an email to ceph-users-le...@ceph.io > > > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Storage class usage stats
Hello, I filed this [1] feature request in the tracker a a while ago. I started looking at the possibilty of helping out with this feature but gave up after digging for a while. Maybe somebody else with more ninja foo can look at it :) [1] https://tracker.ceph.com/issues/47342 > On 28 Oct 2021, at 15:30, Casey Bodley wrote: > > On Thu, Oct 28, 2021 at 3:46 AM Engelmann Florian > wrote: >> >> Is there any PR ongoing to add such counters to bucket stats? rados-level is >> not an option if those counters are needd to do, eg. rating/billing. > > i'm not aware of any work in progress here. a feature request at > https://tracker.ceph.com/projects/rgw/issues/new (set Tracker=Feature) > is probably a good place to start > > there have been some requests to enable user/bucket quotas on a > per-storage-class basis, and that would probably require that buckets > track these per-storage-class stats as well > >> >> From: Casey Bodley >> Sent: Wednesday, September 9, 2020 7:50:12 PM >> To: Tobias Urdin >> Cc: ceph-users@ceph.io >> Subject: [ceph-users] Re: Storage class usage stats >> >> That's right, radosgw doesn't do accounting per storage class. All you >> have to go on is the rados-level pool stats for those storage classes. >> >> On Mon, Sep 7, 2020 at 7:05 AM Tobias Urdin wrote: >>> >>> Hello, >>> >>> Anybody have any feedback or ways they have resolved this issue? >>> >>> Best regards >>> >>> From: Tobias Urdin >>> Sent: Wednesday, August 26, 2020 3:01:49 PM >>> To: ceph-users@ceph.io >>> Subject: [ceph-users] Storage class usage stats >>> >>> Hello, >>> >>> I've been trying to understand if there is any way to get usage information >>> based on storage classes for buckets. >>> >>> Since there is no information available from the "radosgw-admin bucket >>> stats" commands nor any other endpoint I >>> tried to browse the source code but couldn't find any references where the >>> storage class would be exposed in such a way. >>> >>> It also seems that RadosGW today is not saving any counters on amount of >>> objects stored in storage classes when it's >>> collecting usage stats, which means there is no such metadata saved for a >>> bucket. >>> >>> >>> I was hoping it was atleast saved but not exposed because then it would >>> have been a easier fix than adding support to count number of objects in >>> storage classes based on operations which would involve a lot of places and >>> mean writing to the bucket metadata on each op :( >>> >>> >>> Is my assumptions correct that there is no way to retrieve such >>> information, meaning there is no way to measure such usage? >>> >>> If the answer is yes, I assume the only way to get something that could be >>> measured would be to instead have multiple placement >>> targets since that is exposed from in bucket info. The bad things would be >>> though that you lose a lot of functionality related to lifecycle >>> and moving a single object to another storage class. >>> >>> Best regards >>> Tobias >>> ___ >>> ceph-users mailing list -- ceph-users@ceph.io >>> To unsubscribe send an email to ceph-users-le...@ceph.io >>> ___ >>> ceph-users mailing list -- ceph-users@ceph.io >>> To unsubscribe send an email to ceph-users-le...@ceph.io >>> >> ___ >> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe send an email to ceph-users-le...@ceph.io > ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Storage class usage stats
Hello, Anybody have any feedback or ways they have resolved this issue? Best regards From: Tobias Urdin Sent: Wednesday, August 26, 2020 3:01:49 PM To: ceph-users@ceph.io Subject: [ceph-users] Storage class usage stats Hello, I've been trying to understand if there is any way to get usage information based on storage classes for buckets. Since there is no information available from the "radosgw-admin bucket stats" commands nor any other endpoint I tried to browse the source code but couldn't find any references where the storage class would be exposed in such a way. It also seems that RadosGW today is not saving any counters on amount of objects stored in storage classes when it's collecting usage stats, which means there is no such metadata saved for a bucket. I was hoping it was atleast saved but not exposed because then it would have been a easier fix than adding support to count number of objects in storage classes based on operations which would involve a lot of places and mean writing to the bucket metadata on each op :( Is my assumptions correct that there is no way to retrieve such information, meaning there is no way to measure such usage? If the answer is yes, I assume the only way to get something that could be measured would be to instead have multiple placement targets since that is exposed from in bucket info. The bad things would be though that you lose a lot of functionality related to lifecycle and moving a single object to another storage class. Best regards Tobias ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Storage class usage stats
That's right, radosgw doesn't do accounting per storage class. All you have to go on is the rados-level pool stats for those storage classes. On Mon, Sep 7, 2020 at 7:05 AM Tobias Urdin wrote: > > Hello, > > Anybody have any feedback or ways they have resolved this issue? > > Best regards > > From: Tobias Urdin > Sent: Wednesday, August 26, 2020 3:01:49 PM > To: ceph-users@ceph.io > Subject: [ceph-users] Storage class usage stats > > Hello, > > I've been trying to understand if there is any way to get usage information > based on storage classes for buckets. > > Since there is no information available from the "radosgw-admin bucket stats" > commands nor any other endpoint I > tried to browse the source code but couldn't find any references where the > storage class would be exposed in such a way. > > It also seems that RadosGW today is not saving any counters on amount of > objects stored in storage classes when it's > collecting usage stats, which means there is no such metadata saved for a > bucket. > > > I was hoping it was atleast saved but not exposed because then it would have > been a easier fix than adding support to count number of objects in storage > classes based on operations which would involve a lot of places and mean > writing to the bucket metadata on each op :( > > > Is my assumptions correct that there is no way to retrieve such information, > meaning there is no way to measure such usage? > > If the answer is yes, I assume the only way to get something that could be > measured would be to instead have multiple placement > targets since that is exposed from in bucket info. The bad things would be > though that you lose a lot of functionality related to lifecycle > and moving a single object to another storage class. > > Best regards > Tobias > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
