Re: Usage computing removed items
I found a query that maybe a problem:
When usage job start it founds a VOLUME.DELETE in usage_event table and after
that it look for a register in usage_volume table with that query:
SELECT usage_volume.id, usage_volume.zone_id, usage_volume.account_id,
usage_volume.domain_id, usage_volume.volume_id, usage_volume.disk_offering_id,
usage_volume.template_id, usage_volume.size, usage_volume.created,
usage_volume.deleted FROM usage_volume WHERE usage_volume.account_id = 9 AND
usage_volume.id = 110 AND usage_volume.deleted IS NULL;
Empty set (0.00 sec)
but its return empty because the field filter is wrong. I think the query needs
to look for a volume_id field.
SELECT usage_volume.id, usage_volume.zone_id, usage_volume.account_id,
usage_volume.domain_id, usage_volume.volume_id, usage_volume.disk_offering_id,
usage_volume.template_id, usage_volume.size, usage_volume.created,
usage_volume.deleted FROM usage_volume WHERE usage_volume.account_id = 9 AND
usage_volume.volume_id = 110 AND usage_volume.deleted IS NULL;
++-++---+---+--+-+-+-+-+
| id | zone_id | account_id | domain_id | volume_id | disk_offering_id |
template_id | size| created | deleted |
++-++---+---+--+-+-+-+-+
| 74 | 1 | 9 | 2 | 110 | 78 |
NULL | 10737418240 | 2021-06-08 19:19:38 | NULL|
++-++---+---+--+-+-+-+-+
1 row in set (0.00 sec)
WIth this wrong query, the usage_volume line will never be found and will never
be update with correct deleted field.
Is there someone else experiencing this problem?
> On 26 Jul 2022, at 17:30, Matheus Fontes wrote:
>
> Hi,
> Is anyone having problems with usage computing removed items?
> We have a user that reported the problem.
> The volume was deleted since 2022-05-06
>
> mysql> select id,account_id,created,removed,state from volumes where id=3246;
> +--++-+-+--+
> | id | account_id | created | removed | state|
> +--++-+-+--+
> | 3246 |545 | 2021-04-08 17:03:16 | 2022-05-06 15:06:52 | Expunged |
> +--++-+-+--+
>
>
> In usage process we can see it on parsing volume call:
> 2022-07-26 00:18:17,471 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Parsing all Volume usage events for account: 545
> 2022-07-26 00:18:17,472 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Total running time 8640ms
> 2022-07-26 00:18:17,472 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Creating Volume usage record for vol: 3246,
> usage: 24, startDate: Mon Jul 25 00:00:00 BRT 2022, endDate: Mon Jul 25
> 23:59:59 BRT 2022, for account: 545
> 2022-07-26 00:18:17,484 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Total running time 8640ms
> 2022-07-26 00:18:17,484 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Creating Volume usage record for vol: 3246,
> usage: 24, startDate: Mon Jul 25 00:00:00 BRT 2022, endDate: Mon Jul 25
> 23:59:59 BRT 2022, for account: 545
>
>
> And it are being computed to account:
> (ascenty) # > list usagerecords domainid=XXX
> accountid=d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8 type=6 startdate=2022-07-25
> enddate=2022-07-25 filter=account,rawusage,size,startdate,usage,usagetype,
> {
> "count": 2,
> "usagerecord": [
>{
> "accountid": "d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8",
> "rawusage": "24",
> "size": 53687091200,
> "startdate": "2022-07-25'T'00:00:00-03:00",
> "usage": "24 Hrs",
> "usagetype": 6
>},
>{
> "accountid": "d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8",
> "rawusage": "24",
> "size": 53687091200,
> "startdate": "2022-07-25'T'00:00:00-03:00",
> "usage": "24 Hrs",
> "usagetype": 6
>}
> ]
> }
Re: Usage computing removed items
Same here, exactly the same problem with volumes.
I've looked in github source for the SQL query that generates the
aggregated registries to try to understand but I couldn't find.
:/
Appreciate any help too.
Em ter., 26 de jul. de 2022 17:31, Matheus Fontes
escreveu:
> Hi,
> Is anyone having problems with usage computing removed items?
> We have a user that reported the problem.
> The volume was deleted since 2022-05-06
>
> mysql> select id,account_id,created,removed,state from volumes where
> id=3246;
>
> +--++-+-+--+
> | id | account_id | created | removed | state
> |
>
> +--++-+-+--+
> | 3246 |545 | 2021-04-08 17:03:16 | 2022-05-06 15:06:52 | Expunged
> |
>
> +--++-+-+--+
>
>
> In usage process we can see it on parsing volume call:
> 2022-07-26 00:18:17,471 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Parsing all Volume usage events for account: 545
> 2022-07-26 00:18:17,472 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Total running time 8640ms
> 2022-07-26 00:18:17,472 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Creating Volume usage record for vol: 3246,
> usage: 24, startDate: Mon Jul 25 00:00:00 BRT 2022, endDate: Mon Jul 25
> 23:59:59 BRT 2022, for account: 545
> 2022-07-26 00:18:17,484 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Total running time 8640ms
> 2022-07-26 00:18:17,484 DEBUG [usage.parser.VolumeUsageParser]
> (Usage-Job-1:null) (logid:) Creating Volume usage record for vol: 3246,
> usage: 24, startDate: Mon Jul 25 00:00:00 BRT 2022, endDate: Mon Jul 25
> 23:59:59 BRT 2022, for account: 545
>
>
> And it are being computed to account:
> (ascenty) # > list usagerecords domainid=XXX
> accountid=d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8 type=6 startdate=2022-07-25
> enddate=2022-07-25 filter=account,rawusage,size,startdate,usage,usagetype,
> {
> "count": 2,
> "usagerecord": [
> {
> "accountid": "d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8",
> "rawusage": "24",
> "size": 53687091200,
> "startdate": "2022-07-25'T'00:00:00-03:00",
> "usage": "24 Hrs",
> "usagetype": 6
> },
> {
> "accountid": "d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8",
> "rawusage": "24",
> "size": 53687091200,
> "startdate": "2022-07-25'T'00:00:00-03:00",
> "usage": "24 Hrs",
> "usagetype": 6
> }
> ]
> }
--
__
Aviso de confidencialidade
Esta mensagem da
Empresa Brasileira de Pesquisa Agropecuaria (Embrapa), empresa publica
federal regida pelo disposto na Lei Federal no. 5.851, de 7 de dezembro
de 1972, e enviada exclusivamente a seu destinatario e pode conter
informacoes confidenciais, protegidas por sigilo profissional. Sua
utilizacao desautorizada e ilegal e sujeita o infrator as penas da lei.
Se voce a recebeu indevidamente, queira, por gentileza, reenvia-la ao
emitente, esclarecendo o equivoco.
Confidentiality note
This message from
Empresa Brasileira de Pesquisa Agropecuaria (Embrapa), a government
company established under Brazilian law (5.851/72), is directed
exclusively to its addressee and may contain confidential data,
protected under professional secrecy rules. Its unauthorized use is
illegal and may subject the transgressor to the law's penalties. If you
are not the addressee, please send it back, elucidating the failure.
Usage computing removed items
Hi,
Is anyone having problems with usage computing removed items?
We have a user that reported the problem.
The volume was deleted since 2022-05-06
mysql> select id,account_id,created,removed,state from volumes where id=3246;
+--++-+-+--+
| id | account_id | created | removed | state|
+--++-+-+--+
| 3246 |545 | 2021-04-08 17:03:16 | 2022-05-06 15:06:52 | Expunged |
+--++-+-+--+
In usage process we can see it on parsing volume call:
2022-07-26 00:18:17,471 DEBUG [usage.parser.VolumeUsageParser]
(Usage-Job-1:null) (logid:) Parsing all Volume usage events for account: 545
2022-07-26 00:18:17,472 DEBUG [usage.parser.VolumeUsageParser]
(Usage-Job-1:null) (logid:) Total running time 8640ms
2022-07-26 00:18:17,472 DEBUG [usage.parser.VolumeUsageParser]
(Usage-Job-1:null) (logid:) Creating Volume usage record for vol: 3246, usage:
24, startDate: Mon Jul 25 00:00:00 BRT 2022, endDate: Mon Jul 25 23:59:59 BRT
2022, for account: 545
2022-07-26 00:18:17,484 DEBUG [usage.parser.VolumeUsageParser]
(Usage-Job-1:null) (logid:) Total running time 8640ms
2022-07-26 00:18:17,484 DEBUG [usage.parser.VolumeUsageParser]
(Usage-Job-1:null) (logid:) Creating Volume usage record for vol: 3246, usage:
24, startDate: Mon Jul 25 00:00:00 BRT 2022, endDate: Mon Jul 25 23:59:59 BRT
2022, for account: 545
And it are being computed to account:
(ascenty) # > list usagerecords domainid=XXX
accountid=d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8 type=6 startdate=2022-07-25
enddate=2022-07-25 filter=account,rawusage,size,startdate,usage,usagetype,
{
"count": 2,
"usagerecord": [
{
"accountid": "d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8",
"rawusage": "24",
"size": 53687091200,
"startdate": "2022-07-25'T'00:00:00-03:00",
"usage": "24 Hrs",
"usagetype": 6
},
{
"accountid": "d14c8cb9-fd92-43c8-9ddb-f2ed4e9af5a8",
"rawusage": "24",
"size": 53687091200,
"startdate": "2022-07-25'T'00:00:00-03:00",
"usage": "24 Hrs",
"usagetype": 6
}
]
}
