Re: Why RocksDB metrics cache-usage is larger than cache-capacity

[Lei Wang]

Sorry, it was probably  an observation mistake.

I export the metrics to Prometheus and query the result on grafana,
actually the usage will not exceed the capacity

Thanks,
Lei

On Fri, Apr 19, 2024 at 9:55 AM Hangxiang Yu  wrote:

> Hi, Lei.
> It's indeed a bit confusing. Could you share the related rocksdb log which
> may contain more detailed info ?
>
> On Fri, Apr 12, 2024 at 12:49 PM Lei Wang  wrote:
>
>>
>> I enable RocksDB native metrics and do some performance tuning.
>>
>> state.backend.rocksdb.block.cache-size is set to 128m，4 slots for each
>> TaskManager.
>>
>> The observed result for one specific parallel slot:
>> state.backend.rocksdb.metrics.block-cache-capacity  is about 14.5M
>>   state.backend.rocksdb.metrics.block-cache-usage  is about  24M
>>
>> I am a little confused, why usage is larger than capacity?
>>
>> Thanks,
>> Lei
>>
>
>
> --
> Best,
> Hangxiang.
>

Re: Why RocksDB metrics cache-usage is larger than cache-capacity

[Hangxiang Yu]

Hi, Lei.
It's indeed a bit confusing. Could you share the related rocksdb log which
may contain more detailed info ?

On Fri, Apr 12, 2024 at 12:49 PM Lei Wang  wrote:

>
> I enable RocksDB native metrics and do some performance tuning.
>
> state.backend.rocksdb.block.cache-size is set to 128m，4 slots for each
> TaskManager.
>
> The observed result for one specific parallel slot:
> state.backend.rocksdb.metrics.block-cache-capacity  is about 14.5M
>   state.backend.rocksdb.metrics.block-cache-usage  is about  24M
>
> I am a little confused, why usage is larger than capacity?
>
> Thanks,
> Lei
>


-- 
Best,
Hangxiang.

Why RocksDB metrics cache-usage is larger than cache-capacity

[Lei Wang]

I enable RocksDB native metrics and do some performance tuning.

state.backend.rocksdb.block.cache-size is set to 128m，4 slots for each
TaskManager.

The observed result for one specific parallel slot:
state.backend.rocksdb.metrics.block-cache-capacity  is about 14.5M
  state.backend.rocksdb.metrics.block-cache-usage  is about  24M

I am a little confused, why usage is larger than capacity?

Thanks,
Lei