[
https://issues.apache.org/jira/browse/KAFKA-13647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17488169#comment-17488169
]
Sylvain Le Gouellec commented on KAFKA-13647:
---------------------------------------------
Hi [~cadonna] -, First thanks for your support.
Are you sure this is the number of files opened since last measurement ?
No just, the number of files opened actually ?
Sensor : numberOfOpenFilesSensor
numberOfOpenFilesSensor(), addSumMetrics() add metrics new CumulativeSum()
Example (follow your explications):
Time t0 :
0 file opens (reset counter)
0 file closed
numberOfOpenFiles = file opens - file closed = 0
numberOfOpenFilesSensor.record(numberOfOpenFiles);
- metrics number-open-files : CumulativeSum => 0
t+1 :
10 file opens (10 more file opens and reset counter)
0 file closed
numberOfOpenFiles = file opens - file closed = 10
numberOfOpenFilesSensor.record(numberOfOpenFiles);
- metrics number-open-files : CumulativeSum => (previous value)0 + (new
one)10 => 10
t+2 :
2 file opens (2 more file opens)
0 file closed
numberOfOpenFiles = file opens - file closed = 2
numberOfOpenFilesSensor.record(numberOfOpenFiles);
- metrics number-open-files : CumulativeSum => (previous)10 + (new)2 => 12
Ok, in this example, it seems to work.
But if create a simple kafka streams statefull application, and tracking the
number-open-files metrics, compare the number of files inside the state store
folder and lsof return values, it's not the same.
It seems that the number-open-files increase linearly (f(x) = ax).
No ? (see screenshots in bellow)
!image-2022-02-07-16-06-25-304.png!!image-2022-02-07-16-06-53-164.png!
!image-2022-02-07-16-06-39-821.png!
> RocksDb metrics 'number-open-files' is not correct
> --------------------------------------------------
>
> Key: KAFKA-13647
> URL: https://issues.apache.org/jira/browse/KAFKA-13647
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 3.0.0
> Reporter: Sylvain Le Gouellec
> Priority: Major
> Attachments: image-2022-02-07-16-06-25-304.png,
> image-2022-02-07-16-06-39-821.png, image-2022-02-07-16-06-53-164.png
>
>
> We were looking at RocksDB metrics and noticed that the {{number-open-files}}
> metric behaves like a counter, rather than a gauge.
> Looking at the code, we think there is a small error in the type of metric
> for that specific mbean (should be a value metric rather than a sum metric).
> See [
> https://github.com/apache/kafka/blob/ca5d6f9229c170beb23809159113037f05a1120f/streams/src/main/java/org/apache/kafka/streams/state/internals/metrics/RocksDBMetrics.java#L482|https://github.com/apache/kafka/blob/99b9b3e84f4e98c3f07714e1de6a139a004cbc5b/streams/src/main/java/org/apache/kafka/streams/state/internals/metrics/RocksDBMetrics.java#L482]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
