[jira] [Commented] (KAFKA-9675) RocksDB metrics reported always at zero

2020-03-12 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-9675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058178#comment-17058178
 ] 

ASF GitHub Bot commented on KAFKA-9675:
---

vvcephei commented on pull request #8256: KAFKA-9675: Fix bug that prevents 
RocksDB metrics to be updated
URL: https://github.com/apache/kafka/pull/8256
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> RocksDB metrics reported always at zero
> ---
>
> Key: KAFKA-9675
> URL: https://issues.apache.org/jira/browse/KAFKA-9675
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 2.4.0
>Reporter: Michael Viamari
>Assignee: Bruno Cadonna
>Priority: Blocker
> Fix For: 2.6.0, 2.4.2
>
>
> The rocksdb metrics listed under {{stream-state-metrics}} are reported as 
> zero for all metrics and all rocksdb instances. The metrics are present in 
> JMX, but are always zero.
> The streams state is configured with {{MetricsRecordingLevel}} at {{debug}}. 
> I am able to see metrics with appropriate values in the 
> {{stream-rocksdb-window-state-metrics}}, {{stream-record-cache-metrics}}, 
> {{stream-task-metrics}}, and {{stream-processor-node-metrics}}.
> Additionally, my DEBUG logs show the appropriate messages for recording 
> events, i.e.
> {{org.apache.kafka.streams.state.internals.metrics.RocksDBMetricsRecorder 
> [RocksDB Metrics Recorder for agg-store] Recording metrics for store 
> agg-store}}
> It happens that all of my rocksdb instances are windowed stores, not key 
> value stores, so I haven't been able to check if this issue is unique to 
> windowed stores.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (KAFKA-9675) RocksDB metrics reported always at zero

2020-03-09 Thread ASF GitHub Bot (Jira)


[ 
https://issues.apache.org/jira/browse/KAFKA-9675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054948#comment-17054948
 ] 

ASF GitHub Bot commented on KAFKA-9675:
---

cadonna commented on pull request #8256: KAFKA-9675: Fix bug that prevents 
RocksDB metrics to be updated
URL: https://github.com/apache/kafka/pull/8256
 
 
   The root cause of the bug is that the statistics object is passed to the 
RocksDB options after the database is opened. Apparently, the options object is 
copied during the opening process. The solution is to pass the statistics 
object before the database is opened.
   
   Additionally, I added some unit tests to check if the Kafka Streams' metrics 
are updated when the measurements in RocksDB's statistics object are updated.
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> RocksDB metrics reported always at zero
> ---
>
> Key: KAFKA-9675
> URL: https://issues.apache.org/jira/browse/KAFKA-9675
> Project: Kafka
>  Issue Type: Bug
>  Components: streams
>Affects Versions: 2.4.0
>Reporter: Michael Viamari
>Assignee: Bruno Cadonna
>Priority: Major
>
> The rocksdb metrics listed under {{stream-state-metrics}} are reported as 
> zero for all metrics and all rocksdb instances. The metrics are present in 
> JMX, but are always zero.
> The streams state is configured with {{MetricsRecordingLevel}} at {{debug}}. 
> I am able to see metrics with appropriate values in the 
> {{stream-rocksdb-window-state-metrics}}, {{stream-record-cache-metrics}}, 
> {{stream-task-metrics}}, and {{stream-processor-node-metrics}}.
> Additionally, my DEBUG logs show the appropriate messages for recording 
> events, i.e.
> {{org.apache.kafka.streams.state.internals.metrics.RocksDBMetricsRecorder 
> [RocksDB Metrics Recorder for agg-store] Recording metrics for store 
> agg-store}}
> It happens that all of my rocksdb instances are windowed stores, not key 
> value stores, so I haven't been able to check if this issue is unique to 
> windowed stores.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)