[
https://issues.apache.org/jira/browse/SAMZA-25?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13768923#comment-13768923
]
Chris Riccomini commented on SAMZA-25:
--------------------------------------
I've posted an updated patch which covers:
* Add metrics to SamzaContainer/SamzaContainerMetrics
* Add metrics to TaskInstance/TaskInstanceMetrics
* Add metrics to SystemConsumers
* Add metrics to SystemProducers
* Add metrics to KafkaSystemProducer
* Add metrics to KafkaSystemConsumer
* Add metrics to BrokerProxy
* Add metrics to CachedStore
* Add metrics to LoggedStore
* Add metrics to KeyValueStorageEngine
* Add metrics to LevelDbKeyValueStore
It skips the choosers because we're in the processing of refactoring that code
in SAMZA-2.
Decisions that need feedback:
1. I made the counter and gauges injectable in the MetricsRegistry. This allows
us to have gauges that are always up to date (by overriding getValue to just
return myList.size, for example), instead of having to call .set() all the
time. This pattern is mirrored in Kafka's metrics.
2. Added metrics to all storage layers. Arguably, you only need the
read/write/flush/etc metrics in one, since they should theoretically all match
(when they're wired in, which logged/cached aren't always).
3. I probably missed some apache licenses. Speak up if you see any.
4. Refactored BrokerProxy and KafkaSystemConsumer slightly to use this style of
metrics (they had their own, before).
> Add metrics
> -----------
>
> Key: SAMZA-25
> URL: https://issues.apache.org/jira/browse/SAMZA-25
> Project: Samza
> Issue Type: Bug
> Components: container, kafka, kv, metrics, yarn
> Affects Versions: 0.6.0
> Reporter: Chris Riccomini
> Assignee: Chris Riccomini
> Fix For: 0.7.0
>
> Attachments: SAMZA-25.0.patch
>
>
> The latest refactor dropped a bunch of metrics. We need to re-add them.
> Classes that need metrics:
> * Add metrics to SamzaContainer/SamzaContainerMetrics
> * Add metrics to TaskInstance/TaskInstanceMetrics
> * Add metrics to TaskStorageManager
> * Add metrics to MessageChooser
> * Add metrics to DefaultChooser
> * Add metrics to SystemConsumers
> * Add metrics to SystemProducers
> * Add metrics to KafkaSystemProducer
> * Add metrics to KafkaSystemConsumer
> * Add metrics to BrokerProxy
> * Add metrics to CachedStore
> * Add metrics to LoggedStore
> * Add metrics to KeyValueStorageEngine
> * Add metrics to LevelDbKeyValueStore
> We definitely need msgs/sec, bytes/sec at both the container and task level.
> We also need reads/sec writes/sec on the stores. A lot more, as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
