jmpoholarz opened a new pull request #3385: URL: https://github.com/apache/storm/pull/3385
## What is the purpose of the change The Trident Kafka Spout/Emitter does not measure a few metrics that my team has found useful over the years using the spout. This PR adds these couple metrics back to the open source. 1. **Event emit rate** Having a count of how many events are being emitted by the spout can track the rate at which the spout is consuming from the kafka topic(s). If this rate decreases, it signals performance issues to investigate. (@kishorvpatil suggested using a meter here. The initial thought was to just use a Counter or a Gauge which resets to 0 every time getValue() is called) 2. **Kafka Spout Max Lag** This gauge value monitors the backlog of events to determine if the spout is not emitting fast enough to keep up with the incoming data or if the spout is current and Kafka topic delayed. This metric can be used to distinguish between these two cases and determine the source of the slowdown. ## How was the change tested The unit tests for the emitter verify the counts match the expected values. Additionally, my team compiled the storm-kafka-client package and included it as a dependency in our project. After launching our topologies, we verified that the metric values on our YAMAS charts matched the equivalent metrics we had hardcoded into our custom Kafka emitter. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
