[
https://issues.apache.org/jira/browse/KAFKA-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13443553#comment-13443553
]
Jay Kreps commented on KAFKA-203:
---------------------------------
There are some details around how we track request-level stats. The idea was to
begin and end request measurement at the socket server so it includes the full
lifecycle. Each request would have the following phases:
read time - time spent reading data off the network
queue time - time spent waiting for a processing thread to handle the request
api time - time spent in the api layer
queue time - time spent waiting for a network thread to handle the request
write time - time spent writing data (which would include the sendfile time)
The implementation could just be to add to the Request object we have so that
it has something like
request.begin("read")
// do some reading
request.end("read")
It would be nice to have these stats at the client id level, but there are two
problems:
1. That would be a lot of histogram data
2. Currently the socket server is not aware of the client id.
So I recommend we just track this info on a per-api basis for now. We can
revisit in the future to get more in-depth instrumentation.
> Improve Kafka internal metrics
> ------------------------------
>
> Key: KAFKA-203
> URL: https://issues.apache.org/jira/browse/KAFKA-203
> Project: Kafka
> Issue Type: New Feature
> Components: core
> Affects Versions: 0.8
> Reporter: Jay Kreps
> Assignee: Jay Kreps
> Labels: tools
>
> Currently metrics in kafka are using old-school JMX directly. This makes
> adding metrics a pain. It would be good to do one of the following:
> 1. Convert to Coda Hale's metrics package
> (https://github.com/codahale/metrics)
> 2. Write a simple metrics package
> The new metrics package should make metrics easier to add and work with and
> package up the common logic of keeping windowed gauges, histograms, counters,
> etc. JMX should be just one output of this.
> The advantage of the Coda Hale package is that it exists so we don't need to
> write it. The downsides are (1) introduces another client dependency which
> causes conflicts, and (2) seems a bit heavy on design. The good news is that
> the metrics-core package doesn't seem to bring in a lot of dependencies which
> is nice, though the scala wrapper seems to want scala 2.9. I am also a little
> skeptical of the approach for histograms--it does sampling instead of
> bucketing though that may be okay.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
