[
https://issues.apache.org/jira/browse/KAFKA-385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434655#comment-13434655
]
Joel Koshy commented on KAFKA-385:
----------------------------------
I'm about to upload a rebased patch so I'd like to incorporate as much of this
as I can.
> 1. Can we make the reporters pluggable? We shouldn't hard code those, you
> should just give something like
metrics.reporters=com.xyz.MyReporter, com.xyz.YourReporter
These (and ConsoleReporter) are the only reporters currently available. It
would be good to make it pluggable, but their constructors are different. i.e.,
I don't think it is possible to do this without using a spring-like framework.
> 2. Please remove the reference to scala class names from the logging (e.g.
> DelayedProduce)
I don't follow this comment - can you clarify? Which file are you referring to?
> 3. How are you measuring the performance impact of the change you made to
> synchronization?
I did not - I can measure the satisfaction time with and without the
synchronization but I doubt it would add much overhead. Also, we have to add
synchronization one way or the other - either inside the purgatory or outside
(i.e,. burden the client's usage).
> 4. It would be good to cut-and-paste the scala timer class they provide in
> the scala wrapper. That is really nice.
They == ? Not clear on this - can you clarify?
> 5. Size.
I think the overhead is the following:
8 byte pointer to the value
12 byte object header
8 byte value
Total: 28 bytes
> This is too much memory for something that should just be monitoring. I think
> we should not do per-key histograms.
This is certainly a concern. So with the scenario I have in my previous
comment, this would be > 13MB per broker and > double that if I use
exponentially decaying sampling. That said, there is only one per-key histogram
(which is the follower catch up time). OTOH since the main use I can think of
is to see which followers are slower we can achieve that with grep's in the
log. So I guess the visual benefit comes at a prohibitive memory cost.
> RequestPurgatory enhancements - expire/checkSatisfy issue; add jmx beans
> ------------------------------------------------------------------------
>
> Key: KAFKA-385
> URL: https://issues.apache.org/jira/browse/KAFKA-385
> Project: Kafka
> Issue Type: Bug
> Reporter: Joel Koshy
> Assignee: Joel Koshy
> Labels: bugs
> Fix For: 0.8
>
> Attachments: example_dashboard.jpg, graphite_explorer.jpg,
> KAFKA-385-v1.patch, KAFKA-385-v2.patch
>
>
> As discussed in KAFKA-353:
> 1 - There is potential for a client-side race condition in the
> implementations of expire and checkSatisfied. We can just synchronize on the
> DelayedItem.
> 2 - Would be good to add jmx beans to facilitate monitoring RequestPurgatory
> stats.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
