[
https://issues.apache.org/jira/browse/SAMZA-421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200301#comment-14200301
]
Martin Kleppmann commented on SAMZA-421:
----------------------------------------
How many messages did you process during this profiling run? I have no doubt
that the code is fast when it's only processed a small number of messages. My
concern was that it would slow down or run out of memory when it has collected
5 minutes worth of measurements (at 1M msgs/sec, that would be 300M
measurements). Not sure how much memory a ConcurrentSkipListMap<Long, Long>
with 300M entries will take, but probably a couple of gigs. I still feel that's
kinda heavyweight for a metrics system.
The overhead wouldn't necessarily show up in RunLoop.process(), but also when
the metrics system periodically takes a snapshot of the measurements, and in GC.
> Test the performance before and after adding the Timer metric
> -------------------------------------------------------------
>
> Key: SAMZA-421
> URL: https://issues.apache.org/jira/browse/SAMZA-421
> Project: Samza
> Issue Type: Test
> Components: metrics
> Reporter: Yan Fang
> Assignee: Yan Fang
> Labels: newbie
> Attachments: testTimer1.png, update-timer-speed.png
>
>
> As
> [recommended|https://issues.apache.org/jira/browse/SAMZA-349?focusedCommentId=14128365&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14128365]
> by Martin in SAMZA-349, it's worth testing the performance before and after
> implementing the Timer metric.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
