GitHub user revans2 opened a pull request:
https://github.com/apache/storm/pull/816
STORM-1128: Make metrics fast
The metrics in my profiling are contributing significantly to the CPU
utilization of storm, which slows it down. With this patch I was able to go
from 6,500 sentences/second fully counted to 11,000 on a Mac Book Pro running
two workers in the Throughput vs. Latency test. If I turn off sub-sampling of
metrics the performance goes back to about where it was before. There is still
more optimizations that could be done with the metrics, like having the latency
metric also report counts, so we don't have to do an extra lookup and inc
operation on the critical path, but for now this is good enough, and allows me
to start looking at the next bottleneck.
These changes also compliment the batching changes very well. The batching
went from 6,500 sentences/second to 15,000 by itself. This patch increases it
to 25,000 sentences/second. That is almost a 4x improvement by using both
together.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/revans2/incubator-storm fast-counting
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/816.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #816
----
commit e75219b5a4856c39c4031c3d6205105e034d10af
Author: Robert (Bobby) Evans <ev...@yahoo-inc.com>
Date: 2015-10-22T15:48:04Z
STORM-1128: Make metrics fast
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
