Github user kevpeek commented on the issue:
https://github.com/apache/storm/pull/2379
@revans2 I ran some perf tests, and this refactor did slow things down
considerably. I have made a small change that eliminates the main source of
slowness, but it's still not great.
The current version of the code runs in about 150% of the time the original
code does, so certainly not good. Most of the increase in runtime comes from
the more general task selection algorithm. I have been able to speed things up
a little bit more, but I suspect the benefits are not really worth it. Perhaps
that means this code simply doesn't need to be merged in, or it could be added
as a separate grouping, but here are a few observations I made along the way:
* I did not notice the slowdown in our deployment of this code because we
know our key set is only a few thousand elements, allowing us to wrap the
AssignmentCreator in a caching decorator. This trades a larger memory footprint
for better performance. With the caching AssignmentCreator decorator, this
grouping runs in 80% of the time the original takes. This is sort of the point
of the refactor; customizations like this become possible with just a few lines
of code.
* By far, the slowest part of this grouping is the hashing functions in
createAssignment. Perhaps this is one of the optimizations you mention above,
but replacing these with something faster would be a bigger win than optimizing
this refactor.
---
