Github user HeartSaVioR commented on the issue:
https://github.com/apache/storm/pull/2261
Now I introduce 'skip checking update count' to avoid calling
System.currentTimeMillis() every time, but it has clear trade-off, we should
call AtomicInteger.incrementAndGet() every time.
I set the skip checking update index to 100 for starting value, and re-run
tests. Test result is below:
> testBenchmarkLoadAwareShuffleGroupingEvenLoad
Duration: 39352 ms
Duration: 39753 ms
Duration: 40604 ms
> testBenchmarkLoadAwareShuffleGroupingUnevenLoad
Duration: 39615 ms
Duration: 39760 ms
Duration: 40497 ms
> testBenchmarkLoadAwareShuffleGroupingEvenLoadAndMultiThreaded
Max duration among threads is : 113115 ms
Max duration among threads is : 169270 ms
Max duration among threads is : 142175 ms
> testBenchmarkLoadAwareShuffleGroupingUnevenLoadAndMultiThreaded
Max duration among threads is : 168007 ms
Max duration among threads is : 148562 ms
Max duration among threads is : 168780 ms
Now multi-threads hurts more on performance, but still faster than old
LASG's. What we really achieve is performance improvement with single-thread.
we reduced more than half of time than before.
I expect that greater value of 'skip checking update index' would make
single-thread much faster, and maybe multi-threads faster, so there's room to
explore, but the value should be reasonable to answer the question: "Are we OK
to delay updating load information if less than N (the value) calls occurred
within 1 sec?"
Btw, update duration (M secs) is another variable to explore. Maybe also
need to see how often origin load information gets updated, since it is
meaningless that LASG updates the information more often then origin load
information gets updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
