Github user roshannaik commented on the pull request:
https://github.com/apache/storm/pull/1217#issuecomment-197551650
@abhishekagarwal87
This perf hit of 7-9% exists even if user does not enable logging from the
UI. With logging enabled, it should be even higher.
@arunmahadevan
The perf improvement i described here is observed when going from the
default value nil to 0 for topology.eventlogger.executors
**Topology detail:** 1 spout, 1 bolt, 1 acker. Spout generates random
numbers. Bolt does this math on the tuple: (value+1)*2. In effect this is
just a speed of light topology.
The perf hit that i noted is actually due to those very same checking of
flag.
This checking of flags in clojure turns into very expensive lookups in
clojure internals. It internally invokes Java reflection! If you are thinking
'what the hell'.. then yes that was my reaction too.
Specifically this is the problematic lookup for this code path :
'storm-component->debug-atom'
I agree with @arunmahadevan 's concern that this will confuse the users
when they don't see logs after enabling it on the UI.
The alternative fix for this is to change the manner in which this flag is
made available to the code. Basically make it more efficient.
There are some other lookups in the critical path that are also are causing
perf hits... which i plan to address in a separate jira.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
