[ https://issues.apache.org/jira/browse/STORM-3093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Joseph Evans resolved STORM-3093. ---------------------------------------- Resolution: Fixed Thanks [~danny0405], I merged this into master. > Cache the storm id to executors mapping on master to avoid repeat computation > ----------------------------------------------------------------------------- > > Key: STORM-3093 > URL: https://issues.apache.org/jira/browse/STORM-3093 > Project: Apache Storm > Issue Type: Improvement > Components: storm-server > Affects Versions: 2.0.0 > Reporter: Yuzhao Chen > Assignee: Yuzhao Chen > Priority: Major > Labels: pull-request-available > Fix For: 2.0.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Now nimbus will collect all the topologies's conf/topology-ser/storm-base to > compute in a scheduling round, which is a very heavy work. The scheduling > will still take to minutes even we now change to PRC heartbeats and > assignment distribution. > So i decide to redesign the scheduler, so we can only schedule the topologies > that need to: that have dead workers or not enough number workers. > Here i checkout out the code and found that the id->executors mapping is > computed every time for every topology, which is really a heavy computation > and totally not that necessary, because this mapping is fixed invariable for > a topology unless we rebalance or kill it. > So i refactor the code a little here, and this is more powerful after the > scheduler is resigned for delta-scheduling[ which is very lightweight even > there are thousands of topologies on one cluster.] > For now this is enough for us. -- This message was sent by Atlassian JIRA (v7.6.3#76005)