Yuzhao Chen created STORM-3093: ---------------------------------- Summary: Cache the storm id to executors mapping on master to avoid repeat computation Key: STORM-3093 URL: https://issues.apache.org/jira/browse/STORM-3093 Project: Apache Storm Issue Type: Improvement Components: storm-server Affects Versions: 2.0.0 Reporter: Yuzhao Chen Assignee: Yuzhao Chen Fix For: 2.0.0
Now nimbus will collect all the topologies's conf/topology-ser/storm-base to compute in a scheduling round, which is a very heavy work. The scheduling will still take to minutes even we now change to PRC heartbeats and assignment distribution. So i decide to redesign the scheduler, so we can only schedule the topologies that need to: that have dead workers or not enough number workers. Here i checkout out the code and found that the id->executors mapping is computed every time for every topology, which is really a heavy computation and totally not that necessary, because this mapping is fixed invariable for a topology unless we rebalance or kill it. So i refactor the code a little here, and this is more powerful after the scheduler is resigned for delta-scheduling[ which is very lightweight even there are thousands of topologies on one cluster.] -- This message was sent by Atlassian JIRA (v7.6.3#76005)