[
https://issues.apache.org/jira/browse/STORM-190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panfeng Yuan updated STORM-190:
-------------------------------
Description:
we are now testing storm-0.9.0.1 cluster for migrating. When running the
topologies in storm-starter, such as WordCountTopology, ReachTopology, we
found the number of workers, executors, and tasks is unexplainable (according
to
https://github.com/nathanmarz/storm/wiki/Understanding-the-parallelism-of-a-Storm-topology):
1) WordCountTopology:
Name Id Status Uptime Num workers Num
executors Num tasks
wc-test3 wc-test3-3-1388986658 ACTIVE 29m 5s 25 50
50
wc-test2 wc-test2-2-1388986111 ACTIVE 38m 12s 4 29
29
wc-test wc-test-1-1388981297 ACTIVE 1h 58m 26s 3 28
28
This topology's configuration is as follows:
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("spout", new RandomSentenceSpout(), 5);
builder.setBolt("split", new SplitSentence(), 8).shuffleGrouping("spout");
builder.setBolt("count", new WordCount(), 12).fieldsGrouping("split", new
Fields("word"));
......
conf.setNumWorkers(3); #4, 25
2) ReachTopology (which is a drpc topology):
Name Id Status Uptime Num workers Num
executors Num tasks
reach-test reach-test-4-1388986889 ACTIVE 32m 20s 6 35
35
This topology's configuration is as follows:
LinearDRPCTopologyBuilder builder = new LinearDRPCTopologyBuilder("reach");
builder.addBolt(new GetTweeters(), 4);
builder.addBolt(new GetFollowers(), 12).shuffleGrouping();
builder.addBolt(new PartialUniquer(), 6).fieldsGrouping(new Fields("id",
"follower"));
builder.addBolt(new CountAggregator(), 3).fieldsGrouping(new Fields("id"));
......
conf.setNumWorkers(6);
was:
we are now testing storm-0.9.0.1 cluster for migrating. When running the
topologies in storm-starter, such as WordCountTopology, ReachTopology, we
found the number of workers, executors, and tasks is unexplainable (according
to
https://github.com/nathanmarz/storm/wiki/Understanding-the-parallelism-of-a-Storm-topology):
1) WordCountTopology:
Name Id Status Uptime Num workers Num
executors Num tasks
wc-test3 wc-test3-3-1388986658 ACTIVE 29m 5s 25 50
50
wc-test2 wc-test2-2-1388986111 ACTIVE 38m 12s 4 29
29
wc-test wc-test-1-1388981297 ACTIVE 1h 58m 26s 3 28
28
here is the topology's configuration:
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("spout", new RandomSentenceSpout(), 5);
builder.setBolt("split", new SplitSentence(), 8).shuffleGrouping("spout");
builder.setBolt("count", new WordCount(), 12).fieldsGrouping("split", new
Fields("word"));
......
conf.setNumWorkers(3); #4, 25
2) ReachTopology (which is a drpc topology):
Name Id Status Uptime Num workers Num
executors Num tasks
reach-test reach-test-4-1388986889 ACTIVE 32m 20s 6 35
35
here is the topology's configuration:
LinearDRPCTopologyBuilder builder = new LinearDRPCTopologyBuilder("reach");
builder.addBolt(new GetTweeters(), 4);
builder.addBolt(new GetFollowers(), 12).shuffleGrouping();
builder.addBolt(new PartialUniquer(), 6).fieldsGrouping(new Fields("id",
"follower"));
builder.addBolt(new CountAggregator(), 3).fieldsGrouping(new Fields("id"));
......
conf.setNumWorkers(6);
> the allocation of workers, executors and tasks is amazing in our
> storm-0.9.0.1 cluster
> --------------------------------------------------------------------------------------
>
> Key: STORM-190
> URL: https://issues.apache.org/jira/browse/STORM-190
> Project: Apache Storm (Incubating)
> Issue Type: Bug
> Environment: storm-0.9.0.1
> Reporter: Panfeng Yuan
> Labels: executors, tasks, workers
> Original Estimate: 36h
> Remaining Estimate: 36h
>
> we are now testing storm-0.9.0.1 cluster for migrating. When running the
> topologies in storm-starter, such as WordCountTopology, ReachTopology, we
> found the number of workers, executors, and tasks is unexplainable (according
> to
> https://github.com/nathanmarz/storm/wiki/Understanding-the-parallelism-of-a-Storm-topology):
> 1) WordCountTopology:
> Name Id Status Uptime Num workers Num
> executors Num tasks
> wc-test3 wc-test3-3-1388986658 ACTIVE 29m 5s 25 50
> 50
> wc-test2 wc-test2-2-1388986111 ACTIVE 38m 12s 4 29
> 29
> wc-test wc-test-1-1388981297 ACTIVE 1h 58m 26s 3 28
> 28
> This topology's configuration is as follows:
> TopologyBuilder builder = new TopologyBuilder();
> builder.setSpout("spout", new RandomSentenceSpout(), 5);
> builder.setBolt("split", new SplitSentence(), 8).shuffleGrouping("spout");
> builder.setBolt("count", new WordCount(), 12).fieldsGrouping("split", new
> Fields("word"));
> ......
> conf.setNumWorkers(3); #4, 25
> 2) ReachTopology (which is a drpc topology):
> Name Id Status Uptime Num workers Num
> executors Num tasks
> reach-test reach-test-4-1388986889 ACTIVE 32m 20s 6 35
> 35
> This topology's configuration is as follows:
> LinearDRPCTopologyBuilder builder = new
> LinearDRPCTopologyBuilder("reach");
> builder.addBolt(new GetTweeters(), 4);
> builder.addBolt(new GetFollowers(), 12).shuffleGrouping();
> builder.addBolt(new PartialUniquer(), 6).fieldsGrouping(new Fields("id",
> "follower"));
> builder.addBolt(new CountAggregator(), 3).fieldsGrouping(new
> Fields("id"));
> ......
> conf.setNumWorkers(6);
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
