[
https://issues.apache.org/jira/browse/STORM-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16421158#comment-16421158
]
Jungtaek Lim commented on STORM-2983:
-------------------------------------
[~roshan_naik] [~ethanli]
Looks like this is about how we provide users to specify resources which are
needed for running topology.
Other than RAS, we assume there’re homogeneous JVM processes (fixed memory, no
restriction on CPU) so overall available memory for topology is configurable
via how much memory end-users assigns to single worker and how many workers
end-users assigns to topology.
This is no longer true in RAS, and maybe container environment, too. If my
understanding is correct, the major point on RAS / container scheduler is fully
utilizing resources in cluster, so user mainly configures how much resources
each component/worker need, and physical plan and placement is left to
scheduler.
I also agree topology configuration should be considered to public API. So if
such configuration is not effective to RAS, RAS document should mention such
behavior. Even better if we explain why it ignores such configuration.
I’m cc.ing [~revans2] since I think he is the best folk to answer about RAS,
and also cc.ing [~erikdw] to see how storm-mesos works for scheduling.
> Some topologies not working properly
> -------------------------------------
>
> Key: STORM-2983
> URL: https://issues.apache.org/jira/browse/STORM-2983
> Project: Apache Storm
> Issue Type: Bug
> Reporter: Ethan Li
> Assignee: Ethan Li
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> For example,
> {code:java}
> bin/storm jar storm-loadgen-*.jar
> org.apache.storm.loadgen.ThroughputVsLatency --spouts 1 --splitters 2
> --counters 1 -c topology.debug=true
> {code}
> on ResourceAwareScheduler not working properly.
> With default cluster settings, there will be only one __acker-executor and it
> will be on a separate worker. And it looks like the __acker-executor was not
> able to receive messages from spouts and bolts. And spouts and bolts
> continued to retry sending messages to acker. It then led to another problem:
> STORM-2970
> I tried to run on storm right before
> [https://github.com/apache/storm/pull/2502] and right after and confirmed
> that this bug should be related to it
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
