[
https://issues.apache.org/jira/browse/STORM-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233795#comment-14233795
]
John Reilly commented on STORM-561:
-----------------------------------
I was using spring as an example, but it should be equally applicable to any
other config like mechanism that already allows you to instantiate objects.
I see this as possibily being 2 problems.
* instantiation of objects to create the bolts, spouts and constructor params
* creation of the storm topology using these objects.
I see your Argument class as defined above to be something similar to the way
that spring uses a "bean" declaration to create an object. All the other
classes are related to how a storm topology is defined. If object
instantiation is something that someones config framework of choice provides,
it would be nice to be able to use that existing mechanism and just use the
classes related to topology definition.
This is just my 2c though, so take it with a grain of salt since it's not a
feature that I see myself using (at least in the near future).
cheers,
John
> Add ability to create topologies dynamically
> --------------------------------------------
>
> Key: STORM-561
> URL: https://issues.apache.org/jira/browse/STORM-561
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Nathan Leung
> Assignee: Nathan Leung
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> It would be nice if a storm topology could be built dynamically, instead of
> requiring a recompile to change parameters (e.g. number of workers, number of
> tasks, layout, etc).
> I would propose the following data structures for building core storm
> topologies. I haven't done a design for trident yet but the intention would
> be to add trident support when core storm support is complete (or in parallel
> if there are other people working on it):
> {code}
> // fields value and arguments are mutually exclusive
> class Argument {
> String argumentType; // Class used to lookup arguments in
> method/constructor
> String implementationType; // Class used to create this argument
> String value; // String used to construct this argument
> List<Argument> arguments; // arguments used to build this argument
> }
> class Dependency {
> String upstreamComponent; // name of upstream component
> String grouping;
> List<Argument> arguments; // arguments for the grouping
> }
> class StormSpout {
> String name;
> String klazz; // Class of this spout
> List <Argument> arguments;
> int numTasks;
> int numExecutors;
> }
> class StormBolt {
> String name;
> String klazz; // Class of this bolt
> List <Argument> arguments;
> int numTasks;
> int numExecutors;
> List<Dependency> dependencies;
> }
> class StormTopologyRepresentation {
> String name;
> List<StormSpout> spouts;
> List<StormBolt> bolts;
> Map config;
> int numWorkers;
> }
> {code}
> Topology creation will be built on top of the data structures above. The
> benefits:
> * Dependency free. Code to unmarshal from json, xml, etc, can be kept in
> extensions, or as examples, and users can write a different unmarshaller if
> they want to use a different text representation.
> * support arbitrary spout and bolts types
> * support of all groupings, streams, via reflections
> * ability to specify configuration map via config file
> * reification of spout / bolt / dependency arguments
> ** recursive argument reification for complex objects
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
