[ https://issues.apache.org/jira/browse/STORM-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531599#comment-14531599 ]
ASF GitHub Bot commented on STORM-561: -------------------------------------- Github user HeartSaVioR commented on a diff in the pull request: https://github.com/apache/storm/pull/546#discussion_r29810721 --- Diff: external/flux/flux-examples/README.md --- @@ -0,0 +1,68 @@ +# Flux Examples +A collection of examples illustrating various capabilities. + +## Building From Source and Running --- End diff -- It should be changed to clone Storm project and cd to external/flux, and build. (or build whole modules from Storm root) > Add ability to create topologies dynamically > -------------------------------------------- > > Key: STORM-561 > URL: https://issues.apache.org/jira/browse/STORM-561 > Project: Apache Storm > Issue Type: Improvement > Reporter: Nathan Leung > Assignee: P. Taylor Goetz > Original Estimate: 336h > Remaining Estimate: 336h > > It would be nice if a storm topology could be built dynamically, instead of > requiring a recompile to change parameters (e.g. number of workers, number of > tasks, layout, etc). > I would propose the following data structures for building core storm > topologies. I haven't done a design for trident yet but the intention would > be to add trident support when core storm support is complete (or in parallel > if there are other people working on it): > {code} > // fields value and arguments are mutually exclusive > class Argument { > String argumentType; // Class used to lookup arguments in > method/constructor > String implementationType; // Class used to create this argument > String value; // String used to construct this argument > List<Argument> arguments; // arguments used to build this argument > } > class Dependency { > String upstreamComponent; // name of upstream component > String grouping; > List<Argument> arguments; // arguments for the grouping > } > class StormSpout { > String name; > String klazz; // Class of this spout > List <Argument> arguments; > int numTasks; > int numExecutors; > } > class StormBolt { > String name; > String klazz; // Class of this bolt > List <Argument> arguments; > int numTasks; > int numExecutors; > List<Dependency> dependencies; > } > class StormTopologyRepresentation { > String name; > List<StormSpout> spouts; > List<StormBolt> bolts; > Map config; > int numWorkers; > } > {code} > Topology creation will be built on top of the data structures above. The > benefits: > * Dependency free. Code to unmarshal from json, xml, etc, can be kept in > extensions, or as examples, and users can write a different unmarshaller if > they want to use a different text representation. > * support arbitrary spout and bolts types > * support of all groupings, streams, via reflections > * ability to specify configuration map via config file > * reification of spout / bolt / dependency arguments > ** recursive argument reification for complex objects -- This message was sent by Atlassian JIRA (v6.3.4#6332)