[
https://issues.apache.org/jira/browse/STORM-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531156#comment-14531156
]
ASF GitHub Bot commented on STORM-561:
--------------------------------------
GitHub user ptgoetz opened a pull request:
https://github.com/apache/storm/pull/546
STORM-561: Add flux as an external module
For a description of everything this does, it's probably easiest to look at
the README file:
https://github.com/ptgoetz/storm/blob/STORM-561/external/flux/README.md
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ptgoetz/storm STORM-561
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/546.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #546
----
commit 25f97369c36e9ad7dffa8dc2699f9454763b234d
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-04T02:46:33Z
Initial commit
commit 0f28b93d2d2e92b547d02b7e1208c2d0fd9cbff9
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-04T02:49:00Z
Create README.md
commit ab2ba93d73a667fd0d6424b23b3f593cafef78d0
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-06T05:27:15Z
initial commit
commit b3f4b742ad9e6d6e73da02390c8433829069dc28
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-07T04:09:46Z
update model
commit 760f3eade7d66faebf4adcfa694e00ea147ff112
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-07T04:10:59Z
add basic test yaml doc
commit 6344a3df8ddc3912a4ab5ba000346f12018f139c
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-07T04:12:35Z
add parser and executable implementations
commit a99edcf64aaf82b51b9a5e45e2fbf68565d20c51
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-07T04:13:31Z
add maven POM
commit 7f8259ab1e136ba3833d744031b5042f97d5fc41
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-07T06:05:43Z
add logback config
commit 3d9804a0409434834dd8bc9c4bf12d7aa9c68a2a
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-07T06:09:06Z
add sample topology test
commit c0abd701838897de1143227afe3ecf73675f51a3
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-08T05:44:04Z
add logback config file
commit 63264abd953a2d4df161f24774d10040ccff7eb3
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-08T06:14:08Z
validate the topology
commit b776f6554a4f9b66266cdf64e5d99055d1aca70c
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-08T06:25:52Z
add pseudo code for component definitions
commit 9cbed269ad4d24dc635cc1ed98a298a850e4c606
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-08T06:50:47Z
update .gitignore
commit 5bf5ee34decc4cfa5733eb1308bc8552dfe459be
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-09T03:47:26Z
wip poc supporting dynamic constructors and shell components
commit 8ef2af5597e16ac99f63047f93edf80a5fa7f1f7
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-09T04:12:08Z
add initial support for bolt constructors
commit 6d446667264b26afdcc528535facc92042b0eb83
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-09T05:30:20Z
allow output field definition
commit a7619bbdfab3bd34944b85e1b05814167ff168bd
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-09T05:31:56Z
add basic test to make sure shell commands can be run
commit 78b20f1f7014e7444aa96436d949d8d9c6b59839
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-09T05:36:25Z
update unit test file for multilang support
commit 9ae0921efa6d83a2819da31484798cc03c9b2004
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-10T02:34:51Z
move multilang resources to location where Storm can find them
commit 75f909cd48ae8e08381cc7e58829b97bee4142be
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-10T02:36:31Z
update contructor to incude declared fields.
commit 5b7d3275807214dcffa8735b3c9fc72ca8efe3ce
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-10T02:40:36Z
Add environment test to make sure shells can be invoked.
commit 0507f69ec6b0ac6913f3a8d5f394b999e25a413a
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-10T02:44:19Z
allow specification of config file on the command line
commit 997eda9104457daa18908259bf37aa2e85d68ddb
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-10T02:46:13Z
improve comments in shell component example
commit 183d454a27b5f4c13ec8e4427b01790049815ce6
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-10T02:57:54Z
rename shell component test
commit 1489bcb90c0f789825dbdec2a033653fc9f7bf15
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-10T04:58:02Z
cleanup
commit 26e96235d70d01f450bd4e8b5c3c346e40b050d3
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-10T05:05:53Z
add TODO items
commit eff34c04ddea20167fa93303a2892aa384ae4de8
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-11T00:28:31Z
implement groupings as an enum
commit fe273d415b544e620125937145611e8bf07ea998
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-11T00:56:29Z
add javadoc
commit 329037081804a80d418bdf608fb1a2e78bbd0a18
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-11T01:41:27Z
add basic javadoc
commit 3aa41f2712ba101effebf20623e4276296915610
Author: P. Taylor Goetz <[email protected]>
Date: 2015-03-11T04:17:57Z
update docs
----
> Add ability to create topologies dynamically
> --------------------------------------------
>
> Key: STORM-561
> URL: https://issues.apache.org/jira/browse/STORM-561
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Nathan Leung
> Assignee: P. Taylor Goetz
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> It would be nice if a storm topology could be built dynamically, instead of
> requiring a recompile to change parameters (e.g. number of workers, number of
> tasks, layout, etc).
> I would propose the following data structures for building core storm
> topologies. I haven't done a design for trident yet but the intention would
> be to add trident support when core storm support is complete (or in parallel
> if there are other people working on it):
> {code}
> // fields value and arguments are mutually exclusive
> class Argument {
> String argumentType; // Class used to lookup arguments in
> method/constructor
> String implementationType; // Class used to create this argument
> String value; // String used to construct this argument
> List<Argument> arguments; // arguments used to build this argument
> }
> class Dependency {
> String upstreamComponent; // name of upstream component
> String grouping;
> List<Argument> arguments; // arguments for the grouping
> }
> class StormSpout {
> String name;
> String klazz; // Class of this spout
> List <Argument> arguments;
> int numTasks;
> int numExecutors;
> }
> class StormBolt {
> String name;
> String klazz; // Class of this bolt
> List <Argument> arguments;
> int numTasks;
> int numExecutors;
> List<Dependency> dependencies;
> }
> class StormTopologyRepresentation {
> String name;
> List<StormSpout> spouts;
> List<StormBolt> bolts;
> Map config;
> int numWorkers;
> }
> {code}
> Topology creation will be built on top of the data structures above. The
> benefits:
> * Dependency free. Code to unmarshal from json, xml, etc, can be kept in
> extensions, or as examples, and users can write a different unmarshaller if
> they want to use a different text representation.
> * support arbitrary spout and bolts types
> * support of all groupings, streams, via reflections
> * ability to specify configuration map via config file
> * reification of spout / bolt / dependency arguments
> ** recursive argument reification for complex objects
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
