[
https://issues.apache.org/jira/browse/STORM-561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14232767#comment-14232767
]
John Reilly edited comment on STORM-561 at 12/3/14 8:36 AM:
------------------------------------------------------------
I was thinking the object itself, not a serialized representation. I figured
that providing the object would be the responsibility of whatever is using
these classes, probably as part of the deserialization.
One example I can imagine is that these classes are used in spring xml or
something similar
{code}
<bean id="myspout" class="StormSpout">
<property name="name" value="myspout"/>
<property name="klazz" value="com.example.spout.MySpout"/>
<property name="arguments">
<list><bean id="arg1" class="com.example.MyExampleObject"/></list>
</property>
<property name="numTasks" value="2"/>
<property name="numExecutors" value="2"/>
</bean>
{code}
Although...after typing that, it feels like the creation of the spouts and
bolts is even something that could be delegated to the whatever is using these
core classes. e.g.
{code}
<bean id="myspout" class="StormSpout">
<property name="name" value="myspout"/>
<property name="spout">
<bean class="com.example.spout.MySpout">
<constructor-arg index="0">
<bean id="arg1" class="com.example.MyExampleObject"/>
</constructor-arg>
</bean>
</property>
<property name="numTasks" value="2"/>
<property name="numExecutors" value="2"/>
</bean>
where StormSpout looks like
class StormSpout {
String name;
ISpout spout;
int numTasks;
int numExecutors;
}
{code}
was (Author: jr):
I was thinking the object itself, not a serialized representation. I figured
that providing the object would be the responsibility of whatever is using
these classes, probably as part of the deserialization.
One example I can imagine is that these classes are used in spring xml or
something similar
{code}
<bean id="myspout" class="StormSpout">
<property name="name" value="myspout"/>
<property name="klazz" value="com.example.spout.MySpout"/>
<property name="arguments">
<list><bean id="arg1" class="com.example.MyExampleObject"/></list>
</property>
<property name="numTasks" value="2"/>
<property name="numExecutors" value="2"/>
</bean>
{code}
Although...after typing that, it feels like the creation of the spouts and
bolts is even something that could be delegated to the whatever is using these
core classes. e.g.
{code}
<bean id="myspout" class="StormSpout">
<property name="name" value="myspout"/>
<property name="spout">
<bean class="com.example.spout.MySpout">
<constructor-arg index="0">
<bean id="arg1" class="com.example.MyExampleObject"/>
</constructor-arg>
</bean>
</property>
<property name="numTasks" value="2"/>
<property name="numExecutors" value="2"/>
</bean>
and
class StormSpout {
String name;
ISpout spout;
int numTasks;
int numExecutors;
}
{code}
> Add ability to create topologies dynamically
> --------------------------------------------
>
> Key: STORM-561
> URL: https://issues.apache.org/jira/browse/STORM-561
> Project: Apache Storm
> Issue Type: Improvement
> Reporter: Nathan Leung
> Assignee: Nathan Leung
> Original Estimate: 336h
> Remaining Estimate: 336h
>
> It would be nice if a storm topology could be built dynamically, instead of
> requiring a recompile to change parameters (e.g. number of workers, number of
> tasks, layout, etc).
> I would propose the following data structures for building core storm
> topologies. I haven't done a design for trident yet but the intention would
> be to add trident support when core storm support is complete (or in parallel
> if there are other people working on it):
> {code}
> // fields value and arguments are mutually exclusive
> class Argument {
> String argumentType; // Class used to lookup arguments in
> method/constructor
> String implementationType; // Class used to create this argument
> String value; // String used to construct this argument
> List<Argument> arguments; // arguments used to build this argument
> }
> class Dependency {
> String upstreamComponent; // name of upstream component
> String grouping;
> List<Argument> arguments; // arguments for the grouping
> }
> class StormSpout {
> String name;
> String klazz; // Class of this spout
> List <Argument> arguments;
> int numTasks;
> int numExecutors;
> }
> class StormBolt {
> String name;
> String klazz; // Class of this bolt
> List <Argument> arguments;
> int numTasks;
> int numExecutors;
> List<Dependency> dependencies;
> }
> class StormTopologyRepresentation {
> String name;
> List<StormSpout> spouts;
> List<StormBolt> bolts;
> Map config;
> int numWorkers;
> }
> {code}
> Topology creation will be built on top of the data structures above. The
> benefits:
> * Dependency free. Code to unmarshal from json, xml, etc, can be kept in
> extensions, or as examples, and users can write a different unmarshaller if
> they want to use a different text representation.
> * support arbitrary spout and bolts types
> * support of all groupings, streams, via reflections
> * ability to specify configuration map via config file
> * reification of spout / bolt / dependency arguments
> ** recursive argument reification for complex objects
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
