I prefer explicit "modules" section. Just because it's more intuitive.
On Mon, Nov 30, 2015 at 8:21 AM, Thomas Weise <[email protected]> wrote: > +1 > > If there are really module specific constructs, they can be added later. > > Why is the same not supported in the Java API, i.e. why not allow a module > to be added like an operator and handle the distinction in the > implementation? > > On Mon, Nov 30, 2015 at 8:08 AM, Tushar Gosavi <[email protected]> > wrote: > > > Hi All, > > > > I am going to add a feature to allow use of modules in property file and > > json api. > > I am planing to use same "operators" array for adding modules as well as > > operators. The reasoning behind is that user don't have to worry about > the > > component being added to the DAG is actually an operator or a module, as > > the method to specify their properties and to create instance of them is > > same. And it results in less complex syntax for writing application using > > json/property file. > > > > For example the sample Application specified using Json will look like > > below. > > ```json > > { > > "operators": [ > > { > > "name": "operator1", > > "class": "com.datatorrent.lib.operator.Input", > > "properties": { > > "property1": "value1" > > } > > }, > > { > > "name": "module1", > > "class": "com.datatorrent.module.Module1", > > "properties": { > > "property1": "value1" > > } > > }, > > ], > > "streams": [ > > { > > "name": "s1", > > "source": { > > "operatorName": "operator1", > > "portName": "output" > > }, > > "sinks": [ > > { > > "operatorName": "module", > > "portName": "input" > > } > > ] > > } > > ] > > } > > ``` > > > > In above example "module1" is of type Module. While processing the Json > if > > the > > class of component is of type Module then we will use addModule api to > the > > DAG. > > > > Let me know, what do you think about this. > > > > Regards, > > - Tushar. > > > > On Tue, Oct 13, 2015 at 6:32 PM, Tushar Gosavi <[email protected]> > > wrote: > > > > > > > > A module is superset of operator before expansion in logical plan, > except > > > operator > > > attribute are not applicable to modules. As Tim suggested we will have > to > > > add checks > > > later to validate these attributes settings, and not allow top level > > > operator and > > > module to share same name. > > > > > > The same question comes when we want to add module to application using > > > property > > > file API and JSON file API. > > > > > > dt.operator.<operatorname>.classname=<fqcn of operator> > > > > > > One approach we can take is, if class is of type module, then use > > > addModule api > > > to add module to the dag, else use addOperator api. The initialization > of > > > object and applying property is generic can be reused by module. > > > > > > Similarly we have Json API where all operators are specified under > > > operators array > > > > > > { > > > "operators": [ > > > {.. operators .. } > > > ], > > > "streams": [ > > > {.. streams .. } > > > ] > > > } > > > > > > We could add module in the operators array, and based on the object > type > > > we can > > > consider it as module or operator. The mechanism of initializing object > > > and applying > > > properties remains same. > > > > > > - Tushar. > > > > > > > > > > > > > > > > > >
