I feel that adding all operators to dag makes it explicit for a developer who is looking at the code for the first time.
Besides, one might want to do dag.getMeta on operators and ports to set some attributes. I'm not sure how much explicit will that be if we bypass adding operators to the DAG. Though, I totally agree with the intention of simplifying the APIs for which the proposal from Siyuan in another mail is a nice first step. Regards, Chinmay. On Mon, Apr 11, 2016 at 10:48 AM, Sandesh Hegde <[email protected]> wrote: > Yes, that is the idea. > > On Sun, Apr 10, 2016, 9:28 PM Sandeep Deshmukh <[email protected]> > wrote: > > > Are you suggesting we add only the input adapters to the dag and then all > > the connected ones are added by simply traversing the DAG? > > > > Regards, > > Sandeep > > > > On Fri, Apr 8, 2016 at 11:06 PM, Sandesh Hegde <[email protected]> > > wrote: > > > > > Hello Team, > > > > > > As we were talking about High Level api in another thread, here is a > > small > > > variation proposal for our existing apis, > > > > > > Take look at the current code and the proposed code > > > > > > Current Style > > > --------------------------- > > > KafkaSinglePortStringInputOperator kafkaInput = > > > dag.addOperator("kafkaInput", new > KafkaSinglePortStringInputOperator()); > > > DeserializeJSON deserializeJSON = dag.addOperator("deserialize", > new > > > DeserializeJSON()); > > > RedisJoin redisJoin = dag.addOperator("redisJoin", new > RedisJoin()); > > > CampaignProcessorWithApexWindow campaignProcessor = > > > dag.addOperator("campaignProcessor", new > > > CampaignProcessorWithApexWindow()); > > > > > > // Connect the Ports in the Operators > > > dag.addStream("deserialize", kafkaInput.outputPort, > > > deserializeJSON.input) ; > > > dag.addStream("redisJoin", filterFields.output, redisJoin.input) ; > > > dag.addStream("output", redisJoin.output, campaignProcessor.input); > > > ------------------------------------- > > > > > > > > > Proposed Change ( Just create Operators and connect the output port to > > > input ports and then add the input operators to DAG ) > > > ------------------------------------------- > > > KafkaSinglePortStringInputOperator > > kafkaSinglePortStringInputOperator = > > > new KafkaSinglePortStringInputOperator() ; > > > DeserializeJSON deserializeJSON = new DeserializeJSON(); > > > RedisJoin redisJoin = new RedisJoin() ; > > > CampaignProcessor campaignProcessor = new CampaignProcessor() ; > > > > > > kafkaInput.outputPort.connect(deserializeJSON.input); > > > filterFields.output.connect(redisJoin.input) ; > > > redisJoin.output.connect(campaignProcessor.input) ; > > > > > > dag.add(kafkaInput); > > > ----------------------------------------- > > > > > > Things to note : Name can be generated internally or user can specify a > > > name property of the operator ( like Dag.AddOperator("name", ...) ) > > > > > > Let me know your thoughts. > > > > > > Thanks > > > > > >
