Hello Chinmay, Idea is to follow natural order. Suppose you want to set attribute on a port, you will have to write something like below
dag.setInputPortAttribute(deserializeJSON.input, Context.PortContext.PARTITION_PARALLEL, true); How about setting it directly on the ports? deserializeJSON.input.setAttribute(Context.PortContext.PARTITION_PARALLEL, true)); Same thing goes for attributes of the Operators. These new set of APIs will support all the features of our existing API, but in a more natural way. It is not similar to High-Level API, it is just moonlighting idea about our existing API. Thanks On Sun, Apr 10, 2016 at 11:20 PM Chinmay Kolhatkar <[email protected]> wrote: > I feel that adding all operators to dag makes it explicit for a developer > who is looking at the code for the first time. > > Besides, one might want to do dag.getMeta on operators and ports to set > some attributes. > I'm not sure how much explicit will that be if we bypass adding operators > to the DAG. > > Though, I totally agree with the intention of simplifying the APIs for > which the proposal from Siyuan in another mail is a nice first step. > > Regards, > Chinmay. > > > > On Mon, Apr 11, 2016 at 10:48 AM, Sandesh Hegde <[email protected]> > wrote: > > > Yes, that is the idea. > > > > On Sun, Apr 10, 2016, 9:28 PM Sandeep Deshmukh <[email protected]> > > wrote: > > > > > Are you suggesting we add only the input adapters to the dag and then > all > > > the connected ones are added by simply traversing the DAG? > > > > > > Regards, > > > Sandeep > > > > > > On Fri, Apr 8, 2016 at 11:06 PM, Sandesh Hegde < > [email protected]> > > > wrote: > > > > > > > Hello Team, > > > > > > > > As we were talking about High Level api in another thread, here is a > > > small > > > > variation proposal for our existing apis, > > > > > > > > Take look at the current code and the proposed code > > > > > > > > Current Style > > > > --------------------------- > > > > KafkaSinglePortStringInputOperator kafkaInput = > > > > dag.addOperator("kafkaInput", new > > KafkaSinglePortStringInputOperator()); > > > > DeserializeJSON deserializeJSON = dag.addOperator("deserialize", > > new > > > > DeserializeJSON()); > > > > RedisJoin redisJoin = dag.addOperator("redisJoin", new > > RedisJoin()); > > > > CampaignProcessorWithApexWindow campaignProcessor = > > > > dag.addOperator("campaignProcessor", new > > > > CampaignProcessorWithApexWindow()); > > > > > > > > // Connect the Ports in the Operators > > > > dag.addStream("deserialize", kafkaInput.outputPort, > > > > deserializeJSON.input) ; > > > > dag.addStream("redisJoin", filterFields.output, redisJoin.input) > ; > > > > dag.addStream("output", redisJoin.output, > campaignProcessor.input); > > > > ------------------------------------- > > > > > > > > > > > > Proposed Change ( Just create Operators and connect the output port > to > > > > input ports and then add the input operators to DAG ) > > > > ------------------------------------------- > > > > KafkaSinglePortStringInputOperator > > > kafkaSinglePortStringInputOperator = > > > > new KafkaSinglePortStringInputOperator() ; > > > > DeserializeJSON deserializeJSON = new DeserializeJSON(); > > > > RedisJoin redisJoin = new RedisJoin() ; > > > > CampaignProcessor campaignProcessor = new CampaignProcessor() ; > > > > > > > > kafkaInput.outputPort.connect(deserializeJSON.input); > > > > filterFields.output.connect(redisJoin.input) ; > > > > redisJoin.output.connect(campaignProcessor.input) ; > > > > > > > > dag.add(kafkaInput); > > > > ----------------------------------------- > > > > > > > > Things to note : Name can be generated internally or user can > specify a > > > > name property of the operator ( like Dag.AddOperator("name", ...) ) > > > > > > > > Let me know your thoughts. > > > > > > > > Thanks > > > > > > > > > >
