Cool! did not know that :) Will try this approach too! Thanks, Isha
On Wed, Dec 2, 2015 at 10:45 PM, Gaurav Gupta <[email protected]> wrote: > Here is code snippet > > public class DownStreamReceiver extends AbstractFileInputOperator > implements Operator.IdleTimeHandler{ > @Override > public void handleIdleTime() > { > if(upstreamDoneReading){ // this is set to true only after > receiving the trigger from 1st reader > emitTuples(); > } > } > } > > Thanks > - Gaurav > > > On Dec 2, 2015, at 10:41 PM, Gaurav Gupta <[email protected]> > wrote: > > > > Isha, > > > > I know if you add input port it becomes generic operator and for that > you can use IdleTimeHandler and in handleIdleTime call emitTuples only if > you have received trigger from 1st reader. Hope that helps. > > > > Thanks > > - Gaurav > > > >> On Dec 2, 2015, at 10:34 PM, Isha Arkatkar <[email protected] > <mailto:[email protected]>> wrote: > >> > >> Hey Gaurav, > >> > >> No that does not work I am afraid. I tried the same thing first. But > when > >> you have a connected input port even if it is for an Input operator, the > >> Operator type changes from INPUT to GENERIC. emitTuples is invoked in > loop > >> only for InputNode.. So, operator emits nothing if I add a connected > input > >> stream to it. > >> I think, though, this might be nice to have if sending triggers between > >> input operators is a common observed pattern. > >> > >> I will try out the StatsListener approach. > >> > >> Thanks, > >> Isha > >> > >> > >> > >> > >> > >> On Wed, Dec 2, 2015 at 10:10 PM, Gaurav Gupta <[email protected] > <mailto:[email protected]>> > >> wrote: > >> > >>> Can this not be done as follows > >>> > >>> 2nd reader has an input port which is connected to output port of 1st > >>> reader. Once the 1st reader is done reading it can send trigger to 2nd > >>> reader over the output port and 2nd reader starts reading once it gets > >>> trigger. > >>> > >>> Thanks > >>> - Gaurav > >>> > >>>> On Dec 2, 2015, at 7:19 PM, Sandeep Deshmukh <[email protected] > <mailto:[email protected]>> > >>> wrote: > >>>> > >>>> You use StatsListener shared between two operators to trigger this. > >>>> > >>>> Regards, > >>>> Sandeep > >>>> > >>>> On Thu, Dec 3, 2015 at 8:08 AM, Isha Arkatkar <[email protected] > <mailto:[email protected]>> > >>> wrote: > >>>> > >>>>> Hi, > >>>>> > >>>>> Yes, that should work. But suppose first operator rolled back to > >>>>> previous checkpoint due to some fail over, the state of application > >>> would > >>>>> be reset. Except the part that the 2nd file reader which was not > >>> emitting > >>>>> anything in those windows, now will continue to emit tuples. That > will > >>> make > >>>>> the state inconsistent. > >>>>> May be I can create the link from downstream operator after first > >>> reader is > >>>>> done to handle that. > >>>>> > >>>>> Thanks, > >>>>> Isha > >>>>> > >>>>> On Wed, Dec 2, 2015 at 5:02 PM, Munagala Ramanath < > [email protected] <mailto:[email protected]>> > >>>>> wrote: > >>>>> > >>>>>> One way (a bit hacky) is to have the 2nd operator monitor an empty > >>>>>> directory. Then, when the > >>>>>> 1st is done reading, it sends a control tuple to a "file-link" > >>> operator; > >>>>>> that operator creates > >>>>>> symbolic links from the directory monitored by the 2nd operator to > the > >>>>>> actual input files. > >>>>>> That should then trigger the 2nd input operator to do its thing. > >>>>>> > >>>>>> Ram > >>>>>> > >>>>>> On Wed, Dec 2, 2015 at 4:43 PM, Isha Arkatkar <[email protected] > <mailto:[email protected]>> > >>>>>> wrote: > >>>>>> > >>>>>>> Hi all, > >>>>>>> > >>>>>>> I have an application with 2 input file reader operators. In this > >>>>>> case, > >>>>>>> want to trigger start reading from 2nd input location, only after > 1st > >>>>>>> operator is done reading. What is the best way to do this? > >>>>>>> > >>>>>>> Thanks! > >>>>>>> Isha > >>>>>>> > >>>>>> > >>>>> > >>> > >>> > > > >
