Is handleIdleTime guaranteed to to be called? Can there be a case when the machine is loaded and hence don't have any cycles to invoke this extra functionality.
Regards, Sandeep On Thu, Dec 3, 2015 at 12:17 PM, Isha Arkatkar <[email protected]> wrote: > Cool! did not know that :) > Will try this approach too! > > Thanks, > Isha > > On Wed, Dec 2, 2015 at 10:45 PM, Gaurav Gupta <[email protected]> > wrote: > > > Here is code snippet > > > > public class DownStreamReceiver extends AbstractFileInputOperator > > implements Operator.IdleTimeHandler{ > > @Override > > public void handleIdleTime() > > { > > if(upstreamDoneReading){ // this is set to true only after > > receiving the trigger from 1st reader > > emitTuples(); > > } > > } > > } > > > > Thanks > > - Gaurav > > > > > On Dec 2, 2015, at 10:41 PM, Gaurav Gupta <[email protected]> > > wrote: > > > > > > Isha, > > > > > > I know if you add input port it becomes generic operator and for that > > you can use IdleTimeHandler and in handleIdleTime call emitTuples only if > > you have received trigger from 1st reader. Hope that helps. > > > > > > Thanks > > > - Gaurav > > > > > >> On Dec 2, 2015, at 10:34 PM, Isha Arkatkar <[email protected] > > <mailto:[email protected]>> wrote: > > >> > > >> Hey Gaurav, > > >> > > >> No that does not work I am afraid. I tried the same thing first. But > > when > > >> you have a connected input port even if it is for an Input operator, > the > > >> Operator type changes from INPUT to GENERIC. emitTuples is invoked in > > loop > > >> only for InputNode.. So, operator emits nothing if I add a connected > > input > > >> stream to it. > > >> I think, though, this might be nice to have if sending triggers > between > > >> input operators is a common observed pattern. > > >> > > >> I will try out the StatsListener approach. > > >> > > >> Thanks, > > >> Isha > > >> > > >> > > >> > > >> > > >> > > >> On Wed, Dec 2, 2015 at 10:10 PM, Gaurav Gupta <[email protected] > > <mailto:[email protected]>> > > >> wrote: > > >> > > >>> Can this not be done as follows > > >>> > > >>> 2nd reader has an input port which is connected to output port of 1st > > >>> reader. Once the 1st reader is done reading it can send trigger to > 2nd > > >>> reader over the output port and 2nd reader starts reading once it > gets > > >>> trigger. > > >>> > > >>> Thanks > > >>> - Gaurav > > >>> > > >>>> On Dec 2, 2015, at 7:19 PM, Sandeep Deshmukh < > [email protected] > > <mailto:[email protected]>> > > >>> wrote: > > >>>> > > >>>> You use StatsListener shared between two operators to trigger this. > > >>>> > > >>>> Regards, > > >>>> Sandeep > > >>>> > > >>>> On Thu, Dec 3, 2015 at 8:08 AM, Isha Arkatkar <[email protected] > > <mailto:[email protected]>> > > >>> wrote: > > >>>> > > >>>>> Hi, > > >>>>> > > >>>>> Yes, that should work. But suppose first operator rolled back to > > >>>>> previous checkpoint due to some fail over, the state of application > > >>> would > > >>>>> be reset. Except the part that the 2nd file reader which was not > > >>> emitting > > >>>>> anything in those windows, now will continue to emit tuples. That > > will > > >>> make > > >>>>> the state inconsistent. > > >>>>> May be I can create the link from downstream operator after first > > >>> reader is > > >>>>> done to handle that. > > >>>>> > > >>>>> Thanks, > > >>>>> Isha > > >>>>> > > >>>>> On Wed, Dec 2, 2015 at 5:02 PM, Munagala Ramanath < > > [email protected] <mailto:[email protected]>> > > >>>>> wrote: > > >>>>> > > >>>>>> One way (a bit hacky) is to have the 2nd operator monitor an empty > > >>>>>> directory. Then, when the > > >>>>>> 1st is done reading, it sends a control tuple to a "file-link" > > >>> operator; > > >>>>>> that operator creates > > >>>>>> symbolic links from the directory monitored by the 2nd operator to > > the > > >>>>>> actual input files. > > >>>>>> That should then trigger the 2nd input operator to do its thing. > > >>>>>> > > >>>>>> Ram > > >>>>>> > > >>>>>> On Wed, Dec 2, 2015 at 4:43 PM, Isha Arkatkar < > [email protected] > > <mailto:[email protected]>> > > >>>>>> wrote: > > >>>>>> > > >>>>>>> Hi all, > > >>>>>>> > > >>>>>>> I have an application with 2 input file reader operators. In > this > > >>>>>> case, > > >>>>>>> want to trigger start reading from 2nd input location, only after > > 1st > > >>>>>>> operator is done reading. What is the best way to do this? > > >>>>>>> > > >>>>>>> Thanks! > > >>>>>>> Isha > > >>>>>>> > > >>>>>> > > >>>>> > > >>> > > >>> > > > > > > > >
