Here is code snippet
public class DownStreamReceiver extends AbstractFileInputOperator implements
Operator.IdleTimeHandler{
@Override
public void handleIdleTime()
{
if(upstreamDoneReading){ // this is set to true only after receiving
the trigger from 1st reader
emitTuples();
}
}
}
Thanks
- Gaurav
> On Dec 2, 2015, at 10:41 PM, Gaurav Gupta <[email protected]> wrote:
>
> Isha,
>
> I know if you add input port it becomes generic operator and for that you can
> use IdleTimeHandler and in handleIdleTime call emitTuples only if you have
> received trigger from 1st reader. Hope that helps.
>
> Thanks
> - Gaurav
>
>> On Dec 2, 2015, at 10:34 PM, Isha Arkatkar <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>> Hey Gaurav,
>>
>> No that does not work I am afraid. I tried the same thing first. But when
>> you have a connected input port even if it is for an Input operator, the
>> Operator type changes from INPUT to GENERIC. emitTuples is invoked in loop
>> only for InputNode.. So, operator emits nothing if I add a connected input
>> stream to it.
>> I think, though, this might be nice to have if sending triggers between
>> input operators is a common observed pattern.
>>
>> I will try out the StatsListener approach.
>>
>> Thanks,
>> Isha
>>
>>
>>
>>
>>
>> On Wed, Dec 2, 2015 at 10:10 PM, Gaurav Gupta <[email protected]
>> <mailto:[email protected]>>
>> wrote:
>>
>>> Can this not be done as follows
>>>
>>> 2nd reader has an input port which is connected to output port of 1st
>>> reader. Once the 1st reader is done reading it can send trigger to 2nd
>>> reader over the output port and 2nd reader starts reading once it gets
>>> trigger.
>>>
>>> Thanks
>>> - Gaurav
>>>
>>>> On Dec 2, 2015, at 7:19 PM, Sandeep Deshmukh <[email protected]
>>>> <mailto:[email protected]>>
>>> wrote:
>>>>
>>>> You use StatsListener shared between two operators to trigger this.
>>>>
>>>> Regards,
>>>> Sandeep
>>>>
>>>> On Thu, Dec 3, 2015 at 8:08 AM, Isha Arkatkar <[email protected]
>>>> <mailto:[email protected]>>
>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Yes, that should work. But suppose first operator rolled back to
>>>>> previous checkpoint due to some fail over, the state of application
>>> would
>>>>> be reset. Except the part that the 2nd file reader which was not
>>> emitting
>>>>> anything in those windows, now will continue to emit tuples. That will
>>> make
>>>>> the state inconsistent.
>>>>> May be I can create the link from downstream operator after first
>>> reader is
>>>>> done to handle that.
>>>>>
>>>>> Thanks,
>>>>> Isha
>>>>>
>>>>> On Wed, Dec 2, 2015 at 5:02 PM, Munagala Ramanath <[email protected]
>>>>> <mailto:[email protected]>>
>>>>> wrote:
>>>>>
>>>>>> One way (a bit hacky) is to have the 2nd operator monitor an empty
>>>>>> directory. Then, when the
>>>>>> 1st is done reading, it sends a control tuple to a "file-link"
>>> operator;
>>>>>> that operator creates
>>>>>> symbolic links from the directory monitored by the 2nd operator to the
>>>>>> actual input files.
>>>>>> That should then trigger the 2nd input operator to do its thing.
>>>>>>
>>>>>> Ram
>>>>>>
>>>>>> On Wed, Dec 2, 2015 at 4:43 PM, Isha Arkatkar <[email protected]
>>>>>> <mailto:[email protected]>>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I have an application with 2 input file reader operators. In this
>>>>>> case,
>>>>>>> want to trigger start reading from 2nd input location, only after 1st
>>>>>>> operator is done reading. What is the best way to do this?
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Isha
>>>>>>>
>>>>>>
>>>>>
>>>
>>>
>
