No, I don't have an example but several approaches are possible depending
on the
exact requirements, e.g.:
1. How large is the number of directories ?
2. Is the desired sequence a total order or a partial order (i.e. DAG,
https://en.wikipedia.org/wiki/Partially_ordered_set) ?
If the number of directories is small you can use one operator per
directory and link them with ports in the
desired sequence. Each operator sends a control tuple to the next when it
wants the next one to start.
Each operator waits for this trigger and emits tuples in the idle time
handler, for example:
*public class DownStreamReceiver extends AbstractFileInputOperator
implements Operator.IdleTimeHandler{*
* @Override*
* public void handleIdleTime()*
* {*
* if(upstreamDoneReading){ // this is set to true only after
receiving the trigger from 1st reader*
* emitTuples();*
* }*
* }*
*}*
If the number is large, you can explore the earlier partitioned approach
but have each partition look for a trigger
from an external source like a Kafka queue or an entry in a DB to start
processing.
Ram
On Thu, Jun 23, 2016 at 6:11 AM, Mukkamula, Suryavamshivardhan (CWM-NR) <
[email protected]> wrote:
> Hi Ram,
>
> Do you have a sample DT application code for reading multiple directories
> in sequence ?
>
> Or through some light on how would I achieve that with
> AbstractFileInputOperator.
>
> Regards,
> Surya Vamshi
>
>
> _______________________________________________________________________
>
> If you received this email in error, please advise the sender (by return
> email or otherwise) immediately. You have consented to receive the attached
> electronically at the above-noted email address; please retain a copy of
> this confirmation for future reference.
>
> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
> cette confirmation pour les fins de reference future.
>
>
