I am fine if you can show me how to do the Create.of() on the MongoDbIO.read(). It would be nice also to have the status also on the MongoDbIO.write.
again this is the reactive streams pattern that there always is a complete or error path chaim On Tue, Oct 17, 2017 at 2:51 PM, Jean-Baptiste Onofré <[email protected]> wrote: > Good point but I was thinking more about a PTransform with Create.of() if > there's no data. > > Anyway, I can add a DoFn support in the write to update a status (not sure > if it really makes sense). > > Regards > JB > > > On 10/17/2017 01:40 PM, Chaim Turkel wrote: >> >> but if there is no data then >> .apply(ParDo.of(new DoFn() { // check PCollection and set the status })) >> >> will not be called >> >> On Tue, Oct 17, 2017 at 8:33 AM, Jean-Baptiste Onofré <[email protected]> >> wrote: >>> >>> I didn't mean on the read, I meant between the read and write. >>> >>> Basically, your pipeline could look like: >>> >>> pipeline.apply(MongoDbIO.read()...) >>> .apply(ParDo.of(new DoFn() { // check PCollection and set the >>> status >>> })) >>> .apply(MongoDbIO.write()...) >>> >>> Regards >>> JB >>> >>> >>> On 10/16/2017 09:42 PM, Chaim Turkel wrote: >>>> >>>> >>>> how to i add a ParDo on the MongoDbIO.read() if there are no records >>>> read? >>>> >>>> On Mon, Oct 16, 2017 at 6:53 PM, Jean-Baptiste Onofré <[email protected]> >>>> wrote: >>>>> >>>>> >>>>> You can always add your own ParDo(DoFn) where you write the status. >>>>> >>>>> Regards >>>>> JB >>>>> >>>>> >>>>> On 10/16/2017 04:24 PM, Chaim Turkel wrote: >>>>>> >>>>>> >>>>>> >>>>>> that is the problem, i want to write a status that i tried and that >>>>>> there were no records >>>>>> >>>>>> On Mon, Oct 16, 2017 at 3:51 PM, Jean-Baptiste Onofré >>>>>> <[email protected]> >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> Hi Chaim, >>>>>>> >>>>>>> So, you mean you call MongoDBIO.write() with an empty PCollection (no >>>>>>> element in the collection) ? >>>>>>> >>>>>>> The write is basically a DoFn so, it won't do anything if the >>>>>>> PCollection >>>>>>> is >>>>>>> empty. >>>>>>> >>>>>>> Regards >>>>>>> JB >>>>>>> >>>>>>> >>>>>>> On 10/16/2017 01:59 PM, Chaim Turkel wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hi, >>>>>>>> In the case that there are no records to read, is there a way >>>>>>>> to >>>>>>>> get >>>>>>>> called so that i can write the status? >>>>>>>> >>>>>>>> chaim >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Jean-Baptiste Onofré >>>>>>> [email protected] >>>>>>> http://blog.nanthrax.net >>>>>>> Talend - http://www.talend.com >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Jean-Baptiste Onofré >>>>> [email protected] >>>>> http://blog.nanthrax.net >>>>> Talend - http://www.talend.com >>> >>> >>> >>> -- >>> Jean-Baptiste Onofré >>> [email protected] >>> http://blog.nanthrax.net >>> Talend - http://www.talend.com > > > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com
