Good point but I was thinking more about a PTransform with Create.of() if
there's no data.
Anyway, I can add a DoFn support in the write to update a status (not sure if it
really makes sense).
Regards
JB
On 10/17/2017 01:40 PM, Chaim Turkel wrote:
but if there is no data then
.apply(ParDo.of(new DoFn() { // check PCollection and set the status }))
will not be called
On Tue, Oct 17, 2017 at 8:33 AM, Jean-Baptiste Onofré <[email protected]> wrote:
I didn't mean on the read, I meant between the read and write.
Basically, your pipeline could look like:
pipeline.apply(MongoDbIO.read()...)
.apply(ParDo.of(new DoFn() { // check PCollection and set the status
}))
.apply(MongoDbIO.write()...)
Regards
JB
On 10/16/2017 09:42 PM, Chaim Turkel wrote:
how to i add a ParDo on the MongoDbIO.read() if there are no records read?
On Mon, Oct 16, 2017 at 6:53 PM, Jean-Baptiste Onofré <[email protected]>
wrote:
You can always add your own ParDo(DoFn) where you write the status.
Regards
JB
On 10/16/2017 04:24 PM, Chaim Turkel wrote:
that is the problem, i want to write a status that i tried and that
there were no records
On Mon, Oct 16, 2017 at 3:51 PM, Jean-Baptiste Onofré <[email protected]>
wrote:
Hi Chaim,
So, you mean you call MongoDBIO.write() with an empty PCollection (no
element in the collection) ?
The write is basically a DoFn so, it won't do anything if the
PCollection
is
empty.
Regards
JB
On 10/16/2017 01:59 PM, Chaim Turkel wrote:
Hi,
In the case that there are no records to read, is there a way to
get
called so that i can write the status?
chaim
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
