Let me take a quick look and create a Jira if required.
Thanks for the idea !
Regards
JB
On 10/17/2017 01:54 PM, Chaim Turkel wrote:
I am fine if you can show me how to do the Create.of() on the MongoDbIO.read().
It would be nice also to have the status also on the MongoDbIO.write.
again this is the reactive streams pattern that there always is a
complete or error path
chaim
On Tue, Oct 17, 2017 at 2:51 PM, Jean-Baptiste Onofré <[email protected]> wrote:
Good point but I was thinking more about a PTransform with Create.of() if
there's no data.
Anyway, I can add a DoFn support in the write to update a status (not sure
if it really makes sense).
Regards
JB
On 10/17/2017 01:40 PM, Chaim Turkel wrote:
but if there is no data then
.apply(ParDo.of(new DoFn() { // check PCollection and set the status }))
will not be called
On Tue, Oct 17, 2017 at 8:33 AM, Jean-Baptiste Onofré <[email protected]>
wrote:
I didn't mean on the read, I meant between the read and write.
Basically, your pipeline could look like:
pipeline.apply(MongoDbIO.read()...)
.apply(ParDo.of(new DoFn() { // check PCollection and set the
status
}))
.apply(MongoDbIO.write()...)
Regards
JB
On 10/16/2017 09:42 PM, Chaim Turkel wrote:
how to i add a ParDo on the MongoDbIO.read() if there are no records
read?
On Mon, Oct 16, 2017 at 6:53 PM, Jean-Baptiste Onofré <[email protected]>
wrote:
You can always add your own ParDo(DoFn) where you write the status.
Regards
JB
On 10/16/2017 04:24 PM, Chaim Turkel wrote:
that is the problem, i want to write a status that i tried and that
there were no records
On Mon, Oct 16, 2017 at 3:51 PM, Jean-Baptiste Onofré
<[email protected]>
wrote:
Hi Chaim,
So, you mean you call MongoDBIO.write() with an empty PCollection (no
element in the collection) ?
The write is basically a DoFn so, it won't do anything if the
PCollection
is
empty.
Regards
JB
On 10/16/2017 01:59 PM, Chaim Turkel wrote:
Hi,
In the case that there are no records to read, is there a way
to
get
called so that i can write the status?
chaim
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
--
Jean-Baptiste Onofré
[email protected]
http://blog.nanthrax.net
Talend - http://www.talend.com
