Re: [EXTERNAL] Re: FirestoreIO connector [JavaSDK]

[Chamikara Jayalath]

Thanks. Taking a look. It'll probably be helpful if you can write a short
doc on design decisions you took including API and client library choice.
Also, as for any other sink, we should carefully design this so that
duplicate data is not written to the data store when bundles are retried by
runners due to failures.

Thanks,
Cham

On Mon, Dec 2, 2019 at 1:26 AM Stefan Djelekar 
wrote:

> Hi dev team,
>
>
>
> I’ve submitted the pull request
> <https://github.com/apache/beam/pull/10187> for BEAM-8376
> <https://issues.apache.org/jira/browse/BEAM-8376>.
>
>
>
> Getting a review would be nice.
>
>
>
> All the best,
>
> Stefan
>
>
>
> *From:* Chamikara Jayalath 
> *Sent:* Wednesday, November 6, 2019 8:05 PM
> *To:* dev 
> *Subject:* Re: [EXTERNAL] Re: FirestoreIO connector [JavaSDK]
>
>
>
> BTW, FYI, I'm also talking with folks from Google Firestore team regarding
> this. I think they had shown some interest in taking this up but I'm not
> sure.
>
> If they are able to contribute here, and if we can coordinate with some of
> those folks on this effort, it will be great for the long term health and
> maintenance of the connector.
>
>
>
> Thanks,
>
> Cham
>
>
>
> On Wed, Nov 6, 2019 at 1:56 AM Stefan Djelekar 
> wrote:
>
> Thanks for the valuable information.
>
>
>
> We’ve been exactly looking up to that Datastore connector, but there are
> some differences of course.
>
>
>
> I’ll make the PR in the next few days and let’s pick it up from there.
>
>
>
> King regards,
>
> Stefan
>
>
>
>
>
> *From:* Chamikara Jayalath 
> *Sent:* Tuesday, November 5, 2019 2:24 AM
> *To:* dev 
> *Subject:* [EXTERNAL] Re: FirestoreIO connector [JavaSDK]
>
>
>
> Thanks for the contribution. Happy to help with the review.
>
> Also, probably it'll be good to follow patterns employed by the existing
> Datastore connector when applicable:
> https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java
>
>
>
> Thanks,
>
> Cham
>
>
>
> On Mon, Nov 4, 2019 at 2:37 AM Ismaël Mejía  wrote:
>
> Hello,
>
> Please open a PR with the code of the new connector rebased against
> the latest master.
>
> It is worth to take a look at Beam's contribution guide [1] and
> PTransform style guide [2] in advance.
> In any case we can guide you on any issue during the PR review so
> please go ahead.
>
> Regards,
> Ismaël
>
> [1] https://beam.apache.org/contribute/
> [2] https://beam.apache.org/contribute/ptransform-style-guide/
>
>
> On Mon, Nov 4, 2019 at 10:48 AM Stefan Djelekar
>  wrote:
> >
> > Hi beam devs,
> >
> >
> >
> > I'm Stefan from Redbox. We are a customer of GCP and we are in need of
> beam Java connector for Firestore.
> >
> >
> >
> > There is a pending JIRA item for this.
> https://issues.apache.org/jira/browse/BEAM-8376
> >
> >
> >
> > The team inside of the company has been working on this for a while and
> we would like to contribute!
> >
> > What is the best way to do this? Can we perhaps get a review from an
> experienced beam member?
> >
> > Let me know what do you think.
> >
> >
> >
> > All the best,
> >
> >
> >
> > Stefan Đelekar
> >
> > Sofware Engineer
> >
> > stefan.djele...@redbox.com
> >
> > djelekar.com
> >
> >
>
>

RE: [EXTERNAL] Re: FirestoreIO connector [JavaSDK]

[Stefan Djelekar]

Hi dev team,

I’ve submitted the pull request<https://github.com/apache/beam/pull/10187> for 
BEAM-8376<https://issues.apache.org/jira/browse/BEAM-8376>.

Getting a review would be nice.

All the best,
Stefan

From: Chamikara Jayalath 
Sent: Wednesday, November 6, 2019 8:05 PM
To: dev 
Subject: Re: [EXTERNAL] Re: FirestoreIO connector [JavaSDK]

BTW, FYI, I'm also talking with folks from Google Firestore team regarding 
this. I think they had shown some interest in taking this up but I'm not sure.
If they are able to contribute here, and if we can coordinate with some of 
those folks on this effort, it will be great for the long term health and 
maintenance of the connector.

Thanks,
Cham

On Wed, Nov 6, 2019 at 1:56 AM Stefan Djelekar 
mailto:stefan.djele...@redbox.com>> wrote:
Thanks for the valuable information.

We’ve been exactly looking up to that Datastore connector, but there are some 
differences of course.

I’ll make the PR in the next few days and let’s pick it up from there.

King regards,
Stefan


From: Chamikara Jayalath mailto:chamik...@google.com>>
Sent: Tuesday, November 5, 2019 2:24 AM
To: dev mailto:dev@beam.apache.org>>
Subject: [EXTERNAL] Re: FirestoreIO connector [JavaSDK]

Thanks for the contribution. Happy to help with the review.
Also, probably it'll be good to follow patterns employed by the existing 
Datastore connector when applicable: 
https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java<https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java>

Thanks,
Cham

On Mon, Nov 4, 2019 at 2:37 AM Ismaël Mejía 
mailto:ieme...@gmail.com>> wrote:
Hello,

Please open a PR with the code of the new connector rebased against
the latest master.

It is worth to take a look at Beam's contribution guide [1] and
PTransform style guide [2] in advance.
In any case we can guide you on any issue during the PR review so
please go ahead.

Regards,
Ismaël

[1] https://beam.apache.org/contribute/<https://beam.apache.org/contribute/>
[2] 
https://beam.apache.org/contribute/ptransform-style-guide/<https://beam.apache.org/contribute/ptransform-style-guide/>


On Mon, Nov 4, 2019 at 10:48 AM Stefan Djelekar
mailto:stefan.djele...@redbox.com>> wrote:
>
> Hi beam devs,
>
>
>
> I'm Stefan from Redbox. We are a customer of GCP and we are in need of beam 
> Java connector for Firestore.
>
>
>
> There is a pending JIRA item for this. 
> https://issues.apache.org/jira/browse/BEAM-8376<https://issues.apache.org/jira/browse/BEAM-8376>
>
>
>
> The team inside of the company has been working on this for a while and we 
> would like to contribute!
>
> What is the best way to do this? Can we perhaps get a review from an 
> experienced beam member?
>
> Let me know what do you think.
>
>
>
> All the best,
>
>
>
> Stefan Đelekar
>
> Sofware Engineer
>
> stefan.djele...@redbox.com<mailto:stefan.djele...@redbox.com>
>
> djelekar.com<http://djelekar.com>
>
>

Re: [EXTERNAL] Re: FirestoreIO connector [JavaSDK]

[Chamikara Jayalath]

BTW, FYI, I'm also talking with folks from Google Firestore team regarding
this. I think they had shown some interest in taking this up but I'm not
sure.
If they are able to contribute here, and if we can coordinate with some of
those folks on this effort, it will be great for the long term health and
maintenance of the connector.

Thanks,
Cham

On Wed, Nov 6, 2019 at 1:56 AM Stefan Djelekar 
wrote:

> Thanks for the valuable information.
>
>
>
> We’ve been exactly looking up to that Datastore connector, but there are
> some differences of course.
>
>
>
> I’ll make the PR in the next few days and let’s pick it up from there.
>
>
>
> King regards,
>
> Stefan
>
>
>
>
>
> *From:* Chamikara Jayalath 
> *Sent:* Tuesday, November 5, 2019 2:24 AM
> *To:* dev 
> *Subject:* [EXTERNAL] Re: FirestoreIO connector [JavaSDK]
>
>
>
> Thanks for the contribution. Happy to help with the review.
>
> Also, probably it'll be good to follow patterns employed by the existing
> Datastore connector when applicable:
> https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java
>
>
>
> Thanks,
>
> Cham
>
>
>
> On Mon, Nov 4, 2019 at 2:37 AM Ismaël Mejía  wrote:
>
> Hello,
>
> Please open a PR with the code of the new connector rebased against
> the latest master.
>
> It is worth to take a look at Beam's contribution guide [1] and
> PTransform style guide [2] in advance.
> In any case we can guide you on any issue during the PR review so
> please go ahead.
>
> Regards,
> Ismaël
>
> [1] https://beam.apache.org/contribute/
> [2] https://beam.apache.org/contribute/ptransform-style-guide/
>
>
> On Mon, Nov 4, 2019 at 10:48 AM Stefan Djelekar
>  wrote:
> >
> > Hi beam devs,
> >
> >
> >
> > I'm Stefan from Redbox. We are a customer of GCP and we are in need of
> beam Java connector for Firestore.
> >
> >
> >
> > There is a pending JIRA item for this.
> https://issues.apache.org/jira/browse/BEAM-8376
> >
> >
> >
> > The team inside of the company has been working on this for a while and
> we would like to contribute!
> >
> > What is the best way to do this? Can we perhaps get a review from an
> experienced beam member?
> >
> > Let me know what do you think.
> >
> >
> >
> > All the best,
> >
> >
> >
> > Stefan Đelekar
> >
> > Sofware Engineer
> >
> > stefan.djele...@redbox.com
> >
> > djelekar.com
> >
> >
>
>

RE: [EXTERNAL] Re: FirestoreIO connector [JavaSDK]

[Stefan Djelekar]

Thanks for the valuable information.

We’ve been exactly looking up to that Datastore connector, but there are some 
differences of course.

I’ll make the PR in the next few days and let’s pick it up from there.

King regards,
Stefan


From: Chamikara Jayalath 
Sent: Tuesday, November 5, 2019 2:24 AM
To: dev 
Subject: [EXTERNAL] Re: FirestoreIO connector [JavaSDK]

Thanks for the contribution. Happy to help with the review.
Also, probably it'll be good to follow patterns employed by the existing 
Datastore connector when applicable: 
https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java

Thanks,
Cham

On Mon, Nov 4, 2019 at 2:37 AM Ismaël Mejía 
mailto:ieme...@gmail.com>> wrote:
Hello,

Please open a PR with the code of the new connector rebased against
the latest master.

It is worth to take a look at Beam's contribution guide [1] and
PTransform style guide [2] in advance.
In any case we can guide you on any issue during the PR review so
please go ahead.

Regards,
Ismaël

[1] https://beam.apache.org/contribute/
[2] 
https://beam.apache.org/contribute/ptransform-style-guide/


On Mon, Nov 4, 2019 at 10:48 AM Stefan Djelekar
mailto:stefan.djele...@redbox.com>> wrote:
>
> Hi beam devs,
>
>
>
> I'm Stefan from Redbox. We are a customer of GCP and we are in need of beam 
> Java connector for Firestore.
>
>
>
> There is a pending JIRA item for this. 
> https://issues.apache.org/jira/browse/BEAM-8376
>
>
>
> The team inside of the company has been working on this for a while and we 
> would like to contribute!
>
> What is the best way to do this? Can we perhaps get a review from an 
> experienced beam member?
>
> Let me know what do you think.
>
>
>
> All the best,
>
>
>
> Stefan Đelekar
>
> Sofware Engineer
>
> stefan.djele...@redbox.com
>
> djelekar.com
>
>