Hi Philip,
Thanks for expressing interest in the Go SDK! The documentation is indeed
still incomplete (BEAM-3826) and the main design document is probably be
the best starting point right now:
https://s.apache.org/beam-go-sdk-design-rfc
It also contains links to some of the better documented examples in the
repository.
We do plan to prototype Go streaming w/ PubSub on Dataflow in the near
future (BEAM-3854, BEAM-3856). Is that the setup you're mainly looking at
as well? For Datastore reading, Braden even has a PR in flight. So things
are moving along in that direction and there are definitely lots of
opportunities to contribute. The long-term IO story will likely be based on
some combination of Go SplittableDoFns and cross-language IO.
Please feel free to ask questions, open JIRAs or send PRs. Love to hear
your feedback!
Henning
On Thu, Mar 15, 2018 at 7:21 PM Philip Gianfortoni <[email protected]>
wrote:
> Hi dev team,
>
> I am an engineer at Token Transit, a company working on a mobile ticketing
> solution for transit companies. Through Holden (who's working on Py3 w/Beam
> and doing some dev advocacy), we heard about the experimental newly merged
> WIP Go Beam SDK. We are extremely excited about exploring and contributing
> to this new offering from Beam, as almost our entire codebase is written in
> Go, and we are looking for a bulk data processing solution that is
> compatible.
>
> Are there any code pointers, examples, or other docs that someone could
> point me to about this new SDK? Holden pointed us to this repository
> <https://github.com/apache/beam/tree/master/sdks/go>, but it does seem
> that documentation is a bit light at the moment. Any extra examples or docs
> would be greatly appreciated.
>
> Another thing that we were wondering was whether and/or when a PubSub or
> Datastore API will be supported. We are looking at using PubSub to process
> our data incrementally, and in the long term, support for this feature is a
> requirement.
>
> Looking forward to trying out this SDK. Thank you so much!
>
> 8
>
