[ANNOUNCE] Beam 2.54.0 Released

/ and the Github release page https://github.com/apache/beam/releases/tag/v2.54.0 Thanks to everyone who contributed to this release, and we hope you enjoy using Beam 2.54.0. -- Robert Burke, on behalf of the Apache Beam Team.

Beam 2.54.0 Release

* N/A ## Deprecations * N/A ## Bugfixes * Fixed a memory leak affecting some Go SDK since 2.46.0. ( https://github.com/apache/beam/pull/28142) ## Security Fixes * N/A ## Known Issues * N/A >From the blog post: https://beam.apache.org/blog/beam-2.54.0/ Thanks! Robert Burke Beam 2.5

Re: [Request for Feedback] Swift SDK Prototype

ions, I'll leave it to silent consensus. I agree that copying/building the protos isn't a burden, since that's entirely what protos are for. We're already treating them as properly stable and not making breaking proto, so compatibility is maintained by normal proto behavior. Robert Burke Beam G

Re: [ANNOUNCE] Beam 2.50.0 Released

Of course I miss replacing one of them. Thanks! On Wed, Aug 30, 2023 at 11:43 AM Robert Burke wrote: > The Apache Beam Team is pleased to announce the release of version 2.50.0. > > You can download the release here: > > https://beam.apache.org/get-started/downloads/ > >

[ANNOUNCE] Beam 2.50.0 Released

/ and the Github release page https://github.com/apache/beam/releases/tag/v2.50.0 Thanks to everyone who contributed to this release, and we hope you enjoy using Beam 2.50.0. -- Robert Burke, on behalf of the Apache Beam Team.

Re: [Question] Go SDK worker pool run options

t how the Flink portable runner > > works. Is it the case that when I specify LOOPBACK, that the binary sent > > already includes the SDK and is able to spawn the Go routines on the > > TaskManager without any additional config? > > > > On Fri, Mar 24, 2023 at 6:02 PM Rober

Re: [Question] Go SDK worker pool run options

pecific for my use case, is this okay? > > Thanks, > Sherif > > On Thu, Mar 23, 2023 at 3:52 PM Robert Burke wrote: > > > Oh that's very interesting! I have a few comments, but we could end up > > with a new feature for the Go SDK. > > > > As you've

Re: [Question] Go SDK worker pool run options

ted, and help review it. @lostluck on github, if you desired to go this path. Robert Burke Beam Go Busybody On 2023/03/23 15:09:44 Sherif Tolba wrote: > Thank you, Robert, for your detailed response and the resources you shared. > > One thing that I didn't mention is that my goa

Re: [Question] Go SDK worker pool run options

ctions and restrictions. It's currently suitable for smoke testing small pipelines, but the goal is to have a portable reference runner, WRT all facets of beam. Depending on what Beam features you're using, it may not be suitable. I hope this helps! Robert Burke Beam Go Busybody (note, I don't

[ANNOUNCE] Apache Beam 2.42.0 Released

to this release, and we hope you enjoy using Beam 2.42.0. -- Robert Burke, on behalf of The Apache Beam team

Re: [Question] - Side inputs in golang sdk

be anything specific in the pipeline graph for an unbounded side input. We'll investigate this experience and try to make sure it's working for 2.40. On Tue, May 17, 2022, 9:33 AM Robert Burke wrote: > Hello Kane! > > In chatting with Jack, this won't be possible to implement in the Go SD

Re: [Question] - Side inputs in golang sdk

PCollection. I hope this is useful! Robert Burke Beam Go Busybody [1]: https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/PeriodicImpulse.java [2]: https://github.com/apache/beam/blob/master/sdks/python/apache_beam/transforms/periodicsequence.py [3

Re: Apache Beam Go SDK Quickstart bugs

ound a bug with the schema row decoder along the way too. Since the website tracks live, getting the quick start to use Spark 3 doesn't have to happen before the cut, so that's still available to do. I really appreciate the clear errors and repros you provided! Thanks again Robert Burke A On

Re: [Question] Need to write a pipeline in Go consuming events from Kafka

At present, there's no way to write an unbounded datasource with the Go SDK, which would require DoFn Self Checkpointing ( https://issues.apache.org/jira/browse/BEAM-11104) and Watermark Estimation (https://issues.apache.org/jira/browse/BEAM-11105). Daniel is working on wrapping the Java kafka

[ANNOUNCE] Beam 2.26.0 Released

here: https://beam.apache.org/get-started/downloads/ This release includes bug fixes, features, and improvements detailed on the Beam blog: https://beam.apache.org/blog/beam-2.26.0/ Thank you to everyone who contributed to this release, and we hope you enjoy using Beam 2.26.0 Robert Burke 2.26.0

Re: Beam + Dataflow + Go article

here > <https://pasteboard.co/HVf80BU.png>, the "elements added" for input and > output collections was always empty. > > * Wall time always very small > <https://issues.apache.org/jira/browse/BEAM-6375>- Similar to above, my > pipeline would run for hours, but t

[Go SDK] User Defined Coders

atives, or other suggestions and opinions, I'd love to hear them! Otherwise my intent is to get a PR ready by the end of January. Thanks! Robert Burke

Re: Beam + Dataflow + Go article

Thanks for the excellent article! It's great to see what the experience is like from an outside perspective, and it's comforting that it mirrors my own. It means I'm not missing much. It's been on my to-do list to make the Go SDK direct runner more robust, so transitioning to other runners

Want to talk Beam at GopherCon?

1.11 means for the SDK + Harness Performance + Usability and documentation (godoc and Beam site) + Coders + Getting some light hacking done maybe? Looking forward to meeting you! Robert Burke

Re: Go SDK: bad parameter type for reflect.methodValueCall: func(Record)

Registry or similar, we might have a better solution, but the danger will still be there. Please let me know if you need help. Cheers, Robert Burke On 2018/06/18 21:10:33, ed...@gmail.com wrote: > I have the following code:> > > type Record struct {> >Timestamp time.Time