Hello and welcome! On Wed, Nov 23, 2022, 10:40 AM Shivam Singhal <shivamsinghal5...@gmail.com> wrote:
> Hi folks, > > I have been extensively using Java SDK in my daily work, and haven't yet > touched Go SDK. > But I am soon(in ~1month) going to have plenty of time and I would like to > contribute to Go SDK. To help with this, I have a few questions: > > 1. I have cloned the beam repo. Would like to know which particular > code files should I start to read so that I can build familiarity? > > What use cases are you looking to extend or add? There's lots to do, but the best place to start is what *you* would like the SDK to work well with. Is there a usecase you'd like to enable? A specific pipeline you'd like to improve? Contributions for the sake of contributions don't turn out as well as having "I need this to do X so Y works!" as a motivation in my experience. Otherwise, so you aren't left hanging, I refer you to some of the talks I've given on getting started with navigating around the SDK. https://youtu.be/WcuS8ojHfyU It's a bit older now, but still accurate at the scale it's talking about. Beam SDKs are complex, but the Go SDK is pretty easy to navigate due to being Go. My recommendation is just to follow the code call stack (what does it do when beam.ParDo is called? How does the execution harness work, in worker containers, in harness/harness.go) and see where that takes you. Just be aware, ultimately, we do not have the resources to hold your hand and teach you both beam and Go. Not saying if that was your intent, but we do have jobs to do, and our own things to get done. :) That said, I will seek to maintain good Go style through the code review process, and we'll avoid Beam correctness issues through the reviews, so its not that hard a line. > > 1. Is there a roadmap for Go SDK I can look at? Is this Go SDK being > used in production somewhere? > > We tried to keep one up to date, but ultimately stopped. At this point, the roadmap is "what do you want working?" The best place to start is to look at the GitHub issues labeled "go". https://github.com/apache/beam/issues?q=is%3Aissue+is%3Aopen+label%3Ago If it's unassigned, it's fair game to '.take-issue'. Feel free to ping me (@lostluck) if you have any questions about it. You can see a rough view of the SDKs capabilities and direction in the State of the Go SDK 2022 talk: https://youtu.be/e3yx4SwHJK0 Please let me know if there is a better place to ask. > This is the right place to ask! > > Thanks, > Shivam > Robert Burke Beam Go Busybody
