Hi Robert! Thanks for replying with such a detailed explanation and linking to the walkthrough video. I do have some components in mind where I could look to contribute but I think I will first pick up the github issue of increasing the test coverage of the `harness` package just to get my feet wet.
BTW, I loved your talk about the state of the Go SDK, I watched it when it came out. > Just be aware, ultimately, we do not have the resources to hold your hand and teach you both beam and Go. Not saying if that was your intent, but we do have jobs to do, and our own things to get done. :) Of course, this is very true. Apologies if my message came out to be asking for hand holding. I am pretty familiar with Go and I think I should be able to make sense of the codebase. Looking forward to contributing in some time! Thanks, Shivam Singhal On Thu, 24 Nov 2022 at 00:46, Robert Burke <rob...@frantil.com> wrote: > Hello and welcome! > > On Wed, Nov 23, 2022, 10:40 AM Shivam Singhal <shivamsinghal5...@gmail.com> > wrote: > >> Hi folks, >> >> I have been extensively using Java SDK in my daily work, and haven't yet >> touched Go SDK. >> But I am soon(in ~1month) going to have plenty of time and I would like >> to contribute to Go SDK. To help with this, I have a few questions: >> >> 1. I have cloned the beam repo. Would like to know which particular >> code files should I start to read so that I can build familiarity? >> >> What use cases are you looking to extend or add? > > There's lots to do, but the best place to start is what *you* would like > the SDK to work well with. Is there a usecase you'd like to enable? A > specific pipeline you'd like to improve? > > Contributions for the sake of contributions don't turn out as well as > having "I need this to do X so Y works!" as a motivation in my experience. > > Otherwise, so you aren't left hanging, I refer you to some of the talks > I've given on getting started with navigating around the SDK. > > https://youtu.be/WcuS8ojHfyU > > It's a bit older now, but still accurate at the scale it's talking about. > > Beam SDKs are complex, but the Go SDK is pretty easy to navigate due to > being Go. My recommendation is just to follow the code call stack (what > does it do when beam.ParDo is called? How does the execution harness work, > in worker containers, in harness/harness.go) and see where that takes you. > > Just be aware, ultimately, we do not have the resources to hold your hand > and teach you both beam and Go. Not saying if that was your intent, but we > do have jobs to do, and our own things to get done. :) > > That said, I will seek to maintain good Go style through the code review > process, and we'll avoid Beam correctness issues through the reviews, so > its not that hard a line. > >> >> 1. Is there a roadmap for Go SDK I can look at? Is this Go SDK being >> used in production somewhere? >> >> We tried to keep one up to date, but ultimately stopped. At this point, > the roadmap is "what do you want working?" > > The best place to start is to look at the GitHub issues labeled "go". > https://github.com/apache/beam/issues?q=is%3Aissue+is%3Aopen+label%3Ago > > If it's unassigned, it's fair game to '.take-issue'. Feel free to ping me > (@lostluck) if you have any questions about it. > > You can see a rough view of the SDKs capabilities and direction in the > State of the Go SDK 2022 talk: https://youtu.be/e3yx4SwHJK0 > > Please let me know if there is a better place to ask. >> > > This is the right place to ask! > >> >> Thanks, >> Shivam >> > > Robert Burke > Beam Go Busybody >
