On Fri, Feb 15, 2019 at 7:36 AM Can Gencer <[email protected]> wrote: > > We at Hazelcast are looking into writing a Beam runner for Hazelcast Jet > (https://github.com/hazelcast/hazelcast-jet). I wanted to introduce myself as > we'll likely have questions as we start development.
Welcome! Hazelcast looks interesting, a Beam runner for it would be very cool. > Some of the things I'm wondering about currently: > > * Currently there seems to be a guide available at > https://beam.apache.org/contribute/runner-guide/ , is this up to date? Is > there anything in specific to be aware of when starting with a new runner > that's not covered here? That looks like a pretty good starting point. At a quick glance, I don't see anything that looks out of date. Another resource that might be helpful is a talk from last year on writing an SDK (but as it mostly covers the runner-sdk interaction, it's also quite useful for understanding the runner side: https://docs.google.com/presentation/d/1Cso0XP9dmj77OD9Bd53C1M3W1sPJF0ZnA20gzb2BPhE/edit#slide=id.p And please feel free to ask any questions on this list as well; we'd be happy to help. > * Should we be targeting the latest master which is at 2.12-SNAPSHOT or a > stable version? I would target the latest master. > * After a runner is developed, how is the maintenance typically handled, as > the runners seems to be part of Beam codebase? Either is possible. Several runner adapters are part of the Beam codebase, but for example the IMB Streams Beam runner is not. There are certainly pros and cons (certainly early on when the APIs themselves were under heavy development it was easier to keep things in sync in the same codebase, but things have mostly stabilized now). A runner only becomes part of the Beam codebase if there are members of the community committed to maintaining it (which could include you). Both approaches are fine. - Robert
