Given we're well into Portability First, we may want to put all the Portable Stuff at the top of the document these days and clearly mark the Legacy approaches as such.
On Wed, May 11, 2022, 10:42 AM Robert Bradshaw <[email protected]> wrote: > Yes, Beam at its core is designed to have plugable runners. Since I > see that quokka use Ray, you might be interested in connecting with > some folks from that project that are building Beam runner as well. > See, e.g. https://www.mail-archive.com/[email protected]/msg25504.html > > On Wed, May 11, 2022 at 10:35 AM Robert Burke <[email protected]> wrote: > > > > I believe we have a page for that: > > > > https://beam.apache.org/contribute/runner-guide/ > > FYI, I would start at > > https://beam.apache.org/contribute/runner-guide/#writing-an-sdk-independent-runner > > > https://docs.google.com/presentation/d/1Cso0XP9dmj77OD9Bd53C1M3W1sPJF0ZnA20gzb2BPhE/edit > could also be a useful reference. > > > > Likely easiest if you're using Java, more to crib from. But as long as > the language speaks GRPC it's feasible. > > > > On Wed, May 11, 2022, 10:30 AM Tony Wang <[email protected]> wrote: > >> > >> Hello, > >> > >> I am a PhD student at Stanford working on a new data processing > backend, similar to Spark/Flink: https://github.com/marsupialtail/quokka > >> > >> I am wondering how feasible it is to plug it in as a backend for Apache > Beam, i.e. is it even supported, how much work would it be? > >> > >> Best, > >> > >> Tony >
