Hi Kenneth, thanks a lot, this doc is really helpful. Will try to move a bit forward in my tests when I'll get time (hopefully end of the week or next one).
PS: saw some history on google but not sure what was the complete outcome: if there an IRC/slack/git-based chat for beam? Know ASF (infra) can provide #apache-beam on freenode Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://blog-rmannibucau.rhcloud.com> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | JavaEE Factory <https://javaeefactory-rmannibucau.rhcloud.com> 2017-05-24 23:54 GMT+02:00 Kenneth Knowles <k...@google.com>: > Hi Romain, > > These are great topics for discussion, and very good timing. > > I strongly recommend joining dev@beam, as that is where most discussion > about implementing Beam runners takes place. On that list, I recently > shared a guide for implementing a Beam runner at > https://s.apache.org/beam-runner-guide. It will be improved and appear on > the Beam site fairly soon. > > I have also taken the liberty of moving this discussion to dev@beam and > moved user@beam to BCC. > > Kenn > > On Wed, May 24, 2017 at 1:54 PM, Romain Manni-Bucau <rmannibu...@gmail.com > > wrote: > >> Hi guys, >> >> congrats for the 2.0! >> >> I have a few question regarding a custom runner implementation, there is >> no particular order but adding numbers for later references: >> >> 1. why beam doesn't have (yet?) a RunnerAdapter with all primivite listed >> and enforced by API, if I got right the code (shout if not) each runner is >> creating its own processing context and convertion at the moment which kind >> of means beam doesn't abstract the runtime which makes it harder to enter >> into beam model IMHO (vs defining by contract all operations - potentially >> with defaults when compositions are possible) >> 2. close to previous point (still around runner): i find quite hard to >> browse the DAG of beam compared to a plain DAG, is it intended to be so low >> level, can't we get a simple graph model? >> >> Maybe I'm just hitting a not yet extended/defined land and therefore an >> user friendly API is missing or I missed a central concept - in this case >> shout :p. >> >> Any pointers would be very welcomed on how to implement a runner without >> redefining a full transpiler/converter - or is it outside beam scope? >> >> >> FYI here what I have https://github.com/rmannibucau/beam-hazelcast-runner >> and here where I'm stucked (hesitating to redefine a full transpiler since >> I expected beam to help): https://github.com/rman >> nibucau/beam-hazelcast-runner/blob/master/src/main/java/com/ >> github/rmannibucau/beam/runner/hazelcast/HazelcastPipelineVisitor.java >> >> Romain Manni-Bucau >> @rmannibucau <https://twitter.com/rmannibucau> | Blog >> <https://blog-rmannibucau.rhcloud.com> | Old Blog >> <http://rmannibucau.wordpress.com> | Github >> <https://github.com/rmannibucau> | LinkedIn >> <https://www.linkedin.com/in/rmannibucau> | JavaEE Factory >> <https://javaeefactory-rmannibucau.rhcloud.com> >> > >
