+1 It is important to have new runners merged (even if not 100% complete) so they benefit of the fixes going on, and that they can easily (and incrementally) start to track the new portability features as they develop.
What is next then ? Who triggers the green button so this happens? On Sat, Jun 23, 2018 at 6:43 AM Jean-Baptiste Onofré <[email protected]> wrote: > +1 > > As the build is fine, it makes sense to merge pretty fast. > > Thanks, > Regards > JB > > On 22/06/2018 00:14, Xinyu Liu wrote: > > I updated the merge PR with the gradle integration (there was some > > Jenkins Java tests failure with google cloud quota issues. It seems not > > related to this patch). Please feel free to ping me if anything else is > > needed. > > > > Thanks, > > Xinyu > > > > On Mon, Jun 18, 2018 at 5:44 PM, Xinyu Liu <[email protected] > > <mailto:[email protected]>> wrote: > > > > @Kenn: I am going to add the build.gradle. Is there anything else? > > > > @Ahmet, @Robert: here are more details about the samza runner right > now: > > > > - Missing pieces: timer support in ParDo is not there yet and I plan > > to add it soon. SplittableParDo is missing but we don't have a use > > case so far. We are on par with the other runners for the rest of > > the Java features. > > - Work in Progress: implement the portable pipeline runner logic. > > - Future plans: support Python is our next goal. Hopefully we will > > get a prototype working sometime next quarter :). > > > > Btw, thanks everyone for the comments! > > > > Thanks, > > Xinyu > > > > On Mon, Jun 18, 2018 at 4:59 PM, Robert Burke <[email protected] > > <mailto:[email protected]>> wrote: > > > > This is exciting! Is it implemented as a portability framework > > runner too? > > > > > > On Mon, Jun 18, 2018, 4:36 PM Pablo Estrada <[email protected] > > <mailto:[email protected]>> wrote: > > > > It's very exciting to see a new runner making it into > > master. : ) > > > > Best > > -P. > > > > On Mon, Jun 18, 2018 at 3:38 PM Rafael Fernandez > > <[email protected] <mailto:[email protected]>> wrote: > > > > I've just read this and wanted to share my excitement :D > > > > > > > > On Mon, Jun 18, 2018 at 3:10 PM Kenneth Knowles > > <[email protected] <mailto:[email protected]>> wrote: > > > > One thing that will be necessary is porting the > > build to Gradle. > > > > Kenn > > > > On Mon, Jun 18, 2018 at 11:57 AM Xinyu Liu > > <[email protected] > > <mailto:[email protected]>> wrote: > > > > Hi, Folks, > > > > On behalf of the Samza team, I would like to > > propose to merge the samza-runner branch into > > master. The branch was created on Jan when we > > first introduced the Samza Runner [1], and we've > > been adding features and refining it afterwards. > > Now the runner satisfies the criteria outlined > > in [2], and merging it to master will give more > > visibility to other contributors and users. > > > > 1. Have at least 2 contributors interested in > > maintaining it, and 1 committer interested in > > supporting it: *Both Chris and me have been > > making contributions and I am going to sign up > > for the support. There are more folks in the > > Samza team interested in contributing to it. > > Thanks Kenn for all the help and reviews for the > > runner!* > > 2. Provide both end-user and developer-facing > > documentation: *The PR for the samza-runner doc > > has runner user guide, capability matrix, and > > tutorial using WordCount examples.* > > 3. Have at least a basic level of unit test > > coverage: *Unit tests are here [3].* > > 4. Run all existing applicable integration tests > > with other Beam components and create additional > > tests as appropriate: Enabled ValidatesRunner > > tests.* > > 5. Be able to handle a subset of the model that > > addresses a significant set of use cases, such > > as ‘traditional batch’ or ‘processing time > > streaming’: *We have test Beam jobs running in > > Yarn using event-time processing of Kafka > streams.* > > 6. Update the capability matrix with the current > > status. *Same as #2.* > > 7. Add a webpage under documentation/runners. > > *Same as #2.* > > > > The PR for the samza-runner > > merge: https://github.com/apache/beam/pull/5668 > > <https://github.com/apache/beam/pull/5668> > > The PR for the samza-runner > > doc: > https://github.com/apache/beam-site/pull/471 < > https://github.com/apache/beam-site/pull/471> > > > > Thanks, > > Xinyu > > > > [1] > https://issues.apache.org/jira/browse/BEAM-3079 > > <https://issues.apache.org/jira/browse/BEAM-3079 > > > > [2] https://beam.apache.org/contribute/ > > <https://beam.apache.org/contribute/> > > [3] > https://github.com/apache/beam/tree/samza-runner/runners/samza/src/test > > < > https://github.com/apache/beam/tree/samza-runner/runners/samza/src/test> > > > > -- > > Got feedback? go/pabloem-feedback > > > > > > >
