I'll do it. I'm working with Xinyu on the PR. Kenn
On Mon, Jun 25, 2018, 08:24 Ismaël Mejía <[email protected]> wrote: > +1 > > It is important to have new runners merged (even if not 100% complete) so > they benefit of the fixes going on, and that they can easily (and > incrementally) start to track the new portability features as they develop. > > What is next then ? Who triggers the green button so this happens? > > > > > On Sat, Jun 23, 2018 at 6:43 AM Jean-Baptiste Onofré <[email protected]> > wrote: > >> +1 >> >> As the build is fine, it makes sense to merge pretty fast. >> >> Thanks, >> Regards >> JB >> >> On 22/06/2018 00:14, Xinyu Liu wrote: >> > I updated the merge PR with the gradle integration (there was some >> > Jenkins Java tests failure with google cloud quota issues. It seems not >> > related to this patch). Please feel free to ping me if anything else is >> > needed. >> > >> > Thanks, >> > Xinyu >> > >> > On Mon, Jun 18, 2018 at 5:44 PM, Xinyu Liu <[email protected] >> > <mailto:[email protected]>> wrote: >> > >> > @Kenn: I am going to add the build.gradle. Is there anything else? >> > >> > @Ahmet, @Robert: here are more details about the samza runner right >> now: >> > >> > - Missing pieces: timer support in ParDo is not there yet and I plan >> > to add it soon. SplittableParDo is missing but we don't have a use >> > case so far. We are on par with the other runners for the rest of >> > the Java features. >> > - Work in Progress: implement the portable pipeline runner logic. >> > - Future plans: support Python is our next goal. Hopefully we will >> > get a prototype working sometime next quarter :). >> > >> > Btw, thanks everyone for the comments! >> > >> > Thanks, >> > Xinyu >> > >> > On Mon, Jun 18, 2018 at 4:59 PM, Robert Burke <[email protected] >> > <mailto:[email protected]>> wrote: >> > >> > This is exciting! Is it implemented as a portability framework >> > runner too? >> > >> > >> > On Mon, Jun 18, 2018, 4:36 PM Pablo Estrada <[email protected] >> > <mailto:[email protected]>> wrote: >> > >> > It's very exciting to see a new runner making it into >> > master. : ) >> > >> > Best >> > -P. >> > >> > On Mon, Jun 18, 2018 at 3:38 PM Rafael Fernandez >> > <[email protected] <mailto:[email protected]>> wrote: >> > >> > I've just read this and wanted to share my excitement >> :D >> > >> > >> > >> > On Mon, Jun 18, 2018 at 3:10 PM Kenneth Knowles >> > <[email protected] <mailto:[email protected]>> wrote: >> > >> > One thing that will be necessary is porting the >> > build to Gradle. >> > >> > Kenn >> > >> > On Mon, Jun 18, 2018 at 11:57 AM Xinyu Liu >> > <[email protected] >> > <mailto:[email protected]>> wrote: >> > >> > Hi, Folks, >> > >> > On behalf of the Samza team, I would like to >> > propose to merge the samza-runner branch into >> > master. The branch was created on Jan when we >> > first introduced the Samza Runner [1], and we've >> > been adding features and refining it afterwards. >> > Now the runner satisfies the criteria outlined >> > in [2], and merging it to master will give more >> > visibility to other contributors and users. >> > >> > 1. Have at least 2 contributors interested in >> > maintaining it, and 1 committer interested in >> > supporting it: *Both Chris and me have been >> > making contributions and I am going to sign up >> > for the support. There are more folks in the >> > Samza team interested in contributing to it. >> > Thanks Kenn for all the help and reviews for the >> > runner!* >> > 2. Provide both end-user and developer-facing >> > documentation: *The PR for the samza-runner doc >> > has runner user guide, capability matrix, and >> > tutorial using WordCount examples.* >> > 3. Have at least a basic level of unit test >> > coverage: *Unit tests are here [3].* >> > 4. Run all existing applicable integration tests >> > with other Beam components and create additional >> > tests as appropriate: Enabled ValidatesRunner >> > tests.* >> > 5. Be able to handle a subset of the model that >> > addresses a significant set of use cases, such >> > as ‘traditional batch’ or ‘processing time >> > streaming’: *We have test Beam jobs running in >> > Yarn using event-time processing of Kafka >> streams.* >> > 6. Update the capability matrix with the current >> > status. *Same as #2.* >> > 7. Add a webpage under documentation/runners. >> > *Same as #2.* >> > >> > The PR for the samza-runner >> > merge: https://github.com/apache/beam/pull/5668 >> > <https://github.com/apache/beam/pull/5668> >> > The PR for the samza-runner >> > doc: >> https://github.com/apache/beam-site/pull/471 < >> https://github.com/apache/beam-site/pull/471> >> > >> > Thanks, >> > Xinyu >> > >> > [1] >> https://issues.apache.org/jira/browse/BEAM-3079 >> > < >> https://issues.apache.org/jira/browse/BEAM-3079> >> > [2] https://beam.apache.org/contribute/ >> > <https://beam.apache.org/contribute/> >> > [3] >> https://github.com/apache/beam/tree/samza-runner/runners/samza/src/test >> > < >> https://github.com/apache/beam/tree/samza-runner/runners/samza/src/test> >> > >> > -- >> > Got feedback? go/pabloem-feedback >> > >> > >> > >> >
