Thanks a lot, Kenn! Finally we linked our runner in. I will work on the rest of the stuff as you mentioned. Thanks again for everyone's comments, too.
Thanks, Xinyu On Mon, Jun 25, 2018 at 2:48 PM, Kenneth Knowles <[email protected]> wrote: > This is done. Now we need to make sure the build is running, healthy, in > the PR template. > > On Mon, Jun 25, 2018 at 9:10 AM Kenneth Knowles <[email protected]> wrote: > >> I'll do it. I'm working with Xinyu on the PR. >> >> Kenn >> >> On Mon, Jun 25, 2018, 08:24 Ismaël Mejía <[email protected]> wrote: >> >>> +1 >>> >>> It is important to have new runners merged (even if not 100% complete) >>> so they benefit of the fixes going on, and that they can easily (and >>> incrementally) start to track the new portability features as they develop. >>> >>> What is next then ? Who triggers the green button so this happens? >>> >>> >>> >>> >>> On Sat, Jun 23, 2018 at 6:43 AM Jean-Baptiste Onofré <[email protected]> >>> wrote: >>> >>>> +1 >>>> >>>> As the build is fine, it makes sense to merge pretty fast. >>>> >>>> Thanks, >>>> Regards >>>> JB >>>> >>>> On 22/06/2018 00:14, Xinyu Liu wrote: >>>> > I updated the merge PR with the gradle integration (there was some >>>> > Jenkins Java tests failure with google cloud quota issues. It seems >>>> not >>>> > related to this patch). Please feel free to ping me if anything else >>>> is >>>> > needed. >>>> > >>>> > Thanks, >>>> > Xinyu >>>> > >>>> > On Mon, Jun 18, 2018 at 5:44 PM, Xinyu Liu <[email protected] >>>> > <mailto:[email protected]>> wrote: >>>> > >>>> > @Kenn: I am going to add the build.gradle. Is there anything else? >>>> > >>>> > @Ahmet, @Robert: here are more details about the samza runner >>>> right now: >>>> > >>>> > - Missing pieces: timer support in ParDo is not there yet and I >>>> plan >>>> > to add it soon. SplittableParDo is missing but we don't have a use >>>> > case so far. We are on par with the other runners for the rest of >>>> > the Java features. >>>> > - Work in Progress: implement the portable pipeline runner logic. >>>> > - Future plans: support Python is our next goal. Hopefully we will >>>> > get a prototype working sometime next quarter :). >>>> > >>>> > Btw, thanks everyone for the comments! >>>> > >>>> > Thanks, >>>> > Xinyu >>>> > >>>> > On Mon, Jun 18, 2018 at 4:59 PM, Robert Burke <[email protected] >>>> > <mailto:[email protected]>> wrote: >>>> > >>>> > This is exciting! Is it implemented as a portability framework >>>> > runner too? >>>> > >>>> > >>>> > On Mon, Jun 18, 2018, 4:36 PM Pablo Estrada < >>>> [email protected] >>>> > <mailto:[email protected]>> wrote: >>>> > >>>> > It's very exciting to see a new runner making it into >>>> > master. : ) >>>> > >>>> > Best >>>> > -P. >>>> > >>>> > On Mon, Jun 18, 2018 at 3:38 PM Rafael Fernandez >>>> > <[email protected] <mailto:[email protected]>> wrote: >>>> > >>>> > I've just read this and wanted to share my excitement >>>> :D >>>> > >>>> > >>>> > >>>> > On Mon, Jun 18, 2018 at 3:10 PM Kenneth Knowles >>>> > <[email protected] <mailto:[email protected]>> wrote: >>>> > >>>> > One thing that will be necessary is porting the >>>> > build to Gradle. >>>> > >>>> > Kenn >>>> > >>>> > On Mon, Jun 18, 2018 at 11:57 AM Xinyu Liu >>>> > <[email protected] >>>> > <mailto:[email protected]>> wrote: >>>> > >>>> > Hi, Folks, >>>> > >>>> > On behalf of the Samza team, I would like to >>>> > propose to merge the samza-runner branch into >>>> > master. The branch was created on Jan when we >>>> > first introduced the Samza Runner [1], and >>>> we've >>>> > been adding features and refining it >>>> afterwards. >>>> > Now the runner satisfies the criteria outlined >>>> > in [2], and merging it to master will give >>>> more >>>> > visibility to other contributors and users. >>>> > >>>> > 1. Have at least 2 contributors interested in >>>> > maintaining it, and 1 committer interested in >>>> > supporting it: *Both Chris and me have been >>>> > making contributions and I am going to sign up >>>> > for the support. There are more folks in the >>>> > Samza team interested in contributing to it. >>>> > Thanks Kenn for all the help and reviews for >>>> the >>>> > runner!* >>>> > 2. Provide both end-user and developer-facing >>>> > documentation: *The PR for the samza-runner >>>> doc >>>> > has runner user guide, capability matrix, and >>>> > tutorial using WordCount examples.* >>>> > 3. Have at least a basic level of unit test >>>> > coverage: *Unit tests are here [3].* >>>> > 4. Run all existing applicable integration >>>> tests >>>> > with other Beam components and create >>>> additional >>>> > tests as appropriate: Enabled ValidatesRunner >>>> > tests.* >>>> > 5. Be able to handle a subset of the model >>>> that >>>> > addresses a significant set of use cases, such >>>> > as ‘traditional batch’ or ‘processing time >>>> > streaming’: *We have test Beam jobs running in >>>> > Yarn using event-time processing of Kafka >>>> streams.* >>>> > 6. Update the capability matrix with the >>>> current >>>> > status. *Same as #2.* >>>> > 7. Add a webpage under documentation/runners. >>>> > *Same as #2.* >>>> > >>>> > The PR for the samza-runner >>>> > merge: https://github.com/ >>>> apache/beam/pull/5668 >>>> > <https://github.com/apache/beam/pull/5668> >>>> > The PR for the samza-runner >>>> > doc: https://github.com/ >>>> apache/beam-site/pull/471 <https://github.com/apache/beam-site/pull/471 >>>> > >>>> > >>>> > Thanks, >>>> > Xinyu >>>> > >>>> > [1] https://issues.apache. >>>> org/jira/browse/BEAM-3079 >>>> > <https://issues.apache.org/ >>>> jira/browse/BEAM-3079> >>>> > [2] https://beam.apache.org/contribute/ >>>> > <https://beam.apache.org/contribute/> >>>> > [3] https://github.com/ >>>> apache/beam/tree/samza-runner/runners/samza/src/test >>>> > <https://github.com/apache/ >>>> beam/tree/samza-runner/runners/samza/src/test> >>>> > >>>> > -- >>>> > Got feedback? go/pabloem-feedback >>>> <https://goto.google.com/pabloem-feedback> >>>> > >>>> > >>>> > >>>> >>>
