Re: [PROPOSAL] Merge gearpump-runner to master

[Paul Findlay]

Sorry, I missed this email. It appears Ken committed a fix soon after my
email.

On Fri, Aug 11, 2017 at 2:09 PM, Manu Zhang  wrote:

> Hi Paul,
>
> The latest master compiles fine for me. Could you check again ?
> You may also want to check out the contribution guide
> . In short, the
> Apache way is to file a JIRA issue
>  and
> submit a GitHub pull request  to fix
> it.
>
> Thanks,
> Manu
>
> On Tue, Aug 8, 2017 at 1:17 PM Paul Findlay  wrote:
>
> > Cheers team.
> >
> > I have found a compilation issue on master (in
> > CreateGearpumpPCollectionView), attached is a small patch
> >
> > Kind regards,
> >
> > Paul
> >
> > On Tue, Aug 8, 2017 at 4:10 PM, Manu Zhang 
> > wrote:
> >
> >> Thanks Kenn!!! Thanks everyone!!! It's a great achievement for us.
> >>
> >> On Tue, Aug 8, 2017 at 7:54 AM Kenneth Knowles 
> >> wrote:
> >>
> >> > Done!
> >> >
> >> > On Fri, Jul 21, 2017 at 11:08 PM, Jean-Baptiste Onofré <
> j...@nanthrax.net
> >> >
> >> > wrote:
> >> >
> >> > > +1
> >> > >
> >> > > Regards
> >> > > JB
> >> > >
> >> > > On Jul 22, 2017, 05:06, at 05:06, Kenneth Knowles
> >>  >> > >
> >> > > wrote:
> >> > > >+1 to this!
> >> > > >
> >> > > >I really want to call out the longevity of contribution behind
> this,
> >> > > >following many changes in both Beam and Gearpump for over a year.
> >> > > >Here's
> >> > > >the first commit on the branch:
> >> > > >
> >> > > >commit 9478f4117de3a2d0ea40614ed4cb801918610724
> (github/pr/323)
> >> > > >Author: manuzhang 
> >> > > >Date:   Tue Mar 15 16:15:16 2016 +0800
> >> > > >
> >> > > >And here are some numbers, FWIW: 163 non-merge commits, 203 total.
> So
> >> > > >that's a PR and review every couple of weeks.
> >> > > >
> >> > > >The ValidatesRunner capability coverage is very good. The only
> >> skipped
> >> > > >tests are state/timers, metrics, and TestStream, which many runners
> >> > > >have
> >> > > >partial or no support for.
> >> > > >
> >> > > >I'll save practical TODOs like moving ValidatesRunner execution to
> >> > > >postcommit, etc. Pending the results of this discussion, of course.
> >> > > >
> >> > > >Kenn
> >> > > >
> >> > > >
> >> > > >On Fri, Jul 21, 2017 at 12:02 AM, Manu Zhang <
> >> owenzhang1...@gmail.com>
> >> > > >wrote:
> >> > > >
> >> > > >> Guys,
> >> > > >>
> >> > > >> On behalf of the gearpump team, I'd like to propose to merge the
> >> > > >> gearpump-runner branch into master, which will give it more
> >> > > >visibility to
> >> > > >> other contributors and users. The runner satisfies the following
> >> > > >criteria
> >> > > >> outlined in contribution guide [1].
> >> > > >>
> >> > > >>
> >> > > >>1. Have at least 2 contributors interested in maintaining it,
> >> and
> >> > > >1
> >> > > >>committer interested in supporting it: *Both Huafeng and me
> have
> >> > > >been
> >> > > >>making contributions[2] and we will continue to maintain it.
> >> Kenn
> >> > > >and JB
> >> > > >>have been supporting the runner (Thank you, guys!)*
> >> > > >>2. Provide both end-user and developer-facing documentation*:
> >> They
> >> > > >are
> >> > > >>already on the website ([3] and [4]).*
> >> > > >>3. Have at least a basic level of unit test coverage: *We do.*
> >> > > >*[5]*
> >> > > >>4. Run all existing applicable integration tests with other
> Beam
> >> > > >>components and create additional tests as appropriate:
> >> > > >*gearpump-runner
> >> > > >>passes ValidatesRunner tests.*
> >> > > >>
> >> > > >>
> >> > > >> Additionally, as a runner,
> >> > > >>
> >> > > >>
> >> > > >>1. Be able to handle a subset of the model that address a
> >> > > >significant
> >> > > >>set of use cases (aka. ‘traditional batch’ or ‘processing time
> >> > > >> streaming’): *gearpump
> >> > > >>runner is able to handle event time streaming *
> >> > > >>2. Update the capability matrix with the current status: *[4]*
> >> > > >>3. Add a webpage under documentation/runners: *[3]*
> >> > > >>
> >> > > >>
> >> > > >> The PR for the merge: https://github.com/apache/beam/pull/3611
> >> > > >>
> >> > > >> Thanks,
> >> > > >> Manu
> >> > > >>
> >> > > >>
> >> > > >> [1]
> >> > > >
> >> http://beam.apache.org/contribute/contribution-guide/#feature-branches
> >> > > >> [2] https://issues.apache.org/jira/browse/BEAM-79
> >> > > >> [3] https://beam.apache.org/documentation/runners/gearpump/
> >> > > >> [4]
> >> https://beam.apache.org/documentation/runners/capability-matrix/
> >> > > >> [5]
> >> > > >> https://github.com/apache/beam/tree/gearpump-runner/
> >> > > >> runners/gearpump/src/test/java/org/apache/beam/runners/gearpump
> >> > > >>
> >> > >
> >> >
> >>
> >
> >
>

Re: [PROPOSAL] Merge gearpump-runner to master

[Manu Zhang]

Hi Paul,

The latest master compiles fine for me. Could you check again ?
You may also want to check out the contribution guide
. In short, the
Apache way is to file a JIRA issue
 and
submit a GitHub pull request  to fix
it.

Thanks,
Manu

On Tue, Aug 8, 2017 at 1:17 PM Paul Findlay  wrote:

> Cheers team.
>
> I have found a compilation issue on master (in
> CreateGearpumpPCollectionView), attached is a small patch
>
> Kind regards,
>
> Paul
>
> On Tue, Aug 8, 2017 at 4:10 PM, Manu Zhang 
> wrote:
>
>> Thanks Kenn!!! Thanks everyone!!! It's a great achievement for us.
>>
>> On Tue, Aug 8, 2017 at 7:54 AM Kenneth Knowles 
>> wrote:
>>
>> > Done!
>> >
>> > On Fri, Jul 21, 2017 at 11:08 PM, Jean-Baptiste Onofré > >
>> > wrote:
>> >
>> > > +1
>> > >
>> > > Regards
>> > > JB
>> > >
>> > > On Jul 22, 2017, 05:06, at 05:06, Kenneth Knowles
>> > > >
>> > > wrote:
>> > > >+1 to this!
>> > > >
>> > > >I really want to call out the longevity of contribution behind this,
>> > > >following many changes in both Beam and Gearpump for over a year.
>> > > >Here's
>> > > >the first commit on the branch:
>> > > >
>> > > >commit 9478f4117de3a2d0ea40614ed4cb801918610724 (github/pr/323)
>> > > >Author: manuzhang 
>> > > >Date:   Tue Mar 15 16:15:16 2016 +0800
>> > > >
>> > > >And here are some numbers, FWIW: 163 non-merge commits, 203 total. So
>> > > >that's a PR and review every couple of weeks.
>> > > >
>> > > >The ValidatesRunner capability coverage is very good. The only
>> skipped
>> > > >tests are state/timers, metrics, and TestStream, which many runners
>> > > >have
>> > > >partial or no support for.
>> > > >
>> > > >I'll save practical TODOs like moving ValidatesRunner execution to
>> > > >postcommit, etc. Pending the results of this discussion, of course.
>> > > >
>> > > >Kenn
>> > > >
>> > > >
>> > > >On Fri, Jul 21, 2017 at 12:02 AM, Manu Zhang <
>> owenzhang1...@gmail.com>
>> > > >wrote:
>> > > >
>> > > >> Guys,
>> > > >>
>> > > >> On behalf of the gearpump team, I'd like to propose to merge the
>> > > >> gearpump-runner branch into master, which will give it more
>> > > >visibility to
>> > > >> other contributors and users. The runner satisfies the following
>> > > >criteria
>> > > >> outlined in contribution guide [1].
>> > > >>
>> > > >>
>> > > >>1. Have at least 2 contributors interested in maintaining it,
>> and
>> > > >1
>> > > >>committer interested in supporting it: *Both Huafeng and me have
>> > > >been
>> > > >>making contributions[2] and we will continue to maintain it.
>> Kenn
>> > > >and JB
>> > > >>have been supporting the runner (Thank you, guys!)*
>> > > >>2. Provide both end-user and developer-facing documentation*:
>> They
>> > > >are
>> > > >>already on the website ([3] and [4]).*
>> > > >>3. Have at least a basic level of unit test coverage: *We do.*
>> > > >*[5]*
>> > > >>4. Run all existing applicable integration tests with other Beam
>> > > >>components and create additional tests as appropriate:
>> > > >*gearpump-runner
>> > > >>passes ValidatesRunner tests.*
>> > > >>
>> > > >>
>> > > >> Additionally, as a runner,
>> > > >>
>> > > >>
>> > > >>1. Be able to handle a subset of the model that address a
>> > > >significant
>> > > >>set of use cases (aka. ‘traditional batch’ or ‘processing time
>> > > >> streaming’): *gearpump
>> > > >>runner is able to handle event time streaming *
>> > > >>2. Update the capability matrix with the current status: *[4]*
>> > > >>3. Add a webpage under documentation/runners: *[3]*
>> > > >>
>> > > >>
>> > > >> The PR for the merge: https://github.com/apache/beam/pull/3611
>> > > >>
>> > > >> Thanks,
>> > > >> Manu
>> > > >>
>> > > >>
>> > > >> [1]
>> > > >
>> http://beam.apache.org/contribute/contribution-guide/#feature-branches
>> > > >> [2] https://issues.apache.org/jira/browse/BEAM-79
>> > > >> [3] https://beam.apache.org/documentation/runners/gearpump/
>> > > >> [4]
>> https://beam.apache.org/documentation/runners/capability-matrix/
>> > > >> [5]
>> > > >> https://github.com/apache/beam/tree/gearpump-runner/
>> > > >> runners/gearpump/src/test/java/org/apache/beam/runners/gearpump
>> > > >>
>> > >
>> >
>>
>
>

Re: [PROPOSAL] Merge gearpump-runner to master

[Paul Findlay]

Cheers team.

I have found a compilation issue on master (in
CreateGearpumpPCollectionView), attached is a small patch

Kind regards,

Paul

On Tue, Aug 8, 2017 at 4:10 PM, Manu Zhang  wrote:

> Thanks Kenn!!! Thanks everyone!!! It's a great achievement for us.
>
> On Tue, Aug 8, 2017 at 7:54 AM Kenneth Knowles 
> wrote:
>
> > Done!
> >
> > On Fri, Jul 21, 2017 at 11:08 PM, Jean-Baptiste Onofré 
> > wrote:
> >
> > > +1
> > >
> > > Regards
> > > JB
> > >
> > > On Jul 22, 2017, 05:06, at 05:06, Kenneth Knowles
>  > >
> > > wrote:
> > > >+1 to this!
> > > >
> > > >I really want to call out the longevity of contribution behind this,
> > > >following many changes in both Beam and Gearpump for over a year.
> > > >Here's
> > > >the first commit on the branch:
> > > >
> > > >commit 9478f4117de3a2d0ea40614ed4cb801918610724 (github/pr/323)
> > > >Author: manuzhang 
> > > >Date:   Tue Mar 15 16:15:16 2016 +0800
> > > >
> > > >And here are some numbers, FWIW: 163 non-merge commits, 203 total. So
> > > >that's a PR and review every couple of weeks.
> > > >
> > > >The ValidatesRunner capability coverage is very good. The only skipped
> > > >tests are state/timers, metrics, and TestStream, which many runners
> > > >have
> > > >partial or no support for.
> > > >
> > > >I'll save practical TODOs like moving ValidatesRunner execution to
> > > >postcommit, etc. Pending the results of this discussion, of course.
> > > >
> > > >Kenn
> > > >
> > > >
> > > >On Fri, Jul 21, 2017 at 12:02 AM, Manu Zhang  >
> > > >wrote:
> > > >
> > > >> Guys,
> > > >>
> > > >> On behalf of the gearpump team, I'd like to propose to merge the
> > > >> gearpump-runner branch into master, which will give it more
> > > >visibility to
> > > >> other contributors and users. The runner satisfies the following
> > > >criteria
> > > >> outlined in contribution guide [1].
> > > >>
> > > >>
> > > >>1. Have at least 2 contributors interested in maintaining it, and
> > > >1
> > > >>committer interested in supporting it: *Both Huafeng and me have
> > > >been
> > > >>making contributions[2] and we will continue to maintain it. Kenn
> > > >and JB
> > > >>have been supporting the runner (Thank you, guys!)*
> > > >>2. Provide both end-user and developer-facing documentation*:
> They
> > > >are
> > > >>already on the website ([3] and [4]).*
> > > >>3. Have at least a basic level of unit test coverage: *We do.*
> > > >*[5]*
> > > >>4. Run all existing applicable integration tests with other Beam
> > > >>components and create additional tests as appropriate:
> > > >*gearpump-runner
> > > >>passes ValidatesRunner tests.*
> > > >>
> > > >>
> > > >> Additionally, as a runner,
> > > >>
> > > >>
> > > >>1. Be able to handle a subset of the model that address a
> > > >significant
> > > >>set of use cases (aka. ‘traditional batch’ or ‘processing time
> > > >> streaming’): *gearpump
> > > >>runner is able to handle event time streaming *
> > > >>2. Update the capability matrix with the current status: *[4]*
> > > >>3. Add a webpage under documentation/runners: *[3]*
> > > >>
> > > >>
> > > >> The PR for the merge: https://github.com/apache/beam/pull/3611
> > > >>
> > > >> Thanks,
> > > >> Manu
> > > >>
> > > >>
> > > >> [1]
> > > >http://beam.apache.org/contribute/contribution-guide/
> #feature-branches
> > > >> [2] https://issues.apache.org/jira/browse/BEAM-79
> > > >> [3] https://beam.apache.org/documentation/runners/gearpump/
> > > >> [4] https://beam.apache.org/documentation/runners/
> capability-matrix/
> > > >> [5]
> > > >> https://github.com/apache/beam/tree/gearpump-runner/
> > > >> runners/gearpump/src/test/java/org/apache/beam/runners/gearpump
> > > >>
> > >
> >
>
Index: runners/gearpump/src/main/java/org/apache/beam/runners/gearpump/translators/CreateStreamingGearpumpView.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===
--- runners/gearpump/src/main/java/org/apache/beam/runners/gearpump/translators/CreateStreamingGearpumpView.java	(revision 89236e3b588cd8ab5092667b1fe155738c69dbcf)
+++ runners/gearpump/src/main/java/org/apache/beam/runners/gearpump/translators/CreateStreamingGearpumpView.java	(revision )
@@ -18,11 +18,6 @@
 package org.apache.beam.runners.gearpump.translators;
 
 import com.google.common.collect.Iterables;
-
-import java.util.ArrayList;
-import java.util.List;
-import java.util.Map;
-
 import org.apache.beam.runners.core.construction.ReplacementOutputs;
 import org.apache.beam.sdk.coders.Coder;
 import org.apache.beam.sdk.coders.CoderRegistry;
@@ -37,6 +32,10 @@
 import org.apache.beam.sdk.values.PValue;
 import org.apache.beam.sdk.values.TupleTag;
 
+import java.util.ArrayList;
+import java.util.List;
+import 

Re: [PROPOSAL] Merge gearpump-runner to master

[Manu Zhang]

Thanks Kenn!!! Thanks everyone!!! It's a great achievement for us.

On Tue, Aug 8, 2017 at 7:54 AM Kenneth Knowles 
wrote:

> Done!
>
> On Fri, Jul 21, 2017 at 11:08 PM, Jean-Baptiste Onofré 
> wrote:
>
> > +1
> >
> > Regards
> > JB
> >
> > On Jul 22, 2017, 05:06, at 05:06, Kenneth Knowles  >
> > wrote:
> > >+1 to this!
> > >
> > >I really want to call out the longevity of contribution behind this,
> > >following many changes in both Beam and Gearpump for over a year.
> > >Here's
> > >the first commit on the branch:
> > >
> > >commit 9478f4117de3a2d0ea40614ed4cb801918610724 (github/pr/323)
> > >Author: manuzhang 
> > >Date:   Tue Mar 15 16:15:16 2016 +0800
> > >
> > >And here are some numbers, FWIW: 163 non-merge commits, 203 total. So
> > >that's a PR and review every couple of weeks.
> > >
> > >The ValidatesRunner capability coverage is very good. The only skipped
> > >tests are state/timers, metrics, and TestStream, which many runners
> > >have
> > >partial or no support for.
> > >
> > >I'll save practical TODOs like moving ValidatesRunner execution to
> > >postcommit, etc. Pending the results of this discussion, of course.
> > >
> > >Kenn
> > >
> > >
> > >On Fri, Jul 21, 2017 at 12:02 AM, Manu Zhang 
> > >wrote:
> > >
> > >> Guys,
> > >>
> > >> On behalf of the gearpump team, I'd like to propose to merge the
> > >> gearpump-runner branch into master, which will give it more
> > >visibility to
> > >> other contributors and users. The runner satisfies the following
> > >criteria
> > >> outlined in contribution guide [1].
> > >>
> > >>
> > >>1. Have at least 2 contributors interested in maintaining it, and
> > >1
> > >>committer interested in supporting it: *Both Huafeng and me have
> > >been
> > >>making contributions[2] and we will continue to maintain it. Kenn
> > >and JB
> > >>have been supporting the runner (Thank you, guys!)*
> > >>2. Provide both end-user and developer-facing documentation*: They
> > >are
> > >>already on the website ([3] and [4]).*
> > >>3. Have at least a basic level of unit test coverage: *We do.*
> > >*[5]*
> > >>4. Run all existing applicable integration tests with other Beam
> > >>components and create additional tests as appropriate:
> > >*gearpump-runner
> > >>passes ValidatesRunner tests.*
> > >>
> > >>
> > >> Additionally, as a runner,
> > >>
> > >>
> > >>1. Be able to handle a subset of the model that address a
> > >significant
> > >>set of use cases (aka. ‘traditional batch’ or ‘processing time
> > >> streaming’): *gearpump
> > >>runner is able to handle event time streaming *
> > >>2. Update the capability matrix with the current status: *[4]*
> > >>3. Add a webpage under documentation/runners: *[3]*
> > >>
> > >>
> > >> The PR for the merge: https://github.com/apache/beam/pull/3611
> > >>
> > >> Thanks,
> > >> Manu
> > >>
> > >>
> > >> [1]
> > >http://beam.apache.org/contribute/contribution-guide/#feature-branches
> > >> [2] https://issues.apache.org/jira/browse/BEAM-79
> > >> [3] https://beam.apache.org/documentation/runners/gearpump/
> > >> [4] https://beam.apache.org/documentation/runners/capability-matrix/
> > >> [5]
> > >> https://github.com/apache/beam/tree/gearpump-runner/
> > >> runners/gearpump/src/test/java/org/apache/beam/runners/gearpump
> > >>
> >
>

Re: [PROPOSAL] Merge gearpump-runner to master

[Kenneth Knowles]

Done!

On Fri, Jul 21, 2017 at 11:08 PM, Jean-Baptiste Onofré 
wrote:

> +1
>
> Regards
> JB
>
> On Jul 22, 2017, 05:06, at 05:06, Kenneth Knowles 
> wrote:
> >+1 to this!
> >
> >I really want to call out the longevity of contribution behind this,
> >following many changes in both Beam and Gearpump for over a year.
> >Here's
> >the first commit on the branch:
> >
> >commit 9478f4117de3a2d0ea40614ed4cb801918610724 (github/pr/323)
> >Author: manuzhang 
> >Date:   Tue Mar 15 16:15:16 2016 +0800
> >
> >And here are some numbers, FWIW: 163 non-merge commits, 203 total. So
> >that's a PR and review every couple of weeks.
> >
> >The ValidatesRunner capability coverage is very good. The only skipped
> >tests are state/timers, metrics, and TestStream, which many runners
> >have
> >partial or no support for.
> >
> >I'll save practical TODOs like moving ValidatesRunner execution to
> >postcommit, etc. Pending the results of this discussion, of course.
> >
> >Kenn
> >
> >
> >On Fri, Jul 21, 2017 at 12:02 AM, Manu Zhang 
> >wrote:
> >
> >> Guys,
> >>
> >> On behalf of the gearpump team, I'd like to propose to merge the
> >> gearpump-runner branch into master, which will give it more
> >visibility to
> >> other contributors and users. The runner satisfies the following
> >criteria
> >> outlined in contribution guide [1].
> >>
> >>
> >>1. Have at least 2 contributors interested in maintaining it, and
> >1
> >>committer interested in supporting it: *Both Huafeng and me have
> >been
> >>making contributions[2] and we will continue to maintain it. Kenn
> >and JB
> >>have been supporting the runner (Thank you, guys!)*
> >>2. Provide both end-user and developer-facing documentation*: They
> >are
> >>already on the website ([3] and [4]).*
> >>3. Have at least a basic level of unit test coverage: *We do.*
> >*[5]*
> >>4. Run all existing applicable integration tests with other Beam
> >>components and create additional tests as appropriate:
> >*gearpump-runner
> >>passes ValidatesRunner tests.*
> >>
> >>
> >> Additionally, as a runner,
> >>
> >>
> >>1. Be able to handle a subset of the model that address a
> >significant
> >>set of use cases (aka. ‘traditional batch’ or ‘processing time
> >> streaming’): *gearpump
> >>runner is able to handle event time streaming *
> >>2. Update the capability matrix with the current status: *[4]*
> >>3. Add a webpage under documentation/runners: *[3]*
> >>
> >>
> >> The PR for the merge: https://github.com/apache/beam/pull/3611
> >>
> >> Thanks,
> >> Manu
> >>
> >>
> >> [1]
> >http://beam.apache.org/contribute/contribution-guide/#feature-branches
> >> [2] https://issues.apache.org/jira/browse/BEAM-79
> >> [3] https://beam.apache.org/documentation/runners/gearpump/
> >> [4] https://beam.apache.org/documentation/runners/capability-matrix/
> >> [5]
> >> https://github.com/apache/beam/tree/gearpump-runner/
> >> runners/gearpump/src/test/java/org/apache/beam/runners/gearpump
> >>
>

Re: [PROPOSAL] Merge gearpump-runner to master

[Jean-Baptiste Onofré]

+1

Regards
JB

On Jul 22, 2017, 05:06, at 05:06, Kenneth Knowles  
wrote:
>+1 to this!
>
>I really want to call out the longevity of contribution behind this,
>following many changes in both Beam and Gearpump for over a year.
>Here's
>the first commit on the branch:
>
>commit 9478f4117de3a2d0ea40614ed4cb801918610724 (github/pr/323)
>Author: manuzhang 
>Date:   Tue Mar 15 16:15:16 2016 +0800
>
>And here are some numbers, FWIW: 163 non-merge commits, 203 total. So
>that's a PR and review every couple of weeks.
>
>The ValidatesRunner capability coverage is very good. The only skipped
>tests are state/timers, metrics, and TestStream, which many runners
>have
>partial or no support for.
>
>I'll save practical TODOs like moving ValidatesRunner execution to
>postcommit, etc. Pending the results of this discussion, of course.
>
>Kenn
>
>
>On Fri, Jul 21, 2017 at 12:02 AM, Manu Zhang 
>wrote:
>
>> Guys,
>>
>> On behalf of the gearpump team, I'd like to propose to merge the
>> gearpump-runner branch into master, which will give it more
>visibility to
>> other contributors and users. The runner satisfies the following
>criteria
>> outlined in contribution guide [1].
>>
>>
>>1. Have at least 2 contributors interested in maintaining it, and
>1
>>committer interested in supporting it: *Both Huafeng and me have
>been
>>making contributions[2] and we will continue to maintain it. Kenn
>and JB
>>have been supporting the runner (Thank you, guys!)*
>>2. Provide both end-user and developer-facing documentation*: They
>are
>>already on the website ([3] and [4]).*
>>3. Have at least a basic level of unit test coverage: *We do.*
>*[5]*
>>4. Run all existing applicable integration tests with other Beam
>>components and create additional tests as appropriate:
>*gearpump-runner
>>passes ValidatesRunner tests.*
>>
>>
>> Additionally, as a runner,
>>
>>
>>1. Be able to handle a subset of the model that address a
>significant
>>set of use cases (aka. ‘traditional batch’ or ‘processing time
>> streaming’): *gearpump
>>runner is able to handle event time streaming *
>>2. Update the capability matrix with the current status: *[4]*
>>3. Add a webpage under documentation/runners: *[3]*
>>
>>
>> The PR for the merge: https://github.com/apache/beam/pull/3611
>>
>> Thanks,
>> Manu
>>
>>
>> [1]
>http://beam.apache.org/contribute/contribution-guide/#feature-branches
>> [2] https://issues.apache.org/jira/browse/BEAM-79
>> [3] https://beam.apache.org/documentation/runners/gearpump/
>> [4] https://beam.apache.org/documentation/runners/capability-matrix/
>> [5]
>> https://github.com/apache/beam/tree/gearpump-runner/
>> runners/gearpump/src/test/java/org/apache/beam/runners/gearpump
>>

Re: [PROPOSAL] Merge gearpump-runner to master

[Kenneth Knowles]

+1 to this!

I really want to call out the longevity of contribution behind this,
following many changes in both Beam and Gearpump for over a year. Here's
the first commit on the branch:

commit 9478f4117de3a2d0ea40614ed4cb801918610724 (github/pr/323)
Author: manuzhang 
Date:   Tue Mar 15 16:15:16 2016 +0800

And here are some numbers, FWIW: 163 non-merge commits, 203 total. So
that's a PR and review every couple of weeks.

The ValidatesRunner capability coverage is very good. The only skipped
tests are state/timers, metrics, and TestStream, which many runners have
partial or no support for.

I'll save practical TODOs like moving ValidatesRunner execution to
postcommit, etc. Pending the results of this discussion, of course.

Kenn


On Fri, Jul 21, 2017 at 12:02 AM, Manu Zhang 
wrote:

> Guys,
>
> On behalf of the gearpump team, I'd like to propose to merge the
> gearpump-runner branch into master, which will give it more visibility to
> other contributors and users. The runner satisfies the following criteria
> outlined in contribution guide [1].
>
>
>1. Have at least 2 contributors interested in maintaining it, and 1
>committer interested in supporting it: *Both Huafeng and me have been
>making contributions[2] and we will continue to maintain it. Kenn and JB
>have been supporting the runner (Thank you, guys!)*
>2. Provide both end-user and developer-facing documentation*: They are
>already on the website ([3] and [4]).*
>3. Have at least a basic level of unit test coverage: *We do.* *[5]*
>4. Run all existing applicable integration tests with other Beam
>components and create additional tests as appropriate: *gearpump-runner
>passes ValidatesRunner tests.*
>
>
> Additionally, as a runner,
>
>
>1. Be able to handle a subset of the model that address a significant
>set of use cases (aka. ‘traditional batch’ or ‘processing time
> streaming’): *gearpump
>runner is able to handle event time streaming *
>2. Update the capability matrix with the current status: *[4]*
>3. Add a webpage under documentation/runners: *[3]*
>
>
> The PR for the merge: https://github.com/apache/beam/pull/3611
>
> Thanks,
> Manu
>
>
> [1] http://beam.apache.org/contribute/contribution-guide/#feature-branches
> [2] https://issues.apache.org/jira/browse/BEAM-79
> [3] https://beam.apache.org/documentation/runners/gearpump/
> [4] https://beam.apache.org/documentation/runners/capability-matrix/
> [5]
> https://github.com/apache/beam/tree/gearpump-runner/
> runners/gearpump/src/test/java/org/apache/beam/runners/gearpump
>