Cheers team.
I have found a compilation issue on master (in
CreateGearpumpPCollectionView), attached is a small patch
Kind regards,
Paul
On Tue, Aug 8, 2017 at 4:10 PM, Manu Zhang <owenzhang1...@gmail.com> wrote:
> Thanks Kenn!!! Thanks everyone!!! It's a great achievement for us.
>
> On Tue, Aug 8, 2017 at 7:54 AM Kenneth Knowles <k...@google.com.invalid>
> wrote:
>
> > Done!
> >
> > On Fri, Jul 21, 2017 at 11:08 PM, Jean-Baptiste Onofré <j...@nanthrax.net>
> > wrote:
> >
> > > +1
> > >
> > > Regards
> > > JB
> > >
> > > On Jul 22, 2017, 05:06, at 05:06, Kenneth Knowles
> <k...@google.com.INVALID
> > >
> > > wrote:
> > > >+1 to this!
> > > >
> > > >I really want to call out the longevity of contribution behind this,
> > > >following many changes in both Beam and Gearpump for over a year.
> > > >Here's
> > > >the first commit on the branch:
> > > >
> > > > commit 9478f4117de3a2d0ea40614ed4cb801918610724 (github/pr/323)
> > > > Author: manuzhang <owenzhang1...@gmail.com>
> > > > Date: Tue Mar 15 16:15:16 2016 +0800
> > > >
> > > >And here are some numbers, FWIW: 163 non-merge commits, 203 total. So
> > > >that's a PR and review every couple of weeks.
> > > >
> > > >The ValidatesRunner capability coverage is very good. The only skipped
> > > >tests are state/timers, metrics, and TestStream, which many runners
> > > >have
> > > >partial or no support for.
> > > >
> > > >I'll save practical TODOs like moving ValidatesRunner execution to
> > > >postcommit, etc. Pending the results of this discussion, of course.
> > > >
> > > >Kenn
> > > >
> > > >
> > > >On Fri, Jul 21, 2017 at 12:02 AM, Manu Zhang <owenzhang1...@gmail.com
> >
> > > >wrote:
> > > >
> > > >> Guys,
> > > >>
> > > >> On behalf of the gearpump team, I'd like to propose to merge the
> > > >> gearpump-runner branch into master, which will give it more
> > > >visibility to
> > > >> other contributors and users. The runner satisfies the following
> > > >criteria
> > > >> outlined in contribution guide [1].
> > > >>
> > > >>
> > > >> 1. Have at least 2 contributors interested in maintaining it, and
> > > >1
> > > >> committer interested in supporting it: *Both Huafeng and me have
> > > >been
> > > >> making contributions[2] and we will continue to maintain it. Kenn
> > > >and JB
> > > >> have been supporting the runner (Thank you, guys!)*
> > > >> 2. Provide both end-user and developer-facing documentation*:
> They
> > > >are
> > > >> already on the website ([3] and [4]).*
> > > >> 3. Have at least a basic level of unit test coverage: *We do.*
> > > >*[5]*
> > > >> 4. Run all existing applicable integration tests with other Beam
> > > >> components and create additional tests as appropriate:
> > > >*gearpump-runner
> > > >> passes ValidatesRunner tests.*
> > > >>
> > > >>
> > > >> Additionally, as a runner,
> > > >>
> > > >>
> > > >> 1. Be able to handle a subset of the model that address a
> > > >significant
> > > >> set of use cases (aka. ‘traditional batch’ or ‘processing time
> > > >> streaming’): *gearpump
> > > >> runner is able to handle event time streaming *
> > > >> 2. Update the capability matrix with the current status: *[4]*
> > > >> 3. Add a webpage under documentation/runners: *[3]*
> > > >>
> > > >>
> > > >> The PR for the merge: https://github.com/apache/beam/pull/3611
> > > >>
> > > >> Thanks,
> > > >> Manu
> > > >>
> > > >>
> > > >> [1]
> > > >http://beam.apache.org/contribute/contribution-guide/
> #feature-branches
> > > >> [2] https://issues.apache.org/jira/browse/BEAM-79
> > > >> [3] https://beam.apache.org/documentation/runners/gearpump/
> > > >> [4] https://beam.apache.org/documentation/runners/
> capability-matrix/
> > > >> [5]
> > > >> https://github.com/apache/beam/tree/gearpump-runner/
> > > >> runners/gearpump/src/test/java/org/apache/beam/runners/gearpump
> > > >>
> > >
> >
>
Index: runners/gearpump/src/main/java/org/apache/beam/runners/gearpump/translators/CreateStreamingGearpumpView.java
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- runners/gearpump/src/main/java/org/apache/beam/runners/gearpump/translators/CreateStreamingGearpumpView.java (revision 89236e3b588cd8ab5092667b1fe155738c69dbcf)
+++ runners/gearpump/src/main/java/org/apache/beam/runners/gearpump/translators/CreateStreamingGearpumpView.java (revision )
@@ -18,11 +18,6 @@
package org.apache.beam.runners.gearpump.translators;
import com.google.common.collect.Iterables;
-
-import java.util.ArrayList;
-import java.util.List;
-import java.util.Map;
-
import org.apache.beam.runners.core.construction.ReplacementOutputs;
import org.apache.beam.sdk.coders.Coder;
import org.apache.beam.sdk.coders.CoderRegistry;
@@ -37,6 +32,10 @@
import org.apache.beam.sdk.values.PValue;
import org.apache.beam.sdk.values.TupleTag;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+
/** Gearpump streaming overrides for various view (side input) transforms. */
class CreateStreamingGearpumpView<ElemT, ViewT>
extends PTransform<PCollection<ElemT>, PCollection<ElemT>> {
@@ -123,9 +122,8 @@
@Override
public PCollection<List<ElemT>> expand(PCollection<List<ElemT>> input) {
- return PCollection.<List<ElemT>>createPrimitiveOutputInternal(
- input.getPipeline(), input.getWindowingStrategy(), input.isBounded())
- .setCoder(input.getCoder());
+ return PCollection.createPrimitiveOutputInternal(
+ input.getPipeline(), input.getWindowingStrategy(), input.isBounded(), input.getCoder());
}
public PCollectionView<ViewT> getView() {
