I was under the impression that we had several @RunnableOnService integration tests that executed across runners.
Also, doesn't WordCount works on the DirectRunner, Flink and Dataflow? ( https://github.com/apache/incubator-beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/WordCount.java ) You still need to pass the "runner" specific options to get them to run like the GCP project / Flink cluster but this does give you the compile once and run the artifact on different runners. On Tue, Jul 26, 2016 at 2:04 PM, Emanuele Cesena <[email protected]> wrote: > Hi, > > No unfortunately I don’t think there’s currently any such example, > although I guess if you take the Flink runner example and run it with the > local runner it should work. Probably not with Spark though. > > I don’t recall the state issue… but I didn’t have much time lately to > explore, so I guess it’s still in the same *state* :) > > Best, > > > > On Jul 25, 2016, at 3:30 PM, amir bahmanyari <[email protected]> > wrote: > > > > Thanks Emanuele, > > Yes, I know these examples exist. > > I thought there has been one put together that addresses the Runner > agnostic coding specifically without doing any extra work. > > A true "unified" example. > > Did you solve your State issue? I had the same questions sometime ago. > > For now, I use Redis to persist run-time state. Kinda poor man's way :-) > works for now, but doesn't scale as I want it. > > Cheers > > > > From: Emanuele Cesena <[email protected]> > > To: [email protected]; amir bahmanyari <[email protected] > > > > Sent: Monday, July 25, 2016 3:18 PM > > Subject: Re: Example: pass Runner at command line > > > > Hi Amir, > > > > If you’re looking for a runner-independent example, you can find some in > examples/. > > > > If you’re looking for runner-independent code that works on Flink, I > think you should still wait a few iterations. > > > > Or, you can take code that refers to Flink, and remove the dependencies > yourself. > > > > For instance, if you start from this example: > > > https://github.com/ecesena/beam-starter/blob/master/src/main/java/com/dataradiant/beam/examples/WordCount.java > > > > You can remove the setRunner by passing -Drunner=FlinkRunner in pom.xml > > You can remove FlinkPipelineOptions but you have to implement yourself > some of the getter/setter required by Flink, e.g. jobName: > > > https://github.com/apache/incubator-beam/blob/master/runners/flink/runner/src/main/java/org/apache/beam/runners/flink/FlinkPipelineOptions.java > > > > Hope this helps, > > > > > > > On Jul 25, 2016, at 2:59 PM, amir bahmanyari <[email protected]> > wrote: > > > > > > Hi Colleagues, > > > Is there a simple genetic example where the Runner is passed at the > command line, the Beam code sets it in the generic Beam Options.set > Runner(), and Pipeline.create() is? > > > No mention of ANY specific Runner in the code like > FlinkPipelineOptions . > > > > > > Thanks. > > > Amir- > > > > > > -- > > Emanuele Cesena, Data Eng. > > http://www.shopkick.com > > > > Il corpo non ha ideali > > > > > > > > > > > > > > -- > Emanuele Cesena, Data Eng. > http://www.shopkick.com > > Il corpo non ha ideali > > > > >
