This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push: new 3b45ad6 Publishing website 2019/04/18 09:48:28 at commit 0447bf9 3b45ad6 is described below commit 3b45ad611d0b0b0665a9b71642fbb4307a2dc1ff Author: jenkins <bui...@apache.org> AuthorDate: Thu Apr 18 09:48:29 2019 +0000 Publishing website 2019/04/18 09:48:28 at commit 0447bf9 --- .../generated-content/roadmap/portability/index.html | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/website/generated-content/roadmap/portability/index.html b/website/generated-content/roadmap/portability/index.html index 01a56a5..2472ebb 100644 --- a/website/generated-content/roadmap/portability/index.html +++ b/website/generated-content/roadmap/portability/index.html @@ -213,7 +213,7 @@ <li><a href="#issues">Issues</a></li> <li><a href="#status">Status</a> <ul> - <li><a href="#python-on-flink">Running Python wordcount on Flink</a></li> + <li><a href="#python-on-flink">Running Python wordcount on Flink or Spark</a></li> </ul> </li> </ul> @@ -379,18 +379,23 @@ See the <a href="https://s.apache.org/apache-beam-portability-support-table">Portability support table</a> for details.</p> -<h3 id="python-on-flink">Running Python wordcount on Flink</h3> +<h3 id="python-on-flink">Running Python wordcount on Flink or Spark</h3> -<p>The Flink runner is currently the only runner to support portable pipeline execution. -To run a basic Python wordcount (in batch mode) with embedded Flink:</p> +<p>Currently, the Flink and Spark runners support portable pipeline execution. +To run a basic Python wordcount (in batch mode) with embedded Flink or Spark:</p> <ol> <li>Run once to build the SDK harness container: <code class="highlighter-rouge">./gradlew :beam-sdks-python-container:docker</code></li> - <li>Start the Flink portable JobService endpoint: <code class="highlighter-rouge">./gradlew :beam-runners-flink_2.11-job-server:runShadow</code></li> - <li>Submit the wordcount pipeline to above endpoint: <code class="highlighter-rouge">./gradlew :beam-sdks-python:portableWordCount -PjobEndpoint=localhost:8099</code></li> + <li>Choose one: + <ul> + <li>Start the Flink portable JobService endpoint: <code class="highlighter-rouge">./gradlew :beam-runners-flink_2.11-job-server:runShadow</code></li> + <li>Or start the Spark portable JobService endpoint: <code class="highlighter-rouge">./gradlew :beam-runners-spark-job-server:runShadow</code></li> + </ul> + </li> + <li>Submit the wordcount pipeline to above endpoint: <code class="highlighter-rouge">./gradlew :beam-sdks-python:portableWordCount -PjobEndpoint=localhost:8099 -PenvironmentType=LOOPBACK</code></li> </ol> -<p>To run the pipeline in streaming mode: <code class="highlighter-rouge">./gradlew :beam-sdks-python:portableWordCount -PjobEndpoint=localhost:8099 -Pstreaming</code></p> +<p>To run the pipeline in streaming mode (currently only supported on Flink): <code class="highlighter-rouge">./gradlew :beam-sdks-python:portableWordCount -PjobEndpoint=localhost:8099 -Pstreaming</code></p> <p>Please see the <a href="/documentation/runners/flink/">Flink Runner page</a> for more information on how to run portable pipelines on top of Flink.</p>