This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push: new 8cae0b3 Publishing website 2020/06/18 00:03:05 at commit e7ca038 8cae0b3 is described below commit 8cae0b3b3347d1d6b687dacefc1b22046b09d50b Author: jenkins <us...@infra.apache.org> AuthorDate: Thu Jun 18 00:03:06 2020 +0000 Publishing website 2020/06/18 00:03:05 at commit e7ca038 --- website/generated-content/get-started/index.xml | 74 +++++++++++++++++++++- .../get-started/quickstart-java/index.html | 24 ++++++- website/generated-content/sitemap.xml | 2 +- 3 files changed, 95 insertions(+), 5 deletions(-) diff --git a/website/generated-content/get-started/index.xml b/website/generated-content/get-started/index.xml index 7e086da..5cd8418 100644 --- a/website/generated-content/get-started/index.xml +++ b/website/generated-content/get-started/index.xml @@ -978,7 +978,13 @@ limitations under the License. <ul> <li><a href="#set-up-your-development-environment">Set up your Development Environment</a></li> <li><a href="#get-the-wordcount-code">Get the WordCount Code</a></li> -<li><a href="#run-wordcount">Run WordCount</a></li> +<li><a href="#optional-convert-from-maven-to-gradle-project">Optional: Convert from Maven to Gradle Project</a></li> +<li><a href="#run-wordcount">Run WordCount</a> +<ul> +<li><a href="#run-wordcount-using-maven">Run WordCount Using Maven</a></li> +<li><a href="#run-wordcount-using-gradle">Run WordCount Using Gradle</a></li> +</ul> +</li> <li><a href="#inspect-the-results">Inspect the results</a></li> <li><a href="#next-steps">Next Steps</a></li> </ul> @@ -991,6 +997,9 @@ limitations under the License. <li> <p>Download and install <a href="https://maven.apache.org/download.cgi">Apache Maven</a> by following Maven&rsquo;s <a href="https://maven.apache.org/install.html">installation guide</a> for your specific operating system.</p> </li> +<li> +<p>Optional: Install <a href="https://gradle.org/install/">Gradle</a> if you would like to convert your Maven project into Gradle.</p> +</li> </ol> <h2 id="get-the-wordcount-code">Get the WordCount Code</h2> <p>The easiest way to get a copy of the WordCount pipeline is to use the following command to generate a simple Maven project that contains Beam&rsquo;s WordCount examples and builds against the most recent Beam release:</p> @@ -1046,6 +1055,35 @@ d----- 7/19/2018 11:00 PM subprocess -a---- 7/19/2018 11:00 PM 7662 WordCount.java</code></pre> </div> <p>For a detailed introduction to the Beam concepts used in these examples, see the <a href="/get-started/wordcount-example">WordCount Example Walkthrough</a>. Here, we&rsquo;ll just focus on executing <code>WordCount.java</code>.</p> +<h2 id="optional-convert-from-maven-to-gradle-project">Optional: Convert from Maven to Gradle Project</h2> +<p>Ensure you are in the same directory as the <code>pom.xml</code> file generated from the previous step. Automatically convert your project from Maven to Gradle by running: +<pre><code>$ gradle init</code></pre> +</p> +<p>After you have converted the project to Gradle:</p> +<ol> +<li>Edit the generated <code>build.gradle</code> file by adding <code>mavenCentral()</code> under <code>repositories</code>: +<pre><code>repositories { +mavenCentral() +maven { +url = uri(&#39;https://repository.apache.org/content/repositories/snapshots/&#39;) +} +maven { +url = uri(&#39;http://repo.maven.apache.org/maven2&#39;) +} +}</code></pre> +</li> +<li>Add the following task in <code>build.gradle</code> to allow you to execute pipelines with Gradle: +<pre><code>task execute (type:JavaExec) { +main = System.getProperty(&#34;mainClass&#34;) +classpath = sourceSets.main.runtimeClasspath +systemProperties System.getProperties() +args System.getProperty(&#34;exec.args&#34;).split() +}</code></pre> +</li> +<li>Rebuild your project by running: +<pre><code>$ gradle build</code></pre> +</li> +</ol> <h2 id="run-wordcount">Run WordCount</h2> <p>A single Beam pipeline can run on multiple Beam <a href="/documentation#runners">runners</a>, including the <a href="/documentation/runners/apex">ApexRunner</a>, <a href="/documentation/runners/flink">FlinkRunner</a>, <a href="/documentation/runners/spark">SparkRunner</a>, <a href="/documentation/runners/nemo">NemoRunner</a>, <a href="/documentation/runners/jet">JetRunner</a>, or <a href="/documentation/runners/dataflow">DataflowRunner</a>. [...] <p>After you&rsquo;ve chosen which runner you&rsquo;d like to use:</p> @@ -1060,6 +1098,7 @@ d----- 7/19/2018 11:00 PM subprocess </li> <li>Run your first WordCount pipeline.</li> </ol> +<h3 id="run-wordcount-using-maven">Run WordCount Using Maven</h3> <p>For Unix shells:</p> <div class=runner-direct> <pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \ @@ -1151,6 +1190,39 @@ PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examp PS&gt; java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount ` --runner=JetRunner --jetLocalMode=3 --inputFile=$pwd/pom.xml --output=counts</code></pre> </div> +<h3 id="run-wordcount-using-gradle">Run WordCount Using Gradle</h3> +<p>For Unix shells (Instructions currently only available for Direct, Spark, and Dataflow):</p> +<div class=runner-direct> +<pre><code>$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \ +-Dexec.args=&#34;--inputFile=pom.xml --output=counts&#34; -Pdirect-runner</code></pre> +</div> +<div class=runner-apex> +<pre><code>We are working on adding the instruction for this runner!</code></pre> +</div> +<div class=runner-flink-local> +<pre><code>We are working on adding the instruction for this runner!</code></pre> +</div> +<div class=runner-flink-cluster> +<pre><code>We are working on adding the instruction for this runner!</code></pre> +</div> +<div class=runner-spark> +<pre><code>$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \ +-Dexec.args=&#34;--inputFile=pom.xml --output=counts&#34; -Pspark-runner</code></pre> +</div> +<div class=runner-dataflow> +<pre><code>$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \ +-Dexec.args=&#34;--project=&lt;your-gcp-project&gt; --inputFile=gs://apache-beam-samples/shakespeare/* \ +--output=gs://&lt;your-gcs-bucket&gt;/counts&#34; -Pdataflow-runner</code></pre> +</div> +<div class=runner-samza-local> +<pre><code>We are working on adding the instruction for this runner!</code></pre> +</div> +<div class=runner-nemo> +<pre><code>We are working on adding the instruction for this runner!</code></pre> +</div> +<div class=runner-jet> +<pre><code>We are working on adding the instruction for this runner!</code></pre> +</div> <h2 id="inspect-the-results">Inspect the results</h2> <p>Once the pipeline has completed, you can view the output. You&rsquo;ll notice that there may be multiple output files prefixed by <code>count</code>. The exact number of these files is decided by the runner, giving it the flexibility to do efficient, distributed execution.</p> <div class=runner-direct> diff --git a/website/generated-content/get-started/quickstart-java/index.html b/website/generated-content/get-started/quickstart-java/index.html index 14b35a3..057f901 100644 --- a/website/generated-content/get-started/quickstart-java/index.html +++ b/website/generated-content/get-started/quickstart-java/index.html @@ -1,7 +1,7 @@ <!doctype html><html lang=en class=no-js><head><meta charset=utf-8><meta http-equiv=x-ua-compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><title>Beam Quickstart for Java</title><meta name=description content="Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Doma [...] <span class=sr-only>Toggle navigation</span> <span class=icon-bar></span><span class=icon-bar></span><span class=icon-bar></span></button> -<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...] +<a href=/ class=navbar-brand><img alt=Brand style=height:25px src=/images/beam_logo_navbar.png></a></div><div class="navbar-mask closed"></div><div id=navbar class="navbar-container closed"><ul class="nav navbar-nav"><li><a href=/get-started/beam-overview/>Get Started</a></li><li><a href=/documentation/>Documentation</a></li><li><a href=/documentation/sdks/java/>Languages</a></li><li><a href=/documentation/runners/capability-matrix/>RUNNERS</a></li><li><a href=/roadmap/>Roadmap</a></li>< [...] -DarchetypeGroupId=org.apache.beam \ -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \ -DarchetypeVersion=2.22.0 \ @@ -46,7 +46,21 @@ d----- 7/19/2018 11:00 PM subprocess -a---- 7/19/2018 11:00 PM 7073 DebuggingWordCount.java -a---- 7/19/2018 11:00 PM 5945 MinimalWordCount.java -a---- 7/19/2018 11:00 PM 9490 WindowedWordCount.java --a---- 7/19/2018 11:00 PM 7662 WordCount.java</code></pre></div><p>For a detailed introduction to the Beam concepts used in these examples, see the <a href=/get-started/wordcount-example>WordCount Example Walkthrough</a>. Here, we’ll just focus on executing <code>WordCount.java</code>.</p><h2 id=run-wordcount>Run WordCount</h2><p>A single Beam pipeline can run on multiple Beam <a href=/documentation#runners>runners</a>, including the <a href=/documentation/runners [...] +-a---- 7/19/2018 11:00 PM 7662 WordCount.java</code></pre></div><p>For a detailed introduction to the Beam concepts used in these examples, see the <a href=/get-started/wordcount-example>WordCount Example Walkthrough</a>. Here, we’ll just focus on executing <code>WordCount.java</code>.</p><h2 id=optional-convert-from-maven-to-gradle-project>Optional: Convert from Maven to Gradle Project</h2><p>Ensure you are in the same directory as the <code>pom.xml</code> file g [...] + mavenCentral() + maven { + url = uri('https://repository.apache.org/content/repositories/snapshots/') + } + + maven { + url = uri('http://repo.maven.apache.org/maven2') + } +}</code></pre></li><li>Add the following task in <code>build.gradle</code> to allow you to execute pipelines with Gradle:<pre><code>task execute (type:JavaExec) { + main = System.getProperty("mainClass") + classpath = sourceSets.main.runtimeClasspath + systemProperties System.getProperties() + args System.getProperty("exec.args").split() +}</code></pre></li><li>Rebuild your project by running:<pre><code>$ gradle build</code></pre></li></ol><h2 id=run-wordcount>Run WordCount</h2><p>A single Beam pipeline can run on multiple Beam <a href=/documentation#runners>runners</a>, including the <a href=/documentation/runners/apex>ApexRunner</a>, <a href=/documentation/runners/flink>FlinkRunner</a>, <a href=/documentation/runners/spark>SparkRunner</a>, <a href=/documentation/runners/nemo>NemoRunner</a>, <a href=/documentation/runner [...] -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner</code></pre></div><div class=runner-apex><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \ -Dexec.args="--inputFile=pom.xml --output=counts --runner=ApexRunner" -Papex-runner</code></pre></div><div class=runner-flink-local><pre><code>$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \ -Dexec.args="--runner=FlinkRunner --inputFile=pom.xml --output=counts" -Pflink-runner</code></pre></div><div class=runner-flink-cluster><pre><code>$ mvn package exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \ @@ -85,7 +99,11 @@ PS> mvn compile exec:java -D exec.mainClass=org.apache.beam.examples.WordCoun PS> java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount ` --runner=NemoRunner --inputFile=`pwd`/pom.xml --output=counts</code></pre></div><div class=runner-jet><pre><code>PS> mvn package -P jet-runner PS> java -cp target/word-count-beam-bundled-0.1.jar org.apache.beam.examples.WordCount ` - --runner=JetRunner --jetLocalMode=3 --inputFile=$pwd/pom.xml --output=counts</code></pre></div><h2 id=inspect-the-results>Inspect the results</h2><p>Once the pipeline has completed, you can view the output. You’ll notice that there may be multiple output files prefixed by <code>count</code>. The exact number of these files is decided by the runner, giving it the flexibility to do efficient, distributed execution.</p><div class=runner-direct><pre><code>$ ls counts*</code></pre [...] + --runner=JetRunner --jetLocalMode=3 --inputFile=$pwd/pom.xml --output=counts</code></pre></div><h3 id=run-wordcount-using-gradle>Run WordCount Using Gradle</h3><p>For Unix shells (Instructions currently only available for Direct, Spark, and Dataflow):</p><div class=runner-direct><pre><code>$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \ + -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner</code></pre></div><div class=runner-apex><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-flink-local><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-flink-cluster><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-spark><pre><code>$ gradle clean [...] + -Dexec.args="--inputFile=pom.xml --output=counts" -Pspark-runner</code></pre></div><div class=runner-dataflow><pre><code>$ gradle clean execute -DmainClass=org.apache.beam.examples.WordCount \ + -Dexec.args="--project=<your-gcp-project> --inputFile=gs://apache-beam-samples/shakespeare/* \ + --output=gs://<your-gcs-bucket>/counts" -Pdataflow-runner</code></pre></div><div class=runner-samza-local><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-nemo><pre><code>We are working on adding the instruction for this runner!</code></pre></div><div class=runner-jet><pre><code>We are working on adding the instruction for this runner!</code></pre></div><h2 id=inspect-the-results>Inspect the results</h2><p>Once the pip [...] api: 9 bundled: 1 old: 4 diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index b31f7bd..a889e3a 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.22.0/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/b [...] \ No newline at end of file +<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.22.0/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2020-06-08T14:13:37-07:00</lastmod></url><url><loc>/blog/b [...] \ No newline at end of file