This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push: new ec6d621 Publishing website 2022/01/28 06:01:42 at commit 178cb7b ec6d621 is described below commit ec6d6211bca19df023e0a9bca66e3a0ec4cba262 Author: jenkins <bui...@apache.org> AuthorDate: Fri Jan 28 06:01:43 2022 +0000 Publishing website 2022/01/28 06:01:42 at commit 178cb7b --- .../sdks/python-multi-language-pipelines/index.html | 12 ++++++------ website/generated-content/get-started/index.xml | 4 ++-- .../get-started/try-beam-playground/index.html | 4 ++-- website/generated-content/sitemap.xml | 2 +- 4 files changed, 11 insertions(+), 11 deletions(-) diff --git a/website/generated-content/documentation/sdks/python-multi-language-pipelines/index.html b/website/generated-content/documentation/sdks/python-multi-language-pipelines/index.html index 3cac89f..15c50c7 100644 --- a/website/generated-content/documentation/sdks/python-multi-language-pipelines/index.html +++ b/website/generated-content/documentation/sdks/python-multi-language-pipelines/index.html @@ -18,7 +18,7 @@ function addPlaceholder(){$('input:text').attr('placeholder',"What are you looking for?");} function endSearch(){var search=document.querySelector(".searchBar");search.classList.add("disappear");var icons=document.querySelector("#iconsBar");icons.classList.remove("disappear");} function blockScroll(){$("body").toggleClass("fixedPosition");} -function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Languages</span></li><li><span class=section-nav-list-title>Java</span><ul class=section-nav-list><li><a href=/documentation/sdks/java/>Java SDK overvi [...] +function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Languages</span></li><li><span class=section-nav-list-title>Java</span><ul class=section-nav-list><li><a href=/documentation/sdks/java/>Java SDK overvi [...] <span class=kd>final</span> <span class=n>String</span> <span class=n>prefix</span><span class=o>;</span> @@ -42,7 +42,7 @@ function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfi <span class=n>ParDo</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=k>new</span> <span class=n>AddPrefixDoFn</span><span class=o>()));</span> <span class=o>}</span> <span class=o>}</span> -</code></pre></div><p>To make this available as a cross-language transform, you have to add a config object and a builder.</p><blockquote><p><strong>Note:</strong> Starting with Beam 2.34.0, Python SDK users can use some Java transforms without writing additional Java code. To learn more, see <a href=/documentation/programming-guide/#1311-creating-cross-language-java-transforms>Creating cross-language Java transforms</a>.</p></blockquote><p>The config object is a simple Java object (POJO [...] +</code></pre></div><p>To make this available as a cross-language transform, you have to add a config object and a builder.</p><blockquote><p><strong>Note:</strong> Starting with Beam 2.34.0, Python SDK users can use some Java transforms without writing additional Java code. To learn more, see <a href=/documentation/programming-guide/#1311-creating-cross-language-java-transforms>Creating cross-language Java transforms</a>.</p></blockquote><p>The config object is a simple Java object (POJO [...] <span class=n>String</span> <span class=n>prefix</span><span class=o>;</span> @@ -50,7 +50,7 @@ function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfi <span class=k>this</span><span class=o>.</span><span class=na>prefix</span> <span class=o>=</span> <span class=n>prefix</span><span class=o>;</span> <span class=o>}</span> <span class=o>}</span> -</code></pre></div><p>The builder class must implement <a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/transforms/ExternalTransformBuilder.html>ExternalTransformBuilder</a> and override <code>buildExternal</code>, which uses the config object.</p><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=kd>public</span> <span class=kd>class</span> <span class=nc>JavaPrefixBuilder</span> <span class=kd>implements</span> +</code></pre></div><p>The builder class, implemented below as <a href=https://github.com/apache/beam/blob/master/examples/multi-language/src/main/java/org/apache/beam/examples/multilanguage/JavaPrefixBuilder.java>JavaPrefixBuilder</a>, must implement <a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/transforms/ExternalTransformBuilder.html>ExternalTransformBuilder</a> and override <code>buildExternal</code>, which uses the config object.</p><div class=highlight [...] <span class=n>ExternalTransformBuilder</span><span class=o><</span><span class=n>JavaPrefixConfiguration</span><span class=o>,</span> <span class=n>PCollection</span><span class=o><</span><span class=n>String</span><span class=o>>,</span> <span class=n>PCollection</span><span class=o><</span><span class=n>String</span><span class=o>>></span> <span class=o>{</span> <span class=nd>@Override</span> @@ -69,7 +69,8 @@ function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfi <span class=k>return</span> <span class=n>ImmutableMap</span><span class=o>.</span><span class=na>of</span><span class=o>(</span><span class=n>URN</span><span class=o>,</span><span class=k>new</span> <span class=n>JavaPrefixBuilder</span><span class=o>());</span> <span class=o>}</span> <span class=o>}</span> -</code></pre></div><p>The registrar must implement <a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/expansion/ExternalTransformRegistrar.html>ExternalTransformRegistrar</a>, which has one method, <code>knownBuilderInstances</code>. This returns a map that maps a unique URN to an instance of your builder. You can use the <a href=https://github.com/google/auto/tree/master/service>AutoService</a> annotation to register this class with the expansion service.</p><h [...] +</code></pre></div><p>As shown here in <a href=https://github.com/apache/beam/blob/master/examples/multi-language/src/main/java/org/apache/beam/examples/multilanguage/JavaPrefixRegistrar.java>JavaPrefixRegistrar</a>, the registrar must implement <a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/expansion/ExternalTransformRegistrar.html>ExternalTransformRegistrar</a>, which has one method, <code>knownBuilderInstances</code>. This returns a map that maps a unique [...] +</code></pre><p>For instructions on running an example expansion service, see <a href=https://github.com/apache/beam/blob/master/examples/multi-language/README.md>this README</a>.</p><h2 id=create-a-python-pipeline>Create a Python pipeline</h2><p>Your Python pipeline can now use the <a href=https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.external.html#apache_beam.transforms.external.ExternalTransform>ExternalTransform</a> API to configure your cross-language transfo [...] <span class=nb>input</span> <span class=o>=</span> <span class=n>p</span> <span class=o>|</span> <span class=s1>'Read'</span> <span class=o>>></span> <span class=n>ReadFromText</span><span class=p>(</span><span class=n>input_path</span><span class=p>)</span><span class=o>.</span><span class=n>with_output_types</span><span class=p>(</span><span class=nb>str</span><span class=p>)</span> <span class=n>java_output</span> <span class=o>=</span> <span class=p>(</span> @@ -84,8 +85,7 @@ function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfi <span class=n>output</span> <span class=o>=</span> <span class=n>java_output</span> <span class=o>|</span> <span class=s1>'PythonPrefix'</span> <span class=o>>></span> <span class=n>beam</span><span class=o>.</span><span class=n>Map</span><span class=p>(</span><span class=n>python_prefix</span><span class=p>)</span> <span class=n>output</span> <span class=o>|</span> <span class=s1>'Write'</span> <span class=o>>></span> <span class=n>WriteToText</span><span class=p>(</span><span class=n>output_path</span><span class=p>)</span> -</code></pre></div><p><code>ExternalTransform</code> takes three parameters:</p><ul><li>The URN for the cross-language transform</li><li>The payload, either as a byte string or a <a href=https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.external.html#apache_beam.transforms.external.PayloadBuilder>PayloadBuilder</a></li><li>An expansion service</li></ul><p>The URN is simply a unique Beam identifier for the transform, and the expansion service has already been discussed [...] -</code></pre><p>The exact commands for running the Python pipeline will also vary based on your environment. Assuming that your pipeline is coded in a file named <strong>addprefix.py</strong>, the steps should be similar to those below.</p><h3 id=run-with-direct-runner>Run with direct runner</h3><p>In the following command, <code>input1</code> is a file containing lines of text:</p><pre><code>python addprefix.py --runner DirectRunner --environment_type=DOCKER --input input1 --output output +</code></pre></div><p><code>ExternalTransform</code> takes three parameters:</p><ul><li>The URN for the cross-language transform</li><li>The payload, either as a byte string or a <a href=https://beam.apache.org/releases/pydoc/current/apache_beam.transforms.external.html#apache_beam.transforms.external.PayloadBuilder>PayloadBuilder</a></li><li>An expansion service</li></ul><p>The URN is simply a unique Beam identifier for the transform, and the expansion service has already been discussed [...] </code></pre><h3 id=run-with-dataflow-runner>Run with Dataflow runner</h3><p>The following script runs the multi-language pipeline on Dataflow, using example text from a Cloud Storage bucket. You’ll need to adapt the script to your environment.</p><pre><code>#!/bin/bash export GCP_PROJECT=<project> export GCS_BUCKET=<bucket> diff --git a/website/generated-content/get-started/index.xml b/website/generated-content/get-started/index.xml index 70e2b5a..b3a6b23 100644 --- a/website/generated-content/get-started/index.xml +++ b/website/generated-content/get-started/index.xml @@ -4865,9 +4865,9 @@ without having to install Apache Beam in your environment.</p> <a href="https://frontend-beta-dot-apache-beam-testing.appspot.com/">Beam Playground</a>.</p> <h2 id="beam-playground-wordcount-example">Beam Playground WordCount Example</h2> <iframe -src="https://frontend-beta-dot-apache-beam-testing.appspot.com/embedded?example=SDK_JAVA%2fMinimalWordCount" +src="https://frontend-beta-dot-apache-beam-testing.appspot.com/embedded?enabled=true&example=SDK_JAVA%2fMinimalWordCount&code=" width="100%" -height="350px" +height="700px" class="code-snippet playground" allow="clipboard-write"> </iframe> diff --git a/website/generated-content/get-started/try-beam-playground/index.html b/website/generated-content/get-started/try-beam-playground/index.html index 76654aa..dcadb2a 100644 --- a/website/generated-content/get-started/try-beam-playground/index.html +++ b/website/generated-content/get-started/try-beam-playground/index.html @@ -20,7 +20,7 @@ function endSearch(){var search=document.querySelector(".searchBar");search.clas function blockScroll(){$("body").toggleClass("fixedPosition");} function openMenu(){addPlaceholder();blockScroll();}</script><div class="clearfix container-main-content"><div class="section-nav closed" data-offset-top=90 data-offset-bottom=500><span class="section-nav-back glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list data-section-nav><li><span class=section-nav-list-main-title>Get started</span></li><li><a href=/get-started/beam-overview/>Beam Overview</a></li><li><a href=/get-started/tour-of-beam/>Tour of Beam</a></li><li><s [...] without having to install Apache Beam in your environment.</p><p>You can try an Apache Beam examples at -<a href=https://frontend-beta-dot-apache-beam-testing.appspot.com/>Beam Playground</a>.</p><h2 id=beam-playground-wordcount-example>Beam Playground WordCount Example</h2><iframe src="https://frontend-beta-dot-apache-beam-testing.appspot.com/embedded?example=SDK_JAVA%2fMinimalWordCount" width=100% height=350px class="code-snippet playground" allow=clipboard-write></iframe><h2 id=how-to-add-new-examples>How To Add New Examples</h2><p>To add an Apache Beam example/test/kata into Beam Playgr [...] +<a href=https://frontend-beta-dot-apache-beam-testing.appspot.com/>Beam Playground</a>.</p><h2 id=beam-playground-wordcount-example>Beam Playground WordCount Example</h2><iframe src="https://frontend-beta-dot-apache-beam-testing.appspot.com/embedded?enabled=true&example=SDK_JAVA%2fMinimalWordCount&code=" width=100% height=700px class="code-snippet playground" allow=clipboard-write></iframe><h2 id=how-to-add-new-examples>How To Add New Examples</h2><p>To add an Apache Beam example/test/ka [...] add the <code>beam-playground</code> tag into the file to be added. <code>beam-playground</code> tag is a yaml format comment:</p><p><div class="language-java snippet"><div class="notebook-skip code-snippet"><a class=copy type=button data-bs-toggle=tooltip data-bs-placement=bottom title="Copy to clipboard"><img src=/images/copy-icon.svg></a><div class=highlight><pre class=chroma><code class=language-java data-lang=java><span class=c1>// beam-playground: </span><span class=c1>// name: Name of the example/test/kata @@ -60,7 +60,7 @@ If some Beam example/kata/test needs to add a new category, then please submit P After the category has been added, it can be used in the examples.</li></ul><p>More details on examples in Apache Beam Playground can be found <a href="https://docs.google.com/document/d/1LBeGVTYwJHYbtmLt06OjhBR1CJ1Wgz18MEZjvNkuofc/edit?usp=sharing">here</a>.</p><h2 id=next-steps>Next Steps</h2><ul><li>Try examples in <a href=https://frontend-beta-dot-apache-beam-testing.appspot.com/>Apache Beam Playground</a>.</li><li>Submit feedback using “Enjoying Playground?” in <a href=https://frontend-beta-dot-apache-beam-testing.appspot.com/>Apache Beam Playground</a> or via -<a href="https://docs.google.com/forms/d/e/1FAIpQLSd5_5XeOwwW2yjEVHUXmiBad8Lxk-4OtNcgG45pbyAZzd4EbA/viewform?usp=pp_url">this form</a>.</li><li>Join the Beam <a href=/community/contact-us>users@</a> mailing list.</li><li>If you’re interested in contributing to the Apache Beam Playground codebase, see the <a href=/contribute>Contribution Guide</a>.</li></ul><p>Please don’t hesitate to <a href=/community/contact-us>reach out</a> if you encounter any issues!</p><div class=feedba [...] +<a href="https://docs.google.com/forms/d/e/1FAIpQLSd5_5XeOwwW2yjEVHUXmiBad8Lxk-4OtNcgG45pbyAZzd4EbA/viewform?usp=pp_url">this form</a>.</li><li>Join the Beam <a href=/community/contact-us>users@</a> mailing list.</li><li>If you’re interested in contributing to the Apache Beam Playground codebase, see the <a href=/contribute>Contribution Guide</a>.</li></ul><p>Please don’t hesitate to <a href=/community/contact-us>reach out</a> if you encounter any issues!</p><div class=feedba [...] <a href=http://www.apache.org>The Apache Software Foundation</a> | <a href=/privacy_policy>Privacy Policy</a> | <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam logo, and the Apache feather logo are either registered trademarks or trademarks of The Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including The Apache Software Foundation.</div></div></div></div></footer></body></html> \ No newline at end of file diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index afb0369..32a1eaf 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.35.0/</loc><lastmod>2021-12-29T17:49:39-08:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-12-29T17:49:39-08:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-12-29T17:49:39-08:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-12-29T17:49:39-08:00</lastmod></url><url><loc>/blog/b [...] \ No newline at end of file +<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/blog/beam-2.35.0/</loc><lastmod>2021-12-29T17:49:39-08:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-12-29T17:49:39-08:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-12-29T17:49:39-08:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-12-29T17:49:39-08:00</lastmod></url><url><loc>/blog/b [...] \ No newline at end of file