This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push: new c5900e791fa Publishing website 2022/11/14 22:17:16 at commit 2adb68b c5900e791fa is described below commit c5900e791fa2edde71ab7085a6827aa489289bab Author: jenkins <bui...@apache.org> AuthorDate: Mon Nov 14 22:17:16 2022 +0000 Publishing website 2022/11/14 22:17:16 at commit 2adb68b --- .../sdks/java-multi-language-pipelines/index.html | 10 ++++++---- website/generated-content/sitemap.xml | 2 +- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/website/generated-content/documentation/sdks/java-multi-language-pipelines/index.html b/website/generated-content/documentation/sdks/java-multi-language-pipelines/index.html index 620a91e00dd..1df683e5e74 100644 --- a/website/generated-content/documentation/sdks/java-multi-language-pipelines/index.html +++ b/website/generated-content/documentation/sdks/java-multi-language-pipelines/index.html @@ -36,7 +36,8 @@ version 2.41.0 or later installed, and a Python environment. If you don’t already have these environments set up, first complete the <a href=/get-started/quickstart-java/>Apache Beam Java SDK Quickstart</a> and the <a href=/get-started/quickstart-py/>Apache Beam Python SDK Quickstart</a>.</p><p>For running with portable DirectRunner, you need to have Docker installed -locally and the Docker daemon should be running. This is not needed for Dataflow.</p><p>This example relies on Python pandas package 1.4.0 or later which is unavailable +locally and the Docker daemon should be running. This is not needed for Dataflow.</p><p>For running on Dataflow, you need a Google Cloud project with billing enabled and a +<a href=https://cloud.google.com/storage/docs/creating-buckets>Google Cloud Storage bucket</a>.</p><p>This example relies on Python pandas package 1.4.0 or later which is unavailable for Python versions earlier than 3.8. Hence please make sure that the default Python version installed in your system is 3.8 or later.</p><h2 id=specify-a-cross-language-transform>Specify a cross-language transform</h2><p>The Java example pipeline uses the Python <a href=https://github.com/apache/beam/blob/master/sdks/python/apache_beam/dataframe/transforms.py>DataframeTransform</a> @@ -50,7 +51,7 @@ transforms are identified by their fully qualified name. For example, package, so its fully qualified name is <code>apache_beam.dataframe.transforms.DataframeTransform</code>. The example pipeline, -<a href=https://github.com/apache/beam/blob/master/examples/multi-language/src/main/java/org/apache/beam/examples/multilanguage/PythonDataframeWordCount.java>PythonDataframeWordCount</a>, +<a href=https://github.com/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/multilanguage/PythonDataframeWordCount.java>PythonDataframeWordCount</a>, passes this fully qualified name to <a href=https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/extensions/python/PythonExternalTransform.html>PythonExternalTransform</a>.</p><blockquote><p><strong>Note:</strong> The example pipeline is intended to demonstrate the development of Java multi-language pipelines that use arbitrary Python cross-language @@ -96,7 +97,8 @@ a function to <code>DataframeTransform</code>, see <a href=/documentation/dsls/dataframes/overview/#embedding-dataframes-in-a-pipeline>Embedding DataFrames in a pipeline</a>.</p><h2 id=run-the-java-pipeline>Run the Java pipeline</h2><p>If you want to customize the environment or use transforms not available in the default Beam SDK, you might need to run your own expansion service. In such cases, <a href=#advanced-start-an-expansion-service>start the expansion service</a> -before running your pipeline.</p><h3 id=run-with-dataflow-runner-at-head-beam-2410-and-later>Run with Dataflow runner at HEAD (Beam 2.41.0 and later)</h3><blockquote><p><strong>Note:</strong> Due to <a href=https://github.com/apache/beam/issues/23717>issue#23717</a>, +before running your pipeline.</p><p>Before running the pipeline, make sure to perform the +<a href=https://beam.apache.org/get-started/quickstart-java/#run-a-pipeline>runner specific setup</a> for your selected Beam runner.</p><h3 id=run-with-dataflow-runner-at-head-beam-2410-and-later>Run with Dataflow runner at HEAD (Beam 2.41.0 and later)</h3><blockquote><p><strong>Note:</strong> Due to <a href=https://github.com/apache/beam/issues/23717>issue#23717</a>, Beam 2.42.0 requires manually starting up an expansion service (see <a href=https://beam.apache.org/documentation/sdks/java-multi-language-pipelines/#advanced-start-an-expansion-service>these instructions</a>) and using the additional pipeline option <code>--expansionService=localhost:<PORT></code> @@ -146,7 +148,7 @@ or use transforms not available in the default Beam SDK, you might need to run your own expansion service.</p><p>For example, to start the standard expansion service for a Python transform, <a href=https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/expansion_service.py>ExpansionServiceServicer</a>, follow these steps:</p><ol><li><p>Activate a new virtual environment following -<a href=https://beam.apache.org/get-started/quickstart-py/#create-and-activate-a-virtual-environment>these instructions</a>.</p></li><li><p>Install Apache Beam with <code>gcp</code> and <code>dataframe</code> packages.</p></li></ol><pre><code>pip install apache-beam[gcp,dataframe] +<a href=https://beam.apache.org/get-started/quickstart-py/#create-and-activate-a-virtual-environment>these instructions</a>.</p></li><li><p>Install Apache Beam with <code>gcp</code> and <code>dataframe</code> packages.</p></li></ol><pre><code>pip install 'apache-beam[gcp,dataframe]' </code></pre><ol start=4><li>Run the following command</li></ol><pre><code>python -m apache_beam.runners.portability.expansion_service_main -p <PORT> --fully_qualified_name_glob "*" </code></pre><p>The command runs <a href=https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/expansion_service_main.py>expansion_service_main.py</a>, which starts the standard expansion service. When you use diff --git a/website/generated-content/sitemap.xml b/website/generated-content/sitemap.xml index 0c7c6b8ecf7..824ab20c02d 100644 --- a/website/generated-content/sitemap.xml +++ b/website/generated-content/sitemap.xml @@ -1 +1 @@ -<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/categories/blog/</loc><lastmod>2022-11-10T17:26:15-05:00</lastmod></url><url><loc>/blog/</loc><lastmod>2022-11-10T17:26:15-05:00</lastmod></url><url><loc>/categories/</loc><lastmod>2022-11-10T17:26:15-05:00</lastmod></url><url><loc>/blog/ml-resources/</loc><lastmod>2022-11-10T17:26:15-05:00</lastmod></url><url><loc>/categ [...] \ No newline at end of file +<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml"><url><loc>/categories/blog/</loc><lastmod>2022-11-10T17:26:15-05:00</lastmod></url><url><loc>/blog/</loc><lastmod>2022-11-10T17:26:15-05:00</lastmod></url><url><loc>/categories/</loc><lastmod>2022-11-10T17:26:15-05:00</lastmod></url><url><loc>/blog/ml-resources/</loc><lastmod>2022-11-10T17:26:15-05:00</lastmod></url><url><loc>/categ [...] \ No newline at end of file