This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 3612d22b164 Publishing website 2023/04/12 22:15:39 at commit 942bf87
3612d22b164 is described below
commit 3612d22b164f04ad9de143e1ab81a20439b84120
Author: jenkins <bui...@apache.org>
AuthorDate: Wed Apr 12 22:15:40 2023 +0000
Publishing website 2023/04/12 22:15:39 at commit 942bf87
---
.../documentation/sdks/python-pipeline-dependencies/index.html | 8 ++++++--
website/generated-content/sitemap.xml | 2 +-
2 files changed, 7 insertions(+), 3 deletions(-)
diff --git
a/website/generated-content/documentation/sdks/python-pipeline-dependencies/index.html
b/website/generated-content/documentation/sdks/python-pipeline-dependencies/index.html
index da2d998111c..d771e9884af 100644
---
a/website/generated-content/documentation/sdks/python-pipeline-dependencies/index.html
+++
b/website/generated-content/documentation/sdks/python-pipeline-dependencies/index.html
@@ -20,7 +20,7 @@
function addPlaceholder(){$('input:text').attr('placeholder',"What are you
looking for?");}
function endSearch(){var
search=document.querySelector(".searchBar");search.classList.add("disappear");var
icons=document.querySelector("#iconsBar");icons.classList.remove("disappear");}
function blockScroll(){$("body").toggleClass("fixedPosition");}
-function openMenu(){addPlaceholder();blockScroll();}</script><div
class="clearfix container-main-content"><div class="section-nav closed"
data-offset-top=90 data-offset-bottom=500><span class="section-nav-back
glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list
data-section-nav><li><span
class=section-nav-list-main-title>Languages</span></li><li><span
class=section-nav-list-title>Java</span><ul class=section-nav-list><li><a
href=/documentation/sdks/java/>Java SDK overvi [...]
+function openMenu(){addPlaceholder();blockScroll();}</script><div
class="clearfix container-main-content"><div class="section-nav closed"
data-offset-top=90 data-offset-bottom=500><span class="section-nav-back
glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list
data-section-nav><li><span
class=section-nav-list-main-title>Languages</span></li><li><span
class=section-nav-list-title>Java</span><ul class=section-nav-list><li><a
href=/documentation/sdks/java/>Java SDK overvi [...]
</code></pre><p>This command creates a <code>requirements.txt</code> file that
lists all packages that are installed on your machine, regardless of where they
were installed from.</p></li><li><p>Edit the <code>requirements.txt</code> file
and delete all packages that are not relevant to your code.</p></li><li><p>Run
your pipeline with the following command-line option:</p><pre><code>
--requirements_file requirements.txt
</code></pre><p>The runner will use the <code>requirements.txt</code> file to
install your additional dependencies onto the remote
workers.</p></li></ol><blockquote><p><strong>NOTE</strong>: An alternative to
<code>pip freeze</code> is to use a library like <a
href=https://github.com/jazzband/pip-tools>pip-tools</a> to compile all the
dependencies required for the pipeline from a <code>--requirements_file</code>,
where only top-level dependencies are mentioned.</p></blockquote><h2 id=cus
[...]
COPY <path to requirements.txt> /tmp/requirements.txt
@@ -48,7 +48,11 @@ package then you can build the tarball with the following
command:</p><pre><code
</code></pre><p>See the <a
href=https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/complete/juliaset>Juliaset</a>
project for an example that follows this required project
structure.</p></li><li><p>Run your pipeline with the following command-line
option:</p><pre><code> --setup_file /path/to/setup.py
</code></pre></li></ol><p><strong>Note:</strong> Because custom commands
execute after the dependencies for your workflow are installed (by
<code>pip</code>), you should omit the PyPI package dependency from the
pipeline’s <code>requirements.txt</code> file and from the
<code>install_requires</code> parameter in the <code>setuptools.setup()</code>
call of your <code>setup.py</code> file.</p><h2
id=pre-building-sdk-container-image>Pre-building SDK container image</h2><p>In
pipeline [...]
However, it may be possible to pre-build the SDK containers and perform the
dependency installation once before the workers start with
<code>--prebuild_sdk_container_engine</code>. For instructions of how to use
pre-building with Google Cloud
-Dataflow, see <a
href=https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild>Pre-building
the python SDK custom container image with extra
dependencies</a>.</p><p><strong>NOTE</strong>: This feature is available only
for the <code>Dataflow Runner v2</code>.</p></div></div><footer
class=footer><div class=footer__contained><div class=footer__cols><div
class="footer__cols__col footer__cols__col__logos"><div
class=footer__cols__col__logo><img src=/images/beam_logo_cir [...]
+Dataflow, see <a
href=https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild>Pre-building
the python SDK custom container image with extra
dependencies</a>.</p><p><strong>NOTE</strong>: This feature is available only
for the <code>Dataflow Runner v2</code>.</p><h2
id=pickling-and-managing-main-session>Pickling and Managing Main
Session</h2><p>Pickling in the Python SDK is set up to pickle the state of the
global namespace. By default, global imports, functions, an [...]
+Thus, one might encounter an unexpected <code>NameError</code> when running a
<code>DoFn</code> on any remote runner using portability. To resolve this,
manage the main session by
+simply setting the main session. This will load the pickled state of the
global namespace onto the Dataflow workers.
+For example, see <a
href=https://cloud.google.com/dataflow/docs/guides/common-errors#how-do-i-handle-nameerrors>Handling
NameErrors</a> to set the main session on the
<code>DataflowRunner</code>.</p><p><strong>NOTE</strong>: This applies to the
Python SDK executing with the dill pickler on any remote runner using
portability. Therefore, this issue will
+not occur in <code>DirectRunner</code>. Additionally, note that the dill
pickler is the default pickler in the Python SDK.</p></div></div><footer
class=footer><div class=footer__contained><div class=footer__cols><div
class="footer__cols__col footer__cols__col__logos"><div
class=footer__cols__col__logo><img src=/images/beam_logo_circle.svg
class=footer__logo alt="Beam logo"></div><div
class=footer__cols__col__logo><img src=/images/apache_logo_circle.svg
class=footer__logo alt="Apache logo [...]
<a href=https://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam
logo, and the Apache feather logo are either registered trademarks or
trademarks of The Apache Software Foundation. All other products or name brands
are trademarks of their respective holders, including The Apache Software
Foundation.</div></div><div class="footer__cols__col
footer__cols__col__logos"><div class=footer__cols__col--group><div
class=footer__cols__col__logo><a href=https://github.com/apache/beam><im [...]
\ No newline at end of file
diff --git a/website/generated-content/sitemap.xml
b/website/generated-content/sitemap.xml
index 4ef93246939..fb274f8e4e8 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9";
xmlns:xhtml="http://www.w3.org/1999/xhtml";><url><loc>/blog/beam-2.46.0/</loc><lastmod>2023-04-12T18:12:36+03:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2023-04-12T18:12:36+03:00</lastmod></url><url><loc>/blog/</loc><lastmod>2023-04-12T18:12:36+03:00</lastmod></url><url><loc>/categories/</loc><lastmod>2023-04-12T18:12:36+03:00</lastmod></url><url><loc>/catego
[...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9";
xmlns:xhtml="http://www.w3.org/1999/xhtml";><url><loc>/blog/beam-2.46.0/</loc><lastmod>2023-04-12T16:30:14-04:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2023-04-12T16:30:14-04:00</lastmod></url><url><loc>/blog/</loc><lastmod>2023-04-12T16:30:14-04:00</lastmod></url><url><loc>/categories/</loc><lastmod>2023-04-12T16:30:14-04:00</lastmod></url><url><loc>/catego
[...]
\ No newline at end of file
