This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 91ea99deeb9 Publishing website 2022/10/26 22:15:43 at commit 9c44241
91ea99deeb9 is described below
commit 91ea99deeb9dda9079d8f80c8de156f82ca006d3
Author: jenkins <bui...@apache.org>
AuthorDate: Wed Oct 26 22:15:43 2022 +0000
Publishing website 2022/10/26 22:15:43 at commit 9c44241
---
website/generated-content/contribute/index.xml | 24 +++++++-------
.../contribute/release-guide/index.html | 24 +++++++-------
.../sdks/python-machine-learning/index.html | 37 ++++------------------
.../get-started/downloads/index.html | 6 ++--
website/generated-content/get-started/index.xml | 6 ++++
website/generated-content/sitemap.xml | 2 +-
6 files changed, 44 insertions(+), 55 deletions(-)
diff --git a/website/generated-content/contribute/index.xml
b/website/generated-content/contribute/index.xml
index fcbaefbe6ed..edbfaea2df4 100644
--- a/website/generated-content/contribute/index.xml
+++ b/website/generated-content/contribute/index.xml
@@ -1250,7 +1250,7 @@ is perfectly safe since the script does not depend on the
current working tree.&
<ol>
<li>Visit <a
href="https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image";>https://hub.docker.com/u/apache</a></li>
<li>Visit each repository and navigate to <em>tags</em> tab.</li>
-<li>Verify images are pushed with tags: ${RELEASE}_rc{RC_NUM}</li>
+<li>Verify images are pushed with tags:
${RELEASE_VERSION}_rc{RC_NUM}</li>
</ol>
</li>
<li>Verify that third party licenses are included in Docker containers by
logging in to the images.
@@ -1259,12 +1259,12 @@ is perfectly safe since the script does not depend on
the current working tree.&
Please note that dependencies for the SDKs with different Python versions vary.
Need to verify all Python images by replacing <code>${ver}</code> with
each supported Python version <code>X.Y</code>.</li>
</ul>
-<pre><code>docker run --rm -it --entrypoint=/bin/bash
apache/beam_python${ver}_sdk:${RELEASE}_rc{RC_NUM}
+<pre><code>docker run --rm -it --entrypoint=/bin/bash
apache/beam_python${ver}_sdk:${RELEASE_VERSION}_rc{RC_NUM}
ls -al /opt/apache/beam/third_party_licenses/ | wc -l
</code></pre><ul>
<li>For Java SDK images, there should be around 200 dependencies.</li>
</ul>
-<pre><code>docker run --rm -it --entrypoint=/bin/bash
apache/beam_java${ver}_sdk:${RELEASE}_rc{RC_NUM}
+<pre><code>docker run --rm -it --entrypoint=/bin/bash
apache/beam_java${ver}_sdk:${RELEASE_VERSION}_rc{RC_NUM}
ls -al /opt/apache/beam/third_party_licenses/ | wc -l
</code></pre></li>
</ol>
@@ -1413,7 +1413,7 @@ ${CONTRIBUTORS}
<li>Maven artifacts deployed to the staging repository of <a
href="https://repository.apache.org/content/repositories/";>repository.apache.org</a></li>
<li>Source distribution deployed to the dev repository of <a
href="https://dist.apache.org/repos/dist/dev/beam/";>dist.apache.org</a></li>
<li>Website pull request proposed to list the <a
href="https://beam.apache.org/get-started/downloads/";>release</a>, publish
the <a href="https://beam.apache.org/releases/javadoc/";>Java API reference
manual</a>, and publish the <a
href="https://beam.apache.org/releases/pydoc/";>Python API reference
manual</a>.</li>
-<li>Docker images are published to <a
href="https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image";>DockerHub</a>
with tags: {RELEASE}_rc{RC_NUM}.</li>
+<li>Docker images are published to <a
href="https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image";>DockerHub</a>
with tags: {RELEASE_VERSION}_rc{RC_NUM}.</li>
</ol>
<p>You can (optionally) also do additional verification by:</p>
<ol>
@@ -1449,8 +1449,9 @@ The complete staging area is available for your review,
which includes:
* website pull request listing the release [6], the blog post [6], and
publishing the API reference manual [7].
* Java artifacts were built with Gradle GRADLE_VERSION and OpenJDK/Oracle JDK
JDK_VERSION.
* Python artifacts are deployed along with the source release to the
dist.apache.org [2] and PyPI[8].
-* Validation sheet with a tab for 1.2.3 release to help with validation [9].
-* Docker images published to Docker Hub [10].
+* Go artifacts and documentation are available at pkg.go.dev [9]
+* Validation sheet with a tab for 1.2.3 release to help with validation [10].
+* Docker images published to Docker Hub [11].
The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.
For guidelines on how to try the release in your projects, check out our blog
post at https://beam.apache.org/blog/validate-beam-release/.
Thanks,
@@ -1463,8 +1464,9 @@ Release Manager
[6] https://github.com/apache/beam/pull/...
[7] https://github.com/apache/beam-site/pull/...
[8] https://pypi.org/project/apache-beam/1.2.3rc3/
-[9]
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=...
-[10] https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image
+[9] https://pkg.go.dev/github.com/apache/beam/sdks/v2@v1.2.3-RC3/go/pkg/beam
+[10]
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=...
+[11] https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image
</code></pre>
<p>If there are any issues found in the release candidate, reply on the
vote thread to cancel the vote.
There’s no need to wait 72 hours.
@@ -1792,8 +1794,8 @@ All wheels should be published, in addition to the zip of
the release source.
</code></pre><ul>
<li><strong>Verify that:</strong>
<ul>
-<li>Images are published at <a
href="https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image";>DockerHub</a>
with tags {RELEASE} and <em>latest</em>.</li>
-<li>Images with <em>latest</em> tag are pointing to current release
by confirming the digest of the image with <em>latest</em> tag is the
same as the one with {RELEASE} tag.</li>
+<li>Images are published at <a
href="https://hub.docker.com/search?q=apache%2Fbeam&amp;type=image";>DockerHub</a>
with tags {RELEASE_VERSION} and <em>latest</em>.</li>
+<li>Images with <em>latest</em> tag are pointing to current release
by confirming the digest of the image with <em>latest</em> tag is the
same as the one with {RELEASE_VERSION} tag.</li>
</ul>
</li>
</ul>
@@ -1809,7 +1811,7 @@ All wheels should be published, in addition to the zip of
the release source.
<p>Create and push a new signed tag for the released version by copying the
tag for the final release candidate, as follows:</p>
<pre><code># Optional: unlock the signing key by signing an arbitrary
file.
gpg --output ~/doc.sig --sign ~/.bashrc
-VERSION_TAG=&quot;v${RELEASE}&quot;
+VERSION_TAG=&quot;v${RELEASE_VERSION}&quot;
# Tag for Go SDK
git tag -s &quot;sdks/$VERSION_TAG&quot; &quot;$RC_TAG&quot;
git push https://github.com/apache/beam &quot;sdks/$VERSION_TAG&quot;
diff --git a/website/generated-content/contribute/release-guide/index.html
b/website/generated-content/contribute/release-guide/index.html
index 4fd6338478a..06acf40985a 100644
--- a/website/generated-content/contribute/release-guide/index.html
+++ b/website/generated-content/contribute/release-guide/index.html
@@ -143,11 +143,11 @@ The final state of the repository should match this
diagram:</p><p><img src=/ima
adjust the version, and add the tag locally. If it looks good, run it again
with <code>--push-tag</code>.
If you already have a clone that includes the <code>${COMMIT_REF}</code> then
you can omit <code>--clone</code>. This
is perfectly safe since the script does not depend on the current working
tree.</p><p>See the source of the script for more details, or to run commands
manually in case of a problem.</p><h3
id=run-build_release_candidatesh-to-create-a-release-candidate>Run
build_release_candidate.sh to create a release
candidate</h3><ul><li><p><strong>Script:</strong> <a
href=https://github.com/apache/beam/blob/master/release/src/main/scripts/build_release_candidate.sh>build_release_candidate.sh</a></p><
[...]
-</code></pre></li><li><p><strong>The script will:</strong></p><ol><li>Clone
the repo at the selected RC tag.</li><li>Run gradle publish to push java
artifacts into Maven staging repo.</li><li>Stage source release into
dist.apache.org dev <a
href=https://dist.apache.org/repos/dist/dev/beam/>repo</a>.</li><li>Stage, sign
and hash python source distribution and wheels into dist.apache.org dev repo
python dir</li><li>Stage SDK docker images to <a
href="https://hub.docker.com/search?q=apache% [...]
+</code></pre></li><li><p><strong>The script will:</strong></p><ol><li>Clone
the repo at the selected RC tag.</li><li>Run gradle publish to push java
artifacts into Maven staging repo.</li><li>Stage source release into
dist.apache.org dev <a
href=https://dist.apache.org/repos/dist/dev/beam/>repo</a>.</li><li>Stage, sign
and hash python source distribution and wheels into dist.apache.org dev repo
python dir</li><li>Stage SDK docker images to <a
href="https://hub.docker.com/search?q=apache% [...]
Please note that dependencies for the SDKs with different Python versions vary.
-Need to verify all Python images by replacing <code>${ver}</code> with each
supported Python version <code>X.Y</code>.</li></ul><pre><code>docker run --rm
-it --entrypoint=/bin/bash apache/beam_python${ver}_sdk:${RELEASE}_rc{RC_NUM}
+Need to verify all Python images by replacing <code>${ver}</code> with each
supported Python version <code>X.Y</code>.</li></ul><pre><code>docker run --rm
-it --entrypoint=/bin/bash
apache/beam_python${ver}_sdk:${RELEASE_VERSION}_rc{RC_NUM}
ls -al /opt/apache/beam/third_party_licenses/ | wc -l
-</code></pre><ul><li>For Java SDK images, there should be around 200
dependencies.</li></ul><pre><code>docker run --rm -it --entrypoint=/bin/bash
apache/beam_java${ver}_sdk:${RELEASE}_rc{RC_NUM}
+</code></pre><ul><li>For Java SDK images, there should be around 200
dependencies.</li></ul><pre><code>docker run --rm -it --entrypoint=/bin/bash
apache/beam_java${ver}_sdk:${RELEASE_VERSION}_rc{RC_NUM}
ls -al /opt/apache/beam/third_party_licenses/ | wc -l
</code></pre></li></ol></li><li>Publish staging artifacts<ol><li>Log in to the
<a href=https://repository.apache.org/#stagingRepositories>Apache Nexus</a>
website.</li><li>Navigate to Build Promotion -> Staging Repositories (in the
left sidebar).</li><li>Select repository
<code>orgapachebeam-NNNN</code>.</li><li>Click the Close button.</li><li>When
prompted for a description, enter “Apache Beam, version X, release candidate
Y”.</li><li>Review all staged artifacts on <code>https://reposit [...]
They should contain all relevant parts for each module, including
<code>pom.xml</code>, jar, test jar, javadoc, etc.
@@ -235,7 +235,7 @@ For more information on changes in {$RELEASE_VERSION},
check out the [detailed r
According to git shortlog, the following people contributed to the
{$RELEASE_VERSION} release. Thank you to all contributors!
${CONTRIBUTORS}
-</code></pre><h4 id=checklist-to-proceed-to-the-next-step-1>Checklist to
proceed to the next step</h4><ol><li>Maven artifacts deployed to the staging
repository of <a
href=https://repository.apache.org/content/repositories/>repository.apache.org</a></li><li>Source
distribution deployed to the dev repository of <a
href=https://dist.apache.org/repos/dist/dev/beam/>dist.apache.org</a></li><li>Website
pull request proposed to list the <a
href=https://beam.apache.org/get-started/downloads/>re [...]
+</code></pre><h4 id=checklist-to-proceed-to-the-next-step-1>Checklist to
proceed to the next step</h4><ol><li>Maven artifacts deployed to the staging
repository of <a
href=https://repository.apache.org/content/repositories/>repository.apache.org</a></li><li>Source
distribution deployed to the dev repository of <a
href=https://dist.apache.org/repos/dist/dev/beam/>dist.apache.org</a></li><li>Website
pull request proposed to list the <a
href=https://beam.apache.org/get-started/downloads/>re [...]
docker pull apache/beam_python3.7_sdk:2.39.0_rc1
</code></pre><hr><h2 id=10-vote-and-validate-release-candidate>10. Vote and
validate release candidate</h2><p>Once you have built and individually reviewed
the release candidate, please share it for the community-wide review.
Please review foundation-wide <a
href=https://www.apache.org/foundation/voting.html>voting guidelines</a> for
more information.</p><p>Start the review-and-vote thread on the dev@ mailing
list.
@@ -260,8 +260,9 @@ The complete staging area is available for your review,
which includes:
* website pull request listing the release [6], the blog post [6], and
publishing the API reference manual [7].
* Java artifacts were built with Gradle GRADLE_VERSION and OpenJDK/Oracle JDK
JDK_VERSION.
* Python artifacts are deployed along with the source release to the
dist.apache.org [2] and PyPI[8].
-* Validation sheet with a tab for 1.2.3 release to help with validation [9].
-* Docker images published to Docker Hub [10].
+* Go artifacts and documentation are available at pkg.go.dev [9]
+* Validation sheet with a tab for 1.2.3 release to help with validation [10].
+* Docker images published to Docker Hub [11].
The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.
@@ -278,8 +279,9 @@ Release Manager
[6] https://github.com/apache/beam/pull/...
[7] https://github.com/apache/beam-site/pull/...
[8] https://pypi.org/project/apache-beam/1.2.3rc3/
-[9]
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=...
-[10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
+[9] https://pkg.go.dev/github.com/apache/beam/sdks/v2@v1.2.3-RC3/go/pkg/beam
+[10]
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=...
+[11] https://hub.docker.com/search?q=apache%2Fbeam&type=image
</code></pre><p>If there are any issues found in the release candidate, reply
on the vote thread to cancel the vote.
There’s no need to wait 72 hours.
Proceed to the <code>Fix Issues</code> step below and address the problem.
@@ -409,10 +411,10 @@ please follow <a
href=https://help.github.com/articles/creating-a-personal-acces
</code></pre><ul><li>Verify that the files at <a
href=https://pypi.org/project/apache-beam/#files>https://pypi.org/project/apache-beam/#files</a>
are correct.
All wheels should be published, in addition to the zip of the release source.
(Signatures and hashes do <em>not</em> need to be uploaded.)</li></ul><h3
id=deploy-docker-images-to-dockerhub>Deploy docker images to
DockerHub</h3><ul><li><strong>Script:</strong> <a
href=https://github.com/apache/beam/blob/master/release/src/main/scripts/publish_docker_images.sh>publish_docker_images.sh</a></li><li><strong>Usage</strong></li></ul><pre><code>./beam/release/src/main/scripts/publish_docker_images.sh
-</code></pre><ul><li><strong>Verify that:</strong><ul><li>Images are published
at <a
href="https://hub.docker.com/search?q=apache%2Fbeam&type=image";>DockerHub</a>
with tags {RELEASE} and <em>latest</em>.</li><li>Images with <em>latest</em>
tag are pointing to current release by confirming the digest of the image with
<em>latest</em> tag is the same as the one with {RELEASE}
tag.</li></ul></li></ul><p>(Optional) Clean up any unneeded local images
afterward to save disk space.</p><h3 id=me [...]
+</code></pre><ul><li><strong>Verify that:</strong><ul><li>Images are published
at <a
href="https://hub.docker.com/search?q=apache%2Fbeam&type=image";>DockerHub</a>
with tags {RELEASE_VERSION} and <em>latest</em>.</li><li>Images with
<em>latest</em> tag are pointing to current release by confirming the digest of
the image with <em>latest</em> tag is the same as the one with
{RELEASE_VERSION} tag.</li></ul></li></ul><p>(Optional) Clean up any unneeded
local images afterward to save disk spa [...]
gpg --output ~/doc.sig --sign ~/.bashrc
-VERSION_TAG="v${RELEASE}"
+VERSION_TAG="v${RELEASE_VERSION}"
# Tag for Go SDK
git tag -s "sdks/$VERSION_TAG" "$RC_TAG"
@@ -429,7 +431,7 @@ This should happen automatically: <a
href=https://lists.apache.org/thread.html/3
Ask other contributors to do the same.</p><p>Also, update <a
href=https://en.wikipedia.org/wiki/Apache_Beam>the Wikipedia article on Apache
Beam</a>.</p><h3 id=checklist-to-declare-the-process-completed>Checklist to
declare the process completed</h3><ol><li>Release announced on the user@
mailing list.</li><li>Blog post published, if applicable.</li><li>Release
recorded in reporter.apache.org.</li><li>Release announced on social
media.</li><li>Completion declared on the dev@ mailing list. [...]
Once you’ve finished the release, please take a step back and look what areas
of this process and be improved. Perhaps some part of the process can be
simplified.
Perhaps parts of this guide can be clarified.</p><p>If we have specific ideas,
please start a discussion on the dev@ mailing list and/or propose a pull
request to update this guide.
-Thanks!</p><div class=feedback><p class=update>Last updated on
2022/10/24</p><h3>Have you found everything you were looking for?</h3><p
class=description>Was it all useful and clear? Is there anything that you would
like to change? Let us know!</p><button class=load-button><a
href="mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND
FEEDBACK</a></button></div></div></div><footer class=footer><div
class=footer__contained><div class=footer__cols><div class="footer__cols__col
foo [...]
+Thanks!</p><div class=feedback><p class=update>Last updated on
2022/10/26</p><h3>Have you found everything you were looking for?</h3><p
class=description>Was it all useful and clear? Is there anything that you would
like to change? Let us know!</p><button class=load-button><a
href="mailto:d...@beam.apache.org?subject=Beam Website Feedback">SEND
FEEDBACK</a></button></div></div></div><footer class=footer><div
class=footer__contained><div class=footer__cols><div class="footer__cols__col
foo [...]
<a href=https://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam
logo, and the Apache feather logo are either registered trademarks or
trademarks of The Apache Software Foundation. All other products or name brands
are trademarks of their respective holders, including The Apache Software
Foundation.</div></div><div class="footer__cols__col
footer__cols__col__logos"><div class=footer__cols__col--group><div
class=footer__cols__col__logo><a href=https://github.com/apache/beam><im [...]
\ No newline at end of file
diff --git
a/website/generated-content/documentation/sdks/python-machine-learning/index.html
b/website/generated-content/documentation/sdks/python-machine-learning/index.html
index 8000f10c54d..a7c1aa141ce 100644
---
a/website/generated-content/documentation/sdks/python-machine-learning/index.html
+++
b/website/generated-content/documentation/sdks/python-machine-learning/index.html
@@ -19,7 +19,7 @@
function addPlaceholder(){$('input:text').attr('placeholder',"What are you
looking for?");}
function endSearch(){var
search=document.querySelector(".searchBar");search.classList.add("disappear");var
icons=document.querySelector("#iconsBar");icons.classList.remove("disappear");}
function blockScroll(){$("body").toggleClass("fixedPosition");}
-function openMenu(){addPlaceholder();blockScroll();}</script><div
class="clearfix container-main-content"><div class="section-nav closed"
data-offset-top=90 data-offset-bottom=500><span class="section-nav-back
glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list
data-section-nav><li><span
class=section-nav-list-main-title>Languages</span></li><li><span
class=section-nav-list-title>Java</span><ul class=section-nav-list><li><a
href=/documentation/sdks/java/>Java SDK overvi [...]
+function openMenu(){addPlaceholder();blockScroll();}</script><div
class="clearfix container-main-content"><div class="section-nav closed"
data-offset-top=90 data-offset-bottom=500><span class="section-nav-back
glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list
data-section-nav><li><span
class=section-nav-list-main-title>Languages</span></li><li><span
class=section-nav-list-title>Java</span><ul class=section-nav-list><li><a
href=/documentation/sdks/java/>Java SDK overvi [...]
Pydoc</a></td></table><p><br><br><br></p><p>You can use Apache Beam with the
RunInference API to use machine learning (ML) models to do local and remote
inference with batch and streaming pipelines. Starting with Apache Beam 2.40.0,
PyTorch and Scikit-learn frameworks are supported. Tensorflow models are
supported through tfx-bsl.</p><p>You can create multiple types of transforms
using the RunInference API: the API takes multiple types of setup parameters
from model handlers, and the par [...]
<a
href=https://github.com/apache/beam/blob/master/sdks/python/apache_beam/utils/shared.py#L20><code>Shared</code>
class documentation</a>.</p><h3 id=multi-model-pipelines>Multi-model
pipelines</h3><p>The RunInference API can be composed into multi-model
pipelines. Multi-model pipelines can be useful for A/B testing or for building
out cascade models made up of models that perform tokenization, sentence
segmentation, part-of-speech tagging, named entity extraction, language
detection, co [...]
with pipeline as p:
@@ -30,35 +30,12 @@ from apache_beam.ml.inference.sklearn_inference import
SklearnModelHandlerPandas
from apache_beam.ml.inference.pytorch_inference import
PytorchModelHandlerTensor
from apache_beam.ml.inference.pytorch_inference import
PytorchModelHandlerKeyedTensor
from tfx_bsl.public.beam.run_inference import CreateModelHandler
-</code></pre><h3 id=use-pre-trained-models>Use pre-trained models</h3><p>The
section provides requirements for using pre-trained models with PyTorch and
Scikit-learn</p><h4 id=pytorch>PyTorch</h4><p>You need to provide a path to a
file that contains the model’s saved weights. This path must be
accessible by the pipeline. To use pre-trained models with the RunInference API
and the PyTorch framework, complete the following steps:</p><ol><li>Download
the pre-trained weights and host t [...]
+</code></pre><h3 id=use-pre-trained-models>Use pre-trained models</h3><p>The
section provides requirements for using pre-trained models with PyTorch and
Scikit-learn</p><h4 id=pytorch>PyTorch</h4><p>You need to provide a path to a
file that contains the model’s saved weights. This path must be
accessible by the pipeline. To use pre-trained models with the RunInference API
and the PyTorch framework, complete the following steps:</p><ol><li>Download
the pre-trained weights and host t [...]
+that illustrates running PyTorch models with Apache Beam.</p><h4
id=scikit-learn>Scikit-learn</h4><p>You need to provide a path to a file that
contains the pickled Scikit-learn model. This path must be accessible by the
pipeline. To use pre-trained models with the RunInference API and the
Scikit-learn framework, complete the following steps:</p><ol><li>Download the
pickled model class and host it in a location that the pipeline can
access.</li><li>Pass the path of the model to the Sklear [...]
<code>model_uri=<path_to_pickled_file></code> and <code>model_file_type:
<ModelFileType></code>, where you can specify
-<code>ModelFileType.PICKLE</code> or <code>ModelFileType.JOBLIB</code>,
depending on how the model was serialized.</li></ol><h4
id=tensorflow>TensorFlow</h4><p>To use TensorFlow with the RunInference API,
you need to do the following:</p><ul><li>Use <code>tfx_bsl</code> version
1.10.0 or later.</li><li>Create a model handler using
<code>tfx_bsl.public.beam.run_inference.CreateModelHandler()</code>.</li><li>Use
the model handler with the <a href=/releases/pydoc/current/apache_beam.ml.infe
[...]
-from apache_beam.ml.inference.base import RunInference
-from tensorflow_serving.apis import prediction_log_pb2
-from tfx_bsl.public.proto import model_spec_pb2
-from tfx_bsl.public.tfxio import TFExampleRecord
-from tfx_bsl.public.beam.run_inference import CreateModelHandler
-
-pipeline = beam.Pipeline()
-tfexample_beam_record = TFExampleRecord(file_pattern='/path/to/examples')
-saved_model_spec = model_spec_pb2.SavedModelSpec(model_path='/path/to/model')
-inference_spec_type =
model_spec_pb2.InferenceSpecType(saved_model_spec=saved_model_spec)
-model_handler = CreateModelHandler(inference_spec_type)
-with pipeline as p:
- _ = (p | tfexample_beam_record.RawRecordBeamSource()
- | RunInference(model_handler)
- | beam.Map(print)
- )
-</code></pre><p>Note: A model handler that is created with
<code>CreateModelHander()</code> is always unkeyed.</p><h3
id=keyed-model-handlers>Keyed Model Handlers</h3><p>To make a keyed model
handler, wrap any unkeyed model handler in the keyed model handler. For
example:</p><pre><code>from apache_beam.ml.inference.base import RunInference
-from apache_beam.ml.inference.base import KeyedModelHandler
-model_handler = <Instantiate your model handler>
-keyed_model_handler = KeyedModelHandler(model_handler)
-
-with pipeline as p:
- p | ( <Your Pipeline>
- RunInference(keyed_model_handler)
- )
-</code></pre><p>If you are unsure if your data is keyed, you can also use
<code>MaybeKeyedModelHandler</code>.</p><p>For more information, see <a
href=https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.KeyedModelHandler><code>KeyedModelHander</code></a>.</p><h3
id=use-custom-models>Use custom models</h3><p>If you would like to use a model
that isn’t specified by one of the supported frameworks, the RunInference
API is des [...]
+<code>ModelFileType.PICKLE</code> or <code>ModelFileType.JOBLIB</code>,
depending on how the model was serialized.</li></ol><p>See <a
href=https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_sklearn.ipynb>this
notebook</a>
+that illustrates running Scikit-learn models with Apache Beam.</p><h4
id=tensorflow>TensorFlow</h4><p>To use TensorFlow with the RunInference API,
you need to do the following:</p><ul><li>Use <code>tfx_bsl</code> version
1.10.0 or later.</li><li>Create a model handler using
<code>tfx_bsl.public.beam.run_inference.CreateModelHandler()</code>.</li><li>Use
the model handler with the <a
href=/releases/pydoc/current/apache_beam.ml.inference.base.html><code>apache_beam.ml.inference.base.RunInf
[...]
+that illustrates running TensorFlow models with Apache Beam and
tfx-bsl.</p><h3 id=use-custom-models>Use custom models</h3><p>If you would like
to use a model that isn’t specified by one of the supported frameworks,
the RunInference API is designed flexibly to allow you to use any custom
machine learning models.
You only need to create your own <code>ModelHandler</code> or
<code>KeyedModelHandler</code> with logic to load your model and use it to run
the inference.</p><p>A simple example can be found in <a
href=https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_custom_inference.ipynb>this
notebook</a>.
The <code>load_model</code> method shows how to load the model using a popular
<code>spaCy</code> package while <code>run_inference</code> shows how to run
the inference on a batch of examples.</p><h3 id=use-multiple-models>Use
multiple models</h3><p>You can also use the RunInference transform to add
multiple inference models to your pipeline.</p><h4 id=ab-pattern>A/B
Pattern</h4><pre><code>with pipeline as p:
data = p | 'Read' >> beam.ReadFromSource('a_source')
@@ -88,7 +65,7 @@ with pipeline as p:
('img3', torch.tensor([[1,2,3],[4,5,6],...])),
])
predictions = data | RunInference(keyed_model_handler)
-</code></pre><h3 id=use-the-predictionresults-object>Use the PredictionResults
object</h3><p>When doing a prediction in Apache Beam, the output
<code>PCollection</code> includes both the keys of the input examples and the
inferences. Including both these items in the output allows you to find the
input that determined the predictions.</p><p>The <code>PredictionResult</code>
is a <code>NamedTuple</code> object that contains both the input and the
inferences, named <code>example</code> and [...]
+</code></pre><p>If you are unsure if your data is keyed, you can also use
<code>MaybeKeyedModelHandler</code>.</p><p>For more information, see <a
href=https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.KeyedModelHandler><code>KeyedModelHander</code></a>.</p><h3
id=use-the-predictionresults-object>Use the PredictionResults
object</h3><p>When doing a prediction in Apache Beam, the output
<code>PCollection</code> includes both the [...]
def process(self, element: Tuple[str, PredictionResult]):
key, prediction_result = element
inputs = prediction_result.example
diff --git a/website/generated-content/get-started/downloads/index.html
b/website/generated-content/get-started/downloads/index.html
index 80950b44743..8ee09dd6c6a 100644
--- a/website/generated-content/get-started/downloads/index.html
+++ b/website/generated-content/get-started/downloads/index.html
@@ -38,7 +38,9 @@ with the <code>DirectRunner</code>, add the following
dependencies to your <code
with <code>DirectRunner</code>, add the following requirement to your
<code>setup.py</code> file:</p><pre><code>apache-beam==2.42.0
</code></pre><p>Additionally, you may want to depend on additional SDK
modules, such as IO
connectors or other extensions, and additional runners to execute your pipeline
-at scale.</p><h2 id=downloading-source-code>Downloading source code</h2><p>You
can download the source code package for a release from the links in the
+at scale.</p><p>The Go SDK is accessible via Go Modules and calling <code>go
get</code> from a module subdirectory:</p><pre><code> go get
github.com/apache/beam/sdks/v2/go/pkg/beam
+</code></pre><p>Specific versions can be depended on similarly:</p><pre><code>
go get github.com/apache/beam/sdks/v2@v2.42.0/go/pkg/beam
+</code></pre><h2 id=downloading-source-code>Downloading source code</h2><p>You
can download the source code package for a release from the links in the
<a href=#releases>Releases</a> section.</p><h3 id=release-integrity>Release
integrity</h3><p>You <em>must</em> <a
href=https://www.apache.org/info/verification.html>verify</a> the integrity
of downloaded files. We provide OpenPGP signatures for every release file. This
signature should be matched against the
@@ -180,7 +182,7 @@ versions denoted <code>0.x.y</code>.</p><h2
id=releases>Releases</h2><h3 id=2420
<a
href=https://archive.apache.org/dist/beam/2.1.0/apache-beam-2.1.0-source-release.zip.asc>signature</a>.</p><p><a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12340528";>Release
notes</a>.</p><h3 id=200-2017-05-17>2.0.0 (2017-05-17)</h3><p>Official <a
href=https://archive.apache.org/dist/beam/2.0.0/apache-beam-2.0.0-source-release.zip>source
code download</a>.
<a
href=https://archive.apache.org/dist/beam/2.0.0/apache-beam-2.0.0-source-release.zip.sha1>SHA-1</a>.
<a
href=https://archive.apache.org/dist/beam/2.0.0/apache-beam-2.0.0-source-release.zip.md5>MD5</a>.
-<a
href=https://archive.apache.org/dist/beam/2.0.0/apache-beam-2.0.0-source-release.zip.asc>signature</a>.</p><p><a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12339746";>Release
notes</a>.</p><div class=feedback><p class=update>Last updated on
2022/10/17</p><h3>Have you found everything you were looking for?</h3><p
class=description>Was it all useful and clear? Is there anything that you would
like to change? Let us know!</p><button class=load- [...]
+<a
href=https://archive.apache.org/dist/beam/2.0.0/apache-beam-2.0.0-source-release.zip.asc>signature</a>.</p><p><a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12339746";>Release
notes</a>.</p><div class=feedback><p class=update>Last updated on
2022/10/26</p><h3>Have you found everything you were looking for?</h3><p
class=description>Was it all useful and clear? Is there anything that you would
like to change? Let us know!</p><button class=load- [...]
<a href=https://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam
logo, and the Apache feather logo are either registered trademarks or
trademarks of The Apache Software Foundation. All other products or name brands
are trademarks of their respective holders, including The Apache Software
Foundation.</div></div><div class="footer__cols__col
footer__cols__col__logos"><div class=footer__cols__col--group><div
class=footer__cols__col__logo><a href=https://github.com/apache/beam><im [...]
\ No newline at end of file
diff --git a/website/generated-content/get-started/index.xml
b/website/generated-content/get-started/index.xml
index d4f7b99e23f..9c6585f4ee4 100644
--- a/website/generated-content/get-started/index.xml
+++ b/website/generated-content/get-started/index.xml
@@ -1576,6 +1576,12 @@ with <code>DirectRunner</code>, add the following
requirement to your <
<p>Additionally, you may want to depend on additional SDK modules, such as
IO
connectors or other extensions, and additional runners to execute your pipeline
at scale.</p>
+<p>The Go SDK is accessible via Go Modules and calling <code>go
get</code> from a module subdirectory:</p>
+<pre><code> go get github.com/apache/beam/sdks/v2/go/pkg/beam
+</code></pre>
+<p>Specific versions can be depended on similarly:</p>
+<pre><code> go get github.com/apache/beam/sdks/v2@v2.42.0/go/pkg/beam
+</code></pre>
<h2 id="downloading-source-code">Downloading source code</h2>
<p>You can download the source code package for a release from the links in
the
<a href="#releases">Releases</a> section.</p>
diff --git a/website/generated-content/sitemap.xml
b/website/generated-content/sitemap.xml
index a73b53aa8f2..1ff12253687 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9";
xmlns:xhtml="http://www.w3.org/1999/xhtml";><url><loc>/blog/beam-2.42.0/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/catego
[...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9";
xmlns:xhtml="http://www.w3.org/1999/xhtml";><url><loc>/blog/beam-2.42.0/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/blog/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/categories/</loc><lastmod>2022-10-17T09:50:38-07:00</lastmod></url><url><loc>/catego
[...]
\ No newline at end of file
