This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 6763dbb Publishing website 2021/04/28 18:03:38 at commit 1fe677b
6763dbb is described below
commit 6763dbb3ff3b58df3150d99014235575c72d954f
Author: jenkins <bui...@apache.org>
AuthorDate: Wed Apr 28 18:03:38 2021 +0000
Publishing website 2021/04/28 18:03:38 at commit 1fe677b
---
website/generated-content/roadmap/index.xml | 81 ++++------------------
.../roadmap/portability/index.html | 68 +++++-------------
website/generated-content/sitemap.xml | 2 +-
3 files changed, 31 insertions(+), 120 deletions(-)
diff --git a/website/generated-content/roadmap/index.xml
b/website/generated-content/roadmap/index.xml
index 9250c48..50f2c0a 100644
--- a/website/generated-content/roadmap/index.xml
+++ b/website/generated-content/roadmap/index.xml
@@ -355,11 +355,11 @@ limitations under the License.
<h1 id="portability-framework-roadmap">Portability Framework Roadmap</h1>
<h2 id="overview">Overview</h2>
<p>Interoperability between SDKs and runners is a key aspect of Apache
-Beam. So far, however, the reality is that most runners support the
-Java SDK only, because each SDK-runner combination requires non-trivial
-work on both sides. All runners are also currently written in Java,
+Beam. Previously, the reality was that most runners supported the
+Java SDK only, because each SDK-runner combination required non-trivial
+work on both sides. Most runners are also currently written in Java,
which makes support of non-Java SDKs far more expensive. The
-<em>portability framework</em> aims to rectify this situation and provide
+<em>portability framework</em> rectified this situation and provided
full interoperability across the Beam ecosystem.</p>
<p>The portability framework introduces well-defined, language-neutral
data structures and protocols between the SDK and runner. This interop
@@ -410,60 +410,16 @@ the portability API, perhaps exclusively.</p>
<p>If you are interested in digging in to the designs, you can find
them on the <a
href="https://cwiki.apache.org/confluence/display/BEAM/Design+Documents";>Beam
developers&rsquo; wiki</a>.
Another overview can be found <a
href="https://docs.google.com/presentation/d/1Yg8Xm4fb-oRjiLQjwLt5153hpwwTLclZrVOKP2hQifo/edit#slide=id.g42e4c9aad6_1_3070";>here</a>.</p>
-<h2 id="milestones">Milestones</h2>
-<p>The portability framework is a substantial effort that touches every
-Beam component. In addition to the sheer magnitude, a major challenge
-is engineering an interop layer that does not significantly compromise
-performance due to the additional serialization overhead of a
-language-neutral protocol.</p>
-<p>The proposed project phases are roughly as follows and are not
-strictly sequential, as various components will likely move at
-different speeds. Additionally, there have been (and continues to be)
-supporting refactorings that are not always tracked as part of the
-portability effort. Work already done is not tracked here either.</p>
-<ul>
-<li>
-<p><strong>P1 [MVP]</strong>: Implement the fundamental plumbing for
portable SDKs
-and runners for batch and streaming, including containers and the
-ULR
-[<a
href="https://issues.apache.org/jira/browse/BEAM-2899";>BEAM-2899</a>]. Each
-SDK and runner should use the portability framework at least to the
-extent that wordcount
-[<a
href="https://issues.apache.org/jira/browse/BEAM-2896";>BEAM-2896</a>] and
-windowed wordcount
-[<a
href="https://issues.apache.org/jira/browse/BEAM-2941";>BEAM-2941</a>] run
-portably.</p>
-</li>
-<li>
-<p><strong>P2 [Feature complete]</strong>: Design and implement
portability support
-for remaining execution-side features, so that any pipeline from
-any SDK can run portably on any runner. These features include side
-inputs
-[<a
href="https://issues.apache.org/jira/browse/BEAM-2863";>BEAM-2863</a>], User
state [<a
href="https://issues.apache.org/jira/browse/BEAM-2862";>BEAM-2862</a>], User
-timers
-[<a
href="https://issues.apache.org/jira/browse/BEAM-2925";>BEAM-2925</a>],
-Splittable DoFn
-[<a
href="https://issues.apache.org/jira/browse/BEAM-2896";>BEAM-2896</a>] and
-more. Each SDK and runner should use the portability framework at
-least to the extent that the mobile gaming examples
-[<a
href="https://issues.apache.org/jira/browse/BEAM-2940";>BEAM-2940</a>] run
-portably.</p>
-</li>
-<li>
-<p><strong>P3 [Performance]</strong>: Measure and tune performance of
portable
-pipelines using benchmarks such as Nexmark. Features such as
-progress reporting
-[<a
href="https://issues.apache.org/jira/browse/BEAM-2940";>BEAM-2940</a>],
-combiner lifting
-[<a
href="https://issues.apache.org/jira/browse/BEAM-2937";>BEAM-2937</a>] and
-fusion are expected to be needed.</p>
-</li>
-<li>
-<p><strong>P4 [Cross language]</strong>: Design and implement
cross-language
-pipeline support, including how the ecosystem of shared transforms
-should work.</p>
-</li>
-</ul>
+<h2 id="status">Status</h2>
+<p>All SDKs currently support the portability framework.
+There is also a Python Universal Local Runner and shared java runners library.
+Performance is good and multi-language pipelines are supported.
+Currently, the Flink and Spark runners support portable pipeline execution
+(which is used by default for SDKs other than Java),
+as does Dataflow when using the <a
href="https://cloud.google.com/dataflow/docs/guides/deploying-a-pipeline#dataflow-runner-v2";>Dataflow
Runner v2</a>.
+See the
+<a
href="https://s.apache.org/apache-beam-portability-support-table";>Portability
support table</a>
+for details.</p>
<h2 id="issues">Issues</h2>
<p>The portability effort touches every component, so the
&ldquo;portability&rdquo;
label is used to identify all portability-related issues. Pure
@@ -472,15 +428,6 @@ common pattern for new portability features is that the
overall
feature is in &ldquo;beam-model&rdquo; with subtasks for each SDK and
runner in
their respective components.</p>
<p><strong>JIRA:</strong> <a
href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20portability%20order%20by%20priority%20DESC%2Cupdated%20DESC";>query</a></p>
-<h2 id="status">Status</h2>
-<p>MVP, and FeatureCompletness nearly done for
-SDKs, Python ULR, and shared java runners library.
-Performance is good and multi-language pipelines are supported.
-Currently, the Flink and Spark runners support portable pipeline execution,
-as does Dataflow when using the <a
href="https://cloud.google.com/dataflow/docs/guides/deploying-a-pipeline#dataflow-runner-v2";>Dataflow
Runner v2</a>.
-See the
-<a
href="https://s.apache.org/apache-beam-portability-support-table";>Portability
support table</a>
-for details.</p>
<p>Prerequisites: <a
href="https://docs.docker.com/compose/install/";>Docker</a>, <a
href="https://docs.python-guide.org/starting/install3/linux/";>Python</a>,
<a href="https://openjdk.java.net/install/";>Java 8</a></p>
<h3 id="python-on-flink">Running Python wordcount on Flink</h3>
<p>The Beam Flink runner can run Python pipelines in batch and streaming
modes.
diff --git a/website/generated-content/roadmap/portability/index.html
b/website/generated-content/roadmap/portability/index.html
index 22dbf05..af310e9 100644
--- a/website/generated-content/roadmap/portability/index.html
+++ b/website/generated-content/roadmap/portability/index.html
@@ -18,12 +18,12 @@
function addPlaceholder(){$('input:text').attr('placeholder',"What are you
looking for?");}
function endSearch(){var
search=document.querySelector(".searchBar");search.classList.add("disappear");var
icons=document.querySelector("#iconsBar");icons.classList.remove("disappear");}
function blockScroll(){$("body").toggleClass("fixedPosition");}
-function openMenu(){addPlaceholder();blockScroll();}</script><div
class="clearfix container-main-content"><div class="section-nav closed"
data-offset-top=90 data-offset-bottom=500><span class="section-nav-back
glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list
data-section-nav><li><span
class=section-nav-list-main-title>Roadmap</span></li><li><a
href=/roadmap/>Roadmap Highlights</a></li><li><a
href=/roadmap/portability>Portability Framework</a></li><li class=section-na
[...]
-Beam. So far, however, the reality is that most runners support the
-Java SDK only, because each SDK-runner combination requires non-trivial
-work on both sides. All runners are also currently written in Java,
+function openMenu(){addPlaceholder();blockScroll();}</script><div
class="clearfix container-main-content"><div class="section-nav closed"
data-offset-top=90 data-offset-bottom=500><span class="section-nav-back
glyphicon glyphicon-menu-left"></span><nav><ul class=section-nav-list
data-section-nav><li><span
class=section-nav-list-main-title>Roadmap</span></li><li><a
href=/roadmap/>Roadmap Highlights</a></li><li><a
href=/roadmap/portability>Portability Framework</a></li><li class=section-na
[...]
+Beam. Previously, the reality was that most runners supported the
+Java SDK only, because each SDK-runner combination required non-trivial
+work on both sides. Most runners are also currently written in Java,
which makes support of non-Java SDKs far more expensive. The
-<em>portability framework</em> aims to rectify this situation and provide
+<em>portability framework</em> rectified this situation and provided
full interoperability across the Beam ecosystem.</p><p>The portability
framework introduces well-defined, language-neutral
data structures and protocols between the SDK and runner. This interop
layer – called the <em>portability API</em> – ensures that SDKs
and runners
@@ -61,62 +61,26 @@ dependency conflicts. The runner has significant freedom
regarding
how it manages the SDK harness containers.</p></li></ul><p>The goal is that
all (non-direct) runners and SDKs eventually support
the portability API, perhaps exclusively.</p><p>If you are interested in
digging in to the designs, you can find
them on the <a
href=https://cwiki.apache.org/confluence/display/BEAM/Design+Documents>Beam
developers’ wiki</a>.
-Another overview can be found <a
href="https://docs.google.com/presentation/d/1Yg8Xm4fb-oRjiLQjwLt5153hpwwTLclZrVOKP2hQifo/edit#slide=id.g42e4c9aad6_1_3070";>here</a>.</p><h2
id=milestones>Milestones</h2><p>The portability framework is a substantial
effort that touches every
-Beam component. In addition to the sheer magnitude, a major challenge
-is engineering an interop layer that does not significantly compromise
-performance due to the additional serialization overhead of a
-language-neutral protocol.</p><p>The proposed project phases are roughly as
follows and are not
-strictly sequential, as various components will likely move at
-different speeds. Additionally, there have been (and continues to be)
-supporting refactorings that are not always tracked as part of the
-portability effort. Work already done is not tracked here
either.</p><ul><li><p><strong>P1 [MVP]</strong>: Implement the fundamental
plumbing for portable SDKs
-and runners for batch and streaming, including containers and the
-ULR
-[<a href=https://issues.apache.org/jira/browse/BEAM-2899>BEAM-2899</a>]. Each
-SDK and runner should use the portability framework at least to the
-extent that wordcount
-[<a href=https://issues.apache.org/jira/browse/BEAM-2896>BEAM-2896</a>] and
-windowed wordcount
-[<a href=https://issues.apache.org/jira/browse/BEAM-2941>BEAM-2941</a>] run
-portably.</p></li><li><p><strong>P2 [Feature complete]</strong>: Design and
implement portability support
-for remaining execution-side features, so that any pipeline from
-any SDK can run portably on any runner. These features include side
-inputs
-[<a href=https://issues.apache.org/jira/browse/BEAM-2863>BEAM-2863</a>], User
state [<a href=https://issues.apache.org/jira/browse/BEAM-2862>BEAM-2862</a>],
User
-timers
-[<a href=https://issues.apache.org/jira/browse/BEAM-2925>BEAM-2925</a>],
-Splittable DoFn
-[<a href=https://issues.apache.org/jira/browse/BEAM-2896>BEAM-2896</a>] and
-more. Each SDK and runner should use the portability framework at
-least to the extent that the mobile gaming examples
-[<a href=https://issues.apache.org/jira/browse/BEAM-2940>BEAM-2940</a>] run
-portably.</p></li><li><p><strong>P3 [Performance]</strong>: Measure and tune
performance of portable
-pipelines using benchmarks such as Nexmark. Features such as
-progress reporting
-[<a href=https://issues.apache.org/jira/browse/BEAM-2940>BEAM-2940</a>],
-combiner lifting
-[<a href=https://issues.apache.org/jira/browse/BEAM-2937>BEAM-2937</a>] and
-fusion are expected to be needed.</p></li><li><p><strong>P4 [Cross
language]</strong>: Design and implement cross-language
-pipeline support, including how the ecosystem of shared transforms
-should work.</p></li></ul><h2 id=issues>Issues</h2><p>The portability effort
touches every component, so the “portability”
-label is used to identify all portability-related issues. Pure
-design or proto definitions should use the “beam-model” component.
A
-common pattern for new portability features is that the overall
-feature is in “beam-model” with subtasks for each SDK and runner in
-their respective components.</p><p><strong>JIRA:</strong> <a
href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20portability%20order%20by%20priority%20DESC%2Cupdated%20DESC";>query</a></p><h2
id=status>Status</h2><p>MVP, and FeatureCompletness nearly done for
-SDKs, Python ULR, and shared java runners library.
+Another overview can be found <a
href="https://docs.google.com/presentation/d/1Yg8Xm4fb-oRjiLQjwLt5153hpwwTLclZrVOKP2hQifo/edit#slide=id.g42e4c9aad6_1_3070";>here</a>.</p><h2
id=status>Status</h2><p>All SDKs currently support the portability framework.
+There is also a Python Universal Local Runner and shared java runners library.
Performance is good and multi-language pipelines are supported.
-Currently, the Flink and Spark runners support portable pipeline execution,
+Currently, the Flink and Spark runners support portable pipeline execution
+(which is used by default for SDKs other than Java),
as does Dataflow when using the <a
href=https://cloud.google.com/dataflow/docs/guides/deploying-a-pipeline#dataflow-runner-v2>Dataflow
Runner v2</a>.
See the
<a href=https://s.apache.org/apache-beam-portability-support-table>Portability
support table</a>
-for details.</p><p>Prerequisites: <a
href=https://docs.docker.com/compose/install/>Docker</a>, <a
href=https://docs.python-guide.org/starting/install3/linux/>Python</a>, <a
href=https://openjdk.java.net/install/>Java 8</a></p><h3
id=python-on-flink>Running Python wordcount on Flink</h3><p>The Beam Flink
runner can run Python pipelines in batch and streaming modes.
+for details.</p><h2 id=issues>Issues</h2><p>The portability effort touches
every component, so the “portability”
+label is used to identify all portability-related issues. Pure
+design or proto definitions should use the “beam-model” component.
A
+common pattern for new portability features is that the overall
+feature is in “beam-model” with subtasks for each SDK and runner in
+their respective components.</p><p><strong>JIRA:</strong> <a
href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20resolution%20%3D%20Unresolved%20AND%20labels%20%3D%20portability%20order%20by%20priority%20DESC%2Cupdated%20DESC";>query</a></p><p>Prerequisites:
<a href=https://docs.docker.com/compose/install/>Docker</a>, <a
href=https://docs.python-guide.org/starting/install3/linux/>Python</a>, <a
href=https://openjdk.java.net/install/>Java 8</a></p><h3 id=python-on [...]
Please see the <a href=/documentation/runners/flink/>Flink Runner page</a> for
more information on
how to run portable pipelines on top of Flink.</p><h3
id=python-on-spark>Running Python wordcount on Spark</h3><p>The Beam Spark
runner can run Python pipelines in batch mode.
Please see the <a href=/documentation/runners/spark/>Spark Runner page</a> for
more information on
how to run portable pipelines on top of Spark.</p><p>Python streaming mode is
not yet supported on Spark.</p><h2 id=sdk-harness-config>SDK Harness
Configuration</h2><p>See <a
href=/documentation/runtime/sdk-harness-config/>here</a> for more information
on SDK harness deployment options
and <a
href="https://docs.google.com/presentation/d/1Cso0XP9dmj77OD9Bd53C1M3W1sPJF0ZnA20gzb2BPhE/edit?usp=sharing";>here</a>
-for what goes into writing a portable SDK.</p><div class=feedback><p
class=update>Last updated on 2020/07/21</p><h3>Have you found everything you
were looking for?</h3><p class=description>Was it all useful and clear? Is
there anything that you would like to change? Let us know!</p><button
class=load-button><a href="mailto:d...@beam.apache.org?subject=Beam Website
Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=footer><div
class=footer__contained><div class=footer__col [...]
+for what goes into writing a portable SDK.</p><div class=feedback><p
class=update>Last updated on 2021/04/27</p><h3>Have you found everything you
were looking for?</h3><p class=description>Was it all useful and clear? Is
there anything that you would like to change? Let us know!</p><button
class=load-button><a href="mailto:d...@beam.apache.org?subject=Beam Website
Feedback">SEND FEEDBACK</a></button></div></div></div><footer class=footer><div
class=footer__contained><div class=footer__col [...]
<a href=http://www.apache.org>The Apache Software Foundation</a>
| <a href=/privacy_policy>Privacy Policy</a>
| <a href=/feed.xml>RSS Feed</a><br><br>Apache Beam, Apache, Beam, the Beam
logo, and the Apache feather logo are either registered trademarks or
trademarks of The Apache Software Foundation. All other products or name brands
are trademarks of their respective holders, including The Apache Software
Foundation.</div></div></div></div></footer></body></html>
\ No newline at end of file
diff --git a/website/generated-content/sitemap.xml
b/website/generated-content/sitemap.xml
index 2085cb1..218d4a0 100644
--- a/website/generated-content/sitemap.xml
+++ b/website/generated-content/sitemap.xml
@@ -1 +1 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9";
xmlns:xhtml="http://www.w3.org/1999/xhtml";><url><loc>/blog/beam-2.28.0/</loc><lastmod>2021-02-22T11:40:20-08:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/blog/k
[...]
\ No newline at end of file
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><urlset
xmlns="http://www.sitemaps.org/schemas/sitemap/0.9";
xmlns:xhtml="http://www.w3.org/1999/xhtml";><url><loc>/blog/beam-2.28.0/</loc><lastmod>2021-02-22T11:40:20-08:00</lastmod></url><url><loc>/categories/blog/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/blog/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/categories/</loc><lastmod>2021-03-31T21:00:41-04:00</lastmod></url><url><loc>/blog/k
[...]
\ No newline at end of file
