This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit 92a3957d24e04a63c3a4dad8b9959d3712018270
Author: Mergebot <merge...@apache.org>
AuthorDate: Thu Jun 21 23:02:35 2018 -0700
Prepare repository for deployment.
---
content/community/contact-us/index.html | 6 +-
content/contribute/become-a-committer/index.html | 1 +
content/contribute/committer-guide/index.html | 1 +
.../{portability => dependencies}/index.html | 230 +++++----------------
content/contribute/design-documents/index.html | 1 +
content/contribute/docker-images/index.html | 1 +
content/contribute/eclipse/index.html | 1 +
content/contribute/index.html | 1 +
content/contribute/intellij/index.html | 1 +
content/contribute/portability/index.html | 1 +
.../contribute/ptransform-style-guide/index.html | 1 +
content/contribute/release-guide/index.html | 1 +
content/contribute/runner-guide/index.html | 1 +
content/contribute/testing/index.html | 1 +
.../contribute/website-contributions/index.html | 1 +
content/get-started/downloads/index.html | 20 +-
16 files changed, 76 insertions(+), 193 deletions(-)
diff --git a/content/community/contact-us/index.html
b/content/community/contact-us/index.html
index 98242a5..60adf84 100644
--- a/content/community/contact-us/index.html
+++ b/content/community/contact-us/index.html
@@ -154,10 +154,8 @@ whichever one seems best!</p>
</tr>
</tbody>
</table>
-
-If you wish to report a security vulnerability, please contact <a
href="mailto:secur...@apache.org";>secur...@apache.org</a>.
-Apache Beam follows the typical <a
href="https://apache.org/security/committers.html#vulnerability-handling";>Apache
vulnerability handling process</a>.
-
+
+
</div>
</div>
<footer class="footer">
diff --git a/content/contribute/become-a-committer/index.html
b/content/contribute/become-a-committer/index.html
index 3d17119..cc9817e 100644
--- a/content/contribute/become-a-committer/index.html
+++ b/content/contribute/become-a-committer/index.html
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
diff --git a/content/contribute/committer-guide/index.html
b/content/contribute/committer-guide/index.html
index 1206e88..10a5c4d 100644
--- a/content/contribute/committer-guide/index.html
+++ b/content/contribute/committer-guide/index.html
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
diff --git a/content/contribute/portability/index.html
b/content/contribute/dependencies/index.html
similarity index 50%
copy from content/contribute/portability/index.html
copy to content/contribute/dependencies/index.html
index 0e20980..20719ae 100644
--- a/content/contribute/portability/index.html
+++ b/content/contribute/dependencies/index.html
@@ -4,7 +4,7 @@
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
- <title>Portability Framework</title>
+ <title>Dependencies Guide</title>
<meta name="description" content="Apache Beam is an open source, unified
model and set of language-specific SDKs for defining and executing data
processing workflows, and also data ingestion and integration flows, supporting
Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs).
Dataflow pipelines simplify the mechanics of large-scale batch and streaming
data processing and can run on a number of runtimes like Apache Flink, Apache
Spark, and Google Cloud Dataflow [...]
">
<link href="https://fonts.googleapis.com/css?family=Roboto:100,300,400";
rel="stylesheet">
@@ -15,7 +15,7 @@
<script src="/js/fix-menu.js"></script>
<script src="/js/section-nav.js"></script>
<script src="/js/page-nav.js"></script>
- <link rel="canonical" href="https://beam.apache.org/contribute/portability/";
data-proofer-ignore>
+ <link rel="canonical"
href="https://beam.apache.org/contribute/dependencies/"; data-proofer-ignore>
<link rel="shortcut icon" type="image/x-icon" href="/images/favicon.ico">
<link rel="alternate" type="application/rss+xml" title="Apache Beam"
href="https://beam.apache.org/feed.xml";>
<script>
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
@@ -128,209 +129,80 @@
<ul class="nav">
- <li><a href="#overview">Overview</a></li>
- <li><a href="#design">Design</a></li>
- <li><a href="#development">Development</a>
- <ul>
- <li><a href="#roadmap">Roadmap</a></li>
- <li><a href="#issues">Issues</a></li>
- <li><a href="#status">Status</a></li>
- </ul>
- </li>
+ <li><a href="#identifying-outdated-dependencies">Identifying outdated
dependencies</a></li>
+ <li><a href="#upgrading-identified-outdated-dependencies">Upgrading
identified outdated dependencies</a></li>
+ <li><a href="#dependency-updates-and-backwards-compatibility">Dependency
updates and backwards compatibility</a></li>
</ul>
</nav>
<div class="body__contained body__section-nav">
- <h1 id="portability-framework">Portability Framework</h1>
-
-<ul id="markdown-toc">
- <li><a href="#overview" id="markdown-toc-overview">Overview</a></li>
- <li><a href="#design" id="markdown-toc-design">Design</a></li>
- <li><a href="#development" id="markdown-toc-development">Development</a>
<ul>
- <li><a href="#roadmap" id="markdown-toc-roadmap">Roadmap</a></li>
- <li><a href="#issues" id="markdown-toc-issues">Issues</a></li>
- <li><a href="#status" id="markdown-toc-status">Status</a></li>
- </ul>
- </li>
-</ul>
+ <h1 id="dependencies-guide">Dependencies Guide</h1>
-<h2 id="overview">Overview</h2>
-
-<p>Interoperability between SDKs and runners is a key aspect of Apache
-Beam. So far, however, the reality is that most runners support the
-Java SDK only, because each SDK-runner combination requires non-trivial
-work on both sides. All runners are also currently written in Java,
-which makes support of non-Java SDKs far more expensive. The
-<em>portability framework</em> aims to rectify this situation and provide
-full interoperability across the Beam ecosystem.</p>
-
-<p>The portability framework introduces well-defined, language-neutral
-data structures and protocols between the SDK and runner. This interop
-layer – called the <em>portability API</em> – ensures that SDKs and runners
-can work with each other uniformly, reducing the interoperability
-burden for both SDKs and runners to a constant effort. It notably
-ensures that <em>new</em> SDKs automatically work with existing runners and
-vice versa. The framework introduces a new runner, the <em>Universal
-Local Runner (ULR)</em>, as a practical reference implementation that
-complements the direct runners. Finally, it enables cross-language
-pipelines (sharing I/O or transformations across SDKs) and
-user-customized execution environments (“custom containers”).</p>
-
-<p>The portability API consists of a set of smaller contracts that
-isolate SDKs and runners for job submission, management and
-execution. These contracts use protobufs and gRPC for broad language
-support.</p>
+<p>This document describes policies for keeping Beam dependencies up to
date.</p>
-<ul>
- <li>
- <p><strong>Job submission and management</strong>: The <em>Runner API</em>
defines a
-language-neutral pipeline representation with transformations
-specifying the execution environment as a docker container
-image. The latter both allows the execution side to set up the
-right environment as well as opens the door for custom containers
-and cross-environment pipelines. The <em>Job API</em> allows pipeline
-execution and configuration to be managed uniformly.</p>
- </li>
- <li>
- <p><strong>Job execution</strong>: The <em>SDK harness</em> is a
SDK-provided
-program responsible for executing user code and is run separately
-from the runner. The <em>Fn API</em> defines an execution-time binary
-contract between the SDK harness and the runner that describes how
-execution tasks are managed and how data is transferred. In
-addition, the runner needs to handle progress and monitoring in an
-efficient and language-neutral way. SDK harness initialization
-relies on the <em>Provision</em> and <em>Artifact APIs</em> for obtaining
staged
-files, pipeline options and environment information. Docker
-provides isolation between the runner and SDK/user environments to
-the benefit of both as defined by the <em>container contract</em>. The
-containerization of the SDK gives it (and the user, unless the SDK
-is closed) full control over its own environment without risk of
-dependency conflicts. The runner has significant freedom regarding
-how it manages the SDK harness containers.</p>
- </li>
-</ul>
+<p>Old dependencies cause user pain and can result in a system being unusable
for some users. Many users do not use Beam in isolation and bundle other
dependencies in the same deployment. These additional dependencies might pull
in incompatible dependencies to user’s environment which can again result in
broken Beam pipelines, sometimes with undefined behavior. To prevent this,
users will have to update their deployment environment or worse yet may end up
not being able to use Beam along [...]
-<p>The goal is that all (non-direct) runners and SDKs eventually support
-the portability API, perhaps exclusively.</p>
+<p>Beam Java SDK’s Gradle build defines a set of top level <a
href="https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy";>dependencies</a>
and various components (runners, IO connectors, etc) can choose to include
these dependencies. Components usually use the versions defined at the top
level but may choose to override these versions.</p>
-<h2 id="design">Design</h2>
+<p>If a component <em>X</em> chooses to override the version of a dependency
<em>D</em> from <em>a</em> to <em>b</em> and another component <em>Y</em> is
incompatible with version <em>b</em> of <em>D</em>, deployment of a user that
uses both components <em>X</em> and <em>Y</em> will end up in a broken
state.</p>
-<p>The <a href="https://github.com/apache/beam/tree/master/model";>model
protos</a>
-contain all aspects of the portability API and is the truth on the
-ground. The proto definitions supercede any design documents. The main
-design documents are the following:</p>
+<p>A similar issue could arise if two dependencies of Beam depend on a common
library but use incompatible versions of that library.</p>
-<ul>
- <li>
- <p><a href="https://s.apache.org/beam-runner-api";>Runner API</a>. Pipeline
-representation and discussion on primitive/composite transforms and
-optimizations.</p>
- </li>
- <li>
- <p><a href="https://s.apache.org/beam-job-api";>Job API</a>. Job submission
and
-management protocol.</p>
- </li>
- <li>
- <p><a href="https://s.apache.org/beam-fn-api";>Fn API</a>. Execution-side
control
-and data protocols and overview.</p>
- </li>
- <li>
- <p><a href="https://s.apache.org/beam-fn-api-container-contract";>Container
-contract</a>.
-Execution-side docker container invocation and provisioning
-protocols. See
-<a
href="https://github.com/apache/beam/blob/master/sdks/CONTAINERS.md";>CONTAINERS.md</a>
-for how to build container images.</p>
- </li>
-</ul>
+<p>Also, users might not use Beam in isolation, a user that depends on Beam as
well as other libraries in the same environment might run into similar issues
if Beam and the other library share a dependency while using incompatible
versions.</p>
-<p>In discussion:</p>
+<p>Beam Python SDK handles dependencies slightly differently, all dependencies
are defined in a single <a
href="https://github.com/apache/beam/blob/master/sdks/python/setup.py";>setup.py</a>
file and are grouped. One of the groups describes required dependencies while
other groups are for defining dependencies for various optional features. All
Python modules have to use the versions of dependencies defined in <a
href="https://github.com/apache/beam/blob/master/sdks/python/setup.py";>setup
[...]
-<ul>
- <li><a href="https://s.apache.org/beam-mixed-language-pipelines";>Cross
-language</a>. Options
-and tradeoffs for how to handle various kinds of
-multi-language/multi-SDK pipelines.</li>
-</ul>
+<p>This picture can become even more complicated during runtime. Runner
specific code might be incompatible with dependencies included by certain
modules and if these dependencies leak into runtime, a pipeline might end up in
a broken state.</p>
+
+<p>The overall issue is not common to Beam and well known in the industry as
the Diamond <a href="https://en.wikipedia.org/wiki/Dependency_hell";>Dependency
problem (or Dependency Hell)</a>.</p>
-<h2 id="development">Development</h2>
+<p>One common solution for the diamond dependency problem is <a
href="https://semver.org/";>semantic versioning</a>. The basic idea is that
dependencies will be versioned in the form <em>x.y.z</em> where <em>x</em> is
the <em>major version</em>, <em>y</em> is the <em>minor version</em>, and
<em>z</em> is the <em>patch version</em>. A major version change may be
backwards incompatible and is expected to be rare. Minor and patch versions may
be released more regularly but are expected to be [...]
-<p>The portability framework is a substantial effort that touches every
-Beam component. In addition to the sheer magnitude, a major challenge
-is engineering an interop layer that does not significantly compromise
-performance due to the additional serialization overhead of a
-language-neutral protocol.</p>
+<h2 id="identifying-outdated-dependencies">Identifying outdated
dependencies</h2>
-<h3 id="roadmap">Roadmap</h3>
+<p>A big part of keeping dependencies up to date involves identifying outdated
dependencies of Beam that the community should try to upgrade.</p>
-<p>The proposed project phases are roughly as follows and are not
-strictly sequential, as various components will likely move at
-different speeds. Additionally, there have been (and continues to be)
-supporting refactorings that are not always tracked as part of the
-portability effort. Work already done is not tracked here either.</p>
+<p>Beam currently executes a weekly Jenkins job that tries to identify
outdated dependencies for various SDKs. This Jenkins job generates a weekly
report that is shared in Beam dev list. In the future we hope to automatically
create JIRAs based on this report.</p>
+<p>In addition to this, Beam community members might identify other critical
dependency updates that have to be manually performed. For example,</p>
<ul>
- <li>
- <p><strong>P1 [MVP]</strong>: Implement the fundamental plumbing for
portable SDKs
-and runners for batch and streaming, including containers and the
-ULR
-[<a href="https://issues.apache.org/jira/browse/BEAM-2899";>BEAM-2899</a>]. Each
-SDK and runner should use the portability framework at least to the
-extent that wordcount
-[<a href="https://issues.apache.org/jira/browse/BEAM-2896";>BEAM-2896</a>] and
-windowed wordcount
-[<a href="https://issues.apache.org/jira/browse/BEAM-2941";>BEAM-2941</a>] run
-portably.</p>
- </li>
- <li>
- <p><strong>P2 [Feature complete]</strong>: Design and implement
portability support
-for remaining execution-side features, so that any pipeline from
-any SDK can run portably on any runner. These features include side
-inputs
-[<a href="https://issues.apache.org/jira/browse/BEAM-2863";>BEAM-2863</a>], User
-timers
-[<a href="https://issues.apache.org/jira/browse/BEAM-2925";>BEAM-2925</a>],
-Splittable DoFn
-[<a href="https://issues.apache.org/jira/browse/BEAM-2896";>BEAM-2896</a>] and
-more. Each SDK and runner should use the portability framework at
-least to the extent that the mobile gaming examples
-[<a href="https://issues.apache.org/jira/browse/BEAM-2940";>BEAM-2940</a>] run
-portably.</p>
- </li>
- <li>
- <p><strong>P3 [Performance]</strong>: Measure and tune performance of
portable
-pipelines using benchmarks such as Nexmark. Features such as
-progress reporting
-[<a href="https://issues.apache.org/jira/browse/BEAM-2940";>BEAM-2940</a>],
-combiner lifting
-[<a href="https://issues.apache.org/jira/browse/BEAM-2937";>BEAM-2937</a>] and
-fusion are expected to be needed.</p>
- </li>
- <li>
- <p><strong>P4 [Cross language]</strong>: Design and implement
cross-language
-pipeline support, including how the ecosystem of shared transforms
-should work.</p>
- </li>
+ <li>A minor release of a dependency due to a critical security
vulnerability.</li>
+ <li>A dependency conflict that was was triggered by a minor version release
of a Beam dependency (this does not apply to Java SDK that depends on exact
minor versions of dependencies).</li>
</ul>
-<h3 id="issues">Issues</h3>
+<p>These kind of urgently required upgrades might not get automatically picked
up by the Jenkins job for few months. So Beam community has to act to identify
such issues and perform upgrades early.</p>
+
+<h2 id="upgrading-identified-outdated-dependencies">Upgrading identified
outdated dependencies</h2>
+
+<p>After outdated dependencies are identified, Beam community has to act to
upgrade the dependencies regularly. Beam community has agreed on following
policies regarding upgrading dependencies.</p>
+
+<p><strong>Human readable reports on status of Beam dependencies are generated
weekly by an automated Jenkins job and shared with the Beam community through
the dev list.</strong></p>
+
+<p>These reports should be concise and should highlight the cases where the
community has to act on.</p>
+
+<p><strong>Beam components should define dependencies and their versions at
the top level. There can be rare exceptions, but they should come with
explanations.</strong></p>
+
+<p>Components include various Beam runners, IO connectors, etc.
Component-level dependency version declarations should only be performed in
rare cases and should come with a comment explaining the reasoning for
overriding the dependency. For example, dependencies specific to a runner that
are unlikely to be utilized by other components might be defined at the
runner.</p>
+
+<p><strong>A significantly outdated dependency (identified manually or through
the automated Jenkins job) should result in a JIRA that is a blocker for the
next release. Release manager may choose to push the blocker to the subsequent
release or downgrade from a blocker.</strong></p>
+
+<p>This will be a blocker for next major and minor version releases of Beam.
JIRA may be created automatically or manually.</p>
+
+<p>For manually identified critical dependency updates, Beam community members
should create blocking JIRAs for next release. In addition to this Beam
community members may trigger patch releases for any critical dependency fixes
that should be made available to users urgently.</p>
+
+<p><strong>Dependency declarations may identify owners that are responsible
for upgrading respective dependencies.</strong></p>
+
+<p>Owners can be mentioned in a comment. Blocking JIRAs will be initially
assigned to these owners (if available). Release manager may choose to
re-assign these JIRAs. A dependency may have more than one declared owner and
in this case the JIRA will be assigned to one of the owners mentioned.</p>
-<p>The portability effort touches every component, so the “portability”
-label is used to identify all portability-related issues. Pure
-design or proto definitions should use the “beam-model” component. A
-common pattern for new portability features is that the overall
-feature is in “beam-model” with subtasks for each SDK and runner in
-their respective components.</p>
+<p><strong>Dependencies of Java SDK components that may cause issues to other
components if leaked should be vendored.</strong></p>
-<p><strong>JIRA:</strong> <a
href="https://issues.apache.org/jira/issues/?filter=12341256";>query</a></p>
+<p><a
href="https://www.ardanlabs.com/blog/2013/10/manage-dependencies-with-godep.html";>Vendoring</a>
is the process of creating copies of third party dependencies. Combined with
repackaging, vendoring will allow Beam components to depend on third party
libraries without causing conflicts to other components. Vendoring should be
done in a case-by-case basis since this can increase the total number of
dependencies deployed in user’s enviroment.</p>
-<h3 id="status">Status</h3>
+<h2 id="dependency-updates-and-backwards-compatibility">Dependency updates and
backwards compatibility</h2>
-<p>MVP in progress. See the
-<a
href="https://docs.google.com/spreadsheets/d/1KDa_FGn1ShjomGd-UUDOhuh2q73de2tPz6BqHpzqvNI/edit?usp=sharing";>Portability
support table</a>
-for details.</p>
+<p>Beam releases <a
href="https://beam.apache.org/get-started/downloads/";>adhere to</a> semantic
versioning. Hence, community members should take care when updating
dependencies. Minor version updates to dependencies should be backwards
compatible in most cases. Some updates to dependencies though may result in
backwards incompatible API or functionality changes to Beam. PR reviewers and
committers should take care to detect any dependency updates that could
potentially introduce backwar [...]
</div>
</div>
diff --git a/content/contribute/design-documents/index.html
b/content/contribute/design-documents/index.html
index 3e934bb..80563f0 100644
--- a/content/contribute/design-documents/index.html
+++ b/content/contribute/design-documents/index.html
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
diff --git a/content/contribute/docker-images/index.html
b/content/contribute/docker-images/index.html
index 733414a..2b57912 100644
--- a/content/contribute/docker-images/index.html
+++ b/content/contribute/docker-images/index.html
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
diff --git a/content/contribute/eclipse/index.html
b/content/contribute/eclipse/index.html
index 5d426bd..0efe216 100644
--- a/content/contribute/eclipse/index.html
+++ b/content/contribute/eclipse/index.html
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
diff --git a/content/contribute/index.html b/content/contribute/index.html
index 4a8ce28..491e2a3 100644
--- a/content/contribute/index.html
+++ b/content/contribute/index.html
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
diff --git a/content/contribute/intellij/index.html
b/content/contribute/intellij/index.html
index da67921..469fad6 100644
--- a/content/contribute/intellij/index.html
+++ b/content/contribute/intellij/index.html
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
diff --git a/content/contribute/portability/index.html
b/content/contribute/portability/index.html
index 0e20980..119a96b 100644
--- a/content/contribute/portability/index.html
+++ b/content/contribute/portability/index.html
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
diff --git a/content/contribute/ptransform-style-guide/index.html
b/content/contribute/ptransform-style-guide/index.html
index 048fbbd..5795f99 100644
--- a/content/contribute/ptransform-style-guide/index.html
+++ b/content/contribute/ptransform-style-guide/index.html
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
diff --git a/content/contribute/release-guide/index.html
b/content/contribute/release-guide/index.html
index bb0c973..82247d6 100644
--- a/content/contribute/release-guide/index.html
+++ b/content/contribute/release-guide/index.html
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
diff --git a/content/contribute/runner-guide/index.html
b/content/contribute/runner-guide/index.html
index a926f78..6ed5398 100644
--- a/content/contribute/runner-guide/index.html
+++ b/content/contribute/runner-guide/index.html
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
diff --git a/content/contribute/testing/index.html
b/content/contribute/testing/index.html
index 7bd7696..5946c1d 100644
--- a/content/contribute/testing/index.html
+++ b/content/contribute/testing/index.html
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
diff --git a/content/contribute/website-contributions/index.html
b/content/contribute/website-contributions/index.html
index babcfdc..abfec0f 100644
--- a/content/contribute/website-contributions/index.html
+++ b/content/contribute/website-contributions/index.html
@@ -108,6 +108,7 @@
<li><a href="/contribute/portability/">Portability Framework</a></li>
<li><a href="/contribute/docker-images/">Docker images</a></li>
<li><a href="/contribute/design-documents/">Design documents</a></li>
+ <li><a href="/contribute/dependencies/">Dependencies guide</a></li>
</ul>
</li>
<li>
diff --git a/content/get-started/downloads/index.html
b/content/get-started/downloads/index.html
index cfb7dd8..fa3ac0c 100644
--- a/content/get-started/downloads/index.html
+++ b/content/get-started/downloads/index.html
@@ -213,52 +213,52 @@ versions denoted <code
class="highlighter-rouge">0.x.y</code>.</p>
<p><a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608";>Release
notes</a>.</p>
<h3 id="230-2018-01-30">2.3.0 (2018-01-30)</h3>
-<p>Official <a
href="https://www.apache.org/dyn/closer.cgi?filename=beam/2.3.0/apache-beam-2.3.0-source-release.zip&action=download";>source
code download</a>.</p>
+<p>Official <a
href="https://archive.apache.org/dist/beam/2.3.0/apache-beam-2.3.0-source-release.zip";>source
code download</a>.</p>
<p><a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608";>Release
notes</a>.</p>
<h3 id="220-2017-12-02">2.2.0 (2017-12-02)</h3>
-<p>Official <a
href="https://www.apache.org/dyn/closer.cgi?filename=beam/2.2.0/apache-beam-2.2.0-source-release.zip&action=download";>source
code download</a>.</p>
+<p>Official <a
href="https://archive.apache.org/dist/beam/2.2.0/apache-beam-2.2.0-source-release.zip";>source
code download</a>.</p>
<p><a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341044";>Release
notes</a>.</p>
<h3 id="210-2017-08-23">2.1.0 (2017-08-23)</h3>
-<p>Official <a
href="https://www.apache.org/dyn/closer.cgi?filename=beam/2.1.0/apache-beam-2.1.0-source-release.zip&action=download";>source
code download</a>.</p>
+<p>Official <a
href="https://archive.apache.org/dist/beam/2.1.0/apache-beam-2.1.0-source-release.zip";>source
code download</a>.</p>
<p><a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12340528";>Release
notes</a>.</p>
<h3 id="200-2017-05-17">2.0.0 (2017-05-17)</h3>
-<p>Official <a
href="https://www.apache.org/dyn/closer.cgi?filename=beam/2.0.0/apache-beam-2.0.0-source-release.zip&action=download";>source
code download</a>.</p>
+<p>Official <a
href="https://archive.apache.org/dist/beam/2.0.0/apache-beam-2.0.0-source-release.zip";>source
code download</a>.</p>
<p><a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12339746";>Release
notes</a>.</p>
<h3 id="060-2017-03-11">0.6.0 (2017-03-11)</h3>
-<p>Official <a
href="https://www.apache.org/dyn/closer.cgi?filename=beam/0.6.0/apache-beam-0.6.0-source-release.zip&action=download";>source
code download</a>.</p>
+<p>Official <a
href="https://archive.apache.org/dist/beam/0.6.0/apache-beam-0.6.0-source-release.zip";>source
code download</a>.</p>
<p><a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12339256";>Release
notes</a>.</p>
<h3 id="050-2017-02-02">0.5.0 (2017-02-02)</h3>
-<p>Official <a
href="https://www.apache.org/dyn/closer.cgi?filename=beam/0.5.0/apache-beam-0.5.0-source-release.zip&action=download";>source
code download</a>.</p>
+<p>Official <a
href="https://archive.apache.org/dist/beam/0.5.0/apache-beam-0.5.0-source-release.zip";>source
code download</a>.</p>
<p><a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12338859";>Release
notes</a>.</p>
<h3 id="040-2016-12-29">0.4.0 (2016-12-29)</h3>
-<p>Official <a
href="https://www.apache.org/dyn/closer.cgi?filename=beam/0.4.0/apache-beam-0.4.0-source-release.zip&action=download";>source
code download</a>.</p>
+<p>Official <a
href="https://archive.apache.org/dist/beam/0.4.0/apache-beam-0.4.0-source-release.zip";>source
code download</a>.</p>
<p><a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12338590";>Release
notes</a>.</p>
<h3 id="030-incubating-2016-10-31">0.3.0-incubating (2016-10-31)</h3>
-<p>Official <a
href="https://www.apache.org/dyn/closer.cgi?filename=beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip&action=download";>source
code download</a>.</p>
+<p>Official <a
href="https://archive.apache.org/dist/beam/0.3.0-incubating/apache-beam-0.3.0-incubating-source-release.zip";>source
code download</a>.</p>
<p><a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12338051";>Release
notes</a>.</p>
<h3 id="020-incubating-2016-08-08">0.2.0-incubating (2016-08-08)</h3>
-<p>Official <a
href="https://www.apache.org/dyn/closer.cgi?filename=beam/0.2.0-incubating/apache-beam-0.2.0-incubating-source-release.zip&action=download";>source
code download</a>.</p>
+<p>Official <a
href="https://archive.apache.org/dist/beam/0.2.0-incubating/apache-beam-0.2.0-incubating-source-release.zip";>source
code download</a>.</p>
<p><a
href="https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12335766";>Release
notes</a>.</p>
<h3 id="010-incubating-2016-06-15">0.1.0-incubating (2016-06-15)</h3>
-<p>Official <a
href="https://www.apache.org/dyn/closer.cgi?filename=beam/0.1.0-incubating/apache-beam-0.1.0-incubating-source-release.zip&action=download";>source
code download</a>.</p>
+<p>Official <a
href="https://archive.apache.org/dist/beam/0.1.0-incubating/apache-beam-0.1.0-incubating-source-release.zip";>source
code download</a>.</p>
<p>The first incubating release of Apache Beam.</p>
