This is an automated email from the ASF dual-hosted git repository.
mergebot-role pushed a commit to branch mergebot
in repository https://gitbox.apache.org/repos/asf/beam-site.git
commit 8e16d3954c96c59476501cdfa1daf9285765cb2c
Author: Ismaël Mejía <ieme...@gmail.com>
AuthorDate: Tue Aug 22 10:59:27 2017 +0200
[BEAM-1534] Add docker images documentation to the website
---
src/_includes/header.html | 1 +
src/contribute/docker-images.md | 183 ++++++++++++++++++++++++++++++++++++++++
2 files changed, 184 insertions(+)
diff --git a/src/_includes/header.html b/src/_includes/header.html
index 79e72b6..2de94b2 100644
--- a/src/_includes/header.html
+++ b/src/_includes/header.html
@@ -84,6 +84,7 @@
<li><a href="{{ site.baseurl
}}/contribute/design-principles/">Design Principles</a></li>
<li><a href="{{ site.baseurl
}}/contribute/work-in-progress/">Ongoing Projects</a></li>
<li><a href="{{ site.baseurl
}}/contribute/source-repository/">Source Repository</a></li>
+ <li><a href="{{ site.baseurl }}/contribute/docker-images/">Docker
Images</a></li>
<li role="separator" class="divider"></li>
<li class="dropdown-header">Promotion</li>
<li><a href="{{ site.baseurl
}}/contribute/presentation-materials/">Presentation Materials</a></li>
diff --git a/src/contribute/docker-images.md b/src/contribute/docker-images.md
new file mode 100644
index 0000000..0cb19da
--- /dev/null
+++ b/src/contribute/docker-images.md
@@ -0,0 +1,183 @@
+---
+layout: default
+title: 'Beam Docker Images'
+permalink: /contribute/docker-images/
+---
+
+# Docker Images
+
+Docker images allow to create a reproducible environment to build and test
+Beam. You can use the docker images by using the provided [Docker
scripts](https://github.com/apache/beam/tree/master/sdks/java/build-tools/src/main/resources/docker).
+
+In this directory you will find scripts to build and run docker images for
+different purposes:
+
+- [file](#file): Create a Docker container from a Beam source code .zip file
+ in a given environment. It is useful to test a specific version of Beam,
+ for example to validate a release vote.
+
+- [git](#git): Same as file but the Beam source code comes from the git
repository,
+ you can choose a given branch/tag/pull-request. Useful to test in a specific
+ environment.
+
+- [release](#release): It builds an end-user distribution of the latest
version of Beam
+ and its dependencies. Useful for end-users who want to have a ready to use
+ container with Beam (Python only for the moment).
+
+## <a name="file"></a>File based image
+
+If you want to build a container with a ready JDK 8 environment to test Beam:
+
+```
+cd file/openjdk8
+docker build -t beam:openjdk8 .
+```
+
+If you want to build a container with JDK 7:
+
+```
+cd file/openjdk7
+docker build -t beam:openjdk7 .
+```
+
+When you run the image it downloads the specific version of Beam given the
+environment variables. By default it downloads the source code of the latest
+master. If you want to download the latest master and execute the tests:
+
+```
+docker run -it beam:openjdk8 mvn clean verify -Prelease
+```
+
+If you want to have an interactive session you can run a bash prompt:
+
+```
+docker run -it beam:openjdk8 /bin/bash
+```
+
+Inside the container you can test a specific module by running Maven. You
+have to change MODULE_PATH for the given module path. For example to test
+HBaseIO you should write its path `sdks/java/io/hbase` in the place of
+MODULE_PATH:
+
+```
+mvn --projects MODULE_PATH clean verify -Prelease
+```
+
+### Configuring the runtime via the environment variables
+
+You can run different versions of Beam by passing the specific environment
+variables:
+
+`URL`: The URL with the file containing the specific source code.
+`SRC_FILE`: The downloaded file name.
+`SRC_DIR`: The name of the directory inside of the zip file.
+
+For example to run a Docker container with the exact code of the release 2.0.0:
+
+```
+docker run \
+ -e
URL="https://www.apache.org/dyn/closer.cgi?filename=beam/2.0.0/apache-beam-2.0.0-source-release.zip&action=download";
\
+ -e SRC_FILE="apache-beam-2.0.0-source-release.zip" \
+ -e SRC_DIR="apache-beam-2.0.0" \
+ -it beam:openjdk8 /bin/bash
+```
+
+If you want to run a container with a specific version e.g. 2.1.0-RC1:
+
+```
+docker run \
+ -e URL="https://github.com/apache/beam/archive/v2.1.0-RC1.zip"; \
+ -e SRC_FILE="v2.1.0-RC1.zip" \
+ -e SRC_DIR="beam-2.1.0-RC1" \
+ -it beam:openjdk8 /bin/bash
+```
+
+*Notice that SRC_FILE is different from SRC_DIR because github replaces the 'v'
+character in the internal name of the zip*.
+
+To run a container with the source code during a vote:
+
+```
+docker run \
+ -e
URL="https://dist.apache.org/repos/dist/dev/beam/2.1.0/apache-beam-2.1.0-source-release.zip";
\
+ -e SRC_FILE="apache-beam-2.1.0-source-release.zip" \
+ -e SRC_DIR="apache-beam-2.1.0" \
+ -it beam:openjdk8 /bin/bash
+```
+
+### Testing in an specific environment with your own source
+
+You can also overwrite the volume containing the source code of Beam from a
+directory in your host machine. This is useful to test your code with different
+versions of Java.
+
+If you have the code in the ~/workspace/beam directory and you want to quickly
+test it with Beam on Java 7 you can do:
+
+```
+docker run -v ~/workspace/beam:/home/user/beam -it beam:openjdk7 /bin/bash
+```
+
+### Performance improvements
+
+The docker image does **not** contain the Java dependencies and this could make
+the execution of the commands in the image take more time than expected. One
way
+to speed this up is by mounting a directory with the dependencies as a Docker
+volume. The following command shows how to mount the host machine directory
+for Maven dependencies `~/.m2` in the container:
+
+```
+docker run -v ~/.m2:/home/user/.m2 -it beam:openjdk8 /bin/bash
+```
+
+You can also use an empty directory instead of `~/.m2` if you want to create a
+different environment from the local one to share it between runs.
+
+**Maven Improvements:**
+
+You can also speed the execution by adding environment variables to Maven, for
+example the MAVEN_OPTS environment variable may speed up compilation:
+
+```
+docker run \
+ -e MAVEN_OPTS='-client -XX:+TieredCompilation -XX:TieredStopAtLevel=1
-Xverify:none' \
+ -it beam:openjdk8 /bin/bash
+```
+
+Maven does not execute on multiple cores by default; to do this you have to
+enable the threading execution:
+
+```
+mvn --threads 1C ...
+```
+
+## <a name="git"></a>Git based image
+
+It creates a docker container with a cloned version of the git repository, its
+branches and all the pull-requests. When it is run, it updates and checks out
+the specified branch/tag/pull-request.
+
+You can choose the branch to execute via the environment variable:
+
+`BRANCH`: The name of the specific branch, tag or pr to check out.
+
+By default it checks out master from the github repo, but if you want to check
+a specific pull request `ID` you can build the image like this (you must change
+`ID` for the number):
+
+```
+cd git/openjdk8
+docker build -t beam:git .
+docker run -e BRANCH=pr/ID -it beam:git /bin/bash
+```
+
+## <a name="release"></a>Release image
+
+The release image is a container with the needed requirements to run Beam out
of
+the box in a container, or to submit a pipeline to execute on Google Cloud
+Dataflow. To build it run:
+
+```
+cd release/python2
+docker build -t beam:python2 .
+```
--
To stop receiving notification emails like this one, please contact
"commits@beam.apache.org" <commits@beam.apache.org>.
