[ https://issues.apache.org/jira/browse/BEAM-8209?focusedWorklogId=317115&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-317115 ]
ASF GitHub Bot logged work on BEAM-8209: ---------------------------------------- Author: ASF GitHub Bot Created on: 24/Sep/19 01:31 Start Date: 24/Sep/19 01:31 Worklog Time Spent: 10m Work Description: rosetn commented on pull request #9607: [BEAM-8209] Custom container docs URL: https://github.com/apache/beam/pull/9607#discussion_r327393615 ########## File path: website/src/documentation/runtime/environments.md ########## @@ -0,0 +1,187 @@ +--- +layout: section +title: "Runtime environments" +section_menu: section-menu/documentation.html +permalink: /documentation/runtime/environments/ +redirect_from: + - /documentation/execution-model/ +--- +<!-- +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Runtime environments + +Any execution engine can run the Beam SDK beacuse the SDK runtime environment is [containerized](https://s.apache.org/beam-fn-api-container-contract) with [Docker](https://www.docker.com/) and isolated from other runtime systems. This page describes how to build, customize, and push Beam SDK container images. + +## Building container images + +Before building Beam SDK container images: +* Register a [Bintray](https://bintray.com/) account with a Docker repository named `apache`. +* Install [Docker](https://www.docker.com/) on your workstation. + +To build Beam SDK container images: + +<ol> + <li> + Navigate to your local copy of the <a href="https://github.com/apache/beam"><code>beam</code></a> + </li> + <li> + Run Gradle with the <code>docker</code> target: <pre>./gradlew docker</pre> + </li> +</ol> + +> **Note**: It may take a long time to build all of the container images. You can instead build the images for specific SDKs: +> +> ``` +> ./gradlew -p sdks/java/container docker +> ./gradlew -p sdks/python/container docker +> ./gradlew -p sdks/go/container docker +> ``` + +Run `docker images` to examine the containers. For example, if you successfully built the container images, the command prompt displays a response like: + +``` +REPOSITORY TAG IMAGE ID CREATED SIZE +$USER-docker-apache.bintray.io/beam/python latest 4ea515403a1a 3 minutes ago 1.27GB +$USER-docker-apache.bintray.io/beam/java latest 0103512f1d8f 34 minutes ago 780MB +$USER-docker-apache.bintray.io/beam/go latest ce055985808a 35 minutes ago 121MB +``` + +<b>Although the respository names look like URLs, the container images are stored locally on your workstation.</b> After building the container images locally, you can [push](#pushing-container-images) them to an eponymous repository online. + +### Overriding default Docker targets + +The default SDK version is `latest` and the default Docker repository is the following Bintray location: + +``` +$USER-docker-apache.bintray.io/beam +``` + +When you [build SDK container images](#building-container-images), you can override the default version and location. + +To specify an older Python SDK version, like 2.3.0, build the container with the `docker-tag` option: + +``` +./gradlew docker -Pdocker-tag=2.3.0 +``` + +To change the `docker` target, build the container with the `docker-repository-root` option: + +``` +./gradlew docker -Pdocker-repository-root=$LOCATION +``` + +## Customizing container images + +You can add extra dependencies or serialization files to container images so the execution engine doesn't need them. + +To customize a container image, either: +* [Write a new](#writing-new-dockerfiles) [Dockerfile](https://docs.docker.com/engine/reference/builder/) on top of the original +* [Modify](#modifying-dockerfiles) the [original Dockerfile](https://github.com/apache/beam/blob/master/sdks/python/container/Dockerfile) and reimage the container + +It's often easier to write a new Dockerfile, but you can customize anything, including the base OS, by modifying the original. Review comment: "It's often easier to write a new Dockerfile, but by modifying the original Dockerfile, you can customize anything (including the base OS) ." Just a suggestion--I don't really like parentheses, but this sentence is confusing with too many clauses. Another possibility is to break it up. "It's often easier to write a new Dockerfile. However, by modifying the original Dockerfile, you can customize anything, including the base OS." ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 317115) Time Spent: 1h 10m (was: 1h) > Document custom docker containers > --------------------------------- > > Key: BEAM-8209 > URL: https://issues.apache.org/jira/browse/BEAM-8209 > Project: Beam > Issue Type: Sub-task > Components: website > Reporter: Cyrus Maden > Assignee: Cyrus Maden > Priority: Minor > Fix For: 2.16.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)