GitHub user 1ambda opened a pull request: https://github.com/apache/zeppelin/pull/1761
[ZEPPELIN-1711] Create Released Binary Zeppelin Docker Images ### What is this PR for? This PR - added Dockerfiles for released zeppelin binaries (0.6.0, 0.6.1, 0.6.2) - refactored Dockerfiles `scripts/docker/zeppelin-base` so that we can keep small size base images - added missing R, python packages to run tutorials properly - also included `scripts/docker/create-dockerfile.sh` to create docker images for newly released zeppelin binaries #### For Reviewers I have things to be discussed for this PR 1. **Support other linux (centos) or not:** Alpine linux is designed for light weight OS so it doesn't have graphic devices. But usually people use zeppelin to render graph like things. In this case Alpine is not enough 2. **Extracting script for installing required packages into `testing/install_external_dependencies.sh` or not:** Alpine linux has it's own command to install packages like `apk add`. I think this is not reusable. And extracting scripts from Dockerfile causes other problems. For example, we need to add script just for building base images for python, r to copy scripts for external dependencies because Docker command `COPY` and `ADD` doens't support relative paths. It means, we have to copy external dependency scripts before building into the path where Dockerfile is built. 3. **Adding Binary Zeppelin Image to [docker-library/official-images](https://github.com/docker-library/official-images/)**: I haven't experiences about this. ### What type of PR is it? [Feature] ### What is the Jira issue? [ZEPPELIN-1711](https://issues.apache.org/jira/browse/ZEPPELIN-1711) ### How should this be tested? #### Building Base Images ```bash $ cd scripts/docker/zeppelin-base/alpine/java $ docker build . -t zeppelin:alpine-base_java $ cd scripts/docker/zeppelin-base/alpine/python $ docker build . -t zeppelin:alpine-base_python $ cd scripts/docker/zeppelin-base/alpine/r $ docker build . -t zeppelin:alpine-base_r ``` #### Building Zeppelin Images ```bash $ scripts/docker/zeppelin-bin-all/alpine/0.6.2_java $ docker build . -t zeppelin:alpine-0.6.2_java $ scripts/docker/zeppelin-bin-all/alpine/0.6.2_python $ docker build . -t zeppelin:alpine-0.6.2_python $ scripts/docker/zeppelin-bin-all/alpine/0.6.2_r $ docker build . -t zeppelin:alpine-0.6.2_r # build 0.6.1, 0.6.0 images too ``` #### Running Zeppelin Images Make sure you have zeppelin docker images before running containers ```bash $ docker images REPOSITORY TAG IMAGE ID CREATED SIZE zeppelin alpine-base_java 5a24572968d6 5 hours ago 145 MB ``` Then, running containers by replacing tags (0.6.2, 0.6.1, 0.6.0) ``` $ docker run -it --name zeppelin --rm -p 8080:8080 -p 7077:7077 zeppelin:alpine-0.6.2_java $ docker run -it --name zeppelin --rm -p 8080:8080 -p 7077:7077 zeppelin:alpine-0.6.2_python $ docker run -it --name zeppelin --rm -p 8080:8080 -p 7077:7077 zeppelin:alpine-0.6.2_r # run 0.6.1, 0.6.0 also ``` #### Testing Zeppelin Tutorials Here are things you need to know before testing. - `zeppelin:alpine-$TAG_java` images can run **Zeppelin Tutorial** since it has openjdk7 - `zeppelin:alpine-$TAG_python` images can run **Zeppelin Tutorial: Python - matplotlib basic** as well as **Zeppelin Tutorial** since it has python related packages - `zeppelin:alpine-$TAG_r` images can run **R Tutorial** as well as **Zeppelin Tutorial** since it has python related packages - Since alpine linux doesn't have graphical device, some function may not work (e.g plot in R). Some R examples throw invalid images like chunked.png - For the same reason, we have to change some code of **Python - matplotlib basic** before testing otherwise it will throw tkinter errors in the python interpreter ```python // before %python import numpy as np import matplotlib.pyplot as plt // should be %python import numpy as np import matplotlib matplotlib.use('Agg') import matplotlib.pyplot as pet ``` Then, run each zeppelin container and run tutorial while replacing tags by accessing localhost:8080 in your browsers ``` # run: Zeppelin Tutorial $ docker run -it --name zeppelin --rm -p 8080:8080 -p 7077:7077 zeppelin:alpine-0.6.2_java # run: Zeppelin Tutorial, Python - matplotlib basic # and modify the code mentioned above before running paragraphs $ docker run -it --name zeppelin --rm -p 8080:8080 -p 7077:7077 zeppelin:alpine-0.6.2_python # run: Zeppelin Tutorial, R Tutorial $ docker run -it --name zeppelin --rm -p 8080:8080 -p 7077:7077 zeppelin:alpine-0.6.2_r # test 0.6.1, 0.6.0 containers too ``` #### Testing `create-dockerfile.sh` ```bash $ scripts/docker/zeppelin-bin-all/create-dockerfile.sh -h USAGE: ./create-dockerfile.sh version linux platform * version: 0.6.2 (released zeppelin binary version) * linux: [alpine] * platform: [java, python, r] # For example, # This command will create `scripts/docker/zeppelin-bin-all/alpine/0.7.0_java/Dockerfile $ ./create-dockerfile.sh 0.7.0 alpine java ``` ### Screenshots (if appropriate) #### Image Sizes zeppelin base image size added in #1538: 301.3 MB ``` REPOSITORY TAG IMAGE ID CREATED SIZE zeppelin alpine-base_java 5a24572968d6 5 hours ago 145 MB zeppelin alpine-base_python e41dcf04760b 4 hours ago 684.9 MB zeppelin alpine-base_r 80c2d7aa7156 4 hours ago 538.9 MB zeppelin alpine-0.6.0_java 85b4502fd255 2 hours ago 737.7 MB zeppelin alpine-0.6.0_python 709dc585b94c 2 hours ago 1.278 GB zeppelin alpine-0.6.0_r f6fba0c66563 2 hours ago 1.131 GB zeppelin alpine-0.6.1_java ed95ca6204a9 3 hours ago 751.3 MB zeppelin alpine-0.6.1_python 34ceea2aeba8 3 hours ago 1.291 GB zeppelin alpine-0.6.1_r 3886596f3759 3 hours ago 1.145 GB zeppelin alpine-0.6.2_java c65b2f6a1128 3 hours ago 786.4 MB zeppelin alpine-0.6.2_python 16c2c3630545 3 hours ago 1.326 GB zeppelin alpine-0.6.2_r 544311ec1e6a 3 hours ago 1.18 GB ``` ### Questions: * Does the licenses files need update? - NO * Is there breaking changes for older versions? - NO * Does this needs documentation? - YES, I updated `docs/install/docker.md` You can merge this pull request into a Git repository by running: $ git pull https://github.com/1ambda/zeppelin feat/docker-images Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1761.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1761 ---- commit 9f5968339cee5f8c3e03fa73cf5e5b1e10af9ff9 Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T07:17:29Z feat: Install dumb-init using package commit d9b8421499551500a7db9d23fdb504f84cb0c3d3 Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T07:35:04Z feat: Install openjdk7 ref - https://github.com/1ambda/docker-zeppelin/blob/master/alpine/base/Dockerfile commit a9b3d67ec0df1a0b05627550e84f91b9549392ba Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T08:29:18Z chore: Use --no-cache not to remove cache dir commit 085527d85c50533fccf089662fdf7c44fa3cccd2 Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T08:33:05Z chore: Add logging commit 24e15ef207dea4fe064978cbab826e131d2ce641 Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T08:33:22Z fix: Remove exposed ports in base image commit e8d2f5345f010bd5a866224825a351e0881ab01b Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T09:13:50Z feat: Add base_python, base_r commit d7623f76e91182c0c570edf5515235ccce81f32a Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T09:53:49Z chore: Use zeppelin-base/alpine directory structure commit 0730e72fe437248994bcef8e0a5b828d21adb38a Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T10:24:19Z feat: Add create-dockerfile.sh commit ce7b2e42123bf05726dd2000ab37eafd03f73417 Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T12:24:51Z fix: _java postfix commit a10c304984b91af2fc820f9ea4a1917033b16d88 Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T13:22:44Z fix: Base image directory structure commit 03d32d93acd5bb5a03344904d1d7e15348171d27 Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T13:23:04Z docs: Update docker docs commit 0c297af9ff7bb83b10378b13427a4929ccdafbf3 Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T13:35:44Z fix: Update create-dockerfile usage message commit 5df6b7359ef0398a26cab478c1f1ab876fa9574d Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T13:36:13Z feat: Add 0.6.2 bin images commit d960ffd2cba62ee9214100355a1d57502c146da9 Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T13:36:40Z feat: Add 0.6.1 bin images commit 4af7cebe25ec748ca1b04de086970888e6a42abb Author: 1ambda <1am...@gmail.com> Date: 2016-12-14T13:37:00Z feat: Add 0.6.0 bin images ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---