Repository: incubator-hivemall Updated Branches: refs/heads/master 07eb707b7 -> 13e13027b
[HIVEMALL-216] Fix Docker image based on openjdk 8 ## What changes were proposed in this pull request? This PR fixes building Docker image from Docker file. ## What type of PR is it? Bug Fix ## What is the Jira issue? https://issues.apache.org/jira/browse/HIVEMALL-216 ## How was this patch tested? manual tests ## How to use this feature? See [documentation](https://hivemall.incubator.apache.org/userguide/docker/getting_started.html) Author: Aki Ariga <ar...@treasure-data.com> Closes #161 from chezou/fix-dockerfile. Project: http://git-wip-us.apache.org/repos/asf/incubator-hivemall/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-hivemall/commit/13e13027 Tree: http://git-wip-us.apache.org/repos/asf/incubator-hivemall/tree/13e13027 Diff: http://git-wip-us.apache.org/repos/asf/incubator-hivemall/diff/13e13027 Branch: refs/heads/master Commit: 13e13027b9ad601f0ea759ed024fdf25e0bd6616 Parents: 07eb707 Author: Aki Ariga <ar...@treasure-data.com> Authored: Tue Sep 4 15:30:14 2018 +0900 Committer: Makoto Yui <m...@apache.org> Committed: Tue Sep 4 15:30:14 2018 +0900 ---------------------------------------------------------------------- docs/gitbook/docker/getting_started.md | 26 ++++++++++++++------------ resources/docker/Dockerfile | 19 +++++++++++-------- resources/docker/home/.hiverc | 2 +- 3 files changed, 26 insertions(+), 21 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-hivemall/blob/13e13027/docs/gitbook/docker/getting_started.md ---------------------------------------------------------------------- diff --git a/docs/gitbook/docker/getting_started.md b/docs/gitbook/docker/getting_started.md index d4b2991..314c82e 100644 --- a/docs/gitbook/docker/getting_started.md +++ b/docs/gitbook/docker/getting_started.md @@ -17,6 +17,8 @@ under the License. --> +# Getting started with Hivemall on Docker + This page introduces how to run Hivemall on Docker. <!-- toc --> @@ -24,22 +26,22 @@ This page introduces how to run Hivemall on Docker. > #### Caution > This docker image contains a single-node Hadoop enviroment for evaluating > Hivemall. Not suited for production uses. -# Requirements +## Requirements * Docker Engine 1.6+ * Docker Compose 1.10+ -# 1. Build image +## Build image You have two options in order to build a **hivemall** docker image: -## Using `docker-compose` +### Using `docker-compose` ``` $ docker-compose -f resources/docker/docker-compose.yml build ``` -## Using `docker` command +### Using `docker` command ``` $ docker build -f resources/docker/Dockerfile . @@ -48,11 +50,11 @@ $ docker build -f resources/docker/Dockerfile . > #### Note > You can > [skip](./getting_started.html#running-pre-built-docker-image-in-docker-hub) > building images if you try to use a pre-build docker image from Docker Hub. > However, since the Docker Hub repository is experimental one, the > distributed image is NOT built on the "latest" commit in [our master > branch](https://github.com/apache/incubator-hivemall). -# 2. Run container +## Run container If you built an image by yourself, it can be launched by either `docker-compose` or `docker` command: -## By `docker-compose` +### By `docker-compose` ``` $ docker-compose -f resources/docker/docker-compose.yml up -d && docker attach hivemall @@ -68,7 +70,7 @@ volumes: - "/path/to/data/:/root/data/" # mount resources to container-side `/root/data` directory ``` -## By `docker` command +### By `docker` command Find a local docker image by `docker images`, and hit: @@ -84,7 +86,7 @@ Similarly to the `volumes` option in the `docker-compose` file, `docker run` has $ docker run ... -v /path/to/local/hivemall:/opt/hivemall ``` -## Running pre-built Docker image in Docker Hub +### Running pre-built Docker image in Docker Hub > #### Caution > This part is experimental. Hivemall in the pre-built image might be > out-of-date compared to the latest version in [our master > branch](https://github.com/apache/incubator-hivemall). @@ -101,12 +103,12 @@ $ docker pull hivemall/latest:20170517 $ docker run -p 8088:8088 -p 50070:50070 -p 19888:19888 -it hivemall/latest:20170517 ``` -# 3. Run Hivemall on Docker +## Run Hivemall on Docker 1. Type `hive` to run (`.hiverc` automatically loads Hivemall functions) 2. Try your Hivemall queries! -## Accessing Hadoop management GUIs +### Accessing Hadoop management GUIs * YARN http://localhost:8088/ * HDFS http://localhost:50070/ @@ -114,7 +116,7 @@ $ docker run -p 8088:8088 -p 50070:50070 -p 19888:19888 -it hivemall/latest:2017 Note that you need to expose local ports e.g., by `-p 8088:8088 -p 50070:50070 -p 19888:19888` on running docker image. -## Load data into HDFS (optional) +### Load data into HDFS (optional) You can find an example script to load data into HDFS in `$HOME/bin/prepare_iris.sh`. The script loads iris dataset into `iris` database: @@ -137,7 +139,7 @@ OK Once you prepared the `iris` database, you are ready to move on to [our multi-class classification tutorial](../multiclass/iris_dataset.html). -## Build Hivemall (optional) +### Build Hivemall (optional) In the container, Hivemall resource is stored in `$HIVEMALL_PATH`. You can build Hivemall package by `cd $HIVEMALL_PATH && ./bin/build.sh`. http://git-wip-us.apache.org/repos/asf/incubator-hivemall/blob/13e13027/resources/docker/Dockerfile ---------------------------------------------------------------------- diff --git a/resources/docker/Dockerfile b/resources/docker/Dockerfile index 6332f4c..39dc3f4 100644 --- a/resources/docker/Dockerfile +++ b/resources/docker/Dockerfile @@ -17,13 +17,13 @@ # under the License. # -FROM openjdk:7 +FROM openjdk:8 WORKDIR /root/ ARG PREBUILD=true -ARG HADOOP_VERSION=2.7.3 -ARG HIVE_VERSION=2.1.1 +ARG HADOOP_VERSION=2.7.7 +ARG HIVE_VERSION=2.3.3 ENV BASE_URL='https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename=' ENV HADOOP_HOME='/usr/local/hadoop' @@ -34,13 +34,16 @@ ENV HADOOP_OPTS=' \ -Dsystem:user.name=root \ -Dderby.stream.error.file=/root/derby.log' ENV PATH="${HADOOP_HOME}/bin:${HIVE_HOME}/bin:${PATH}" +ENV JAVA_HOME='/usr/lib/jvm/java-8-openjdk-amd64' +ENV JAVA8_HOME='/usr/lib/jvm/java-8-openjdk-amd64' COPY . ${HIVEMALL_PATH}/ RUN set -eux && \ - apt update && \ - apt install -y --no-install-recommends openssh-server maven g++ make ruby npm && \ - ln -s /usr/bin/nodejs /usr/bin/node && \ + apt-get update && \ + apt-get install -y --no-install-recommends openssh-server maven g++ make ruby && \ + curl -sL https://deb.nodesource.com/setup_10.x | bash - && \ + apt-get install -y nodejs && \ npm install -g gitbook-cli && \ \ wget ${BASE_URL}hadoop/common/hadoop-${HADOOP_VERSION}/hadoop-${HADOOP_VERSION}.tar.gz -O - \ @@ -72,8 +75,8 @@ RUN set -eux && \ -exec sh -c 'f={} && ln -s $f /root/bin/${f##*/}' \; && \ ln -s ${HIVEMALL_PATH}/resources/docker/home/.hiverc /root && \ ln -s ${HIVEMALL_PATH}/resources/ddl/define-all.hive /root/hivemall/define-all.hive && \ - ln -s ${HIVEMALL_PATH}/target/hivemall-core-${HIVEMALL_VERSION}-with-dependencies.jar \ - /root/hivemall/hivemall-core-with-dependencies.jar && \ + ln -s ${HIVEMALL_PATH}/target/hivemall-all-${HIVEMALL_VERSION}.jar \ + /root/hivemall/hivemall-all.jar && \ rm -rf /var/cache/apt/archives/* /var/lib/apt/lists/* /root/.m2/* /root/.npm/* VOLUME ["/opt/hivemall/", "/root/data/"] http://git-wip-us.apache.org/repos/asf/incubator-hivemall/blob/13e13027/resources/docker/home/.hiverc ---------------------------------------------------------------------- diff --git a/resources/docker/home/.hiverc b/resources/docker/home/.hiverc index 0030939..0260b22 100644 --- a/resources/docker/home/.hiverc +++ b/resources/docker/home/.hiverc @@ -1,2 +1,2 @@ -add jar /root/hivemall/hivemall-core-with-dependencies.jar; +add jar /root/hivemall/hivemall-all.jar; source /root/hivemall/define-all.hive;
