Tim Armstrong has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/12074 )
Change subject: IMPALA-7948: part 1: initial docker container build ...................................................................... IMPALA-7948: part 1: initial docker container build This builds an impala_base container that has all of the build artifacts required to run the impala processes, then builds impalad, catalogd and statestore containers based on that with the right ports exposed. The images are based on the Ubuntu 16.04 image to align with the most common development environment. The container build process is integrated with CMake and is designed to integrate with the rest of the build so that the container build depends on the artifacts that will go into the container. You can build the images with the following command, which will create images called "impala_base", "impalad", "catalogd" and "statestored": ninja -j $IMPALA_BUILD_THREADS docker_images The images need some refinement to be truly useful. The following will be done in future patches: * IMPALA-7947 - integrate with start-impala-cluster.py to automatically create docker network with containers running on it * Mechanism to pass in command-line flags * Mechanisms to update the various config files to point to the docker host rather than "localhost", which doesn't point to the right thing inside the container. * Mechanisms to set mem_limit, JVM heap sizes, etc, automatically. * Mapping /etc/localtime from host Testing: Manually started up the containers connected to a user-defined bridge network, tweaked the configurations to point to the HMS/HDFS/etc running on my host. I then used "docker ps" to figure out the port mappings for beeswax and debug webserver. Confirmed that I could run a query and access debug pages: $ impala-shell.sh -i localhost:32860 -q "select coordinator()" Starting Impala Shell without Kerberos authentication Opened TCP connection to localhost:32860 Connected to localhost:32860 Server version: impalad version 3.1.0-SNAPSHOT DEBUG (build d7870fe03645490f95bd5ffd4a2177f90eb2f3c0) Query: select coordinator() Query submitted at: 2018-12-11 15:51:04 (Coordinator: http://8063e77ce999:25000) Query progress can be monitored at: http://8063e77ce999:25000/query_plan?query_id=1b4d03f0f0f1fcfb:b0b37e5000000000 +---------------+ | coordinator() | +---------------+ | 8063e77ce999 | +---------------+ Fetched 1 row(s) in 0.11s Change-Id: Ifea707aa3cc23e4facda8ac374160c6de23ffc4e --- M .gitignore M CMakeLists.txt M be/src/service/CMakeLists.txt M bin/start-catalogd.sh M bin/start-statestored.sh A docker/CMakeLists.txt M docker/README.md A docker/catalogd/Dockerfile A docker/impala_base/Dockerfile A docker/impalad/Dockerfile A docker/run_with_classpath.sh A docker/setup_build_context.sh A docker/statestored/Dockerfile M fe/CMakeLists.txt 14 files changed, 391 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/12074/4 -- To view, visit http://gerrit.cloudera.org:8080/12074 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ifea707aa3cc23e4facda8ac374160c6de23ffc4e Gerrit-Change-Number: 12074 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Philip Zeyliger <phi...@cloudera.com>
