Repository: metron Updated Branches: refs/heads/master d07833a25 -> 0f8a323dd
METRON-1333 Ansible-Docker can no longer build metron (ottobackwards) closes apache/metron#848 Project: http://git-wip-us.apache.org/repos/asf/metron/repo Commit: http://git-wip-us.apache.org/repos/asf/metron/commit/0f8a323d Tree: http://git-wip-us.apache.org/repos/asf/metron/tree/0f8a323d Diff: http://git-wip-us.apache.org/repos/asf/metron/diff/0f8a323d Branch: refs/heads/master Commit: 0f8a323ddabf93617da514ccba66ea8a408893bd Parents: d07833a Author: ottobackwards <ottobackwa...@gmail.com> Authored: Tue Nov 28 06:49:13 2017 -0500 Committer: otto <o...@apache.org> Committed: Tue Nov 28 06:49:13 2017 -0500 ---------------------------------------------------------------------- README.md | 6 +++ .../packaging/docker/ansible-docker/Dockerfile | 19 +++++++++- .../packaging/docker/ansible-docker/README.md | 39 ++++++++++++++++---- 3 files changed, 54 insertions(+), 10 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/metron/blob/0f8a323d/README.md ---------------------------------------------------------------------- diff --git a/README.md b/README.md index f3b2765..91bef3d 100644 --- a/README.md +++ b/README.md @@ -87,6 +87,7 @@ $ mvn clean install -PHDP-2.5.0.0 You can swap "install" for "package" in the commands above if you don't want to deploy the artifacts to your local .m2 repo. + # Build Metron Reporting To build and run reporting with code coverage: @@ -102,6 +103,11 @@ $ mvn clean install -DskipTests site site:stage-deploy site:deploy The staged site is deployed to /tmp/metron/site/index.html, and can be viewed by opening the file in a browser. +## Building with Docker + +A Docker container with all the required software, with the proper versions, is available to be used as well. +see [ansible-docker](metron-deployment/packaging/docker/ansible-docker) + # Navigating the Architecture Metron is at its core a Kappa architecture with Apache Storm as the processing http://git-wip-us.apache.org/repos/asf/metron/blob/0f8a323d/metron-deployment/packaging/docker/ansible-docker/Dockerfile ---------------------------------------------------------------------- diff --git a/metron-deployment/packaging/docker/ansible-docker/Dockerfile b/metron-deployment/packaging/docker/ansible-docker/Dockerfile index 3fc8de5..ebf002a 100644 --- a/metron-deployment/packaging/docker/ansible-docker/Dockerfile +++ b/metron-deployment/packaging/docker/ansible-docker/Dockerfile @@ -14,13 +14,18 @@ # See the License for the specific language governing permissions and # limitations under the License. # -FROM centos:centos6 +FROM centos:centos6.9 MAINTAINER Apache Metron RUN yum install -y tar RUN yum install -y wget +# base development tools required RUN yum groupinstall -y "Development tools" +# newer cpp 11 support required for building node modules +RUN yum install -y centos-release-scl +RUN yum install -y devtoolset-4-gcc-c++ devtoolset-4-gcc RUN yum install -y zlib-dev openssl-devel sqlite-devel bzip2-devel libffi-devel +# install python 2.7.11 but do not make it the default RUN wget https://www.python.org/ftp/python/2.7.11/Python-2.7.11.tgz -O /usr/src/Python-2.7.11.tgz WORKDIR /usr/src RUN tar xvf Python-2.7.11.tgz @@ -33,18 +38,28 @@ RUN tar xvf setuptools-11.3.tar.gz WORKDIR /usr/src/setuptools-11.3 RUN python2.7 setup.py install RUN easy_install-2.7 pip +# install ansible and set the configuration var RUN pip2.7 install ansible==2.0.0.2 RUN pip2.7 install boto COPY ansible.cfg /root/ ENV ANSIBLE_CONFIG /root/ansible.cfg +# java RUN yum install -y java-1.8.0-openjdk java-1.8.0-openjdk-devel RUN yum install -y which RUN yum install -y nss WORKDIR /usr/src +# setup maven RUN wget http://apache.cs.utah.edu/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz RUN tar xzvf apache-maven-3.3.9-bin.tar.gz RUN mv apache-maven-3.3.9 /opt/maven RUN ln -s /opt/maven/bin/mvn /usr/bin/mvn -RUN yum -y install asciidoc rpm-build rpm2cpio tar unzip xmlto zip rpmlint && yum clean all +# install rpm tools required to build rpms +RUN yum -y install asciidoc rpm-build rpm2cpio tar unzip xmlto zip rpmlint make && yum clean all +# create a .bashrc for root, enabling the cpp 11 toolset +RUN touch /root/.bashrc \ + && cat '/opt/rh/devtoolset-4/enable' >> /root/.bashrc +# install node so that the node dependencies can be packaged into the RPMs +RUN curl --silent --location https://rpm.nodesource.com/setup_6.x | bash - +RUN yum -y install nodejs WORKDIR /root http://git-wip-us.apache.org/repos/asf/metron/blob/0f8a323d/metron-deployment/packaging/docker/ansible-docker/README.md ---------------------------------------------------------------------- diff --git a/metron-deployment/packaging/docker/ansible-docker/README.md b/metron-deployment/packaging/docker/ansible-docker/README.md index 3d50d23..0c41d10 100644 --- a/metron-deployment/packaging/docker/ansible-docker/README.md +++ b/metron-deployment/packaging/docker/ansible-docker/README.md @@ -1,17 +1,40 @@ # Overview -The Metron ansible-docker container is provided in an effort reduce the installation burden of deploying Metron in a live envirionment. -It is provisioned with software required to sucessfully run the deployment scripts. +The Metron ansible-docker container is provided in an effort reduce the installation burden of building Metron. +It may also be used to deploy Metron in a live environment. +It is provisioned with software required to sucessfully build metron and run the deployment scripts. ## Building the Container 1. Install Docker ( https://www.docker.com/products/overview ) 2. Navigate to \<project-directory\>/metron-deployment/packaging/docker/ansible-docker 3. Build the container `docker build -t ansible-docker:2.0.0.2 .` -## Using the Container -Full instructions are found on the wiki at https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65144361 +## Using the Container to build metron +anytime after building the container you can run it with the following command + +`docker run -it -v \<project-directory\>:/root/metron ansible-docker:2.0.0.2 bash` + +If you are going to build metron multiple times, you may want to map the /root/.m2 maven +repo from outside of the container so that you don't start with an empty repo every build and have to download +the world. + +`docker run -it -v \<project-directory\>:/root/metron -v \<your .m2 directory\>:/root/.m2 ansible-docker:2.0.0.2 bash` + +After running the container: + +1. cd /root/metron +2. run build commands, for example: + - build metron without tests : `mvn clean package -DskipTests` + - build metron and build the rpms as well : `mvn clean install && cd metron-deployment && mvn package -P build-rpms` + +If you wish to use this build with a vagrant instance, then after building with rpms as above, modify +your usual vagrant up command to skip the build role, as so: -tl;dr: +`vagrant --ansible-skip-tags="build,quick_dev" up` + + +## Using the Container for deployment + +> Note these instructions are outdated + +Full instructions are found on the wiki at https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65144361 -1. docker run -it -v \<project-directory\>:/root/metron ansible-docker:2.0.0.2 bash -2. cd /root/metron -3. mvn clean package -DskipTests