This is an automated email from the ASF dual-hosted git repository. abhay pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/ranger.git
The following commit(s) were added to refs/heads/master by this push: new 9181c20c2 RANGER:3777 Execute permissions required in init scripts to run containers 9181c20c2 is described below commit 9181c20c2b461ab413bad0c485d99b518352d107 Author: Abhishek Kumar <abhishekkumar100...@gmail.com> AuthorDate: Tue May 31 08:57:39 2022 -0700 RANGER:3777 Execute permissions required in init scripts to run containers --- dev-support/ranger-docker/Dockerfile.ranger | 2 +- dev-support/ranger-docker/Dockerfile.ranger-hadoop | 4 ++- dev-support/ranger-docker/Dockerfile.ranger-hbase | 3 +- dev-support/ranger-docker/Dockerfile.ranger-hive | 3 +- dev-support/ranger-docker/Dockerfile.ranger-kafka | 3 +- dev-support/ranger-docker/Dockerfile.ranger-knox | 3 +- .../ranger-docker/Dockerfile.ranger-tagsync | 3 +- .../ranger-docker/Dockerfile.ranger-usersync | 3 +- dev-support/ranger-docker/README.md | 34 +++++++++++++++------- 9 files changed, 39 insertions(+), 19 deletions(-) diff --git a/dev-support/ranger-docker/Dockerfile.ranger b/dev-support/ranger-docker/Dockerfile.ranger index f5a1ed93f..b050b13c4 100644 --- a/dev-support/ranger-docker/Dockerfile.ranger +++ b/dev-support/ranger-docker/Dockerfile.ranger @@ -33,7 +33,7 @@ RUN tar xvfz /home/ranger/dist/ranger-${RANGER_VERSION}-admin.tar.gz --direct && cp -f ${RANGER_SCRIPTS}/ranger-admin-install.properties ${RANGER_HOME}/admin/install.properties \ && mkdir -p /var/run/ranger \ && mkdir -p /var/log/ranger \ - && chown -R ranger:ranger ${RANGER_HOME}/admin/ /var/run/ranger/ /var/log/ranger/ \ + && chown -R ranger:ranger ${RANGER_HOME}/admin/ ${RANGER_SCRIPTS}/ /var/run/ranger/ /var/log/ranger/ \ && mkdir -p /usr/share/java/ FROM ranger AS ranger_postgres diff --git a/dev-support/ranger-docker/Dockerfile.ranger-hadoop b/dev-support/ranger-docker/Dockerfile.ranger-hadoop index f25bc0d8d..9970c7cb3 100644 --- a/dev-support/ranger-docker/Dockerfile.ranger-hadoop +++ b/dev-support/ranger-docker/Dockerfile.ranger-hadoop @@ -41,7 +41,9 @@ RUN tar xvfz /home/ranger/dist/hadoop-${HADOOP_VERSION}.tar.gz --directory=/opt/ tar xvfz /home/ranger/dist/ranger-${RANGER_VERSION}-yarn-plugin.tar.gz --directory=/opt/ranger && \ ln -s /opt/ranger/ranger-${RANGER_VERSION}-yarn-plugin /opt/ranger/ranger-yarn-plugin && \ rm -f /home/ranger/dist/ranger-${RANGER_VERSION}-yarn-plugin.tar.gz && \ - cp -f /home/ranger/scripts/ranger-yarn-plugin-install.properties /opt/ranger/ranger-yarn-plugin/install.properties + cp -f /home/ranger/scripts/ranger-yarn-plugin-install.properties /opt/ranger/ranger-yarn-plugin/install.properties && \ + chmod 744 ${RANGER_SCRIPTS}/ranger-hadoop-setup.sh ${RANGER_SCRIPTS}/ranger-hadoop.sh ${RANGER_SCRIPTS}/ranger-hadoop-mkdir.sh && \ + chown hdfs:hadoop ${RANGER_SCRIPTS}/ranger-hadoop-mkdir.sh ENV HADOOP_HOME /opt/hadoop ENV HADOOP_CONF_DIR /opt/hadoop/etc/hadoop diff --git a/dev-support/ranger-docker/Dockerfile.ranger-hbase b/dev-support/ranger-docker/Dockerfile.ranger-hbase index 5a2f056a9..1a2eea461 100644 --- a/dev-support/ranger-docker/Dockerfile.ranger-hbase +++ b/dev-support/ranger-docker/Dockerfile.ranger-hbase @@ -35,7 +35,8 @@ RUN tar xvfz /home/ranger/dist/hbase-${HBASE_VERSION}-bin.tar.gz --directory=/op tar xvfz /home/ranger/dist/ranger-${RANGER_VERSION}-hbase-plugin.tar.gz --directory=/opt/ranger && \ ln -s /opt/ranger/ranger-${RANGER_VERSION}-hbase-plugin /opt/ranger/ranger-hbase-plugin && \ rm -f /home/ranger/dist/ranger-${RANGER_VERSION}-hbase-plugin.tar.gz && \ - cp -f /home/ranger/scripts/ranger-hbase-plugin-install.properties /opt/ranger/ranger-hbase-plugin/install.properties + cp -f /home/ranger/scripts/ranger-hbase-plugin-install.properties /opt/ranger/ranger-hbase-plugin/install.properties && \ + chmod 744 ${RANGER_SCRIPTS}/ranger-hbase-setup.sh ${RANGER_SCRIPTS}/ranger-hbase.sh ENV HBASE_HOME /opt/hbase ENV PATH /usr/java/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/hbase/bin diff --git a/dev-support/ranger-docker/Dockerfile.ranger-hive b/dev-support/ranger-docker/Dockerfile.ranger-hive index 9ef89b59a..fc09fdc38 100644 --- a/dev-support/ranger-docker/Dockerfile.ranger-hive +++ b/dev-support/ranger-docker/Dockerfile.ranger-hive @@ -43,7 +43,8 @@ RUN tar xvfz /home/ranger/dist/apache-hive-${HIVE_VERSION}-bin.tar.gz --director tar xvfz /home/ranger/dist/ranger-${RANGER_VERSION}-hive-plugin.tar.gz --directory=/opt/ranger && \ ln -s /opt/ranger/ranger-${RANGER_VERSION}-hive-plugin /opt/ranger/ranger-hive-plugin && \ rm -f /home/ranger/dist/ranger-${RANGER_VERSION}-hive-plugin.tar.gz && \ - cp -f /home/ranger/scripts/ranger-hive-plugin-install.properties /opt/ranger/ranger-hive-plugin/install.properties + cp -f /home/ranger/scripts/ranger-hive-plugin-install.properties /opt/ranger/ranger-hive-plugin/install.properties && \ + chmod 744 ${RANGER_SCRIPTS}/ranger-hive-setup.sh ${RANGER_SCRIPTS}/ranger-hive.sh ENV HIVE_HOME /opt/hive ENV HADOOP_HOME /opt/hadoop diff --git a/dev-support/ranger-docker/Dockerfile.ranger-kafka b/dev-support/ranger-docker/Dockerfile.ranger-kafka index 83c41b277..24269b6cd 100644 --- a/dev-support/ranger-docker/Dockerfile.ranger-kafka +++ b/dev-support/ranger-docker/Dockerfile.ranger-kafka @@ -34,7 +34,8 @@ RUN tar xvfz /home/ranger/dist/kafka_2.12-${KAFKA_VERSION}.tgz --directory=/opt/ tar xvfz /home/ranger/dist/ranger-${RANGER_VERSION}-kafka-plugin.tar.gz --directory=/opt/ranger && \ ln -s /opt/ranger/ranger-${RANGER_VERSION}-kafka-plugin /opt/ranger/ranger-kafka-plugin && \ rm -f /home/ranger/dist/ranger-${RANGER_VERSION}-kafka-plugin.tar.gz && \ - cp -f /home/ranger/scripts/ranger-kafka-plugin-install.properties /opt/ranger/ranger-kafka-plugin/install.properties + cp -f /home/ranger/scripts/ranger-kafka-plugin-install.properties /opt/ranger/ranger-kafka-plugin/install.properties && \ + chmod 744 ${RANGER_SCRIPTS}/ranger-kafka-setup.sh ${RANGER_SCRIPTS}/ranger-kafka.sh ENV KAFKA_HOME /opt/kafka ENV PATH /usr/java/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/kafka/bin diff --git a/dev-support/ranger-docker/Dockerfile.ranger-knox b/dev-support/ranger-docker/Dockerfile.ranger-knox index bc78af9cc..df4671149 100644 --- a/dev-support/ranger-docker/Dockerfile.ranger-knox +++ b/dev-support/ranger-docker/Dockerfile.ranger-knox @@ -38,7 +38,8 @@ RUN apt-get update && apt-get install -y expect && \ ln -s /opt/ranger/ranger-${RANGER_VERSION}-knox-plugin /opt/ranger/ranger-knox-plugin && \ rm -f /home/ranger/dist/ranger-${RANGER_VERSION}-knox-plugin.tar.gz && \ cp -f /home/ranger/scripts/ranger-knox-plugin-install.properties /opt/ranger/ranger-knox-plugin/install.properties && \ - cp -f /home/ranger/scripts/ranger-knox-sandbox.xml /opt/knox/conf/topologies/sandbox.xml + cp -f /home/ranger/scripts/ranger-knox-sandbox.xml /opt/knox/conf/topologies/sandbox.xml && \ + chmod 744 ${RANGER_SCRIPTS}/ranger-knox-setup.sh ${RANGER_SCRIPTS}/ranger-knox.sh ${RANGER_SCRIPTS}/ranger-knox-expect.sh ENV KNOX_HOME /opt/knox ENV PATH /usr/java/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/knox/bin diff --git a/dev-support/ranger-docker/Dockerfile.ranger-tagsync b/dev-support/ranger-docker/Dockerfile.ranger-tagsync index 28dadd424..1c9ad67da 100644 --- a/dev-support/ranger-docker/Dockerfile.ranger-tagsync +++ b/dev-support/ranger-docker/Dockerfile.ranger-tagsync @@ -41,7 +41,8 @@ RUN tar xvfz /home/ranger/dist/ranger-${RANGER_VERSION}-tagsync.tar.gz --directo ln -s /etc/init.d/ranger-tagsync /etc/rc3.d/S99ranger-tagsync && \ ln -s /etc/init.d/ranger-tagsync /etc/rc3.d/K00ranger-tagsync && \ ln -s ${RANGER_HOME}/tagsync/ranger-tagsync-services.sh /usr/bin/ranger-tagsync-services.sh && \ - chown -R ranger:ranger ${RANGER_HOME}/tagsync/ /var/run/ranger/ /var/log/ranger/ /etc/ranger /etc/init.d/ranger-tagsync + chown -R ranger:ranger ${RANGER_HOME}/tagsync/ ${RANGER_SCRIPTS}/ /var/run/ranger/ /var/log/ranger/ /etc/ranger /etc/init.d/ranger-tagsync && \ + chmod 744 ${RANGER_SCRIPTS}/ranger-tagsync.sh USER ranger diff --git a/dev-support/ranger-docker/Dockerfile.ranger-usersync b/dev-support/ranger-docker/Dockerfile.ranger-usersync index ba7859ede..c29e04c24 100644 --- a/dev-support/ranger-docker/Dockerfile.ranger-usersync +++ b/dev-support/ranger-docker/Dockerfile.ranger-usersync @@ -38,7 +38,8 @@ RUN tar xvfz /home/ranger/dist/ranger-${RANGER_VERSION}-usersync.tar.gz --direct ln -s /etc/init.d/ranger-usersync /etc/rc3.d/S99ranger-usersync && \ ln -s /etc/init.d/ranger-usersync /etc/rc3.d/K00ranger-usersync && \ ln -s ${RANGER_HOME}/usersync/ranger-usersync-services.sh /usr/bin/ranger-usersync && \ - chown -R ranger:ranger ${RANGER_HOME}/usersync/ /var/run/ranger/ /var/log/ranger/ /etc/ranger /etc/init.d/ranger-usersync + chown -R ranger:ranger ${RANGER_HOME}/usersync/ ${RANGER_SCRIPTS}/ /var/run/ranger/ /var/log/ranger/ /etc/ranger /etc/init.d/ranger-usersync && \ + chmod 744 ${RANGER_SCRIPTS}/ranger-usersync.sh USER ranger diff --git a/dev-support/ranger-docker/README.md b/dev-support/ranger-docker/README.md index 248ab1fc9..e18b0e7ff 100644 --- a/dev-support/ranger-docker/README.md +++ b/dev-support/ranger-docker/README.md @@ -26,30 +26,42 @@ Docker files in this folder create docker images and run them to build Apache Ra 1. Ensure that you have recent version of Docker installed from [docker.io](http://www.docker.io) (as of this writing: Engine 20.10.5, Compose 1.28.5). Make sure to configure docker with at least 6gb of memory. -2. Set this folder as your working directory. +2. Update environment variables in ```.env``` file, if necessary -3. Update environment variables in .env file, if necessary +3. Set ```dev-support/ranger-docker``` as your working directory. 4. Execute following command to download necessary archives to setup Ranger/HDFS/Hive/HBase/Kafka/Knox services: - ./download-archives.sh -5. Build and deploy Apache Ranger in containers using docker-compose + ```chmod +x download-archives.sh && ./download-archives.sh``` - 5.1. Execute following command to build Apache Ranger: +5. Build Apache Ranger in containers using docker-compose - docker-compose -f docker-compose.ranger-base.yml -f docker-compose.ranger-build.yml up + 1. Execute following command to build Apache Ranger: - Time taken to complete the build might vary (upto an hour), depending on status of ${HOME}/.m2 directory cache. + ```docker-compose -f docker-compose.ranger-base.yml -f docker-compose.ranger-build.yml up``` - 5.2. Execute following command to start Ranger, Ranger enabled HDFS/YARN/HBase/Hive/Kafka/Knox and dependent services (Solr, DB) in containers: + Time taken to complete the build might vary (upto an hour), depending on status of ${HOME}/.m2 directory cache. + + 2. Alternatively, the following commands can be executed from the parent directory + 1. To generate tarballs:```mvn clean package -DskipTests``` + + 2. Copy the tarballs and version file to ```dev-support/ranger-docker/dist``` + ~~~ + cp target/ranger-* dev-support/ranger-docker/dist/ + cp target/version dev-support/ranger-docker/dist/ + ~~~ + 3. Build the ranger-base image: + ```docker-compose -f docker-compose.ranger-base.yml build --no-cache``` + +6. Execute following command to start Ranger, Ranger enabled HDFS/YARN/HBase/Hive/Kafka/Knox and dependent services (Solr, DB) in containers: export RANGER_DB_TYPE=postgres docker-compose -f docker-compose.ranger-base.yml -f docker-compose.ranger.yml -f docker-compose.ranger-${RANGER_DB_TYPE}.yml -f docker-compose.ranger-usersync.yml -f docker-compose.ranger-tagsync.yml -f docker-compose.ranger-hadoop.yml -f docker-compose.ranger-hbase.yml -f docker-compose.ranger-kafka.yml -f docker-compose.ranger-hive.yml -f docker-compose.ranger-knox.yml up -d - valid values for RANGER_DB_TYPE: mysql or postgres - - 5.3. To rebuild specific images and start containers with the new image, use following command: + +7. To rebuild specific images and start containers with the new image, use following command: docker-compose -f docker-compose.ranger-base.yml -f docker-compose.ranger.yml -f docker-compose.ranger-usersync.yml -f docker-compose.ranger-tagsync.yml -f docker-compose.ranger-hadoop.yml -f docker-compose.ranger-hbase.yml -f docker-compose.ranger-kafka.yml -f docker-compose.ranger-hive.yml -f docker-compose.ranger-knox.yml up -d --no-deps --force-recreate --build <service-1> <service-2> -6. Ranger Admin can be accessed at http://localhost:6080 (admin/rangerR0cks!) +8. Ranger Admin can be accessed at http://localhost:6080 (admin/rangerR0cks!)