Copilot commented on code in PR #654:
URL: https://github.com/apache/atlas/pull/654#discussion_r3329662731
##########
dev-support/atlas-docker/config/init_postgres.sh:
##########
@@ -16,21 +16,74 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+set -euo pipefail
-set -e
+: "${POSTGRES_HOST:=atlas-db}"
+: "${POSTGRES_PORT:=5432}"
+: "${POSTGRES_USER:=postgres}"
+: "${POSTGRES_DB:=postgres}"
+: "${POSTGRES_PASSWORD:?POSTGRES_PASSWORD must be set}"
+: "${HIVE_DB_PASSWORD:?HIVE_DB_PASSWORD must be set}"
+: "${ATLAS_DB_PASSWORD:?ATLAS_DB_PASSWORD must be set}"
+: "${ATLAS_SCHEMA_FILE:?ATLAS_SCHEMA_FILE must be set}"
-psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB"
<<-EOSQL
- CREATE USER hive WITH PASSWORD 'atlasR0cks!';
- CREATE DATABASE hive;
- GRANT ALL PRIVILEGES ON DATABASE hive TO hive;
+export PGPASSWORD="${POSTGRES_PASSWORD}"
- CREATE USER atlas WITH PASSWORD 'atlasR0cks!';
- CREATE DATABASE atlas;
- GRANT ALL PRIVILEGES ON DATABASE atlas TO atlas;
+psql_cmd=(
+ psql
+ -v ON_ERROR_STOP=1
+ --host "${POSTGRES_HOST}"
+ --port "${POSTGRES_PORT}"
+ --username "${POSTGRES_USER}"
+ --dbname "${POSTGRES_DB}"
+)
- \c hive
- GRANT ALL ON SCHEMA public TO public;
+atlas_psql_cmd=(
+ psql
+ -v ON_ERROR_STOP=1
+ --host "${POSTGRES_HOST}"
+ --port "${POSTGRES_PORT}"
+ --username atlas
+ --dbname atlas
+)
- \c atlas
- GRANT ALL ON SCHEMA public TO public;
+create_role() {
+ local role_name=$1
+ local role_password=$2
+
+ "${psql_cmd[@]}" <<EOSQL
+DO \$\$
+BEGIN
+ IF NOT EXISTS (SELECT FROM pg_catalog.pg_roles WHERE rolname =
'${role_name}') THEN
+ CREATE ROLE ${role_name} WITH LOGIN PASSWORD '${role_password}';
+ ELSE
+ ALTER ROLE ${role_name} WITH LOGIN PASSWORD '${role_password}';
+ END IF;
+END
+\$\$;
EOSQL
Review Comment:
`create_role()` interpolates `role_name` and especially `role_password`
directly into SQL. If the password contains a single quote (or other special
chars), the script will fail; it also makes the SQL harder to safely reuse. Use
`psql --set` variables plus `format(%I/%L)` in `EXECUTE` to safely quote
identifiers and literals.
##########
dev-support/atlas-docker/docker-compose.atlas-postgres.yml:
##########
@@ -0,0 +1,34 @@
+services:
+ atlas:
+ depends_on:
+ atlas-db-init:
+ condition: service_completed_successfully
+
+ atlas-db-init:
+ image: postgres:13.21
+ container_name: atlas-db-init
+ hostname: atlas-db-init.example.com
+ networks:
+ - atlas
+ depends_on:
+ atlas-backend:
+ condition: service_healthy
+ environment:
+ POSTGRES_HOST: atlas-db
+ POSTGRES_PORT: 5432
+ POSTGRES_USER: postgres
+ POSTGRES_DB: postgres
+ POSTGRES_PASSWORD: atlasR0cks!
+ HIVE_DB_PASSWORD: atlasR0cks!
+ ATLAS_DB_PASSWORD: atlasR0cks!
+ ATLAS_SCHEMA_FILE: /home/atlas/create_schema.sql
+ volumes:
+ - ./config/init_postgres.sh:/home/atlas/init_postgres.sh:ro
+ -
../../graphdb/janusgraph-rdbms/src/main/resources/META-INF/postgres/create_schema.sql:/home/atlas/create_schema.sql:ro
Review Comment:
The official `postgres` image does not provide a `/home/atlas` directory by
default; bind-mounting files into `/home/atlas/...` will fail at container
creation time because the parent directory won’t exist in the container. Mount
these files into a guaranteed-existing directory (e.g. `/tmp`) instead.
##########
dev-support/atlas-docker/README.md:
##########
@@ -31,35 +31,55 @@ Docker files in this folder create docker images and run
them to build Apache At
3. Update environment variables in .env file, if necessary
4. Execute following command to download necessary archives to setup
Atlas/HDFS/HBase/Kafka services:
- ~~~
+
+ ```shell
chmod +x download-archives.sh
./download-archives.sh
- ~~~
+ ```
+
+5. Execute following commands to set environment variables to build Apache
Atlas docker containers:
-5. Execute following commands to set environment variables to build Apache
Atlas docker containers:
- ~~~
+ ```shell
export DOCKER_BUILDKIT=1
export COMPOSE_DOCKER_CLI_BUILD=1
- ~~~
+ ```
6. Build and deploy Apache Atlas in containers using docker compose
- 6.1. Build atlas-base image with the following command:
- docker compose -f docker-compose.atlas-base.yml build
+ Atlas server configuration is mounted from
`config/atlas/${ATLAS_BACKEND}/atlas-application.properties`.
+ The file authentication credentials are mounted from
`config/atlas/users-credentials.properties`.
+
+ 1. Build atlas-base image with the following command:
+
+ ```shell
+ docker compose -f docker-compose.atlas-base.yml build
+ ```
- 6.2. Ensure that the `${HOME}/.m2` directory exists and execute following
command to build Apache Atlas:
- mkdir -p ${HOME}/.m2
- docker compose -f docker-compose.atlas-build.yml up
+ 2. Ensure that the `${HOME}/.m2` directory exists and execute following
command to build Apache Atlas:
+
+ ```shell
+ mkdir -p ${HOME}/.m2
+ docker compose -f docker-compose.atlas-build.yml up
+ ```
Time taken to complete the build might vary (upto an hour), depending on
status of ${HOME}/.m2 directory cache.
- 6.3. To install and start Atlas using Postgres as backend store, execute
following commands:
- export ATLAS_BACKEND=postgres
- docker compose -f docker-compose.atlas.yml up -d --wait
+ 3. To install and start Atlas using Postgres as backend store, execute
following commands:
+
+ ```shell
+ export ATLAS_BACKEND=postgres
+ docker compose -f docker-compose.atlas.yml -f
docker-compose.atlas-postgres.yml up -d --wait
+ ```
+
+ The Postgres overlay runs `config/init_postgres.sh` as a one-shot
initialization service before Atlas starts.
+ This creates the required roles, databases, and Atlas RDBMS schema.
+
+ 4. To install and start Atlas using HBase as backend store, execute
following commands:
- 6.4. To install and start Atlas using HBase as backend store, execute
following commands:
- export ATLAS_BACKEND=hbase
- docker compose -f docker-compose.atlas.yml -f
docker-compose.atlas-hadoop.yml up -d --wait
+ ```shell
+ export ATLAS_BACKEND=hbase
+ docker compose -f docker-compose.atlas.yml -f
docker-compose.atlas-hadoop.yml up -d --wait
+ ```
Apache Atlas will be installed at /opt/atlas/, and logs are at
/var/logs/atlas directory.
Review Comment:
The log directory path here doesn’t match what the Docker image sets up: the
Dockerfiles create `/var/log/atlas` (no trailing `s`) and symlink
`${ATLAS_HOME}/logs` there. Update this path to avoid sending users to a
non-existent directory.
##########
dev-support/atlas-docker/docker-compose.atlas-postgres.yml:
##########
@@ -0,0 +1,34 @@
+services:
+ atlas:
+ depends_on:
+ atlas-db-init:
+ condition: service_completed_successfully
+
+ atlas-db-init:
+ image: postgres:13.21
+ container_name: atlas-db-init
+ hostname: atlas-db-init.example.com
+ networks:
+ - atlas
+ depends_on:
+ atlas-backend:
+ condition: service_healthy
+ environment:
+ POSTGRES_HOST: atlas-db
+ POSTGRES_PORT: 5432
+ POSTGRES_USER: postgres
+ POSTGRES_DB: postgres
+ POSTGRES_PASSWORD: atlasR0cks!
+ HIVE_DB_PASSWORD: atlasR0cks!
+ ATLAS_DB_PASSWORD: atlasR0cks!
+ ATLAS_SCHEMA_FILE: /home/atlas/create_schema.sql
+ volumes:
+ - ./config/init_postgres.sh:/home/atlas/init_postgres.sh:ro
+ -
../../graphdb/janusgraph-rdbms/src/main/resources/META-INF/postgres/create_schema.sql:/home/atlas/create_schema.sql:ro
+ command:
+ - /bin/bash
+ - /home/atlas/init_postgres.sh
Review Comment:
The init `command` references `/home/atlas/init_postgres.sh`, which won’t
exist if the script is mounted into a different directory (and `/home/atlas`
typically doesn’t exist in the `postgres` image). Update the command to match
the mount target (e.g. `/tmp/init_postgres.sh`).
##########
dev-support/atlas-docker/docker-compose.atlas-postgres.yml:
##########
@@ -0,0 +1,34 @@
+services:
+ atlas:
+ depends_on:
+ atlas-db-init:
+ condition: service_completed_successfully
+
+ atlas-db-init:
+ image: postgres:13.21
+ container_name: atlas-db-init
+ hostname: atlas-db-init.example.com
+ networks:
+ - atlas
+ depends_on:
+ atlas-backend:
+ condition: service_healthy
+ environment:
+ POSTGRES_HOST: atlas-db
+ POSTGRES_PORT: 5432
+ POSTGRES_USER: postgres
+ POSTGRES_DB: postgres
+ POSTGRES_PASSWORD: atlasR0cks!
+ HIVE_DB_PASSWORD: atlasR0cks!
+ ATLAS_DB_PASSWORD: atlasR0cks!
+ ATLAS_SCHEMA_FILE: /home/atlas/create_schema.sql
Review Comment:
`ATLAS_SCHEMA_FILE` points to `/home/atlas/create_schema.sql`, but the
`postgres` image doesn’t have `/home/atlas` by default. Point this to the same
guaranteed-existing mount point used for the schema file (e.g.
`/tmp/create_schema.sql`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
