o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1367561202
########## airflow/providers/amazon/aws/executors/ecs/Dockerfile: ########## @@ -0,0 +1,86 @@ +# hadolint ignore=DL3007 +FROM apache/airflow:latest +USER root +RUN apt-get update \ + && apt-get install -y --no-install-recommends unzip \ + # The below helps to keep the image size down + && apt-get clean \ + && rm -rf /var/lib/apt/lists/* +RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip"; -o "awscliv2.zip" +RUN unzip awscliv2.zip && ./aws/install + +# Add a script to run the aws s3 sync command when the container is run +COPY <<"EOF" /entrypoint.sh +#!/bin/bash + +echo "Downloading DAGs from S3 bucket" +aws s3 sync "$S3_URL" "$CONTAINER_DAG_PATH" + +exec "$@" +EOF + +RUN chmod +x /entrypoint.sh + +USER airflow + +## Installing Python Dependencies +# Python dependencies can be installed by providing a requirements.txt. +# If the file is in a different location, use the requirements_path build argument to specify +# the file path. +ARG requirements_path=./requirements.txt +ENV REQUIREMENTS_PATH=$requirements_path + +# Uncomment the two lines below to copy the requirements.txt file to the container, and +# install the dependencies. +# COPY --chown=airflow:root $REQUIREMENTS_PATH /opt/airflow/requirements.txt +# RUN pip install --no-cache-dir -r /opt/airflow/requirements.txt + + +## AWS Authentication +# The image requires access to AWS services. This Dockerfile supports 2 ways to authenticate with AWS. +# The first is using build arguments where you can provide the AWS credentials as arguments +# passed when building the image. The other option is to copy the ~/.aws folder to the container, +# and authenticate using the credentials in that folder. +# If you would like to use an alternative method of authentication, feel free to make the +# necessary changes to this file. + +# Use these arguments to provide AWS authentication information +ARG aws_access_key_id +ARG aws_secret_access_key +ARG aws_default_region +ARG aws_session_token + +ENV AWS_ACCESS_KEY_ID=$aws_access_key_id Review Comment: We've documented and tested using ECS task roles and advocate that users make use of this mechanism as the most secure approach and we have bumped providing credentials as the backup approach and note in the documentation that it should not be used for production purposes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
