ashb commented on a change in pull request #6266: [AIRFLOW-2439] Production Docker image support including refactoring of build scripts - depends on [AIRFLOW-5704] URL: https://github.com/apache/airflow/pull/6266#discussion_r336916999
########## File path: Dockerfile ########## @@ -334,56 +380,52 @@ COPY --chown=airflow:airflow airflow/version.py ${AIRFLOW_SOURCES}/airflow/versi COPY --chown=airflow:airflow airflow/__init__.py ${AIRFLOW_SOURCES}/airflow/__init__.py COPY --chown=airflow:airflow airflow/bin/airflow ${AIRFLOW_SOURCES}/airflow/bin/airflow -# The goal of this line is to install the dependencies from the most current setup.py from sources -# This will be usually incremental small set of packages in CI optimized build, so it will be very fast -# In non-CI optimized build this will install all dependencies before installing sources. -RUN pip install -e ".[${AIRFLOW_EXTRAS}]" - +# Setting to 1 speeds up building the image. Cassandra driver without CYTHON saves around 10 minutes +# But might not be suitable for production image +ENV CASS_DRIVER_NO_CYTHON="" +ENV CASS_DRIVER_BUILD_CONCURRENCY="8" -WORKDIR ${AIRFLOW_SOURCES}/airflow/www - -# Copy all www files here so that we can run npm building for production -COPY --chown=airflow:airflow airflow/www/ ${AIRFLOW_SOURCES}/airflow/www/ +ENV PATH="/home/airflow/.local/bin:/home/airflow:${PATH}" -# Package NPM for production -RUN gosu ${AIRFLOW_USER} npm run prod +# The goal of this line is to install the dependencies from the most current setup.py from sources +# This will be usually incremental small set of packages in CI optimized build, so it will be very fast +# For production optimised build it is the first time dependencies are installed so it will be slower +RUN pip install --user ".[${AIRFLOW_PROD_EXTRAS}]" \ + && pip uninstall --yes apache-airflow snakebite # Cache for this line will be automatically invalidated if any # of airflow sources change COPY --chown=airflow:airflow . ${AIRFLOW_SOURCES}/ -WORKDIR ${AIRFLOW_SOURCES} - -# Finally install the requirements from the latest sources -RUN pip install -e ".[${AIRFLOW_EXTRAS}]" +# Reinstall airflow again - this time with sources and remove the sources after installation +RUN pip install --user ".[${AIRFLOW_PROD_EXTRAS}]" Review comment: I don't see where we remove the sources? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services