potiuk commented on PR #53770: URL: https://github.com/apache/airflow/pull/53770#issuecomment-3218006683
Hey @aritra24 -> I finally got a deep look and added some fixups. The main reason things were not working well was that there were no symbolic links in /usr/local/bin/ in the "main" image (and airflow has been installed using those). However I noticed that the size of the image (specifically python layers) is quite a bit bigger: 200 MB instead of ~50 MB in the original image. And I started to look a bit closer how the 'official` python image is built - and I copied over the things i found there: https://github.com/docker-library/python/blob/master/Dockerfile-linux.template - a bit more flags added to compilation and some cleanup instructions I realized that instead of checking out sources from repo, it's much better to - similarly as in the original image - we can download released python packages (and verify them with checksums and keys of the release managers - similarly as we do with our packages). Also since 3.0.5 has been yanked, I switched all the base airflow versions to 3.0.4 (we will change it to 3.0.6 once we release it): So my fixup contains a number of changes: * 3.0.5 -> 3.0.6 airflow version change * I used released official packages rather than git repo to install Python * I added wget (it was added in the original image to pull python packages so I added it for compatibility) * I added those flags to python build (same as in the original build): ``` --with-ensurepip --build="$gnuArch" \ --enable-loadable-sqlite-extensions --enable-option-checking=fatal --enable-shared --with-lto ``` * added cleanup of apt after installing packages * added removal of .pyc/.test etc. files (this is the main reason why we had 200 MB instead of 49MB) * added relinking of symbolic links from /usr/python/bin to /usr/local/bin in the "main" image as well as in the build image. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
