If it’s not user facing then that is an even strong case for removing it. A lot of the things in the Dockerfile are to support those flags.
> how they can get optimized image This is _precisely_ my point. Either they do `docker pull apache/airflow`, or they learn it themselves. Us carrying that complexity for years to support a few people is not worth it IMO. As far as I can look from my quick scan, many of the features I called out in my list are not used directly in Airflow’s CI (they are exercised in tests of breeze etc, but not used in Airflow’s main CI workflows); The fact that we have tests for a feature is not, by itself, an argument for retaining that feature. I will look at the CI jobs in more detail and build a list of exactly which CI/GHA jobs (i.e. not just the ci.yaml job) consume docker images, and what variants they currently test against. -ash > On 26 Oct 2026, at 12:37, Jarek Potiuk <[email protected]> wrote: > > Also - the various flags of `brreze prod-image` are used to run various > tests in our CI - so by removing those we remove a number of test cases > that are needed for our tests as well. If we want to remove `breeze > prod-build` (as CI /test command) we will have to find out which of those > tests to drop or implement differently. We should conduct a detailed > analysis if we propose that change. > > J. > > > On Mon, Oct 26, 2026 at 1:34 PM Jarek Potiuk <[email protected]> wrote: > >> Actually there is one serious misunderstanding here, Ash. >> >> The `bree prod-image build` is not a user facing feature today. It's only >> used for our CI and tests. >> >> The user facing documentation uses purely `docker build` features and it >> is described here https://airflow.apache.org/docs/docker-stack/build.html >> . Breeze is never mentioned there. And we also have tests covering various >> scenarios of building te imate. >> >> I think it's a useful thing to have and many of our users asked how they >> can get optimized image - and that page describe various scenarios in >> detail. >> >> On Mon, Oct 26, 2026 at 11:59 AM Ash Berlin-Taylor <[email protected]> wrote: >> >>> Hi all, >>> >>> Spawned by the other thread about what version of images we should >>> publish in the 3.4 Docker images, I have a proposal for 3.5: >>> >>> We remove any and all user-facing documentation about using `breeze >>> prod-image build`, and remove all features from that which are not directly >>> used in our own build or test pipelines. We should also remove as much of >>> the complexity from the Dockerfile itself as possible. >>> >>> For downstream users, the supported interface becomes `FROM >>> apache/airflow:…`, not building their own image using our Dockerfile. >>> >>> Right now `breeze prod-image build` has 58 different options, including >>> things like these: >>> >>> ``` >>> │ --airflow-extras Extras to install by default. >>> [default: │ >>> │ --additional-airflow-extras Additional extra package while >>> installing Airflow in the image. (TEXT) │ >>> │ --additional-python-deps Additional python dependencies to use >>> when building the images. (TEXT) │ >>> │ --dev-apt-deps Apt dev dependencies to use when >>> building the images. (TEXT) │ >>> │ --additional-dev-apt-deps Additional apt dev dependencies to >>> use when building the images. (TEXT) │ >>> │ --dev-apt-command Command executed before dev apt deps >>> are installed. (TEXT) │ >>> │ --additional-dev-apt-command Additional command executed before >>> dev apt deps are installed. (TEXT) │ >>> │ --additional-dev-apt-env Additional environment variables set >>> when adding dev dependencies. (TEXT) │ >>> │ --runtime-apt-deps Apt runtime dependencies to use when >>> building the images. (TEXT) │ >>> │ --additional-runtime-apt-deps Additional apt runtime dependencies >>> to use when building the images. (TEXT) │ >>> │ --runtime-apt-command Command executed before runtime apt >>> deps are installed. (TEXT) │ >>> │ --additional-runtime-apt-command Additional command executed before >>> runtime apt deps are installed. (TEXT) │ >>> │ --additional-runtime-apt-env Additional environment variables set >>> when adding runtime dependencies. (TEXT) │ >>> │ --installation-method Install Airflow from: >>> sources (.) or packages (apache-airflow). [default: .] (. │ >>> │ --install-airflow-reference Install Airflow using >>> GitHub tag or branch. (TEXT) │ >>> ``` >>> >>> My reason for suggesting we simplify are: >>> - It is trying to fit all such use case, including letting users build >>> their own image. We should not support that; if you want to customise the >>> image you run: a Dockerfile is right there. >>> - that the entire thing is way too complicated, and has essentially grown >>> the kitchen sink of features. >>> - The Dockerfile is 2526 lines. Each one of those carries a cognitive >>> burden (either you as a human, or extra tokens for your LLM to understand) >>> >>> This is all just too complicated and I have not been happy with this >>> status quo for a long time. Additionally and I have never seen any other >>> project do this. >>> >>> TL;DR: We should maintain the tooling needed to build, test and publish >>> _our_ production images, not a general-purpose framework for building >>> customised Airflow images. That should let us substantially simplify both >>> the Dockerfile and Breeze, without losing anything we actually need for CI >>> or releases. >>> >>> My concrete proposals (not exhaustive: >>> >>> 1. Reduce the Dockerfile to essentially “add uv, install apache-airflow >>> with constraints”. >>> >>> 2. Remove the ability to add airflow extras, or python deps >>> >>> These can be done just as easily by users by doing “FROM >>> apache/airflow; RUN uv pip install apache-airflow[my-extra] some-other-dep”. >>> >>> 3. Similarly, remove all apt envs/commands/deps, as the same FROM+RUN >>> works just as well there. >>> >>> 4. Only use UV for building the image. >>> >>> 5. Remove the `—python-versions` argument. We publish a fixed set of >>> python versions, if users want a different one, then they should build >>> their own image from scratch. >>> >>> 6. Explicitly remove the “Dockerfile must be standalone”, and stop >>> inlining all the scripts. This only has merit if as long as “Dockerfile is >>> meant for users to build the kind of image they want”. If we stop doing >>> that, we remove the need for it being standalone. >>> >>> 7. --installation-method is gone/simplified, and the only options are >>> “published pypi release”, or local dir of .whl files. >>> >>> 8. Everything --additional-* is just removed. >>> >>> 9. --disable-$db-* is removed. Don’t want it: build your own Dockerfile >>> >>> 10. Separate build-time and runtime constraints. >>> We should continue using full constraints to build and test our images, >>> but we shouldn't enforce those constraints when users extend the image, as >>> that would prevent them from upgrading dependencies to pick up security >>> fixes. >>> >>> Instead, we should provide a minimal constraints file that pins only >>> the Airflow distributions installed in the image (`apache-airflow`, >>> `apache-airflow-core`, `apache-airflow-task-sdk`) to their actual installed >>> versions. Users wanting to upgrade Airflow itself should upgrade their base >>> image instead. Everything else should be free to upgrade if the user needs >>> or wants to, subject to package compatibility and uv’s existing rules and >>> checks. >>> >>> 11. Remove the build-time configurable UID (`ARG AIRFLOW_UID`), but keep >>> the runtime arbitrary UID support (needed for OpenShift and a few other >>> cases etc) >>> >>> This is not an exhaustive list, and it is not an all-or-nothing list >>> either, if someone can articulate a good case why we should keep a point. >>> >>> This would remove the ability to use Airflow's Dockerfile as a supported >>> framework for producing size-optimised custom images. Users with those >>> requirements can maintain their own multi-stage Dockerfiles. I don't think >>> Apache Airflow needs to own that general-purpose build framework. >>> >>> -ash >> >> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
