potiuk opened a new pull request #20664:
URL: https://github.com/apache/airflow/pull/20664
The "buildkit" is much more modern docker build mechanism and supports
multiarchitecture builds which makes it suitable for our future ARM
support, it also has nicer UI and much more sophisticated caching
mechanisms as well as supports better multi-segment builds.
BuildKit has been promoted to official for quite a while and it is
rather stable now. Also we can now install BuildKit Plugin to docker
that add capabilities of building and managin cache using dedicated
builders (previously BuildKit cache was managed using rather
complex external tools).
This gives us an opportunity to vastly
simplify our build scripts, because it has now much more robust caching
mechanism than the old docker build (which forced us to pull images
before using them as cache).
We had a lot of complexity involved in efficient caching
but with BuildKit all that can be vastly simplified and we can
get rid of:
* keeping base python images in our registry
* keeping build segments for prod image in our registry
* keeping manifest images in our registry
* deciding when to pull or pull&build image (not needed now, we can
always build image with --cache-from and buildkit will pull cached
layers as needed
* building the image when performing pre-commit (rather than that
we simply encourage users to rebuild the image via breeze command)
* pulling the images before building
* separate 'build' cache kept in our registry (not needed any more
as buildkit allows to keep cache for all segments of multi-segmented
build in a single cache
* the nice animated tty UI of buildkit eliminates the need of manual
spinner
* and a number of other complexities.
Depends on #20238
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
for more information.
In case of fundamental code change, Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in
[UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
