potiuk commented on pull request #22258: URL: https://github.com/apache/airflow/pull/22258#issuecomment-1073838947
This is an optimization (10x speedup) of building our mutliplatform images on CI as cache for other builds. I run it manually on my Macbook so far because of the time it took to build with emulation, but with this one the maximum total time of building the multiplatform images (both CI and PROD) on our CI is down to ~25 minutes from > 3 hours. Completed, succesful run (I limited it and removed the regular tests there) can be found here: https://github.com/apache/airflow/runs/5626562720?check_suite_focus=true With this approach we can actually even start thinking about running selected tests on ARM in "main" without actually spinning an ARM runner - that would be a nice step to follow but it is not implemented yet). Github Actions are also close to supporting ARM public runners it seems, so we can eventually get it running for both self-hosted and public runners. I added several protections against spinnig up too many instances (including self-termination after 50 minutes) , but I will keep watching if we are not running too many of those once we merge it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
