potiuk commented on code in PR #37134:
URL: https://github.com/apache/airflow/pull/37134#discussion_r1475865072
##########
scripts/ci/images/ci_start_arm_instance_and_connect_to_docker.sh:
##########
@@ -20,7 +20,7 @@ SCRIPTS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd)"
WORKING_DIR="/tmp/armdocker"
INSTANCE_INFO="${WORKING_DIR}/instance_info.json"
ARM_AMI="ami-0e43196369d299715" # AMI ID of latest arm-docker-ami-v*
Review Comment:
The 16x times I last saw (you can was mostly for CI image builds - which
have much higher proportion of what needed to be compiled comparing to a number
of downloads and it was done at the time when there were many more packages
that needed to be compiled on top - many of them, when we first switched to
ARM. I think that 16x was what I saw at that time as the raw "emulator"
overhead in some of those
It's likely heavily outdated and maybe they were in some specific
circumstances. Last time I looked at it was like 1.5 years ago. So yyeah 4x/5x
is likely good estimate for PROD images.
But also the ARM instances are not 10x more expensive. This is just the max
price we are willing to pay. Usually the spot price is way lower than that -
last time I checked the AMD/ARM prices were comparable (certainly not 10x) and
we are talking about total $15(!) USD / month for all the time of ARM instances
running and that. So saving few USD is proably not worth even thinking about
it :D.
The good thing about using ARM instances for that is that we can build the
multi-platform images easily by running the two in parallel
https://github.com/apache/airflow/actions/runs/7739332335/job/21102024869#step:11:7730
- building both ARM and AMD image using buildx multi-builder capabilities
takes 5-8 minutes, depends on Python version. And in this case the "elapsed
time" trumps the cost (but I still think that even if it is 4x longer it's
still far cheaper to use the ARM spot instances for that).
But -- of course - maybe you want to look at the setup and try to optimize
it :).
In the future we could (and this is something if we would like to use public
ARM instance) much complicate the build process and build images for AMD and
ARM in separate jobs and combine them as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
