Oops, the container link got mangled, it was supposed to be < https://github.com/apache/trafficcontrol/pkgs/container/trafficcontrol/ci/trafficserver-alpine >.
On Tue, Dec 6, 2022 at 9:38 AM Zach Hoffman <zrhoff...@apache.org> wrote: > in Traffic Control, we build an Alpine image < > https://github.com/apache/trafficcontrol/pkgs/container/trafficcontrol%2Fci%2Ftrafficserver-alpine>, > with Traffic Server baked into it, for amd64 and arm64 . Rather than > building them both from the same machine, the amd64 and arm64 images in > separate GitHub Actions jobs, then combined using `docker manifest` in a > third GitHub Actions job < > https://github.com/apache/trafficcontrol/blob/master/.github/workflows/container-trafficserver-alpine.yml > >. > > The arm64 part still takes much longer (over 2 hours, compare to 10 > minutes for amd64), but this approach eliminates the need to build for more > than one platform at a time, so if the arm64 job were to run on an arm64 > runner in the future, that wouldn't need to further complicate the other > jobs in order to take advantage of that speedup. > > -Zach > > On Tue, Dec 6, 2022 at 9:02 AM Jarek Potiuk <ja...@potiuk.com> wrote: > >> In Airflow we have a bit more complex setting (we are building 2x5x2 >> different images and they are different sets of them for different >> branches), Building images for Airflow takes quite some time (installing >> many dependencies) so qemu was out of the question (several hours to build >> single image). Unfortunately qemu is ~ 15 times (!) slower than hardware >> builds from our experience. Good enough for small images, but unacceptable >> if your image normally takes time >> >> The built-in actions like the one mentioned by Jacob have the limitations >> that they depende on qemu - I have not yet found an easy "Action" that >> could utilise AMD and ARM hardware together. But maybe there are other >> ways. >> >> We developed our own scripting and tooling: >> >> * we have self-hosted runners in GitHub and we only use those to build >> images (with Astronomer money and Amazon-sponsored credits on AWS) >> * for the time when we build multi-platform build we start ARM instances >> (they have built-in timed auto-kill-switch so that they are not run >> indefinitely and take resources) >> * we configured ssh-forwarded docker port from our AMD instance to forward >> docker socket >> * we configured two builder with buildx (local for AMD and the forwarded >> one for ARM) >> * we run multi-platform buildx build to use both builders. >> * we have our own Python build/development environment (breeze) that wraps >> the docker commands that are executed so the "code" doing it is not easy >> to >> copy elsewhere sometimes, but I copied below the actual "meat" what >> happens >> under the hood. >> * we run it as part of our Github Action workflows >> >> Examples: >> >> * Starting ARM instance and configuring builders: >> https://github.com/apache/airflow/actions/runs/3603432288/jobs/6080260757 >> >> This is the relevant part of establishing the worker for buildx (I omit >> starting the instance as it is amazon-specific): >> >> export AUTOSSH_LOGFILE="${WORKING_DIR}/autossh.log" >> autossh -f "-L12357:/var/run/docker.sock" \ >> -N -o "IdentitiesOnly=yes" -o "StrictHostKeyChecking=no" \ >> -i "${WORKING_DIR}/my_key" "${EC2_USER}@${INSTANCE_PRIVATE_DNS_NAME}" >> >> bash -c 'echo -n "Waiting port 12357 .."; for _ in `seq 1 40`; do echo -n >> .; sleep 0.25; nc -z localhost 12357 && echo " Open." && exit ; done; echo >> " Timeout!" >&2; exit 1' >> >> docker buildx rm --force airflow_cache || true >> docker buildx create --name airflow_cache >> docker buildx create --name airflow_cache --append localhost:12357 >> >> * Releasing image: >> https://github.com/apache/airflow/actions/runs/3603432288/jobs/6080260776 >> >> This is the command that is run under-the-hood >> >> docker buildx build --builder airflow_cache --build-arg >> PYTHON_BASE_IMAGE=python:3.10-slim-bullseye --build-arg >> AIRFLOW_VERSION=2.5.0 --platform linux/amd64,linux/arm64 . -t >> apache/airflow:2.5.0-python3.10 --push >> >> J. >> >> >> >> On Tue, Dec 6, 2022 at 2:10 PM Jacob Wujciak >> <ja...@voltrondata.com.invalid> >> wrote: >> >> > Hello Robert, >> > >> > I would suggest using GItHub Actions, there you can use the official >> suite >> > of docker actions to build multiplatform images with little need for >> custom >> > scripting [1]. >> > Feel free to ping me in the ASF slack. >> > >> > Best >> > Jacob >> > >> > [1]: https://github.com/docker/build-push-action >> > >> > On Tue, Dec 6, 2022 at 1:43 PM Robert Munteanu <romb...@apache.org> >> wrote: >> > >> > > Hi, >> > > >> > > We had a user report that our official Docker image does not support >> > > architectures other than AMD64 [1]. M1 Macs and Raspberry Pis can't >> run >> > > the image with our current setup. >> > > >> > > On our side, we set up automated builds on Docker Hub using automated >> > > builds. Unforunately, Docker Hub autobuilds don't support `docker >> > > buildx` or another form of multi-arch builds. It is not the roadmap >> > > though [2], but here is no guarantee on when (or if) it will become >> > > available. >> > > >> > > I see two alternatives so far: >> > > >> > > 1. Moving to GitHub actions >> > > 2. Use hooks to install qemu and 'fake' a multi-arch build on Docker >> > > Hub >> > > >> > > To be honest, none is too appealing to me, we have a simple process >> > > that works. >> > > >> > > How are other projects handling this? Or does anyone have any ideas >> > > that they can share? >> > > >> > > Thanks, >> > > Robert >> > > >> > > [1]: https://issues.apache.org/jira/browse/SLING-11714 >> > > [2]: https://github.com/docker/roadmap/issues/109 >> > > >> > >> >
