Go cross compilation is as simple as setting the right flag env variables [1], but can be as complicated as requiring a cross compiling GCC instance installed if CGO[2] is necessary. I think we're probably clear on just needing the flag though for the various Boot executables.
For go pipelines we'd need to update the shared runner code to support selecting the cross compiled worker binary environment. I believe it's hard set to amd64 linux at present, but that's a separate issue. [1] https://golangcookbook.com/chapters/running/cross-compiling/ [2] https://golang.org/cmd/cgo/ On Tue, Jan 26, 2021, 10:25 AM Robert Bradshaw <rober...@google.com> wrote: > +1 > > I don't think it would be that hard to build and release arm-based docker > images. (Perhaps just a matter of changing the docker file to depend on a > different base, and doing some cross-compile. That would suss out whether > we're inadvertently taking on any incompatible dependencies.) > > Theoretically, if one does that and manually specifies the container, it > could just work for Python (assuming no wheel files are specified as manual > dependencies). For Java, if one builds/deploys an uberjar (on a different > architecture), there may be issues in any transitive dependency that has > JNI code (us or users). I'd imagine this issue is common to and being > explored by many of the other Java big data systems in use; it'd be > interesting to know what solutions are out there. > > For go, the executable is uploaded directly into the container. We'd > probably have to do something fancier like cross-compiling the executable > (and making sure the UserFn references, which I think are just pointers > into the binary, still work if the launcher is one architecture and the > workers another). > > Definitely worth exploring. > > > > > On Tue, Jan 26, 2021 at 10:09 AM Ismaël Mejía <ieme...@gmail.com> wrote: > >> I stumbled today on this user request: >> BEAM-10982 Wheel support for linux aarch64 >> >> It made me wonder if with the advent of ARM64 processors not only in >> the client but server side (Graviton and others) if it is worth that >> we start to think about having support for this architecture on the >> python installers and in the docker images. It seems that for the >> latter it should not be that difficult given that our parent images >> are already multi-arch. >> >> Are there some possible issues or binary/platform specific >> dependencies that impede us from doing this? >> >
