I upgraded the docker version on Jenkins workers and the tests passed. (also installed Python 3.11 so we are ready for that)
On Tue, Feb 14, 2023 at 3:21 PM Kenneth Knowles <k...@apache.org> wrote: > SGTM. I asked on the PR if this could impact users, but having read the > docker release calendar I am not concerned. The last update to the old > version was in 2019, and the introduction of compatible versions was 2020. > > On Tue, Feb 14, 2023 at 3:01 PM Byron Ellis via user <user@beam.apache.org> > wrote: > >> FWIW I am Team Upgrade Docker :-) >> >> On Tue, Feb 14, 2023 at 2:53 PM Luke Cwik via user <user@beam.apache.org> >> wrote: >> >>> I made some progress in testing the container and did hit an issue where >>> Ubuntu 22.04 "Jammy" is dependent on the version of Docker installed. It >>> turns out that our boot.go crashes with "runtime/cgo: pthread_create >>> failed: Operation not permitted" because the Ubuntu 22.04 is using new >>> syscalls that Docker 18.09.4 doesn't have a seccomp policy for (and uses a >>> default of deny). We have a couple of choices here: >>> 1) upgrade the version of docker on Jenkins and require users to >>> similarly use a new enough version of Docker so that this isn't an issue >>> for them >>> 2) use Ubuntu 20.04 "Focal" as the docker container >>> >>> I was using Docker 20.10.21 which is why I didn't hit this issue when >>> testing the change locally. >>> >>> We could also do these but they same strictly worse then either of the >>> two options discussed above: >>> A) disable the seccomp policy on Jenkins >>> B) use a custom seccomp policy on Jenkins >>> >>> My suggestion is to upgrade Docker versions on Jenkins and use Ubuntu >>> 22.04 as it will have LTS releases till 2027 and then security patches till >>> 2032 which gives everyone the longest runway till we need to swap OS >>> versions again for users of Apache Beam. Any concerns or ideas? >>> >>> >>> >>> On Thu, Feb 9, 2023 at 10:20 AM Luke Cwik <lc...@google.com> wrote: >>> >>>> Our current container java 8 container is 262 MiBs and layers on top of >>>> openjdk:8-bullseye which is 226 MiBs compressed while eclipse-temurin:8 is >>>> 92 MiBs compressed and eclipse-temurin:8-alpine is 65 MiBs compressed. >>>> >>>> I would rather not get into issues with C library differences caused by >>>> the alpine project so I would stick with the safer option and let users >>>> choose alpine when building their custom container if they feel it provides >>>> a large win for them. We can always swap to alpine in the future as well if >>>> the C library differences become a non-issue. >>>> >>>> So swapping to eclipse-temurin will save us a bunch on the container >>>> size which should help with container transfer and hopefully for startup >>>> times as well. >>>> >>>> On Tue, Feb 7, 2023 at 5:41 PM Andrew Pilloud <apill...@google.com> >>>> wrote: >>>> >>>>> This sounds reasonable to me as well. >>>>> >>>>> I've made swaps like this in the past, the base image of each is >>>>> probably a bigger factor than the JDK. The openjdk images were based on >>>>> Debian 11. The default eclipse-temurin images are based on Ubuntu 22.04 >>>>> with an alpine option. Ubuntu is a Debian derivative but the versions and >>>>> package names aren't exact matches and Ubuntu tends to update a little >>>>> faster. For most users I don't think this will matter but users building >>>>> custom containers may need to make minor changes. The alpine option will >>>>> be >>>>> much smaller (which could be a significant improvement) but would be a >>>>> more >>>>> significant change to the environment. >>>>> >>>>> On Tue, Feb 7, 2023 at 5:18 PM Robert Bradshaw via dev < >>>>> d...@beam.apache.org> wrote: >>>>> >>>>>> Seams reasonable to me. >>>>>> >>>>>> On Tue, Feb 7, 2023 at 4:19 PM Luke Cwik via user < >>>>>> user@beam.apache.org> wrote: >>>>>> > >>>>>> > As per [1], the JDK8 and JDK11 containers that Apache Beam uses >>>>>> have stopped being built and supported since July 2022. I have filed [2] >>>>>> to >>>>>> track the resolution of this issue. >>>>>> > >>>>>> > Based upon [1], almost everyone is swapping to the eclipse-temurin >>>>>> container[3] as their base based upon the linked issues from the >>>>>> deprecation notice[1]. The eclipse-temurin container is released under >>>>>> these licenses: >>>>>> > Apache License, Version 2.0 >>>>>> > Eclipse Distribution License 1.0 (BSD) >>>>>> > Eclipse Public License 2.0 >>>>>> > 一 (Secondary) GNU General Public License, version 2 with OpenJDK >>>>>> Assembly Exception >>>>>> > 一 (Secondary) GNU General Public License, version 2 with the GNU >>>>>> Classpath Exception >>>>>> > >>>>>> > I propose that we swap all our containers to the eclipse-temurin >>>>>> containers[3]. >>>>>> > >>>>>> > Open to other ideas and also would be great to hear about your >>>>>> experience in any other projects that you have had to make a similar >>>>>> decision. >>>>>> > >>>>>> > 1: https://github.com/docker-library/openjdk/issues/505 >>>>>> > 2: https://github.com/apache/beam/issues/25371 >>>>>> > 3: https://hub.docker.com/_/eclipse-temurin >>>>>> >>>>>
