The ASF only counts the source releases the official releases. All binaries including docker images should be for convenience only.
Even the image needs to go through a vote, you can of course build and stage the image with any method in hand. The problem is where we can share a build environment - Since that building pulsar-all is a bit resource consuming, it can requires extra platform resource. Zixuan Liu <node...@gmail.com>于2023年6月1日 周四23:47写道： > > The best solution is to make the build repeatable. IIUC, that will let > us build all of the artifacts using CI instead of personal machines, > which removes this class of errors. > > I agree! But I don't know if Apache rules allow this operation, and if > so, I recommend using the CI to publish our project. > > Thanks, > Zixuan > > > Zike Yang <z...@apache.org> 于2023年6月1日周四 18:45写道： > > > > > The best solution is to make the build repeatable. IIUC, that will let > > us build all of the artifacts using CI instead of personal machines, > > which removes this class of errors. > > > > I don't think this issue is related to where the docker image was > > built. It has nothing to do with caching and nothing to do with the > > machine on which it is built. It's just because the build uses the > > `latest` tag, which points to an older version. > > I'm + 0.5 for this solution. I am currently wondering how much reward > > it will bring. > > > > > If we are able to prepare the docker image from the RELEASED tarbal > > and connectors then I am fine with it. > > > > We currently build our images using compiled tarball rather than > > rebuilding from sources. You can check the code here: > > > https://github.com/apache/pulsar/blob/5e6e6cebcdbeec32ed49729f658f2d5cd0d98347/docker/pulsar/Dockerfile#L25-L26 > > > > BR, > > Zike Yang > > > > On Thu, Jun 1, 2023 at 4:52 PM Asaf Mesika <asaf.mes...@gmail.com> > wrote: > > > > > > I understand that you can't build a release process due to Apache > > > foundation rules, that makes it mandatory to release something you > built on > > > your own machine. > > > > > > > > > On Wed, May 31, 2023 at 5:54 PM Enrico Olivelli <eolive...@gmail.com> > wrote: > > > > > > > Il giorno mer 31 mag 2023 alle ore 16:50 Michael Marshall > > > > <mmarsh...@apache.org> ha scritto: > > > > > > > > > > The best solution is to make the build repeatable. IIUC, that will > let > > > > > us build all of the artifacts using CI instead of personal > machines, > > > > > which removes this class of errors. > > > > > > > > > > That being said, I don't know how much effort it would be to > achieve. > > > > > > > > > > Thanks, > > > > > Michael > > > > > > > > > > On Tue, May 30, 2023 at 2:24 AM Zike Yang <z...@apache.org> wrote: > > > > > > > > > > > > Hi, Enrico > > > > > > > > > > > > > When we ran the VOTE and we provided the docker images, were > they > > > > > > already broken ? > > > > > > > > > > > > Actually, they are not broken unless we use the new features of > Pulsar > > > > 3.0.0. > > > > > > > > It is not Pulsar 3 but Pulsar 2.11. > > > > So for users this is a problem > > > > > > > > If we are able to prepare the docker image from the RELEASED tarbal > > > > and connectors then I am fine with it. > > > > If you have to rebuild from the sources I strongly believe that we > > > > cannot do it without a proper release process > > > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > > > I think we need something like verification test scripts, to > verify > > > > > > the release candidate. For example, we use the image provided in > the > > > > > > RC to run the integration tests. And we need to make sure that > we have > > > > > > tested the newly added feature for the RC docker image. > > > > > > > > > > > > What do you think? > > > > > > > > > > > > BR, > > > > > > Zike Yang > > > > > > > > > > > > On Tue, May 30, 2023 at 3:14 PM Zike Yang <z...@apache.org> > wrote: > > > > > > > > > > > > > > Hi, Asaf > > > > > > > > > > > > > > > How do you suggest we prevent it from happening next time? > > > > > > > > > > > > > > I have pushed a PR to fix it: > > > > https://github.com/apache/pulsar/pull/20435 > > > > > > > This PR specifies the correct image name for `pulsar` image to > build > > > > pulsar-all. > > > > > > > > > > > > > > Note that, in the release of Pulsar 3.0, we build the docker > image by > > > > > > > executing the following command instead of the > `docker/build.sh`: > > > > > > > ``` > > > > > > > mvn install -DUBUNTU_MIRROR= > http://azure.archive.ubuntu.com/ubuntu/ > > > > \ > > > > > > > -DskipTests \ > > > > > > > -Pdocker -Pdocker-push \ > > > > > > > -Ddocker.platforms=linux/amd64,linux/arm64 \ > > > > > > > -Ddocker.organization=snzkyang \ > > > > > > > -pl docker/pulsar,docker/pulsar-all > > > > > > > ``` > > > > > > > I think to take it a step further, we could fix these > > > > scripts(build.sh > > > > > > > and publish.sh) and use the shell scripts to build the image. > > > > > > > > > > > > > > I have verified the PR, and it works well. Please see more > detail in > > > > > > > the PR description. > > > > > > > > > > > > > > Thanks, > > > > > > > Zike Yang > > > > > > > > > > > > > > On Mon, May 29, 2023 at 9:50 PM Enrico Olivelli < > eolive...@gmail.com> > > > > wrote: > > > > > > > > > > > > > > > > I am really worried about the process. > > > > > > > > > > > > > > > > When we ran the VOTE and we provided the docker images, were > they > > > > > > > > already broken ? > > > > > > > > > > > > > > > > In any case we cannot overwrite those images, they have been > cached > > > > > > > > all over the world now. > > > > > > > > > > > > > > > > It is safer to cut a new 3.0.1 release and run a VOTE. > > > > > > > > > > > > > > > > Maybe we can remove the old images, forever > > > > > > > > > > > > > > > > Enrico > > > > > > > > > > > > > > > > Il giorno lun 29 mag 2023 alle ore 13:55 Asaf Mesika > > > > > > > > <asaf.mes...@gmail.com> ha scritto: > > > > > > > > > > > > > > > > > > Good catch! > > > > > > > > > > > > > > > > > > How do you suggest we prevent it from happening next time? > > > > > > > > > > > > > > > > > > On Mon, May 29, 2023 at 1:34 PM Zike Yang <z...@apache.org > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi, all > > > > > > > > > > > > > > > > > > > > Recently, we found an issue with the `pulsar-all:3.0.0` > image. > > > > The > > > > > > > > > > pulsar library included in `pulsar-all:3.0.0` is the > version of > > > > > > > > > > 2.11.0: > > > > > > > > > > > > > > > > > > > > ``` > > > > > > > > > > docker run apachepulsar/pulsar-all:3.0.0 ls lib/ | grep > > > > pulsar-broker > > > > > > > > > > > > > > > > > > > > org.apache.pulsar-pulsar-broker-2.11.0.jar > > > > > > > > > > org.apache.pulsar-pulsar-broker-auth-sasl-2.11.0.jar > > > > > > > > > > org.apache.pulsar-pulsar-broker-common-2.11.0.jar > > > > > > > > > > ``` > > > > > > > > > > > > > > > > > > > > The root cause is that we use > `apachepulsar/pulsar:latest` to > > > > build > > > > > > > > > > the `pulsar-all` image. But at the time of building > Pulsar > > > > 3.0.0, > > > > > > > > > > `apachepulsar/pulsar:latest` was pointing to version > 2.11.0. > > > > > > > > > > > > > > > > > > > > Therefore, the `pulsar-all:3.0.0` is actually a version > 2.11.0 > > > > of > > > > > > > > > > Pulsar but with 3.0.0 connectors and offloaders. > > > > > > > > > > > > > > > > > > > > Please see more detail in this issue: > > > > > > > > > > https://github.com/apache/pulsar/issues/20420 > > > > > > > > > > > > > > > > > > > > I have rebuilt the `pulsar-all:3.0.0` image: > > > > > > > > > > > > > > > > > > > > > > > > > https://hub.docker.com/layers/snzkyang/pulsar-all/3.0.0/images/sha256-833ea988bce8c704b179cc4c9c38fac8980e108b0bc67454e06c22927990b169?context=explore > > > > > > > > > > > > > > > > > > > > Please help and verify it. And check if there are any > other > > > > problems > > > > > > > > > > with the image. > > > > > > > > > > > > > > > > > > > > I'm going to publish the image to the `apachepulsar` > > > > organization to > > > > > > > > > > replace the old one. But before we do that, do we need a > Vote > > > > or other > > > > > > > > > > ways to reach a consensus? Is there any problem if we > replace > > > > the old > > > > > > > > > > image? > > > > > > > > > > > > > > > > > > > > Besides, I will also fix the docker build script to avoid > > > > similar issues. > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > Zike Yang > > > > > > > > > > > > > > > -- Best, tison.
