Great summary Houston! Could also be that docker team is willing to provide a “link” from official _/solr to apache/solr if we can convince them of solid quality. Think they do this for elastic images already.
Since Docker images contain Linux and Java, which we would not be allowed to release as part of Solr, I have seen discussions in various ASF lists stating that it can be argued that the only binaries we do “release” are the layers built by out Dockerfile, i.e. what comes after the (runtime) FROM line. So we should be careful with what extra software we add in dockerfile. I did a check earlier and think we are in good shape. We currently re-build a bunch of older solr docked images every time we release a new, but I don’t think there is any automatic refresh of images outside a release. Great idea to kick off refresh of images from Jenkins. We can also publish nightly “master” images but since they are not officially voted releases they must be clearly labelled as unofficial and not advertised on the web page, only for dev purposes. Jan Høydahl > 16. jan. 2021 kl. 00:36 skrev Timothy Potter <thelabd...@gmail.com>: > > I'm curious about how tags will work when updating the base image for a > released image? The image for a tag should be immutable (IMHO), and I think > people would be surprised if 8.8.0 suddenly changed even if it was for a good > reason such as fixing a CVE in the base image. But based on what Kevin said, > perhaps there's already precedence for this with the official images? > > On Fri, Jan 15, 2021 at 1:51 PM Houston Putman <houstonput...@gmail.com> > wrote: >> Thanks for bringing up this issue Kevin. >> >> Periodically re-building docker images is certainly a feature we could >> support, and probably should to automatically keep up with security fixes. >> We could even automate it pretty easily in Jenkins. >> >> We could also build in support in the gradle commands to instead of building >> a TGZ from source, download and verify the "official" TGZ, to build the >> image with. That way release images are always built with the same exact >> binaries. The Dockerfile wouldn't need to change at all between local and >> release, it still merely expects a TGZ to be passed in the context; gradle >> can determine if it needs to be built from scratch or downloaded. >> >> This still likely wouldn't be good enough to make the image an "official >> docker image" but it gets us to essentially the same end-state image. The >> only difference is the downloading and verification are happening in gradle >> instead of the Dockerfile. >> >> - Houston >> >> On Fri, Jan 15, 2021 at 2:05 PM Kevin Risden <kris...@apache.org> wrote: >>>> Currently the solr-docker-image, and a majority of "Docker Official >>>> Images", the officially released Solr binaries are downloaded from mirrors >>>> and validated within the Dockerfiles. This makes it easy to ensure to >>>> users that the 9.0 solr docker image contains the 9.0 solr release. This >>>> process doesn't fit very well with local builds, because there is nowhere >>>> to download local builds from, and validation isn't required. >>>> >>>> The current opinion in the community is to abandon the "Docker Official >>>> Images" style process of downloading and validating official binaries, and >>>> instead having the release manager use the local-build image creation with >>>> the final release source. This should result in the same docker image in >>>> the end, however there is no trust built into the docker image itself. >>>> Instead we are likely going to document a way for users to verify the >>>> docker-image contents themselves. >>> >>> Before we abandon the official process of downloading/validating official >>> binaries, I think there is a good reason to keep the ability to download an >>> "official" Apache Solr release and use it in the "official" Solr >>> convenience Docker image. >>> >>> Docker images are static point in time copies of an OS and all supporting >>> packages (like Java) when built. Periodically Docker images should be >>> rebuilt to pick up the latest security and bug enhancements in the base >>> image. Just like any OS should run `apt upgrade` or `yum update` >>> periodically to ensure it is up to date. >>> >>> My proposal is to periodically rebuild the "official" Solr convenience >>> Docker image based on the "official" Solr release to ensure we keep the >>> Docker images up to date. The idea being that we have a list of "supported" >>> versions of Solr (ie: 8.5, 8.6, 8.7) and periodically (ie: daily, weekly) >>> the Docker images are rebuilt. Once a new release is made (ie: 8.8) it gets >>> added to this rebuilding matrix. This ensures that the "official" Solr >>> convenience Docker image is reasonably up to date with regards to the base >>> image security updates. >>> >>> My understanding (from a few years ago) is that the Docker official images >>> are rebuilt when the base image is updated. This was automatic from what I >>> understand. If we move away from the "Docker official image" way of doing >>> things, we should still ensure that we can provide high quality secure >>> Docker images to the community. >>> >>> We should not need a new Apache Solr release (ie: 8.7.1, 8.8) to update the >>> Docker image to pick up the latest base image changes. >>> >>> It is important to be able to build and test locally with a Docker image >>> that matches what an "official" Docker image is. This should still be a >>> goal, but we should be able to rebuild the Solr docker image without >>> rebuilding all of Solr. >>> >>> PS - I chatted a bit with Houston on Slack about this topic and hopefully >>> captured all the context correctly. >>> >>> Kevin Risden >>> >>> >>> On Fri, Jan 15, 2021 at 12:45 PM Houston Putman <houstonput...@gmail.com> >>> wrote: >>>> There's a few decisions that need to be ironed out around the Solr docker >>>> image before 9.0 is released. This is because the community has decided >>>> that Solr should start releasing it's own docker images starting with 9.0. >>>> >>>> Below is the current state of the ongoing discussions for the Solr Docker >>>> image. Please feel free to correct me or fill in any information I may be >>>> missing. >>>> >>>> Where does this image live? >>>> >>>> There are two options for this really. >>>> _/solr - docker run solr:9.0 (Official Docker Image) >>>> apache/solr - docker run apache/solr:9.0 >>>> The benefits of the first are 1) the nice usability of being able to >>>> plainly specify "solr" and 2) the "Docker Official Images" badge on >>>> DockerHub. The downsides are that there are very strict requirements with >>>> creating Official Docker Images, which would complicate and require >>>> separating the way that we build release docker images and local docker >>>> images. >>>> >>>> The benefits of using the apache namespace is that we can build the image >>>> in any way that we want. We would be able to build release and local >>>> docker images the exact same way. The downside is the loss of the "Docker >>>> Official Images" badge. >>>> >>>> I think there is some consensus that choosing the "apache/solr" location >>>> is fine, and worth the added flexibility we get in the build process. >>>> >>>> Legal Stuff >>>> >>>> There are a few legal questions we need to keep in mind when creating this >>>> process and doing a release for the first time. >>>> Source release - The apache policy is: (from Michael Sokolov) >>>>> “Every ASF release MUST contain one or more source packages, which MUST >>>>> be sufficient for a user to build and test the release provided they have >>>>> access to the appropriate platform and tools.” >>>> For the docker build this is fine as long as the solr/docker gradle module >>>> is included in the source release. As one can always rebuild the same >>>> image running gradlew docker using the source. >>>> Jan Høydahl mentioned that the docker file layers should be limited, but >>>> I'm not exactly sure what this means or entails. Maybe he can expand on >>>> this. >>>> Artifacts within the Image >>>> >>>> As mentioned above in the "Image Location" section, the goal of including >>>> the docker build process inside the Solr project is to make development >>>> easier by providing an easy way to build the official-style docker image >>>> with local source code. In order to achieve "official-style" images for >>>> local builds, we want to make the build process for local images as close >>>> as possible to the process for building official release images. >>>> >>>> Currently the solr-docker-image, and a majority of "Docker Official >>>> Images", the officially released Solr binaries are downloaded from mirrors >>>> and validated within the Dockerfiles. This makes it easy to ensure to >>>> users that the 9.0 solr docker image contains the 9.0 solr release. This >>>> process doesn't fit very well with local builds, because there is nowhere >>>> to download local builds from, and validation isn't required. >>>> >>>> The current opinion in the community is to abandon the "Docker Official >>>> Images" style process of downloading and validating official binaries, and >>>> instead having the release manager use the local-build image creation with >>>> the final release source. This should result in the same docker image in >>>> the end, however there is no trust built into the docker image itself. >>>> Instead we are likely going to document a way for users to verify the >>>> docker-image contents themselves. >>>> >>>> I am not sure what the user-side verification process would look like for >>>> the image, but I definitely think it is something that we should look >>>> into. Maybe Docker will allow us to use these as official images if we can >>>> script out this verification and make it easy for them to do? Just a >>>> thought, I'm not sure if that would actually work.
