Hey Chris, Once we provide the definitions to docker, they should take care of everything from there. They mentioned here <https://github.com/docker-library/official-images?tab=readme-ov-file#library-definition-files> that the image will be rebuilt when the base image is updated. Hence active rebuilds won't require any changes from our side. If we are packaging something which may contain a CVE, like some jar, then the onus will be on us to patch it, but it will be upto us whether we consider the threat severe enough to fix and when we want to provide the fixed version. Having Docker Official Image will not impact the frequency of our releases. It will be the Apache Kafka community's call on when a release goes and Docker Official Image will be released accordingly as per the KIP. source <https://github.com/docker-library/faq?tab=readme-ov-file#image-building>
As mentioned in Docker's documentation as well "In essence we strive to heed upstream's recommendations on how they intend for their software to be consumed." source <https://github.com/docker-library/official-images?tab=readme-ov-file#what-are-official-images> Docker Official Image will rely on upstream's recommendation for functionality. But I do agree that since Docker's stance on this might change in future it makes sense to put a safeguard that will not allow any functionality changes get incorporated as part of the vetting process. I have updated the KIP to reflect the same. KIP-975 has a well defined public interface based on how configs can be supplied and how it can be used. I am not sure if we put that label on it during discussions. I am happy to have a separate email thread on it to iron things out. I hope this addresses all of your concerns! Thanks and regards, Vedarth On Mon, May 13, 2024 at 10:55 PM Chris Egerton <chr...@aiven.io.invalid> wrote: > Thanks both for your responses! Friendly reminder: again, better to provide > a quote instead of just a link :) > > I've seen a bit about image rebuilding to handle CVEs but I'm a little > unclear on how this would work in practice, and I couldn't find any > concrete details in any of the links. Does Docker do this automatically for > DOIs? Or will the onus be on us to put out patched images? Would this lead > to us putting out images more quickly than we put out standard releases? As > a plus, it does look like DOIs get the benefit of Docker Scout [1] for > free, which is nice, but it's still unclear who'd be doing the rest of the > work on that front. > > As far as this point from Vedarth goes: > > > By incorporating the source code of the Docker Official Image into our > > AK ecosystem, we gain control over its functionality, ensuring alignment > > with the OSS Docker image. This ensures a seamless experience for users > who > > may need to transition between these images. > > This captures my concern with the KIP pretty well. If there's any > significant divergence in behavior (not just build methodology) between the > apache/kafka image and what Docker requires for a Kafka DOI, how are we > going to vet these changes moving forward? Under the "Post Release Process > - if Dockerhub folks suggest changes to the Dockerfiles:" header, this KIP > proposes that we port all suggested changes for the DOI to > the docker/jvm/Dockerfile image, but this seems a bit too permissive. As an > alternative, we could state that all build-related changes can be done with > a PR on the apache/kafka GitHub repo (which will require approval from a > single committer), but any functional changes will require a KIP. > > Finally, during KIP-975 was there discussion on what we would count as the > public interface for the apache/kafka image? If not, it'd be nice to get > that ironed out since it may make future discussions around our Docker > images quicker, but I don't think this is necessary for KIP-1028. > > [1] - https://www.docker.com/products/docker-scout/ > > On Mon, May 13, 2024 at 4:37 AM Prabha Manepalli > <mpra...@confluent.io.invalid> wrote: > > > Hi Chris, > > > > Sharing the requested links explaining why Docker Official images are > > considered more secure - > > > > > https://www.docker.com/blog/enhancing-security-and-transparency-with-docker-official-images/ > > and > > > > > https://github.com/docker-library/faq#why-does-my-security-scanner-show-that-an-image-has-cves > > > > I hope these links help you understand why we need Docker Official images > > for organisations with stringent security compliance requirements for > their > > Kafka workloads. > > > > Thank you. > > > > > > > > On Sun, May 12, 2024 at 3:33 PM Vedarth Sharma <vedarth.sha...@gmail.com > > > > wrote: > > > > > Hey Chris! > > > > > > Functionality wise, we don't intend to have any differences between OSS > > > Docker Image and Docker Official Image. > > > The Docker Official Image will be the recommended one. > > > Since the Docker Official Image might be delayed due to review done by > > > Docker, images on apache/kafka (OSS Docker Image) can be used by users. > > > > > > 1) I read https://docs.docker.com/trusted-content/official-images/ and > > > > didn't find anything on that page or immediately around it that > > explains > > > > what compliance requirements might be satisfied by a DOI that > couldn't > > be > > > > satisfied by the existing apache/kafka image. Can you elaborate? Feel > > > free > > > > to provide another link, but please also quote the relevant sections > > from > > > > it (as StackOverflow likes to say, links can grow stale over time). > > > > > > > > > - If a user's selection is confined solely to Docker Official > Images, > > > this Docker Image will ensure their access to Kafka. > > > - Details on specific advantages of Docker Official Images can be > > found > > > at > > > > > > > > > https://github.com/docker-library/official-images?tab=readme-ov-file#what-are-official-images > > > . > > > - The Docker Official Image will be actively rebuilt for updates and > > > security fixes. > > > - It's true we can provide exactly the same benefits in the OSS > Docker > > > Image as well. But it won't have the Docker Official Image badge and > > it > > > will add additional overhead for Apache Kafka community. > > > - The fact that it will have Docker Official Image badge will set it > > > apart from the OSS Docker Image. > > > - Also the ability to do just docker pull kafka to get the kafka > > docker > > > image will only be possible with Docker Official Image. > > > > > > > > > 2) It would be great to see a brief summary of the differences in these > > > > images included in the KIP, in order to try to gauge how this would > > look > > > to > > > > our users. > > > > > > > > > - Functionally, both Docker images will remain identical. > > > - The variance lies primarily in the methodologies of building and > > > validation, as outlined in the updated KIP. > > > > > > > > > 3) What I suggested last time was not a separate apache/apache-docker > > > > repository, but a repository controlled entirely by Docker. The DOI > > docs > > > > [1] state that "While it's preferable to have upstream software > authors > > > > maintaining their Docker Official Images, this isn't a strict > > > requirement", > > > > which I take to mean that it's not required for an Apache Kafka DOI > to > > > live > > > > under the apache organization on GitHub. It also seems like there's > > > > precedent for this: images for MySQL [2] and PHP [3] already exist > > under > > > > the control of Docker. The reason I think this is worth considering > is > > > that > > > > Docker can arbitrarily change the eligibility requirements for their > > > > official images at any time, and it doesn't seem like there's a clear > > > > process in the KIP for judging how we should respond to these changes > > (in > > > > fact, it seems like the idea in the KIP is that we should make any > > change > > > > required with no further vetting beyond possibly a pull request on > > > > apache/kafka, which would require approval by a committer). By > hosting > > > the > > > > DOI definitions ourselves (either in apache/kafka, or in a > theoretical > > > > apache/docker-kafka repository), we take responsibility for the > image, > > > even > > > > if the owner on Docker Hub is Docker, not Apache. If the code lives > > > > elsewhere, then (as long as basic trademark and possibly security > > > > guidelines are respected) Apache doesn't have to concern itself at > all > > > with > > > > the image and the maintainers are free to make whatever changes they > > want > > > > to it in order to meet Docker's requirements. > > > > > > > > > - By incorporating the source code of the Docker Official Image into > > our > > > AK ecosystem, we gain control over its functionality, ensuring > > alignment > > > with the OSS Docker image. This ensures a seamless experience for > > users > > > who > > > may need to transition between these images. > > > - Maintaining both images within the same community facilitates ease > > of > > > management and fosters a singular source of truth. > > > - While Apache may not retain ownership of the hosted Docker > Official > > > Image, we are, in essence, providing Docker with a foundation that > > > aligns > > > with their established guidelines as well as remains consistent with > > OSS > > > Docker Image apache/kafka. > > > - Any future alterations to the functionality can be seamlessly > > > propagated across both the OSS and Official Docker Images. > > > > > > I think these reasons must be why a lot of the Apache projects choose > to > > > host the docker definitions themselves. The responsibility of owning > the > > > definitions comes with benefits as well that we should also consider. > > > > > > Let us know if you have any further questions! > > > > > > Thanks and regards, > > > Vedarth > > > > > > On Fri, May 10, 2024 at 7:56 PM Chris Egerton <chr...@aiven.io.invalid > > > > > wrote: > > > > > > > Hi Krish and Prabha, > > > > > > > > Thanks for your replies. I still have some follow-up questions: > > > > > > > > 1) I read https://docs.docker.com/trusted-content/official-images/ > and > > > > didn't find anything on that page or immediately around it that > > explains > > > > what compliance requirements might be satisfied by a DOI that > couldn't > > be > > > > satisfied by the existing apache/kafka image. Can you elaborate? Feel > > > free > > > > to provide another link, but please also quote the relevant sections > > from > > > > it (as StackOverflow likes to say, links can grow stale over time). > > > > > > > > 2) It would be great to see a brief summary of the differences in > these > > > > images included in the KIP, in order to try to gauge how this would > > look > > > to > > > > our users. > > > > > > > > 3) What I suggested last time was not a separate apache/apache-docker > > > > repository, but a repository controlled entirely by Docker. The DOI > > docs > > > > [1] state that "While it's preferable to have upstream software > authors > > > > maintaining their Docker Official Images, this isn't a strict > > > requirement", > > > > which I take to mean that it's not required for an Apache Kafka DOI > to > > > live > > > > under the apache organization on GitHub. It also seems like there's > > > > precedent for this: images for MySQL [2] and PHP [3] already exist > > under > > > > the control of Docker. The reason I think this is worth considering > is > > > that > > > > Docker can arbitrarily change the eligibility requirements for their > > > > official images at any time, and it doesn't seem like there's a clear > > > > process in the KIP for judging how we should respond to these changes > > (in > > > > fact, it seems like the idea in the KIP is that we should make any > > change > > > > required with no further vetting beyond possibly a pull request on > > > > apache/kafka, which would require approval by a committer). By > hosting > > > the > > > > DOI definitions ourselves (either in apache/kafka, or in a > theoretical > > > > apache/docker-kafka repository), we take responsibility for the > image, > > > even > > > > if the owner on Docker Hub is Docker, not Apache. If the code lives > > > > elsewhere, then (as long as basic trademark and possibly security > > > > guidelines are respected) Apache doesn't have to concern itself at > all > > > with > > > > the image and the maintainers are free to make whatever changes they > > want > > > > to it in order to meet Docker's requirements. > > > > > > > > [1] - > > > > > https://docs.docker.com/trusted-content/official-images/contributing/, > > > > second paragraph under "Contributing to Docker Official Images" > > > > [2] - https://github.com/docker-library/mysql > > > > [3] - https://github.com/docker-library/php > > > > > > > > Cheers, > > > > > > > > Chris > > > > > > > > On Fri, May 10, 2024 at 7:46 AM Krish Vora <krishvor...@gmail.com> > > > wrote: > > > > > > > > > Hey Chris, > > > > > > > > > > We have responded to the initial round of queries. And as you > > mentioned > > > > in > > > > > your comment, you may have more questions related to this KIP. > Please > > > let > > > > > us know if you have any. > > > > > We want to start the voting for this KIP soon, as we intend to > > include > > > it > > > > > in the 3.8.0 release. > > > > > > > > > > Thanks. > > > > > > > > > > Regards, > > > > > > > > > > Krish. > > > > > > > > > > On Wed, May 8, 2024 at 6:19 PM Krish Vora <krishvor...@gmail.com> > > > wrote: > > > > > > > > > > > Hi Chris. Thanks for the questions. > > > > > > > > > > > > 3. Would a separate Docker-owned repository be out of the > question? > > > I'm > > > > > >> guessing there are some trademark issues that might get in the > > way, > > > > but > > > > > >> it's worth exploring since the entire purpose of this KIP seems > to > > > be > > > > to > > > > > >> provide images that are vetted and designed by Docker more than > by > > > the > > > > > >> Apache Kafka contributors/committers/PMC. > > > > > > > > > > > > > > > > > > > > > > > > - The process for introducing a Docker Official Image involves > > > > > > - Hosting the Dockerfile in the Apache Kafka repository and > > > > > > - Providing the path to this Dockerfile to Docker Hub in > > Docker > > > > > > Hub’s own repo > > > > > > < > > > > > > > https://github.com/docker-library/official-images/tree/master/library> > > > > > > . > > > > > > - This ensures that any updates to the Dockerfile in the AK > > > > repository > > > > > > are directly applicable to the docker official images > available > > on > > > > > Docker > > > > > > Hub. > > > > > > > > > > > > > > > > > > - We also did not find any added advantage to create a > separate > > > > > > repository named apache-docker within the Apache GitHub > > > > organization. > > > > > > > > > > > > Thanks, > > > > > > Krish. > > > > > > > > > > > > On Wed, May 8, 2024 at 6:05 PM Prabha Manepalli > > > > > > <mpra...@confluent.io.invalid> wrote: > > > > > > > > > > > >> Hi Chris, I would like to add more context to this KIP's > > > motivation. > > > > > >> Vedarth and Krish, please weigh in with your inputs. > > > > > >> > > > > > >> In the motivation section it's stated that "Several other Apache > > > > > projects, > > > > > >> > like Flink, Spark, Solr, have already released Docker Official > > > > Images, > > > > > >> with > > > > > >> > download figures ranging from 50 million to over 1 billion. > > These > > > > > >> numbers > > > > > >> > highlight the significant demand among users." But then > > > immediately > > > > > >> > afterwards, we learn that "Also the Docker Official Images are > > > > always > > > > > >> the > > > > > >> > top 1 search result, irrespective of the number of downloads." > > > > > Wouldn't > > > > > >> a > > > > > >> > high number of downloads for an image naturally follow from > > being > > > > the > > > > > >> top > > > > > >> > search result? It seems like we can't necessarily assume that > > > Docker > > > > > >> > Official Images are inherently more desirable for users based > > > solely > > > > > on > > > > > >> > download statistics. > > > > > >> > > > > > > >> > > > > > >> *My thoughts: *Unlike the Sponsored OSS image, the Docker > Official > > > > image > > > > > >> is > > > > > >> more desirable for workloads that have stringent compliance > > > > > requirements. > > > > > >> More details on why official images are more trusted are > > documented > > > > here > > > > > >> <https://docs.docker.com/trusted-content/official-images/>. The > > > > Docker > > > > > >> Official image would also help an absolutely new Kafka beginner > > who > > > > > might > > > > > >> not know about Apache or the concept of Sponsored images. We > want > > to > > > > > make > > > > > >> it easier for Kafka beginners to discover the Kafka image > through > > > > > >> DockerHub. > > > > > >> > > > > > >> > > > > > >> Can you elaborate on the value that these new images would add > > from > > > a > > > > > >> > user's perspective? I'm hesitant to introduce another image, > > since > > > > it > > > > > >> adds > > > > > >> > to the cognitive burden of people who will inevitably have to > > > answer > > > > > the > > > > > >> > question of "What are the differences between all of the > > available > > > > > >> images > > > > > >> > and which one is best for my use case?" > > > > > >> > > > > > > >> > > > > > >> > > > > > >> *My thoughts: *This is a valid concern to address. The response > to > > > the > > > > > >> above question addresses the value-add this new Docker Official > > > image > > > > > >> would > > > > > >> provide. I also agree we need a clear distinction between each > of > > > > these > > > > > >> images to be well documented. We plan to update the AK website > > with > > > > > >> details > > > > > >> on how, why, and when a developer would want to use each of > these > > > > > >> particular images(KIP-974,975,1028). > > > > > >> > > > > > >> Thanks, > > > > > >> Prabha. > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > > >> On Tue, Apr 30, 2024 at 9:41 PM Chris Egerton > > > <chr...@aiven.io.invalid > > > > > > > > > > >> wrote: > > > > > >> > > > > > >> > Hi Vedarth and Krish, > > > > > >> > > > > > > >> > Thanks for the KIP! I have to admit I'm a little skeptical; > > > > hopefully > > > > > >> you > > > > > >> > can help me understand the need for these additional images. > > > > > >> > > > > > > >> > 1) In the motivation section it's stated that "Several other > > > Apache > > > > > >> > projects, like Flink, Spark, Solr, have already released > Docker > > > > > Official > > > > > >> > Images, with download figures ranging from 50 million to over > 1 > > > > > billion. > > > > > >> > These numbers highlight the significant demand among users." > But > > > > then > > > > > >> > immediately afterwards, we learn that "Also the Docker > Official > > > > Images > > > > > >> are > > > > > >> > always the top 1 search result, irrespective of the number of > > > > > >> downloads." > > > > > >> > Wouldn't a high number of downloads for an image naturally > > follow > > > > from > > > > > >> > being the top search result? It seems like we can't > necessarily > > > > assume > > > > > >> that > > > > > >> > Docker Official Images are inherently more desirable for users > > > based > > > > > >> solely > > > > > >> > on download statistics. > > > > > >> > > > > > > >> > 2) Can you elaborate on the value that these new images would > > add > > > > > from a > > > > > >> > user's perspective? I'm hesitant to introduce another image, > > since > > > > it > > > > > >> adds > > > > > >> > to the cognitive burden of people who will inevitably have to > > > answer > > > > > the > > > > > >> > question of "What are the differences between all of the > > available > > > > > >> images > > > > > >> > and which one is best for my use case?" > > > > > >> > > > > > > >> > 3) Would a separate Docker-owned repository be out of the > > > question? > > > > > I'm > > > > > >> > guessing there are some trademark issues that might get in the > > > way, > > > > > but > > > > > >> > it's worth exploring since the entire purpose of this KIP > seems > > to > > > > be > > > > > to > > > > > >> > provide images that are vetted and designed by Docker more > than > > by > > > > the > > > > > >> > Apache Kafka contributors/committers/PMC. > > > > > >> > > > > > > >> > I may have more questions later but wanted to get this initial > > > round > > > > > out > > > > > >> > now without trying to list everything first. > > > > > >> > > > > > > >> > Looking forward to your thoughts! > > > > > >> > > > > > > >> > Cheers, > > > > > >> > > > > > > >> > Chris > > > > > >> > > > > > > >> > On Mon, Apr 22, 2024 at 2:14 PM Vedarth Sharma < > > > > > >> vedarth.sha...@gmail.com> > > > > > >> > wrote: > > > > > >> > > > > > > >> > > Hey folks, > > > > > >> > > > > > > > >> > > Thanks a lot for reviewing the KIP and providing feedback. > > > > > >> > > The discussion thread seems resolved and KIP has been > updated > > > > > >> > accordingly. > > > > > >> > > We will be starting the voting thread for this KIP in the > next > > > few > > > > > >> days. > > > > > >> > > Please take a look at the KIP and let us know if any further > > > > > >> discussion > > > > > >> > > is needed. > > > > > >> > > > > > > > >> > > Thanks and regards, > > > > > >> > > Vedarth > > > > > >> > > > > > > > >> > > On Fri, Apr 19, 2024 at 1:33 PM Manikumar < > > > > > manikumar.re...@gmail.com> > > > > > >> > > wrote: > > > > > >> > > > > > > > >> > > > Thanks Krish. KIP looks good to me. > > > > > >> > > > > > > > > >> > > > On Wed, Apr 17, 2024 at 1:38 PM Krish Vora < > > > > krishvor...@gmail.com > > > > > > > > > > > >> > > wrote: > > > > > >> > > > > > > > > > >> > > > > Hi Manikumar, > > > > > >> > > > > > > > > > >> > > > > Thanks for the comments. > > > > > >> > > > > > > > > > >> > > > > Maybe as part of the release process, RM can create a > JIRA > > > for > > > > > >> this > > > > > >> > > > > > task. This can be taken by RM or any comitter or any > > > > > contributor > > > > > >> > > (with > > > > > >> > > > > > some help from commiters to run "Docker Image > > Preparation > > > > via > > > > > >> > GitHub > > > > > >> > > > > > Actions:" > > > > > >> > > > > > > > > > >> > > > > This sounds like a good idea. This step would be > > beneficial. > > > > By > > > > > >> > > creating > > > > > >> > > > a > > > > > >> > > > > JIRA ticket, it will also serve as a reminder to > complete > > > the > > > > > >> > > > post-release > > > > > >> > > > > steps for the Docker official images. Have updated the > KIP > > > > with > > > > > >> this > > > > > >> > > > step. > > > > > >> > > > > > > > > > >> > > > > Is this using GitHub Actions workflow? or manual > testing? > > > > > >> > > > > > > > > > >> > > > > This will be done by a Github Actions workflow, which > will > > > > test > > > > > >> the > > > > > >> > > > static > > > > > >> > > > > Docker Official Image assets for a specific release > > version. > > > > > >> > > > > > > > > > >> > > > > Is it mandatory for RM/comitters to raise the PR to > Docker > > > > Hub’s > > > > > >> > > > > > official images repository (or) can it be done by any > > > > > >> contributor. > > > > > >> > > > > > > > > > >> > > > > I believe that it can be done by any contributor (ref: > > This > > > > link > > > > > >> > > > > < > > > > > >> > > > > > > https://docs.docker.com/trusted-content/official-images/contributing/ > > > > > >> > > > > > > > > >> > > > > quotes "*Anyone can provide feedback, contribute code, > > > suggest > > > > > >> > process > > > > > >> > > > > changes, or even propose a new Official Image.*") > > > > > >> > > > > > > > > > >> > > > > Also I was thinking, once the KIP gets voted, we should > > try > > > to > > > > > >> > release > > > > > >> > > > > > kafka:3.7.0 (or 3.7.1) Docker Official image. This > will > > > help > > > > > us > > > > > >> to > > > > > >> > > > > > validate the process and allow us to fix any changes > > > > suggested > > > > > >> by > > > > > >> > > > > > Dockerhub before the 3.8.0 release. > > > > > >> > > > > > > > > > >> > > > > This sounds like a great idea. This KIP proposes release > > of > > > > DOI > > > > > >> as a > > > > > >> > > > > post-release process, which can be done anytime post > > > release. > > > > > >> Since > > > > > >> > > 3.7.0 > > > > > >> > > > > is already released, we can perform these steps for that > > > > release > > > > > >> too. > > > > > >> > > By > > > > > >> > > > > the time the KIP gets implemented, if 3.7.1 is released, > > we > > > > > could > > > > > >> do > > > > > >> > > > these > > > > > >> > > > > steps for 3.7.1, instead of 3.7.0. This would allow us > to > > > make > > > > > >> > changes > > > > > >> > > to > > > > > >> > > > > the Dockerfiles and other assets based on feedback from > > > Docker > > > > > Hub > > > > > >> > > before > > > > > >> > > > > the release of version 3.8.0. > > > > > >> > > > > > > > > > >> > > > > Thanks, > > > > > >> > > > > Krish. > > > > > >> > > > > > > > > > >> > > > > On Mon, Apr 15, 2024 at 12:59 PM Manikumar < > > > > > >> > manikumar.re...@gmail.com> > > > > > >> > > > > wrote: > > > > > >> > > > > > > > > > >> > > > > > Hi Krish, > > > > > >> > > > > > > > > > > >> > > > > > Thanks for the updated KIP. a few comments below. > > > > > >> > > > > > > > > > > >> > > > > > > "These actions can be carried out by the RM or any > > > > > contributor > > > > > >> > post > > > > > >> > > > the > > > > > >> > > > > > release process." > > > > > >> > > > > > Maybe as part of the release process, RM can create a > > JIRA > > > > for > > > > > >> this > > > > > >> > > > > > task. This can be taken by RM or any comitter or any > > > > > contributor > > > > > >> > > (with > > > > > >> > > > > > some help from commiters to run "Docker Image > > Preparation > > > > via > > > > > >> > GitHub > > > > > >> > > > > > Actions:" > > > > > >> > > > > > > > > > > >> > > > > > > "Perform Docker build tests to ensure image > integrity" > > > > > >> > > > > > Is this using GitHub Actions workflow? or manual > > testing? > > > > > >> > > > > > > > > > > >> > > > > > > "The RM will manually raise the final PR to Docker > > Hub’s > > > > > >> official > > > > > >> > > > images > > > > > >> > > > > > repository using the contents of the generated file" > > > > > >> > > > > > Is it mandatory for RM/comitters to raise the PR to > > > Docker > > > > > >> Hub’s > > > > > >> > > > > > official images repository (or) can it be done by any > > > > > >> contributor. > > > > > >> > > > > > > > > > > >> > > > > > Also I was thinking, once the KIP gets voted, we > should > > > try > > > > to > > > > > >> > > release > > > > > >> > > > > > kafka:3.7.0 (or 3.7.1) Docker Official image. This > will > > > help > > > > > us > > > > > >> to > > > > > >> > > > > > validate the process and allow us to fix any changes > > > > suggested > > > > > >> by > > > > > >> > > > > > Dockerhub before the 3.8.0 release. > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > Thanks, > > > > > >> > > > > > > > > > > >> > > > > > On Mon, Apr 8, 2024 at 2:33 PM Krish Vora < > > > > > >> krishvor...@gmail.com> > > > > > >> > > > wrote: > > > > > >> > > > > > > > > > > > >> > > > > > > Hi Manikumar and Luke. > > > > > >> > > > > > > Thanks for the questions. > > > > > >> > > > > > > > > > > > >> > > > > > > 1. No, the Docker inventory files and configurations > > > will > > > > > not > > > > > >> be > > > > > >> > > the > > > > > >> > > > same > > > > > >> > > > > > > for Open Source Software (OSS) Images and Docker > > > Official > > > > > >> Images > > > > > >> > > > (DOI). > > > > > >> > > > > > > > > > > > >> > > > > > > For OSS images, the Dockerfile located in > > > > > >> docker/jvm/dockerfile > > > > > >> > is > > > > > >> > > > > > > utilized. This process is integrated with the > existing > > > > > release > > > > > >> > > > pipeline > > > > > >> > > > > > as > > > > > >> > > > > > > outlined in KIP-975 > > > > > >> > > > > > > < > > > > > >> > > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka#KIP975:DockerImageforApacheKafka-Status > > > > > >> > > > > > >, > > > > > >> > > > > > > where the Kafka URL is provided as a build argument. > > > This > > > > > >> method > > > > > >> > > > allows > > > > > >> > > > > > for > > > > > >> > > > > > > building, testing, and releasing OSS images > > dynamically. > > > > The > > > > > >> OSS > > > > > >> > > > images > > > > > >> > > > > > > will continue to be released under the standard > > release > > > > > >> process . > > > > > >> > > > > > > > > > > > >> > > > > > > In contrast, the release process for DOIs requires > > > > providing > > > > > >> the > > > > > >> > > > Docker > > > > > >> > > > > > Hub > > > > > >> > > > > > > team with a specific directory for each version > > release > > > > that > > > > > >> > > > contains a > > > > > >> > > > > > > standalone Dockerfile. These Dockerfiles are > designed > > to > > > > be > > > > > >> > > > > > > self-sufficient, hence require hardcoded values > > instead > > > of > > > > > >> > relying > > > > > >> > > on > > > > > >> > > > > > build > > > > > >> > > > > > > arguments. To accommodate this, in our proposed > > > approach, > > > > a > > > > > >> new > > > > > >> > > > directory > > > > > >> > > > > > > named docker_official_images has been created. This > > > > > directory > > > > > >> > > > contains > > > > > >> > > > > > > version-specific directories, having Dockerfiles > with > > > > > >> hardcoded > > > > > >> > > > > > > configurations for each release, acting as the > source > > of > > > > > truth > > > > > >> > for > > > > > >> > > > DOI > > > > > >> > > > > > > releases. The hardcoded dockerfiles will be created > > > using > > > > > the > > > > > >> > > > > > > docker/jvm/dockerfile as a template. Thus, as part > of > > > post > > > > > >> > release > > > > > >> > > we > > > > > >> > > > > > will > > > > > >> > > > > > > be creating a Dockerfile that will be reviewed by > the > > > > > >> Dockerhub > > > > > >> > > > community > > > > > >> > > > > > > and might need changes as per their review. This > > > approach > > > > > >> ensures > > > > > >> > > > that > > > > > >> > > > > > DOIs > > > > > >> > > > > > > are built consistently and meet the specific > > > requirements > > > > > set > > > > > >> by > > > > > >> > > > Docker > > > > > >> > > > > > Hub. > > > > > >> > > > > > > > > > > > >> > > > > > > 2. Yes Manikumar, transitioning the release of > Docker > > > > > Official > > > > > >> > > Images > > > > > >> > > > > > (DOI) > > > > > >> > > > > > > to a post-release activity does address the concerns > > > about > > > > > >> > > > complicating > > > > > >> > > > > > the > > > > > >> > > > > > > release process. Initially, we considered > > incorporating > > > > DOI > > > > > >> > release > > > > > >> > > > > > > directly into Kafka's release workflow. However, > this > > > > > approach > > > > > >> > > > > > > significantly increased the RMs workload due to the > > > > addition > > > > > >> of > > > > > >> > > > numerous > > > > > >> > > > > > > steps, complicating the process. By designating the > > DOI > > > > > >> release > > > > > >> > as > > > > > >> > > a > > > > > >> > > > > > > post-release task, we maintain the original release > > > > process. > > > > > >> This > > > > > >> > > > > > > adjustment allows for the DOI release to be done > after > > > the > > > > > >> main > > > > > >> > > > release. > > > > > >> > > > > > We > > > > > >> > > > > > > have revised the KIP to reflect that DOI releases > will > > > now > > > > > >> occur > > > > > >> > > > after > > > > > >> > > > > > the > > > > > >> > > > > > > main release phase. Please review the updated > document > > > and > > > > > >> > provide > > > > > >> > > > any > > > > > >> > > > > > > feedback you might have. > > > > > >> > > > > > > > > > > > >> > > > > > > Thanks, > > > > > >> > > > > > > Krish. > > > > > >> > > > > > > > > > > > >> > > > > > > On Wed, Apr 3, 2024 at 3:35 PM Luke Chen < > > > > show...@gmail.com > > > > > > > > > > > >> > > wrote: > > > > > >> > > > > > > > > > > > >> > > > > > > > Hi Krishna, > > > > > >> > > > > > > > > > > > > >> > > > > > > > I also have the same question as Manikumar raised: > > > > > >> > > > > > > > 1. Will the Docker inventory files/etc are the > same > > > for > > > > > OSS > > > > > >> > Image > > > > > >> > > > and > > > > > >> > > > > > > > Docker Official Images? > > > > > >> > > > > > > > If no, then why not? Could we make them identical > so > > > > that > > > > > we > > > > > >> > > don't > > > > > >> > > > > > have to > > > > > >> > > > > > > > build 2 images for each release? > > > > > >> > > > > > > > > > > > > >> > > > > > > > Thank you. > > > > > >> > > > > > > > Luke > > > > > >> > > > > > > > > > > > > >> > > > > > > > On Wed, Apr 3, 2024 at 12:41 AM Manikumar < > > > > > >> > > > manikumar.re...@gmail.com> > > > > > >> > > > > > > > wrote: > > > > > >> > > > > > > > > > > > > >> > > > > > > > > Hi Krishna, > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Thanks for the KIP. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > I think Docker Official Images will be > beneficial > > to > > > > the > > > > > >> > Kafka > > > > > >> > > > > > community. > > > > > >> > > > > > > > > Few queries below. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > 1. Will the Docker inventory files/etc are the > > same > > > > for > > > > > >> OSS > > > > > >> > > > Image and > > > > > >> > > > > > > > > Docker Official Images > > > > > >> > > > > > > > > 2. I am a bit worried about the new steps to the > > > > release > > > > > >> > > process. > > > > > >> > > > > > Maybe > > > > > >> > > > > > > > we > > > > > >> > > > > > > > > should consider Docker Official Images release > as > > > > > >> > Post-Release > > > > > >> > > > > > activity. > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Thanks, > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > On Fri, Mar 22, 2024 at 3:29 PM Krish Vora < > > > > > >> > > > krishvor...@gmail.com> > > > > > >> > > > > > > > wrote: > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > Hi Hector, > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > Thanks for reaching out. This KIP builds on > top > > of > > > > > >> KIP-975 > > > > > >> > > > > > > > > > < > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > and > > > > > >> > > > > > > > > > aims to introduce a JVM-based Docker Official > > > Image > > > > > (DOI > > > > > >> > > > > > > > > > < > > > > > >> https://docs.docker.com/trusted-content/official-images/ > > > > > >> > >) > > > > > >> > > > for > > > > > >> > > > > > Apache > > > > > >> > > > > > > > > > Kafka that will be visible under Docker > Official > > > > > Images > > > > > >> > > > > > > > > > < > > > > > https://hub.docker.com/search?image_filter=official&q= > > > > > >> >. > > > > > >> > > Once > > > > > >> > > > > > > > > implemented > > > > > >> > > > > > > > > > for Apache Kafka, for each release, there will > > be > > > > one > > > > > >> more > > > > > >> > > > > > JVM-based > > > > > >> > > > > > > > > Docker > > > > > >> > > > > > > > > > image available to users. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > Currently, we already have an OSS sponsored > > image, > > > > > which > > > > > >> > was > > > > > >> > > > > > introduced > > > > > >> > > > > > > > > via > > > > > >> > > > > > > > > > KIP-975 (apache/kafka < > > > > > >> > > > https://hub.docker.com/r/apache/kafka/tags > > > > > >> > > > > > >) > > > > > >> > > > > > > > > which > > > > > >> > > > > > > > > > comes under The Apache Software Foundation < > > > > > >> > > > > > > > > > https://hub.docker.com/u/apache> in > > > > > >> > > > > > > > > > Docker Hub. The new Docker Image is the Docker > > > > > Official > > > > > >> > Image > > > > > >> > > > > > (DOI), > > > > > >> > > > > > > > > which > > > > > >> > > > > > > > > > will be built and maintained by Docker > > Community. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > For example, for a release version like 3.8.0 > we > > > > will > > > > > >> have > > > > > >> > > two > > > > > >> > > > JVM > > > > > >> > > > > > > > based > > > > > >> > > > > > > > > > docker images:- > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > - apache/kafka:3.8.0 (OSS sponsored image) > > > > > >> > > > > > > > > > - kafka:3.8.0 (Docker Official image) > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > I have added the same in the KIP too for > > > everyone's > > > > > >> > > reference. > > > > > >> > > > > > > > > > Thanks, > > > > > >> > > > > > > > > > Krish. > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > On Fri, Mar 22, 2024 at 2:50 AM Hector > Geraldino > > > > > >> > (BLOOMBERG/ > > > > > >> > > > 919 > > > > > >> > > > > > 3RD > > > > > >> > > > > > > > A) < > > > > > >> > > > > > > > > > hgerald...@bloomberg.net> wrote: > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > Hi, > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > What is the difference between this KIP and > > > > KIP-975: > > > > > >> > Docker > > > > > >> > > > > > Image for > > > > > >> > > > > > > > > > > Apache Kafka? > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > From: dev@kafka.apache.org At: 03/21/24 > > > 07:30:07 > > > > > >> > > UTC-4:00To: > > > > > >> > > > > > > > > > > dev@kafka.apache.org > > > > > >> > > > > > > > > > > Subject: [DISCUSS] KIP-1028: Docker Official > > > Image > > > > > for > > > > > >> > > Apache > > > > > >> > > > > > Kafka > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > Hi everyone, > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > I would like to start the discussion on > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1028%3A+Docker+Official+Im > > > > > >> > > > > > > > > > > age+for+Apache+Kafka > > > > > >> > > > > > > > > > > . > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > This KIP aims to introduce JVM based Docker > > > > Official > > > > > >> > Image > > > > > >> > > > (DOI) > > > > > >> > > > > > for > > > > > >> > > > > > > > > > Apache > > > > > >> > > > > > > > > > > Kafka. > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > Regards, > > > > > >> > > > > > > > > > > Krish. > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > >> -- > > > > > >> > > > > > >> [image: Confluent] <https://www.confluent.io> > > > > > >> Prabha Manepalli > > > > > >> Product Manager for Confluent Platform Security, Docker > > > > > >> linkedin.com/in/prabhamanepalli > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > [image: Confluent] <https://www.confluent.io> > > Prabha Manepalli > > Product Manager for Confluent Platform Security, Docker Containers > > linkedin.com/in/prabhamanepalli > > >
