Hi Vedarth, This looks great, thank you for helping me thoroughly understand the motivation and benefits for the KIP. Looks good to me.
Regarding public interface for the images--everything in the "Public Interface" section in KIP-975 does qualify as public interface for the images IMO, but I don't think it's comprehensive. If we were asked to, for example, change the port in the EXPOSE directive in our Dockerfile, that would probably qualify as a change to public interface too. With the strict language in the latest draft of this KIP that ensures that any functional changes to our Docker images go through another follow-up KIP, we should be fine without having to identify a comprehensive list of everything that constitutes public interface for our images. Cheers, and thanks again for the KIP, Chris On Mon, May 13, 2024 at 3:07 PM Vedarth Sharma <vedarth.sha...@gmail.com> wrote: > Hey Chris, > > Once we provide the definitions to docker, they should take care of > everything from there. They mentioned here > < > https://github.com/docker-library/official-images?tab=readme-ov-file#library-definition-files > > > that > the image will be rebuilt when the base image is updated. Hence active > rebuilds won't require any changes from our side. > If we are packaging something which may contain a CVE, like some jar, then > the onus will be on us to patch it, but it will be upto us whether we > consider the threat severe enough to fix and when we want to provide the > fixed version. Having Docker Official Image will not impact the frequency > of our releases. It will be the Apache Kafka community's call on when a > release goes and Docker Official Image will be released accordingly as per > the KIP. source > <https://github.com/docker-library/faq?tab=readme-ov-file#image-building> > > As mentioned in Docker's documentation as well "In essence we strive to > heed upstream's recommendations on how they intend for their software to be > consumed." source > < > https://github.com/docker-library/official-images?tab=readme-ov-file#what-are-official-images > > > Docker Official Image will rely on upstream's recommendation for > functionality. But I do agree that since Docker's stance on this might > change in future it makes sense to put a safeguard that will not allow any > functionality changes get incorporated as part of the vetting process. I > have updated the KIP to reflect the same. > > KIP-975 has a well defined public interface based on how configs can be > supplied and how it can be used. I am not sure if we put that label on it > during discussions. I am happy to have a separate email thread on it to > iron things out. > > I hope this addresses all of your concerns! > > Thanks and regards, > Vedarth > > On Mon, May 13, 2024 at 10:55 PM Chris Egerton <chr...@aiven.io.invalid> > wrote: > > > Thanks both for your responses! Friendly reminder: again, better to > provide > > a quote instead of just a link :) > > > > I've seen a bit about image rebuilding to handle CVEs but I'm a little > > unclear on how this would work in practice, and I couldn't find any > > concrete details in any of the links. Does Docker do this automatically > for > > DOIs? Or will the onus be on us to put out patched images? Would this > lead > > to us putting out images more quickly than we put out standard releases? > As > > a plus, it does look like DOIs get the benefit of Docker Scout [1] for > > free, which is nice, but it's still unclear who'd be doing the rest of > the > > work on that front. > > > > As far as this point from Vedarth goes: > > > > > By incorporating the source code of the Docker Official Image into our > > > AK ecosystem, we gain control over its functionality, ensuring > alignment > > > with the OSS Docker image. This ensures a seamless experience for users > > who > > > may need to transition between these images. > > > > This captures my concern with the KIP pretty well. If there's any > > significant divergence in behavior (not just build methodology) between > the > > apache/kafka image and what Docker requires for a Kafka DOI, how are we > > going to vet these changes moving forward? Under the "Post Release > Process > > - if Dockerhub folks suggest changes to the Dockerfiles:" header, this > KIP > > proposes that we port all suggested changes for the DOI to > > the docker/jvm/Dockerfile image, but this seems a bit too permissive. As > an > > alternative, we could state that all build-related changes can be done > with > > a PR on the apache/kafka GitHub repo (which will require approval from a > > single committer), but any functional changes will require a KIP. > > > > Finally, during KIP-975 was there discussion on what we would count as > the > > public interface for the apache/kafka image? If not, it'd be nice to get > > that ironed out since it may make future discussions around our Docker > > images quicker, but I don't think this is necessary for KIP-1028. > > > > [1] - https://www.docker.com/products/docker-scout/ > > > > On Mon, May 13, 2024 at 4:37 AM Prabha Manepalli > > <mpra...@confluent.io.invalid> wrote: > > > > > Hi Chris, > > > > > > Sharing the requested links explaining why Docker Official images are > > > considered more secure - > > > > > > > > > https://www.docker.com/blog/enhancing-security-and-transparency-with-docker-official-images/ > > > and > > > > > > > > > https://github.com/docker-library/faq#why-does-my-security-scanner-show-that-an-image-has-cves > > > > > > I hope these links help you understand why we need Docker Official > images > > > for organisations with stringent security compliance requirements for > > their > > > Kafka workloads. > > > > > > Thank you. > > > > > > > > > > > > On Sun, May 12, 2024 at 3:33 PM Vedarth Sharma < > vedarth.sha...@gmail.com > > > > > > wrote: > > > > > > > Hey Chris! > > > > > > > > Functionality wise, we don't intend to have any differences between > OSS > > > > Docker Image and Docker Official Image. > > > > The Docker Official Image will be the recommended one. > > > > Since the Docker Official Image might be delayed due to review done > by > > > > Docker, images on apache/kafka (OSS Docker Image) can be used by > users. > > > > > > > > 1) I read https://docs.docker.com/trusted-content/official-images/ > and > > > > > didn't find anything on that page or immediately around it that > > > explains > > > > > what compliance requirements might be satisfied by a DOI that > > couldn't > > > be > > > > > satisfied by the existing apache/kafka image. Can you elaborate? > Feel > > > > free > > > > > to provide another link, but please also quote the relevant > sections > > > from > > > > > it (as StackOverflow likes to say, links can grow stale over time). > > > > > > > > > > > > - If a user's selection is confined solely to Docker Official > > Images, > > > > this Docker Image will ensure their access to Kafka. > > > > - Details on specific advantages of Docker Official Images can be > > > found > > > > at > > > > > > > > > > > > > > https://github.com/docker-library/official-images?tab=readme-ov-file#what-are-official-images > > > > . > > > > - The Docker Official Image will be actively rebuilt for updates > and > > > > security fixes. > > > > - It's true we can provide exactly the same benefits in the OSS > > Docker > > > > Image as well. But it won't have the Docker Official Image badge > and > > > it > > > > will add additional overhead for Apache Kafka community. > > > > - The fact that it will have Docker Official Image badge will set > it > > > > apart from the OSS Docker Image. > > > > - Also the ability to do just docker pull kafka to get the kafka > > > docker > > > > image will only be possible with Docker Official Image. > > > > > > > > > > > > 2) It would be great to see a brief summary of the differences in > these > > > > > images included in the KIP, in order to try to gauge how this would > > > look > > > > to > > > > > our users. > > > > > > > > > > > > - Functionally, both Docker images will remain identical. > > > > - The variance lies primarily in the methodologies of building and > > > > validation, as outlined in the updated KIP. > > > > > > > > > > > > 3) What I suggested last time was not a separate apache/apache-docker > > > > > repository, but a repository controlled entirely by Docker. The DOI > > > docs > > > > > [1] state that "While it's preferable to have upstream software > > authors > > > > > maintaining their Docker Official Images, this isn't a strict > > > > requirement", > > > > > which I take to mean that it's not required for an Apache Kafka DOI > > to > > > > live > > > > > under the apache organization on GitHub. It also seems like there's > > > > > precedent for this: images for MySQL [2] and PHP [3] already exist > > > under > > > > > the control of Docker. The reason I think this is worth considering > > is > > > > that > > > > > Docker can arbitrarily change the eligibility requirements for > their > > > > > official images at any time, and it doesn't seem like there's a > clear > > > > > process in the KIP for judging how we should respond to these > changes > > > (in > > > > > fact, it seems like the idea in the KIP is that we should make any > > > change > > > > > required with no further vetting beyond possibly a pull request on > > > > > apache/kafka, which would require approval by a committer). By > > hosting > > > > the > > > > > DOI definitions ourselves (either in apache/kafka, or in a > > theoretical > > > > > apache/docker-kafka repository), we take responsibility for the > > image, > > > > even > > > > > if the owner on Docker Hub is Docker, not Apache. If the code lives > > > > > elsewhere, then (as long as basic trademark and possibly security > > > > > guidelines are respected) Apache doesn't have to concern itself at > > all > > > > with > > > > > the image and the maintainers are free to make whatever changes > they > > > want > > > > > to it in order to meet Docker's requirements. > > > > > > > > > > > > - By incorporating the source code of the Docker Official Image > into > > > our > > > > AK ecosystem, we gain control over its functionality, ensuring > > > alignment > > > > with the OSS Docker image. This ensures a seamless experience for > > > users > > > > who > > > > may need to transition between these images. > > > > - Maintaining both images within the same community facilitates > ease > > > of > > > > management and fosters a singular source of truth. > > > > - While Apache may not retain ownership of the hosted Docker > > Official > > > > Image, we are, in essence, providing Docker with a foundation that > > > > aligns > > > > with their established guidelines as well as remains consistent > with > > > OSS > > > > Docker Image apache/kafka. > > > > - Any future alterations to the functionality can be seamlessly > > > > propagated across both the OSS and Official Docker Images. > > > > > > > > I think these reasons must be why a lot of the Apache projects choose > > to > > > > host the docker definitions themselves. The responsibility of owning > > the > > > > definitions comes with benefits as well that we should also consider. > > > > > > > > Let us know if you have any further questions! > > > > > > > > Thanks and regards, > > > > Vedarth > > > > > > > > On Fri, May 10, 2024 at 7:56 PM Chris Egerton > <chr...@aiven.io.invalid > > > > > > > wrote: > > > > > > > > > Hi Krish and Prabha, > > > > > > > > > > Thanks for your replies. I still have some follow-up questions: > > > > > > > > > > 1) I read https://docs.docker.com/trusted-content/official-images/ > > and > > > > > didn't find anything on that page or immediately around it that > > > explains > > > > > what compliance requirements might be satisfied by a DOI that > > couldn't > > > be > > > > > satisfied by the existing apache/kafka image. Can you elaborate? > Feel > > > > free > > > > > to provide another link, but please also quote the relevant > sections > > > from > > > > > it (as StackOverflow likes to say, links can grow stale over time). > > > > > > > > > > 2) It would be great to see a brief summary of the differences in > > these > > > > > images included in the KIP, in order to try to gauge how this would > > > look > > > > to > > > > > our users. > > > > > > > > > > 3) What I suggested last time was not a separate > apache/apache-docker > > > > > repository, but a repository controlled entirely by Docker. The DOI > > > docs > > > > > [1] state that "While it's preferable to have upstream software > > authors > > > > > maintaining their Docker Official Images, this isn't a strict > > > > requirement", > > > > > which I take to mean that it's not required for an Apache Kafka DOI > > to > > > > live > > > > > under the apache organization on GitHub. It also seems like there's > > > > > precedent for this: images for MySQL [2] and PHP [3] already exist > > > under > > > > > the control of Docker. The reason I think this is worth considering > > is > > > > that > > > > > Docker can arbitrarily change the eligibility requirements for > their > > > > > official images at any time, and it doesn't seem like there's a > clear > > > > > process in the KIP for judging how we should respond to these > changes > > > (in > > > > > fact, it seems like the idea in the KIP is that we should make any > > > change > > > > > required with no further vetting beyond possibly a pull request on > > > > > apache/kafka, which would require approval by a committer). By > > hosting > > > > the > > > > > DOI definitions ourselves (either in apache/kafka, or in a > > theoretical > > > > > apache/docker-kafka repository), we take responsibility for the > > image, > > > > even > > > > > if the owner on Docker Hub is Docker, not Apache. If the code lives > > > > > elsewhere, then (as long as basic trademark and possibly security > > > > > guidelines are respected) Apache doesn't have to concern itself at > > all > > > > with > > > > > the image and the maintainers are free to make whatever changes > they > > > want > > > > > to it in order to meet Docker's requirements. > > > > > > > > > > [1] - > > > > > > > https://docs.docker.com/trusted-content/official-images/contributing/, > > > > > second paragraph under "Contributing to Docker Official Images" > > > > > [2] - https://github.com/docker-library/mysql > > > > > [3] - https://github.com/docker-library/php > > > > > > > > > > Cheers, > > > > > > > > > > Chris > > > > > > > > > > On Fri, May 10, 2024 at 7:46 AM Krish Vora <krishvor...@gmail.com> > > > > wrote: > > > > > > > > > > > Hey Chris, > > > > > > > > > > > > We have responded to the initial round of queries. And as you > > > mentioned > > > > > in > > > > > > your comment, you may have more questions related to this KIP. > > Please > > > > let > > > > > > us know if you have any. > > > > > > We want to start the voting for this KIP soon, as we intend to > > > include > > > > it > > > > > > in the 3.8.0 release. > > > > > > > > > > > > Thanks. > > > > > > > > > > > > Regards, > > > > > > > > > > > > Krish. > > > > > > > > > > > > On Wed, May 8, 2024 at 6:19 PM Krish Vora <krishvor...@gmail.com > > > > > > wrote: > > > > > > > > > > > > > Hi Chris. Thanks for the questions. > > > > > > > > > > > > > > 3. Would a separate Docker-owned repository be out of the > > question? > > > > I'm > > > > > > >> guessing there are some trademark issues that might get in the > > > way, > > > > > but > > > > > > >> it's worth exploring since the entire purpose of this KIP > seems > > to > > > > be > > > > > to > > > > > > >> provide images that are vetted and designed by Docker more > than > > by > > > > the > > > > > > >> Apache Kafka contributors/committers/PMC. > > > > > > > > > > > > > > > > > > > > > > > > > > > > - The process for introducing a Docker Official Image > involves > > > > > > > - Hosting the Dockerfile in the Apache Kafka repository > and > > > > > > > - Providing the path to this Dockerfile to Docker Hub in > > > Docker > > > > > > > Hub’s own repo > > > > > > > < > > > > > > > > > https://github.com/docker-library/official-images/tree/master/library> > > > > > > > . > > > > > > > - This ensures that any updates to the Dockerfile in the AK > > > > > repository > > > > > > > are directly applicable to the docker official images > > available > > > on > > > > > > Docker > > > > > > > Hub. > > > > > > > > > > > > > > > > > > > > > - We also did not find any added advantage to create a > > separate > > > > > > > repository named apache-docker within the Apache GitHub > > > > > organization. > > > > > > > > > > > > > > Thanks, > > > > > > > Krish. > > > > > > > > > > > > > > On Wed, May 8, 2024 at 6:05 PM Prabha Manepalli > > > > > > > <mpra...@confluent.io.invalid> wrote: > > > > > > > > > > > > > >> Hi Chris, I would like to add more context to this KIP's > > > > motivation. > > > > > > >> Vedarth and Krish, please weigh in with your inputs. > > > > > > >> > > > > > > >> In the motivation section it's stated that "Several other > Apache > > > > > > projects, > > > > > > >> > like Flink, Spark, Solr, have already released Docker > Official > > > > > Images, > > > > > > >> with > > > > > > >> > download figures ranging from 50 million to over 1 billion. > > > These > > > > > > >> numbers > > > > > > >> > highlight the significant demand among users." But then > > > > immediately > > > > > > >> > afterwards, we learn that "Also the Docker Official Images > are > > > > > always > > > > > > >> the > > > > > > >> > top 1 search result, irrespective of the number of > downloads." > > > > > > Wouldn't > > > > > > >> a > > > > > > >> > high number of downloads for an image naturally follow from > > > being > > > > > the > > > > > > >> top > > > > > > >> > search result? It seems like we can't necessarily assume > that > > > > Docker > > > > > > >> > Official Images are inherently more desirable for users > based > > > > solely > > > > > > on > > > > > > >> > download statistics. > > > > > > >> > > > > > > > >> > > > > > > >> *My thoughts: *Unlike the Sponsored OSS image, the Docker > > Official > > > > > image > > > > > > >> is > > > > > > >> more desirable for workloads that have stringent compliance > > > > > > requirements. > > > > > > >> More details on why official images are more trusted are > > > documented > > > > > here > > > > > > >> <https://docs.docker.com/trusted-content/official-images/>. > The > > > > > Docker > > > > > > >> Official image would also help an absolutely new Kafka > beginner > > > who > > > > > > might > > > > > > >> not know about Apache or the concept of Sponsored images. We > > want > > > to > > > > > > make > > > > > > >> it easier for Kafka beginners to discover the Kafka image > > through > > > > > > >> DockerHub. > > > > > > >> > > > > > > >> > > > > > > >> Can you elaborate on the value that these new images would add > > > from > > > > a > > > > > > >> > user's perspective? I'm hesitant to introduce another image, > > > since > > > > > it > > > > > > >> adds > > > > > > >> > to the cognitive burden of people who will inevitably have > to > > > > answer > > > > > > the > > > > > > >> > question of "What are the differences between all of the > > > available > > > > > > >> images > > > > > > >> > and which one is best for my use case?" > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> *My thoughts: *This is a valid concern to address. The > response > > to > > > > the > > > > > > >> above question addresses the value-add this new Docker > Official > > > > image > > > > > > >> would > > > > > > >> provide. I also agree we need a clear distinction between each > > of > > > > > these > > > > > > >> images to be well documented. We plan to update the AK website > > > with > > > > > > >> details > > > > > > >> on how, why, and when a developer would want to use each of > > these > > > > > > >> particular images(KIP-974,975,1028). > > > > > > >> > > > > > > >> Thanks, > > > > > > >> Prabha. > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> On Tue, Apr 30, 2024 at 9:41 PM Chris Egerton > > > > <chr...@aiven.io.invalid > > > > > > > > > > > > >> wrote: > > > > > > >> > > > > > > >> > Hi Vedarth and Krish, > > > > > > >> > > > > > > > >> > Thanks for the KIP! I have to admit I'm a little skeptical; > > > > > hopefully > > > > > > >> you > > > > > > >> > can help me understand the need for these additional images. > > > > > > >> > > > > > > > >> > 1) In the motivation section it's stated that "Several other > > > > Apache > > > > > > >> > projects, like Flink, Spark, Solr, have already released > > Docker > > > > > > Official > > > > > > >> > Images, with download figures ranging from 50 million to > over > > 1 > > > > > > billion. > > > > > > >> > These numbers highlight the significant demand among users." > > But > > > > > then > > > > > > >> > immediately afterwards, we learn that "Also the Docker > > Official > > > > > Images > > > > > > >> are > > > > > > >> > always the top 1 search result, irrespective of the number > of > > > > > > >> downloads." > > > > > > >> > Wouldn't a high number of downloads for an image naturally > > > follow > > > > > from > > > > > > >> > being the top search result? It seems like we can't > > necessarily > > > > > assume > > > > > > >> that > > > > > > >> > Docker Official Images are inherently more desirable for > users > > > > based > > > > > > >> solely > > > > > > >> > on download statistics. > > > > > > >> > > > > > > > >> > 2) Can you elaborate on the value that these new images > would > > > add > > > > > > from a > > > > > > >> > user's perspective? I'm hesitant to introduce another image, > > > since > > > > > it > > > > > > >> adds > > > > > > >> > to the cognitive burden of people who will inevitably have > to > > > > answer > > > > > > the > > > > > > >> > question of "What are the differences between all of the > > > available > > > > > > >> images > > > > > > >> > and which one is best for my use case?" > > > > > > >> > > > > > > > >> > 3) Would a separate Docker-owned repository be out of the > > > > question? > > > > > > I'm > > > > > > >> > guessing there are some trademark issues that might get in > the > > > > way, > > > > > > but > > > > > > >> > it's worth exploring since the entire purpose of this KIP > > seems > > > to > > > > > be > > > > > > to > > > > > > >> > provide images that are vetted and designed by Docker more > > than > > > by > > > > > the > > > > > > >> > Apache Kafka contributors/committers/PMC. > > > > > > >> > > > > > > > >> > I may have more questions later but wanted to get this > initial > > > > round > > > > > > out > > > > > > >> > now without trying to list everything first. > > > > > > >> > > > > > > > >> > Looking forward to your thoughts! > > > > > > >> > > > > > > > >> > Cheers, > > > > > > >> > > > > > > > >> > Chris > > > > > > >> > > > > > > > >> > On Mon, Apr 22, 2024 at 2:14 PM Vedarth Sharma < > > > > > > >> vedarth.sha...@gmail.com> > > > > > > >> > wrote: > > > > > > >> > > > > > > > >> > > Hey folks, > > > > > > >> > > > > > > > > >> > > Thanks a lot for reviewing the KIP and providing feedback. > > > > > > >> > > The discussion thread seems resolved and KIP has been > > updated > > > > > > >> > accordingly. > > > > > > >> > > We will be starting the voting thread for this KIP in the > > next > > > > few > > > > > > >> days. > > > > > > >> > > Please take a look at the KIP and let us know if any > further > > > > > > >> discussion > > > > > > >> > > is needed. > > > > > > >> > > > > > > > > >> > > Thanks and regards, > > > > > > >> > > Vedarth > > > > > > >> > > > > > > > > >> > > On Fri, Apr 19, 2024 at 1:33 PM Manikumar < > > > > > > manikumar.re...@gmail.com> > > > > > > >> > > wrote: > > > > > > >> > > > > > > > > >> > > > Thanks Krish. KIP looks good to me. > > > > > > >> > > > > > > > > > >> > > > On Wed, Apr 17, 2024 at 1:38 PM Krish Vora < > > > > > krishvor...@gmail.com > > > > > > > > > > > > > >> > > wrote: > > > > > > >> > > > > > > > > > > >> > > > > Hi Manikumar, > > > > > > >> > > > > > > > > > > >> > > > > Thanks for the comments. > > > > > > >> > > > > > > > > > > >> > > > > Maybe as part of the release process, RM can create a > > JIRA > > > > for > > > > > > >> this > > > > > > >> > > > > > task. This can be taken by RM or any comitter or any > > > > > > contributor > > > > > > >> > > (with > > > > > > >> > > > > > some help from commiters to run "Docker Image > > > Preparation > > > > > via > > > > > > >> > GitHub > > > > > > >> > > > > > Actions:" > > > > > > >> > > > > > > > > > > >> > > > > This sounds like a good idea. This step would be > > > beneficial. > > > > > By > > > > > > >> > > creating > > > > > > >> > > > a > > > > > > >> > > > > JIRA ticket, it will also serve as a reminder to > > complete > > > > the > > > > > > >> > > > post-release > > > > > > >> > > > > steps for the Docker official images. Have updated the > > KIP > > > > > with > > > > > > >> this > > > > > > >> > > > step. > > > > > > >> > > > > > > > > > > >> > > > > Is this using GitHub Actions workflow? or manual > > testing? > > > > > > >> > > > > > > > > > > >> > > > > This will be done by a Github Actions workflow, which > > will > > > > > test > > > > > > >> the > > > > > > >> > > > static > > > > > > >> > > > > Docker Official Image assets for a specific release > > > version. > > > > > > >> > > > > > > > > > > >> > > > > Is it mandatory for RM/comitters to raise the PR to > > Docker > > > > > Hub’s > > > > > > >> > > > > > official images repository (or) can it be done by > any > > > > > > >> contributor. > > > > > > >> > > > > > > > > > > >> > > > > I believe that it can be done by any contributor (ref: > > > This > > > > > link > > > > > > >> > > > > < > > > > > > >> > > > > > > > > https://docs.docker.com/trusted-content/official-images/contributing/ > > > > > > >> > > > > > > > > > >> > > > > quotes "*Anyone can provide feedback, contribute code, > > > > suggest > > > > > > >> > process > > > > > > >> > > > > changes, or even propose a new Official Image.*") > > > > > > >> > > > > > > > > > > >> > > > > Also I was thinking, once the KIP gets voted, we > should > > > try > > > > to > > > > > > >> > release > > > > > > >> > > > > > kafka:3.7.0 (or 3.7.1) Docker Official image. This > > will > > > > help > > > > > > us > > > > > > >> to > > > > > > >> > > > > > validate the process and allow us to fix any changes > > > > > suggested > > > > > > >> by > > > > > > >> > > > > > Dockerhub before the 3.8.0 release. > > > > > > >> > > > > > > > > > > >> > > > > This sounds like a great idea. This KIP proposes > release > > > of > > > > > DOI > > > > > > >> as a > > > > > > >> > > > > post-release process, which can be done anytime post > > > > release. > > > > > > >> Since > > > > > > >> > > 3.7.0 > > > > > > >> > > > > is already released, we can perform these steps for > that > > > > > release > > > > > > >> too. > > > > > > >> > > By > > > > > > >> > > > > the time the KIP gets implemented, if 3.7.1 is > released, > > > we > > > > > > could > > > > > > >> do > > > > > > >> > > > these > > > > > > >> > > > > steps for 3.7.1, instead of 3.7.0. This would allow us > > to > > > > make > > > > > > >> > changes > > > > > > >> > > to > > > > > > >> > > > > the Dockerfiles and other assets based on feedback > from > > > > Docker > > > > > > Hub > > > > > > >> > > before > > > > > > >> > > > > the release of version 3.8.0. > > > > > > >> > > > > > > > > > > >> > > > > Thanks, > > > > > > >> > > > > Krish. > > > > > > >> > > > > > > > > > > >> > > > > On Mon, Apr 15, 2024 at 12:59 PM Manikumar < > > > > > > >> > manikumar.re...@gmail.com> > > > > > > >> > > > > wrote: > > > > > > >> > > > > > > > > > > >> > > > > > Hi Krish, > > > > > > >> > > > > > > > > > > > >> > > > > > Thanks for the updated KIP. a few comments below. > > > > > > >> > > > > > > > > > > > >> > > > > > > "These actions can be carried out by the RM or any > > > > > > contributor > > > > > > >> > post > > > > > > >> > > > the > > > > > > >> > > > > > release process." > > > > > > >> > > > > > Maybe as part of the release process, RM can create > a > > > JIRA > > > > > for > > > > > > >> this > > > > > > >> > > > > > task. This can be taken by RM or any comitter or any > > > > > > contributor > > > > > > >> > > (with > > > > > > >> > > > > > some help from commiters to run "Docker Image > > > Preparation > > > > > via > > > > > > >> > GitHub > > > > > > >> > > > > > Actions:" > > > > > > >> > > > > > > > > > > > >> > > > > > > "Perform Docker build tests to ensure image > > integrity" > > > > > > >> > > > > > Is this using GitHub Actions workflow? or manual > > > testing? > > > > > > >> > > > > > > > > > > > >> > > > > > > "The RM will manually raise the final PR to Docker > > > Hub’s > > > > > > >> official > > > > > > >> > > > images > > > > > > >> > > > > > repository using the contents of the generated file" > > > > > > >> > > > > > Is it mandatory for RM/comitters to raise the PR to > > > > Docker > > > > > > >> Hub’s > > > > > > >> > > > > > official images repository (or) can it be done by > any > > > > > > >> contributor. > > > > > > >> > > > > > > > > > > > >> > > > > > Also I was thinking, once the KIP gets voted, we > > should > > > > try > > > > > to > > > > > > >> > > release > > > > > > >> > > > > > kafka:3.7.0 (or 3.7.1) Docker Official image. This > > will > > > > help > > > > > > us > > > > > > >> to > > > > > > >> > > > > > validate the process and allow us to fix any changes > > > > > suggested > > > > > > >> by > > > > > > >> > > > > > Dockerhub before the 3.8.0 release. > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > Thanks, > > > > > > >> > > > > > > > > > > > >> > > > > > On Mon, Apr 8, 2024 at 2:33 PM Krish Vora < > > > > > > >> krishvor...@gmail.com> > > > > > > >> > > > wrote: > > > > > > >> > > > > > > > > > > > > >> > > > > > > Hi Manikumar and Luke. > > > > > > >> > > > > > > Thanks for the questions. > > > > > > >> > > > > > > > > > > > > >> > > > > > > 1. No, the Docker inventory files and > configurations > > > > will > > > > > > not > > > > > > >> be > > > > > > >> > > the > > > > > > >> > > > same > > > > > > >> > > > > > > for Open Source Software (OSS) Images and Docker > > > > Official > > > > > > >> Images > > > > > > >> > > > (DOI). > > > > > > >> > > > > > > > > > > > > >> > > > > > > For OSS images, the Dockerfile located in > > > > > > >> docker/jvm/dockerfile > > > > > > >> > is > > > > > > >> > > > > > > utilized. This process is integrated with the > > existing > > > > > > release > > > > > > >> > > > pipeline > > > > > > >> > > > > > as > > > > > > >> > > > > > > outlined in KIP-975 > > > > > > >> > > > > > > < > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka#KIP975:DockerImageforApacheKafka-Status > > > > > > >> > > > > > >, > > > > > > >> > > > > > > where the Kafka URL is provided as a build > argument. > > > > This > > > > > > >> method > > > > > > >> > > > allows > > > > > > >> > > > > > for > > > > > > >> > > > > > > building, testing, and releasing OSS images > > > dynamically. > > > > > The > > > > > > >> OSS > > > > > > >> > > > images > > > > > > >> > > > > > > will continue to be released under the standard > > > release > > > > > > >> process . > > > > > > >> > > > > > > > > > > > > >> > > > > > > In contrast, the release process for DOIs requires > > > > > providing > > > > > > >> the > > > > > > >> > > > Docker > > > > > > >> > > > > > Hub > > > > > > >> > > > > > > team with a specific directory for each version > > > release > > > > > that > > > > > > >> > > > contains a > > > > > > >> > > > > > > standalone Dockerfile. These Dockerfiles are > > designed > > > to > > > > > be > > > > > > >> > > > > > > self-sufficient, hence require hardcoded values > > > instead > > > > of > > > > > > >> > relying > > > > > > >> > > on > > > > > > >> > > > > > build > > > > > > >> > > > > > > arguments. To accommodate this, in our proposed > > > > approach, > > > > > a > > > > > > >> new > > > > > > >> > > > directory > > > > > > >> > > > > > > named docker_official_images has been created. > This > > > > > > directory > > > > > > >> > > > contains > > > > > > >> > > > > > > version-specific directories, having Dockerfiles > > with > > > > > > >> hardcoded > > > > > > >> > > > > > > configurations for each release, acting as the > > source > > > of > > > > > > truth > > > > > > >> > for > > > > > > >> > > > DOI > > > > > > >> > > > > > > releases. The hardcoded dockerfiles will be > created > > > > using > > > > > > the > > > > > > >> > > > > > > docker/jvm/dockerfile as a template. Thus, as part > > of > > > > post > > > > > > >> > release > > > > > > >> > > we > > > > > > >> > > > > > will > > > > > > >> > > > > > > be creating a Dockerfile that will be reviewed by > > the > > > > > > >> Dockerhub > > > > > > >> > > > community > > > > > > >> > > > > > > and might need changes as per their review. This > > > > approach > > > > > > >> ensures > > > > > > >> > > > that > > > > > > >> > > > > > DOIs > > > > > > >> > > > > > > are built consistently and meet the specific > > > > requirements > > > > > > set > > > > > > >> by > > > > > > >> > > > Docker > > > > > > >> > > > > > Hub. > > > > > > >> > > > > > > > > > > > > >> > > > > > > 2. Yes Manikumar, transitioning the release of > > Docker > > > > > > Official > > > > > > >> > > Images > > > > > > >> > > > > > (DOI) > > > > > > >> > > > > > > to a post-release activity does address the > concerns > > > > about > > > > > > >> > > > complicating > > > > > > >> > > > > > the > > > > > > >> > > > > > > release process. Initially, we considered > > > incorporating > > > > > DOI > > > > > > >> > release > > > > > > >> > > > > > > directly into Kafka's release workflow. However, > > this > > > > > > approach > > > > > > >> > > > > > > significantly increased the RMs workload due to > the > > > > > addition > > > > > > >> of > > > > > > >> > > > numerous > > > > > > >> > > > > > > steps, complicating the process. By designating > the > > > DOI > > > > > > >> release > > > > > > >> > as > > > > > > >> > > a > > > > > > >> > > > > > > post-release task, we maintain the original > release > > > > > process. > > > > > > >> This > > > > > > >> > > > > > > adjustment allows for the DOI release to be done > > after > > > > the > > > > > > >> main > > > > > > >> > > > release. > > > > > > >> > > > > > We > > > > > > >> > > > > > > have revised the KIP to reflect that DOI releases > > will > > > > now > > > > > > >> occur > > > > > > >> > > > after > > > > > > >> > > > > > the > > > > > > >> > > > > > > main release phase. Please review the updated > > document > > > > and > > > > > > >> > provide > > > > > > >> > > > any > > > > > > >> > > > > > > feedback you might have. > > > > > > >> > > > > > > > > > > > > >> > > > > > > Thanks, > > > > > > >> > > > > > > Krish. > > > > > > >> > > > > > > > > > > > > >> > > > > > > On Wed, Apr 3, 2024 at 3:35 PM Luke Chen < > > > > > show...@gmail.com > > > > > > > > > > > > > >> > > wrote: > > > > > > >> > > > > > > > > > > > > >> > > > > > > > Hi Krishna, > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > I also have the same question as Manikumar > raised: > > > > > > >> > > > > > > > 1. Will the Docker inventory files/etc are the > > same > > > > for > > > > > > OSS > > > > > > >> > Image > > > > > > >> > > > and > > > > > > >> > > > > > > > Docker Official Images? > > > > > > >> > > > > > > > If no, then why not? Could we make them > identical > > so > > > > > that > > > > > > we > > > > > > >> > > don't > > > > > > >> > > > > > have to > > > > > > >> > > > > > > > build 2 images for each release? > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > Thank you. > > > > > > >> > > > > > > > Luke > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > On Wed, Apr 3, 2024 at 12:41 AM Manikumar < > > > > > > >> > > > manikumar.re...@gmail.com> > > > > > > >> > > > > > > > wrote: > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > Hi Krishna, > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > Thanks for the KIP. > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > I think Docker Official Images will be > > beneficial > > > to > > > > > the > > > > > > >> > Kafka > > > > > > >> > > > > > community. > > > > > > >> > > > > > > > > Few queries below. > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > 1. Will the Docker inventory files/etc are the > > > same > > > > > for > > > > > > >> OSS > > > > > > >> > > > Image and > > > > > > >> > > > > > > > > Docker Official Images > > > > > > >> > > > > > > > > 2. I am a bit worried about the new steps to > the > > > > > release > > > > > > >> > > process. > > > > > > >> > > > > > Maybe > > > > > > >> > > > > > > > we > > > > > > >> > > > > > > > > should consider Docker Official Images release > > as > > > > > > >> > Post-Release > > > > > > >> > > > > > activity. > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > Thanks, > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > On Fri, Mar 22, 2024 at 3:29 PM Krish Vora < > > > > > > >> > > > krishvor...@gmail.com> > > > > > > >> > > > > > > > wrote: > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > Hi Hector, > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > Thanks for reaching out. This KIP builds on > > top > > > of > > > > > > >> KIP-975 > > > > > > >> > > > > > > > > > < > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > and > > > > > > >> > > > > > > > > > aims to introduce a JVM-based Docker > Official > > > > Image > > > > > > (DOI > > > > > > >> > > > > > > > > > < > > > > > > >> https://docs.docker.com/trusted-content/official-images/ > > > > > > >> > >) > > > > > > >> > > > for > > > > > > >> > > > > > Apache > > > > > > >> > > > > > > > > > Kafka that will be visible under Docker > > Official > > > > > > Images > > > > > > >> > > > > > > > > > < > > > > > > https://hub.docker.com/search?image_filter=official&q= > > > > > > >> >. > > > > > > >> > > Once > > > > > > >> > > > > > > > > implemented > > > > > > >> > > > > > > > > > for Apache Kafka, for each release, there > will > > > be > > > > > one > > > > > > >> more > > > > > > >> > > > > > JVM-based > > > > > > >> > > > > > > > > Docker > > > > > > >> > > > > > > > > > image available to users. > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > Currently, we already have an OSS sponsored > > > image, > > > > > > which > > > > > > >> > was > > > > > > >> > > > > > introduced > > > > > > >> > > > > > > > > via > > > > > > >> > > > > > > > > > KIP-975 (apache/kafka < > > > > > > >> > > > https://hub.docker.com/r/apache/kafka/tags > > > > > > >> > > > > > >) > > > > > > >> > > > > > > > > which > > > > > > >> > > > > > > > > > comes under The Apache Software Foundation < > > > > > > >> > > > > > > > > > https://hub.docker.com/u/apache> in > > > > > > >> > > > > > > > > > Docker Hub. The new Docker Image is the > Docker > > > > > > Official > > > > > > >> > Image > > > > > > >> > > > > > (DOI), > > > > > > >> > > > > > > > > which > > > > > > >> > > > > > > > > > will be built and maintained by Docker > > > Community. > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > For example, for a release version like > 3.8.0 > > we > > > > > will > > > > > > >> have > > > > > > >> > > two > > > > > > >> > > > JVM > > > > > > >> > > > > > > > based > > > > > > >> > > > > > > > > > docker images:- > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > - apache/kafka:3.8.0 (OSS sponsored > image) > > > > > > >> > > > > > > > > > - kafka:3.8.0 (Docker Official image) > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > I have added the same in the KIP too for > > > > everyone's > > > > > > >> > > reference. > > > > > > >> > > > > > > > > > Thanks, > > > > > > >> > > > > > > > > > Krish. > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > On Fri, Mar 22, 2024 at 2:50 AM Hector > > Geraldino > > > > > > >> > (BLOOMBERG/ > > > > > > >> > > > 919 > > > > > > >> > > > > > 3RD > > > > > > >> > > > > > > > A) < > > > > > > >> > > > > > > > > > hgerald...@bloomberg.net> wrote: > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > Hi, > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > What is the difference between this KIP > and > > > > > KIP-975: > > > > > > >> > Docker > > > > > > >> > > > > > Image for > > > > > > >> > > > > > > > > > > Apache Kafka? > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > From: dev@kafka.apache.org At: 03/21/24 > > > > 07:30:07 > > > > > > >> > > UTC-4:00To: > > > > > > >> > > > > > > > > > > dev@kafka.apache.org > > > > > > >> > > > > > > > > > > Subject: [DISCUSS] KIP-1028: Docker > Official > > > > Image > > > > > > for > > > > > > >> > > Apache > > > > > > >> > > > > > Kafka > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > Hi everyone, > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > I would like to start the discussion on > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1028%3A+Docker+Official+Im > > > > > > >> > > > > > > > > > > age+for+Apache+Kafka > > > > > > >> > > > > > > > > > > . > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > This KIP aims to introduce JVM based > Docker > > > > > Official > > > > > > >> > Image > > > > > > >> > > > (DOI) > > > > > > >> > > > > > for > > > > > > >> > > > > > > > > > Apache > > > > > > >> > > > > > > > > > > Kafka. > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > Regards, > > > > > > >> > > > > > > > > > > Krish. > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > > >> > > > > > > > > > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> -- > > > > > > >> > > > > > > >> [image: Confluent] <https://www.confluent.io> > > > > > > >> Prabha Manepalli > > > > > > >> Product Manager for Confluent Platform Security, Docker > > > > > > >> linkedin.com/in/prabhamanepalli > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > [image: Confluent] <https://www.confluent.io> > > > Prabha Manepalli > > > Product Manager for Confluent Platform Security, Docker Containers > > > linkedin.com/in/prabhamanepalli > > > > > >
