Hey Chris, We have responded to the initial round of queries. And as you mentioned in your comment, you may have more questions related to this KIP. Please let us know if you have any. We want to start the voting for this KIP soon, as we intend to include it in the 3.8.0 release.
Thanks. Regards, Krish. On Wed, May 8, 2024 at 6:19 PM Krish Vora <krishvor...@gmail.com> wrote: > Hi Chris. Thanks for the questions. > > 3. Would a separate Docker-owned repository be out of the question? I'm >> guessing there are some trademark issues that might get in the way, but >> it's worth exploring since the entire purpose of this KIP seems to be to >> provide images that are vetted and designed by Docker more than by the >> Apache Kafka contributors/committers/PMC. > > > > - The process for introducing a Docker Official Image involves > - Hosting the Dockerfile in the Apache Kafka repository and > - Providing the path to this Dockerfile to Docker Hub in Docker > Hub’s own repo > <https://github.com/docker-library/official-images/tree/master/library> > . > - This ensures that any updates to the Dockerfile in the AK repository > are directly applicable to the docker official images available on Docker > Hub. > > > - We also did not find any added advantage to create a separate > repository named apache-docker within the Apache GitHub organization. > > Thanks, > Krish. > > On Wed, May 8, 2024 at 6:05 PM Prabha Manepalli > <mpra...@confluent.io.invalid> wrote: > >> Hi Chris, I would like to add more context to this KIP's motivation. >> Vedarth and Krish, please weigh in with your inputs. >> >> In the motivation section it's stated that "Several other Apache projects, >> > like Flink, Spark, Solr, have already released Docker Official Images, >> with >> > download figures ranging from 50 million to over 1 billion. These >> numbers >> > highlight the significant demand among users." But then immediately >> > afterwards, we learn that "Also the Docker Official Images are always >> the >> > top 1 search result, irrespective of the number of downloads." Wouldn't >> a >> > high number of downloads for an image naturally follow from being the >> top >> > search result? It seems like we can't necessarily assume that Docker >> > Official Images are inherently more desirable for users based solely on >> > download statistics. >> > >> >> *My thoughts: *Unlike the Sponsored OSS image, the Docker Official image >> is >> more desirable for workloads that have stringent compliance requirements. >> More details on why official images are more trusted are documented here >> <https://docs.docker.com/trusted-content/official-images/>. The Docker >> Official image would also help an absolutely new Kafka beginner who might >> not know about Apache or the concept of Sponsored images. We want to make >> it easier for Kafka beginners to discover the Kafka image through >> DockerHub. >> >> >> Can you elaborate on the value that these new images would add from a >> > user's perspective? I'm hesitant to introduce another image, since it >> adds >> > to the cognitive burden of people who will inevitably have to answer the >> > question of "What are the differences between all of the available >> images >> > and which one is best for my use case?" >> > >> >> >> *My thoughts: *This is a valid concern to address. The response to the >> above question addresses the value-add this new Docker Official image >> would >> provide. I also agree we need a clear distinction between each of these >> images to be well documented. We plan to update the AK website with >> details >> on how, why, and when a developer would want to use each of these >> particular images(KIP-974,975,1028). >> >> Thanks, >> Prabha. >> >> >> >> >> >> On Tue, Apr 30, 2024 at 9:41 PM Chris Egerton <chr...@aiven.io.invalid> >> wrote: >> >> > Hi Vedarth and Krish, >> > >> > Thanks for the KIP! I have to admit I'm a little skeptical; hopefully >> you >> > can help me understand the need for these additional images. >> > >> > 1) In the motivation section it's stated that "Several other Apache >> > projects, like Flink, Spark, Solr, have already released Docker Official >> > Images, with download figures ranging from 50 million to over 1 billion. >> > These numbers highlight the significant demand among users." But then >> > immediately afterwards, we learn that "Also the Docker Official Images >> are >> > always the top 1 search result, irrespective of the number of >> downloads." >> > Wouldn't a high number of downloads for an image naturally follow from >> > being the top search result? It seems like we can't necessarily assume >> that >> > Docker Official Images are inherently more desirable for users based >> solely >> > on download statistics. >> > >> > 2) Can you elaborate on the value that these new images would add from a >> > user's perspective? I'm hesitant to introduce another image, since it >> adds >> > to the cognitive burden of people who will inevitably have to answer the >> > question of "What are the differences between all of the available >> images >> > and which one is best for my use case?" >> > >> > 3) Would a separate Docker-owned repository be out of the question? I'm >> > guessing there are some trademark issues that might get in the way, but >> > it's worth exploring since the entire purpose of this KIP seems to be to >> > provide images that are vetted and designed by Docker more than by the >> > Apache Kafka contributors/committers/PMC. >> > >> > I may have more questions later but wanted to get this initial round out >> > now without trying to list everything first. >> > >> > Looking forward to your thoughts! >> > >> > Cheers, >> > >> > Chris >> > >> > On Mon, Apr 22, 2024 at 2:14 PM Vedarth Sharma < >> vedarth.sha...@gmail.com> >> > wrote: >> > >> > > Hey folks, >> > > >> > > Thanks a lot for reviewing the KIP and providing feedback. >> > > The discussion thread seems resolved and KIP has been updated >> > accordingly. >> > > We will be starting the voting thread for this KIP in the next few >> days. >> > > Please take a look at the KIP and let us know if any further >> discussion >> > > is needed. >> > > >> > > Thanks and regards, >> > > Vedarth >> > > >> > > On Fri, Apr 19, 2024 at 1:33 PM Manikumar <manikumar.re...@gmail.com> >> > > wrote: >> > > >> > > > Thanks Krish. KIP looks good to me. >> > > > >> > > > On Wed, Apr 17, 2024 at 1:38 PM Krish Vora <krishvor...@gmail.com> >> > > wrote: >> > > > > >> > > > > Hi Manikumar, >> > > > > >> > > > > Thanks for the comments. >> > > > > >> > > > > Maybe as part of the release process, RM can create a JIRA for >> this >> > > > > > task. This can be taken by RM or any comitter or any contributor >> > > (with >> > > > > > some help from commiters to run "Docker Image Preparation via >> > GitHub >> > > > > > Actions:" >> > > > > >> > > > > This sounds like a good idea. This step would be beneficial. By >> > > creating >> > > > a >> > > > > JIRA ticket, it will also serve as a reminder to complete the >> > > > post-release >> > > > > steps for the Docker official images. Have updated the KIP with >> this >> > > > step. >> > > > > >> > > > > Is this using GitHub Actions workflow? or manual testing? >> > > > > >> > > > > This will be done by a Github Actions workflow, which will test >> the >> > > > static >> > > > > Docker Official Image assets for a specific release version. >> > > > > >> > > > > Is it mandatory for RM/comitters to raise the PR to Docker Hub’s >> > > > > > official images repository (or) can it be done by any >> contributor. >> > > > > >> > > > > I believe that it can be done by any contributor (ref: This link >> > > > > < >> > https://docs.docker.com/trusted-content/official-images/contributing/ >> > > > >> > > > > quotes "*Anyone can provide feedback, contribute code, suggest >> > process >> > > > > changes, or even propose a new Official Image.*") >> > > > > >> > > > > Also I was thinking, once the KIP gets voted, we should try to >> > release >> > > > > > kafka:3.7.0 (or 3.7.1) Docker Official image. This will help us >> to >> > > > > > validate the process and allow us to fix any changes suggested >> by >> > > > > > Dockerhub before the 3.8.0 release. >> > > > > >> > > > > This sounds like a great idea. This KIP proposes release of DOI >> as a >> > > > > post-release process, which can be done anytime post release. >> Since >> > > 3.7.0 >> > > > > is already released, we can perform these steps for that release >> too. >> > > By >> > > > > the time the KIP gets implemented, if 3.7.1 is released, we could >> do >> > > > these >> > > > > steps for 3.7.1, instead of 3.7.0. This would allow us to make >> > changes >> > > to >> > > > > the Dockerfiles and other assets based on feedback from Docker Hub >> > > before >> > > > > the release of version 3.8.0. >> > > > > >> > > > > Thanks, >> > > > > Krish. >> > > > > >> > > > > On Mon, Apr 15, 2024 at 12:59 PM Manikumar < >> > manikumar.re...@gmail.com> >> > > > > wrote: >> > > > > >> > > > > > Hi Krish, >> > > > > > >> > > > > > Thanks for the updated KIP. a few comments below. >> > > > > > >> > > > > > > "These actions can be carried out by the RM or any contributor >> > post >> > > > the >> > > > > > release process." >> > > > > > Maybe as part of the release process, RM can create a JIRA for >> this >> > > > > > task. This can be taken by RM or any comitter or any contributor >> > > (with >> > > > > > some help from commiters to run "Docker Image Preparation via >> > GitHub >> > > > > > Actions:" >> > > > > > >> > > > > > > "Perform Docker build tests to ensure image integrity" >> > > > > > Is this using GitHub Actions workflow? or manual testing? >> > > > > > >> > > > > > > "The RM will manually raise the final PR to Docker Hub’s >> official >> > > > images >> > > > > > repository using the contents of the generated file" >> > > > > > Is it mandatory for RM/comitters to raise the PR to Docker >> Hub’s >> > > > > > official images repository (or) can it be done by any >> contributor. >> > > > > > >> > > > > > Also I was thinking, once the KIP gets voted, we should try to >> > > release >> > > > > > kafka:3.7.0 (or 3.7.1) Docker Official image. This will help us >> to >> > > > > > validate the process and allow us to fix any changes suggested >> by >> > > > > > Dockerhub before the 3.8.0 release. >> > > > > > >> > > > > > >> > > > > > Thanks, >> > > > > > >> > > > > > On Mon, Apr 8, 2024 at 2:33 PM Krish Vora < >> krishvor...@gmail.com> >> > > > wrote: >> > > > > > > >> > > > > > > Hi Manikumar and Luke. >> > > > > > > Thanks for the questions. >> > > > > > > >> > > > > > > 1. No, the Docker inventory files and configurations will not >> be >> > > the >> > > > same >> > > > > > > for Open Source Software (OSS) Images and Docker Official >> Images >> > > > (DOI). >> > > > > > > >> > > > > > > For OSS images, the Dockerfile located in >> docker/jvm/dockerfile >> > is >> > > > > > > utilized. This process is integrated with the existing release >> > > > pipeline >> > > > > > as >> > > > > > > outlined in KIP-975 >> > > > > > > < >> > > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka#KIP975:DockerImageforApacheKafka-Status >> > > > > > >, >> > > > > > > where the Kafka URL is provided as a build argument. This >> method >> > > > allows >> > > > > > for >> > > > > > > building, testing, and releasing OSS images dynamically. The >> OSS >> > > > images >> > > > > > > will continue to be released under the standard release >> process . >> > > > > > > >> > > > > > > In contrast, the release process for DOIs requires providing >> the >> > > > Docker >> > > > > > Hub >> > > > > > > team with a specific directory for each version release that >> > > > contains a >> > > > > > > standalone Dockerfile. These Dockerfiles are designed to be >> > > > > > > self-sufficient, hence require hardcoded values instead of >> > relying >> > > on >> > > > > > build >> > > > > > > arguments. To accommodate this, in our proposed approach, a >> new >> > > > directory >> > > > > > > named docker_official_images has been created. This directory >> > > > contains >> > > > > > > version-specific directories, having Dockerfiles with >> hardcoded >> > > > > > > configurations for each release, acting as the source of truth >> > for >> > > > DOI >> > > > > > > releases. The hardcoded dockerfiles will be created using the >> > > > > > > docker/jvm/dockerfile as a template. Thus, as part of post >> > release >> > > we >> > > > > > will >> > > > > > > be creating a Dockerfile that will be reviewed by the >> Dockerhub >> > > > community >> > > > > > > and might need changes as per their review. This approach >> ensures >> > > > that >> > > > > > DOIs >> > > > > > > are built consistently and meet the specific requirements set >> by >> > > > Docker >> > > > > > Hub. >> > > > > > > >> > > > > > > 2. Yes Manikumar, transitioning the release of Docker Official >> > > Images >> > > > > > (DOI) >> > > > > > > to a post-release activity does address the concerns about >> > > > complicating >> > > > > > the >> > > > > > > release process. Initially, we considered incorporating DOI >> > release >> > > > > > > directly into Kafka's release workflow. However, this approach >> > > > > > > significantly increased the RMs workload due to the addition >> of >> > > > numerous >> > > > > > > steps, complicating the process. By designating the DOI >> release >> > as >> > > a >> > > > > > > post-release task, we maintain the original release process. >> This >> > > > > > > adjustment allows for the DOI release to be done after the >> main >> > > > release. >> > > > > > We >> > > > > > > have revised the KIP to reflect that DOI releases will now >> occur >> > > > after >> > > > > > the >> > > > > > > main release phase. Please review the updated document and >> > provide >> > > > any >> > > > > > > feedback you might have. >> > > > > > > >> > > > > > > Thanks, >> > > > > > > Krish. >> > > > > > > >> > > > > > > On Wed, Apr 3, 2024 at 3:35 PM Luke Chen <show...@gmail.com> >> > > wrote: >> > > > > > > >> > > > > > > > Hi Krishna, >> > > > > > > > >> > > > > > > > I also have the same question as Manikumar raised: >> > > > > > > > 1. Will the Docker inventory files/etc are the same for OSS >> > Image >> > > > and >> > > > > > > > Docker Official Images? >> > > > > > > > If no, then why not? Could we make them identical so that we >> > > don't >> > > > > > have to >> > > > > > > > build 2 images for each release? >> > > > > > > > >> > > > > > > > Thank you. >> > > > > > > > Luke >> > > > > > > > >> > > > > > > > On Wed, Apr 3, 2024 at 12:41 AM Manikumar < >> > > > manikumar.re...@gmail.com> >> > > > > > > > wrote: >> > > > > > > > >> > > > > > > > > Hi Krishna, >> > > > > > > > > >> > > > > > > > > Thanks for the KIP. >> > > > > > > > > >> > > > > > > > > I think Docker Official Images will be beneficial to the >> > Kafka >> > > > > > community. >> > > > > > > > > Few queries below. >> > > > > > > > > >> > > > > > > > > 1. Will the Docker inventory files/etc are the same for >> OSS >> > > > Image and >> > > > > > > > > Docker Official Images >> > > > > > > > > 2. I am a bit worried about the new steps to the release >> > > process. >> > > > > > Maybe >> > > > > > > > we >> > > > > > > > > should consider Docker Official Images release as >> > Post-Release >> > > > > > activity. >> > > > > > > > > >> > > > > > > > > Thanks, >> > > > > > > > > >> > > > > > > > > On Fri, Mar 22, 2024 at 3:29 PM Krish Vora < >> > > > krishvor...@gmail.com> >> > > > > > > > wrote: >> > > > > > > > > >> > > > > > > > > > Hi Hector, >> > > > > > > > > > >> > > > > > > > > > Thanks for reaching out. This KIP builds on top of >> KIP-975 >> > > > > > > > > > < >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka >> > > > > > > > > > > >> > > > > > > > > > and >> > > > > > > > > > aims to introduce a JVM-based Docker Official Image (DOI >> > > > > > > > > > < >> https://docs.docker.com/trusted-content/official-images/ >> > >) >> > > > for >> > > > > > Apache >> > > > > > > > > > Kafka that will be visible under Docker Official Images >> > > > > > > > > > <https://hub.docker.com/search?image_filter=official&q= >> >. >> > > Once >> > > > > > > > > implemented >> > > > > > > > > > for Apache Kafka, for each release, there will be one >> more >> > > > > > JVM-based >> > > > > > > > > Docker >> > > > > > > > > > image available to users. >> > > > > > > > > > >> > > > > > > > > > Currently, we already have an OSS sponsored image, which >> > was >> > > > > > introduced >> > > > > > > > > via >> > > > > > > > > > KIP-975 (apache/kafka < >> > > > https://hub.docker.com/r/apache/kafka/tags >> > > > > > >) >> > > > > > > > > which >> > > > > > > > > > comes under The Apache Software Foundation < >> > > > > > > > > > https://hub.docker.com/u/apache> in >> > > > > > > > > > Docker Hub. The new Docker Image is the Docker Official >> > Image >> > > > > > (DOI), >> > > > > > > > > which >> > > > > > > > > > will be built and maintained by Docker Community. >> > > > > > > > > > >> > > > > > > > > > For example, for a release version like 3.8.0 we will >> have >> > > two >> > > > JVM >> > > > > > > > based >> > > > > > > > > > docker images:- >> > > > > > > > > > >> > > > > > > > > > - apache/kafka:3.8.0 (OSS sponsored image) >> > > > > > > > > > - kafka:3.8.0 (Docker Official image) >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > I have added the same in the KIP too for everyone's >> > > reference. >> > > > > > > > > > Thanks, >> > > > > > > > > > Krish. >> > > > > > > > > > >> > > > > > > > > > On Fri, Mar 22, 2024 at 2:50 AM Hector Geraldino >> > (BLOOMBERG/ >> > > > 919 >> > > > > > 3RD >> > > > > > > > A) < >> > > > > > > > > > hgerald...@bloomberg.net> wrote: >> > > > > > > > > > >> > > > > > > > > > > Hi, >> > > > > > > > > > > >> > > > > > > > > > > What is the difference between this KIP and KIP-975: >> > Docker >> > > > > > Image for >> > > > > > > > > > > Apache Kafka? >> > > > > > > > > > > >> > > > > > > > > > > From: dev@kafka.apache.org At: 03/21/24 07:30:07 >> > > UTC-4:00To: >> > > > > > > > > > > dev@kafka.apache.org >> > > > > > > > > > > Subject: [DISCUSS] KIP-1028: Docker Official Image for >> > > Apache >> > > > > > Kafka >> > > > > > > > > > > >> > > > > > > > > > > Hi everyone, >> > > > > > > > > > > >> > > > > > > > > > > I would like to start the discussion on >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > >> > > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1028%3A+Docker+Official+Im >> > > > > > > > > > > age+for+Apache+Kafka >> > > > > > > > > > > . >> > > > > > > > > > > >> > > > > > > > > > > This KIP aims to introduce JVM based Docker Official >> > Image >> > > > (DOI) >> > > > > > for >> > > > > > > > > > Apache >> > > > > > > > > > > Kafka. >> > > > > > > > > > > >> > > > > > > > > > > Regards, >> > > > > > > > > > > Krish. >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > >> > > >> > >> >> >> -- >> >> [image: Confluent] <https://www.confluent.io> >> Prabha Manepalli >> Product Manager for Confluent Platform Security, Docker >> linkedin.com/in/prabhamanepalli >> >
