Hi Vedarth and Krish, Thanks for the KIP! I have to admit I'm a little skeptical; hopefully you can help me understand the need for these additional images.
1) In the motivation section it's stated that "Several other Apache projects, like Flink, Spark, Solr, have already released Docker Official Images, with download figures ranging from 50 million to over 1 billion. These numbers highlight the significant demand among users." But then immediately afterwards, we learn that "Also the Docker Official Images are always the top 1 search result, irrespective of the number of downloads." Wouldn't a high number of downloads for an image naturally follow from being the top search result? It seems like we can't necessarily assume that Docker Official Images are inherently more desirable for users based solely on download statistics. 2) Can you elaborate on the value that these new images would add from a user's perspective? I'm hesitant to introduce another image, since it adds to the cognitive burden of people who will inevitably have to answer the question of "What are the differences between all of the available images and which one is best for my use case?" 3) Would a separate Docker-owned repository be out of the question? I'm guessing there are some trademark issues that might get in the way, but it's worth exploring since the entire purpose of this KIP seems to be to provide images that are vetted and designed by Docker more than by the Apache Kafka contributors/committers/PMC. I may have more questions later but wanted to get this initial round out now without trying to list everything first. Looking forward to your thoughts! Cheers, Chris On Mon, Apr 22, 2024 at 2:14 PM Vedarth Sharma <vedarth.sha...@gmail.com> wrote: > Hey folks, > > Thanks a lot for reviewing the KIP and providing feedback. > The discussion thread seems resolved and KIP has been updated accordingly. > We will be starting the voting thread for this KIP in the next few days. > Please take a look at the KIP and let us know if any further discussion > is needed. > > Thanks and regards, > Vedarth > > On Fri, Apr 19, 2024 at 1:33 PM Manikumar <manikumar.re...@gmail.com> > wrote: > > > Thanks Krish. KIP looks good to me. > > > > On Wed, Apr 17, 2024 at 1:38 PM Krish Vora <krishvor...@gmail.com> > wrote: > > > > > > Hi Manikumar, > > > > > > Thanks for the comments. > > > > > > Maybe as part of the release process, RM can create a JIRA for this > > > > task. This can be taken by RM or any comitter or any contributor > (with > > > > some help from commiters to run "Docker Image Preparation via GitHub > > > > Actions:" > > > > > > This sounds like a good idea. This step would be beneficial. By > creating > > a > > > JIRA ticket, it will also serve as a reminder to complete the > > post-release > > > steps for the Docker official images. Have updated the KIP with this > > step. > > > > > > Is this using GitHub Actions workflow? or manual testing? > > > > > > This will be done by a Github Actions workflow, which will test the > > static > > > Docker Official Image assets for a specific release version. > > > > > > Is it mandatory for RM/comitters to raise the PR to Docker Hub’s > > > > official images repository (or) can it be done by any contributor. > > > > > > I believe that it can be done by any contributor (ref: This link > > > <https://docs.docker.com/trusted-content/official-images/contributing/ > > > > > quotes "*Anyone can provide feedback, contribute code, suggest process > > > changes, or even propose a new Official Image.*") > > > > > > Also I was thinking, once the KIP gets voted, we should try to release > > > > kafka:3.7.0 (or 3.7.1) Docker Official image. This will help us to > > > > validate the process and allow us to fix any changes suggested by > > > > Dockerhub before the 3.8.0 release. > > > > > > This sounds like a great idea. This KIP proposes release of DOI as a > > > post-release process, which can be done anytime post release. Since > 3.7.0 > > > is already released, we can perform these steps for that release too. > By > > > the time the KIP gets implemented, if 3.7.1 is released, we could do > > these > > > steps for 3.7.1, instead of 3.7.0. This would allow us to make changes > to > > > the Dockerfiles and other assets based on feedback from Docker Hub > before > > > the release of version 3.8.0. > > > > > > Thanks, > > > Krish. > > > > > > On Mon, Apr 15, 2024 at 12:59 PM Manikumar <manikumar.re...@gmail.com> > > > wrote: > > > > > > > Hi Krish, > > > > > > > > Thanks for the updated KIP. a few comments below. > > > > > > > > > "These actions can be carried out by the RM or any contributor post > > the > > > > release process." > > > > Maybe as part of the release process, RM can create a JIRA for this > > > > task. This can be taken by RM or any comitter or any contributor > (with > > > > some help from commiters to run "Docker Image Preparation via GitHub > > > > Actions:" > > > > > > > > > "Perform Docker build tests to ensure image integrity" > > > > Is this using GitHub Actions workflow? or manual testing? > > > > > > > > > "The RM will manually raise the final PR to Docker Hub’s official > > images > > > > repository using the contents of the generated file" > > > > Is it mandatory for RM/comitters to raise the PR to Docker Hub’s > > > > official images repository (or) can it be done by any contributor. > > > > > > > > Also I was thinking, once the KIP gets voted, we should try to > release > > > > kafka:3.7.0 (or 3.7.1) Docker Official image. This will help us to > > > > validate the process and allow us to fix any changes suggested by > > > > Dockerhub before the 3.8.0 release. > > > > > > > > > > > > Thanks, > > > > > > > > On Mon, Apr 8, 2024 at 2:33 PM Krish Vora <krishvor...@gmail.com> > > wrote: > > > > > > > > > > Hi Manikumar and Luke. > > > > > Thanks for the questions. > > > > > > > > > > 1. No, the Docker inventory files and configurations will not be > the > > same > > > > > for Open Source Software (OSS) Images and Docker Official Images > > (DOI). > > > > > > > > > > For OSS images, the Dockerfile located in docker/jvm/dockerfile is > > > > > utilized. This process is integrated with the existing release > > pipeline > > > > as > > > > > outlined in KIP-975 > > > > > < > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka#KIP975:DockerImageforApacheKafka-Status > > > > >, > > > > > where the Kafka URL is provided as a build argument. This method > > allows > > > > for > > > > > building, testing, and releasing OSS images dynamically. The OSS > > images > > > > > will continue to be released under the standard release process . > > > > > > > > > > In contrast, the release process for DOIs requires providing the > > Docker > > > > Hub > > > > > team with a specific directory for each version release that > > contains a > > > > > standalone Dockerfile. These Dockerfiles are designed to be > > > > > self-sufficient, hence require hardcoded values instead of relying > on > > > > build > > > > > arguments. To accommodate this, in our proposed approach, a new > > directory > > > > > named docker_official_images has been created. This directory > > contains > > > > > version-specific directories, having Dockerfiles with hardcoded > > > > > configurations for each release, acting as the source of truth for > > DOI > > > > > releases. The hardcoded dockerfiles will be created using the > > > > > docker/jvm/dockerfile as a template. Thus, as part of post release > we > > > > will > > > > > be creating a Dockerfile that will be reviewed by the Dockerhub > > community > > > > > and might need changes as per their review. This approach ensures > > that > > > > DOIs > > > > > are built consistently and meet the specific requirements set by > > Docker > > > > Hub. > > > > > > > > > > 2. Yes Manikumar, transitioning the release of Docker Official > Images > > > > (DOI) > > > > > to a post-release activity does address the concerns about > > complicating > > > > the > > > > > release process. Initially, we considered incorporating DOI release > > > > > directly into Kafka's release workflow. However, this approach > > > > > significantly increased the RMs workload due to the addition of > > numerous > > > > > steps, complicating the process. By designating the DOI release as > a > > > > > post-release task, we maintain the original release process. This > > > > > adjustment allows for the DOI release to be done after the main > > release. > > > > We > > > > > have revised the KIP to reflect that DOI releases will now occur > > after > > > > the > > > > > main release phase. Please review the updated document and provide > > any > > > > > feedback you might have. > > > > > > > > > > Thanks, > > > > > Krish. > > > > > > > > > > On Wed, Apr 3, 2024 at 3:35 PM Luke Chen <show...@gmail.com> > wrote: > > > > > > > > > > > Hi Krishna, > > > > > > > > > > > > I also have the same question as Manikumar raised: > > > > > > 1. Will the Docker inventory files/etc are the same for OSS Image > > and > > > > > > Docker Official Images? > > > > > > If no, then why not? Could we make them identical so that we > don't > > > > have to > > > > > > build 2 images for each release? > > > > > > > > > > > > Thank you. > > > > > > Luke > > > > > > > > > > > > On Wed, Apr 3, 2024 at 12:41 AM Manikumar < > > manikumar.re...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Hi Krishna, > > > > > > > > > > > > > > Thanks for the KIP. > > > > > > > > > > > > > > I think Docker Official Images will be beneficial to the Kafka > > > > community. > > > > > > > Few queries below. > > > > > > > > > > > > > > 1. Will the Docker inventory files/etc are the same for OSS > > Image and > > > > > > > Docker Official Images > > > > > > > 2. I am a bit worried about the new steps to the release > process. > > > > Maybe > > > > > > we > > > > > > > should consider Docker Official Images release as Post-Release > > > > activity. > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > On Fri, Mar 22, 2024 at 3:29 PM Krish Vora < > > krishvor...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > Hi Hector, > > > > > > > > > > > > > > > > Thanks for reaching out. This KIP builds on top of KIP-975 > > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka > > > > > > > > > > > > > > > > > and > > > > > > > > aims to introduce a JVM-based Docker Official Image (DOI > > > > > > > > <https://docs.docker.com/trusted-content/official-images/>) > > for > > > > Apache > > > > > > > > Kafka that will be visible under Docker Official Images > > > > > > > > <https://hub.docker.com/search?image_filter=official&q=>. > Once > > > > > > > implemented > > > > > > > > for Apache Kafka, for each release, there will be one more > > > > JVM-based > > > > > > > Docker > > > > > > > > image available to users. > > > > > > > > > > > > > > > > Currently, we already have an OSS sponsored image, which was > > > > introduced > > > > > > > via > > > > > > > > KIP-975 (apache/kafka < > > https://hub.docker.com/r/apache/kafka/tags > > > > >) > > > > > > > which > > > > > > > > comes under The Apache Software Foundation < > > > > > > > > https://hub.docker.com/u/apache> in > > > > > > > > Docker Hub. The new Docker Image is the Docker Official Image > > > > (DOI), > > > > > > > which > > > > > > > > will be built and maintained by Docker Community. > > > > > > > > > > > > > > > > For example, for a release version like 3.8.0 we will have > two > > JVM > > > > > > based > > > > > > > > docker images:- > > > > > > > > > > > > > > > > - apache/kafka:3.8.0 (OSS sponsored image) > > > > > > > > - kafka:3.8.0 (Docker Official image) > > > > > > > > > > > > > > > > > > > > > > > > I have added the same in the KIP too for everyone's > reference. > > > > > > > > Thanks, > > > > > > > > Krish. > > > > > > > > > > > > > > > > On Fri, Mar 22, 2024 at 2:50 AM Hector Geraldino (BLOOMBERG/ > > 919 > > > > 3RD > > > > > > A) < > > > > > > > > hgerald...@bloomberg.net> wrote: > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > What is the difference between this KIP and KIP-975: Docker > > > > Image for > > > > > > > > > Apache Kafka? > > > > > > > > > > > > > > > > > > From: dev@kafka.apache.org At: 03/21/24 07:30:07 > UTC-4:00To: > > > > > > > > > dev@kafka.apache.org > > > > > > > > > Subject: [DISCUSS] KIP-1028: Docker Official Image for > Apache > > > > Kafka > > > > > > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > > > I would like to start the discussion on > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1028%3A+Docker+Official+Im > > > > > > > > > age+for+Apache+Kafka > > > > > > > > > . > > > > > > > > > > > > > > > > > > This KIP aims to introduce JVM based Docker Official Image > > (DOI) > > > > for > > > > > > > > Apache > > > > > > > > > Kafka. > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > Krish. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
