Hi Chris. Thanks for the questions.
3. Would a separate Docker-owned repository be out of the question? I'm
> guessing there are some trademark issues that might get in the way, but
> it's worth exploring since the entire purpose of this KIP seems to be to
> provide images that are vetted and designed by Docker more than by the
> Apache Kafka contributors/committers/PMC.
- The process for introducing a Docker Official Image involves
- Hosting the Dockerfile in the Apache Kafka repository and
- Providing the path to this Dockerfile to Docker Hub in Docker Hub’s
own repo
<https://github.com/docker-library/official-images/tree/master/library>
.
- This ensures that any updates to the Dockerfile in the AK repository
are directly applicable to the docker official images available on Docker
Hub.
- We also did not find any added advantage to create a separate
repository named apache-docker within the Apache GitHub organization.
Thanks,
Krish.
On Wed, May 8, 2024 at 6:05 PM Prabha Manepalli
<mpra...@confluent.io.invalid> wrote:
> Hi Chris, I would like to add more context to this KIP's motivation.
> Vedarth and Krish, please weigh in with your inputs.
>
> In the motivation section it's stated that "Several other Apache projects,
> > like Flink, Spark, Solr, have already released Docker Official Images,
> with
> > download figures ranging from 50 million to over 1 billion. These numbers
> > highlight the significant demand among users." But then immediately
> > afterwards, we learn that "Also the Docker Official Images are always the
> > top 1 search result, irrespective of the number of downloads." Wouldn't a
> > high number of downloads for an image naturally follow from being the top
> > search result? It seems like we can't necessarily assume that Docker
> > Official Images are inherently more desirable for users based solely on
> > download statistics.
> >
>
> *My thoughts: *Unlike the Sponsored OSS image, the Docker Official image is
> more desirable for workloads that have stringent compliance requirements.
> More details on why official images are more trusted are documented here
> <https://docs.docker.com/trusted-content/official-images/>. The Docker
> Official image would also help an absolutely new Kafka beginner who might
> not know about Apache or the concept of Sponsored images. We want to make
> it easier for Kafka beginners to discover the Kafka image through
> DockerHub.
>
>
> Can you elaborate on the value that these new images would add from a
> > user's perspective? I'm hesitant to introduce another image, since it
> adds
> > to the cognitive burden of people who will inevitably have to answer the
> > question of "What are the differences between all of the available images
> > and which one is best for my use case?"
> >
>
>
> *My thoughts: *This is a valid concern to address. The response to the
> above question addresses the value-add this new Docker Official image would
> provide. I also agree we need a clear distinction between each of these
> images to be well documented. We plan to update the AK website with details
> on how, why, and when a developer would want to use each of these
> particular images(KIP-974,975,1028).
>
> Thanks,
> Prabha.
>
>
>
>
>
> On Tue, Apr 30, 2024 at 9:41 PM Chris Egerton <chr...@aiven.io.invalid>
> wrote:
>
> > Hi Vedarth and Krish,
> >
> > Thanks for the KIP! I have to admit I'm a little skeptical; hopefully you
> > can help me understand the need for these additional images.
> >
> > 1) In the motivation section it's stated that "Several other Apache
> > projects, like Flink, Spark, Solr, have already released Docker Official
> > Images, with download figures ranging from 50 million to over 1 billion.
> > These numbers highlight the significant demand among users." But then
> > immediately afterwards, we learn that "Also the Docker Official Images
> are
> > always the top 1 search result, irrespective of the number of downloads."
> > Wouldn't a high number of downloads for an image naturally follow from
> > being the top search result? It seems like we can't necessarily assume
> that
> > Docker Official Images are inherently more desirable for users based
> solely
> > on download statistics.
> >
> > 2) Can you elaborate on the value that these new images would add from a
> > user's perspective? I'm hesitant to introduce another image, since it
> adds
> > to the cognitive burden of people who will inevitably have to answer the
> > question of "What are the differences between all of the available images
> > and which one is best for my use case?"
> >
> > 3) Would a separate Docker-owned repository be out of the question? I'm
> > guessing there are some trademark issues that might get in the way, but
> > it's worth exploring since the entire purpose of this KIP seems to be to
> > provide images that are vetted and designed by Docker more than by the
> > Apache Kafka contributors/committers/PMC.
> >
> > I may have more questions later but wanted to get this initial round out
> > now without trying to list everything first.
> >
> > Looking forward to your thoughts!
> >
> > Cheers,
> >
> > Chris
> >
> > On Mon, Apr 22, 2024 at 2:14 PM Vedarth Sharma <vedarth.sha...@gmail.com
> >
> > wrote:
> >
> > > Hey folks,
> > >
> > > Thanks a lot for reviewing the KIP and providing feedback.
> > > The discussion thread seems resolved and KIP has been updated
> > accordingly.
> > > We will be starting the voting thread for this KIP in the next few
> days.
> > > Please take a look at the KIP and let us know if any further discussion
> > > is needed.
> > >
> > > Thanks and regards,
> > > Vedarth
> > >
> > > On Fri, Apr 19, 2024 at 1:33 PM Manikumar <manikumar.re...@gmail.com>
> > > wrote:
> > >
> > > > Thanks Krish. KIP looks good to me.
> > > >
> > > > On Wed, Apr 17, 2024 at 1:38 PM Krish Vora <krishvor...@gmail.com>
> > > wrote:
> > > > >
> > > > > Hi Manikumar,
> > > > >
> > > > > Thanks for the comments.
> > > > >
> > > > > Maybe as part of the release process, RM can create a JIRA for this
> > > > > > task. This can be taken by RM or any comitter or any contributor
> > > (with
> > > > > > some help from commiters to run "Docker Image Preparation via
> > GitHub
> > > > > > Actions:"
> > > > >
> > > > > This sounds like a good idea. This step would be beneficial. By
> > > creating
> > > > a
> > > > > JIRA ticket, it will also serve as a reminder to complete the
> > > > post-release
> > > > > steps for the Docker official images. Have updated the KIP with
> this
> > > > step.
> > > > >
> > > > > Is this using GitHub Actions workflow? or manual testing?
> > > > >
> > > > > This will be done by a Github Actions workflow, which will test the
> > > > static
> > > > > Docker Official Image assets for a specific release version.
> > > > >
> > > > > Is it mandatory for RM/comitters to raise the PR to Docker Hub’s
> > > > > > official images repository (or) can it be done by any
> contributor.
> > > > >
> > > > > I believe that it can be done by any contributor (ref: This link
> > > > > <
> > https://docs.docker.com/trusted-content/official-images/contributing/
> > > >
> > > > > quotes "*Anyone can provide feedback, contribute code, suggest
> > process
> > > > > changes, or even propose a new Official Image.*")
> > > > >
> > > > > Also I was thinking, once the KIP gets voted, we should try to
> > release
> > > > > > kafka:3.7.0 (or 3.7.1) Docker Official image. This will help us
> to
> > > > > > validate the process and allow us to fix any changes suggested by
> > > > > > Dockerhub before the 3.8.0 release.
> > > > >
> > > > > This sounds like a great idea. This KIP proposes release of DOI as
> a
> > > > > post-release process, which can be done anytime post release. Since
> > > 3.7.0
> > > > > is already released, we can perform these steps for that release
> too.
> > > By
> > > > > the time the KIP gets implemented, if 3.7.1 is released, we could
> do
> > > > these
> > > > > steps for 3.7.1, instead of 3.7.0. This would allow us to make
> > changes
> > > to
> > > > > the Dockerfiles and other assets based on feedback from Docker Hub
> > > before
> > > > > the release of version 3.8.0.
> > > > >
> > > > > Thanks,
> > > > > Krish.
> > > > >
> > > > > On Mon, Apr 15, 2024 at 12:59 PM Manikumar <
> > manikumar.re...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Krish,
> > > > > >
> > > > > > Thanks for the updated KIP. a few comments below.
> > > > > >
> > > > > > > "These actions can be carried out by the RM or any contributor
> > post
> > > > the
> > > > > > release process."
> > > > > > Maybe as part of the release process, RM can create a JIRA for
> this
> > > > > > task. This can be taken by RM or any comitter or any contributor
> > > (with
> > > > > > some help from commiters to run "Docker Image Preparation via
> > GitHub
> > > > > > Actions:"
> > > > > >
> > > > > > > "Perform Docker build tests to ensure image integrity"
> > > > > > Is this using GitHub Actions workflow? or manual testing?
> > > > > >
> > > > > > > "The RM will manually raise the final PR to Docker Hub’s
> official
> > > > images
> > > > > > repository using the contents of the generated file"
> > > > > > Is it mandatory for RM/comitters to raise the PR to Docker Hub’s
> > > > > > official images repository (or) can it be done by any
> contributor.
> > > > > >
> > > > > > Also I was thinking, once the KIP gets voted, we should try to
> > > release
> > > > > > kafka:3.7.0 (or 3.7.1) Docker Official image. This will help us
> to
> > > > > > validate the process and allow us to fix any changes suggested by
> > > > > > Dockerhub before the 3.8.0 release.
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > On Mon, Apr 8, 2024 at 2:33 PM Krish Vora <krishvor...@gmail.com
> >
> > > > wrote:
> > > > > > >
> > > > > > > Hi Manikumar and Luke.
> > > > > > > Thanks for the questions.
> > > > > > >
> > > > > > > 1. No, the Docker inventory files and configurations will not
> be
> > > the
> > > > same
> > > > > > > for Open Source Software (OSS) Images and Docker Official
> Images
> > > > (DOI).
> > > > > > >
> > > > > > > For OSS images, the Dockerfile located in docker/jvm/dockerfile
> > is
> > > > > > > utilized. This process is integrated with the existing release
> > > > pipeline
> > > > > > as
> > > > > > > outlined in KIP-975
> > > > > > > <
> > > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka#KIP975:DockerImageforApacheKafka-Status
> > > > > > >,
> > > > > > > where the Kafka URL is provided as a build argument. This
> method
> > > > allows
> > > > > > for
> > > > > > > building, testing, and releasing OSS images dynamically. The
> OSS
> > > > images
> > > > > > > will continue to be released under the standard release
> process .
> > > > > > >
> > > > > > > In contrast, the release process for DOIs requires providing
> the
> > > > Docker
> > > > > > Hub
> > > > > > > team with a specific directory for each version release that
> > > > contains a
> > > > > > > standalone Dockerfile. These Dockerfiles are designed to be
> > > > > > > self-sufficient, hence require hardcoded values instead of
> > relying
> > > on
> > > > > > build
> > > > > > > arguments. To accommodate this, in our proposed approach, a new
> > > > directory
> > > > > > > named docker_official_images has been created. This directory
> > > > contains
> > > > > > > version-specific directories, having Dockerfiles with hardcoded
> > > > > > > configurations for each release, acting as the source of truth
> > for
> > > > DOI
> > > > > > > releases. The hardcoded dockerfiles will be created using the
> > > > > > > docker/jvm/dockerfile as a template. Thus, as part of post
> > release
> > > we
> > > > > > will
> > > > > > > be creating a Dockerfile that will be reviewed by the Dockerhub
> > > > community
> > > > > > > and might need changes as per their review. This approach
> ensures
> > > > that
> > > > > > DOIs
> > > > > > > are built consistently and meet the specific requirements set
> by
> > > > Docker
> > > > > > Hub.
> > > > > > >
> > > > > > > 2. Yes Manikumar, transitioning the release of Docker Official
> > > Images
> > > > > > (DOI)
> > > > > > > to a post-release activity does address the concerns about
> > > > complicating
> > > > > > the
> > > > > > > release process. Initially, we considered incorporating DOI
> > release
> > > > > > > directly into Kafka's release workflow. However, this approach
> > > > > > > significantly increased the RMs workload due to the addition of
> > > > numerous
> > > > > > > steps, complicating the process. By designating the DOI release
> > as
> > > a
> > > > > > > post-release task, we maintain the original release process.
> This
> > > > > > > adjustment allows for the DOI release to be done after the main
> > > > release.
> > > > > > We
> > > > > > > have revised the KIP to reflect that DOI releases will now
> occur
> > > > after
> > > > > > the
> > > > > > > main release phase. Please review the updated document and
> > provide
> > > > any
> > > > > > > feedback you might have.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Krish.
> > > > > > >
> > > > > > > On Wed, Apr 3, 2024 at 3:35 PM Luke Chen <show...@gmail.com>
> > > wrote:
> > > > > > >
> > > > > > > > Hi Krishna,
> > > > > > > >
> > > > > > > > I also have the same question as Manikumar raised:
> > > > > > > > 1. Will the Docker inventory files/etc are the same for OSS
> > Image
> > > > and
> > > > > > > > Docker Official Images?
> > > > > > > > If no, then why not? Could we make them identical so that we
> > > don't
> > > > > > have to
> > > > > > > > build 2 images for each release?
> > > > > > > >
> > > > > > > > Thank you.
> > > > > > > > Luke
> > > > > > > >
> > > > > > > > On Wed, Apr 3, 2024 at 12:41 AM Manikumar <
> > > > manikumar.re...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Krishna,
> > > > > > > > >
> > > > > > > > > Thanks for the KIP.
> > > > > > > > >
> > > > > > > > > I think Docker Official Images will be beneficial to the
> > Kafka
> > > > > > community.
> > > > > > > > > Few queries below.
> > > > > > > > >
> > > > > > > > > 1. Will the Docker inventory files/etc are the same for OSS
> > > > Image and
> > > > > > > > > Docker Official Images
> > > > > > > > > 2. I am a bit worried about the new steps to the release
> > > process.
> > > > > > Maybe
> > > > > > > > we
> > > > > > > > > should consider Docker Official Images release as
> > Post-Release
> > > > > > activity.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > On Fri, Mar 22, 2024 at 3:29 PM Krish Vora <
> > > > krishvor...@gmail.com>
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Hector,
> > > > > > > > > >
> > > > > > > > > > Thanks for reaching out. This KIP builds on top of
> KIP-975
> > > > > > > > > > <
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka
> > > > > > > > > > >
> > > > > > > > > > and
> > > > > > > > > > aims to introduce a JVM-based Docker Official Image (DOI
> > > > > > > > > > <
> https://docs.docker.com/trusted-content/official-images/
> > >)
> > > > for
> > > > > > Apache
> > > > > > > > > > Kafka that will be visible under Docker Official Images
> > > > > > > > > > <https://hub.docker.com/search?image_filter=official&q=
> >.
> > > Once
> > > > > > > > > implemented
> > > > > > > > > > for Apache Kafka, for each release, there will be one
> more
> > > > > > JVM-based
> > > > > > > > > Docker
> > > > > > > > > > image available to users.
> > > > > > > > > >
> > > > > > > > > > Currently, we already have an OSS sponsored image, which
> > was
> > > > > > introduced
> > > > > > > > > via
> > > > > > > > > > KIP-975 (apache/kafka <
> > > > https://hub.docker.com/r/apache/kafka/tags
> > > > > > >)
> > > > > > > > > which
> > > > > > > > > > comes under The Apache Software Foundation <
> > > > > > > > > > https://hub.docker.com/u/apache> in
> > > > > > > > > > Docker Hub. The new Docker Image is the Docker Official
> > Image
> > > > > > (DOI),
> > > > > > > > > which
> > > > > > > > > > will be built and maintained by Docker Community.
> > > > > > > > > >
> > > > > > > > > > For example, for a release version like 3.8.0 we will
> have
> > > two
> > > > JVM
> > > > > > > > based
> > > > > > > > > > docker images:-
> > > > > > > > > >
> > > > > > > > > > - apache/kafka:3.8.0 (OSS sponsored image)
> > > > > > > > > > - kafka:3.8.0 (Docker Official image)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I have added the same in the KIP too for everyone's
> > > reference.
> > > > > > > > > > Thanks,
> > > > > > > > > > Krish.
> > > > > > > > > >
> > > > > > > > > > On Fri, Mar 22, 2024 at 2:50 AM Hector Geraldino
> > (BLOOMBERG/
> > > > 919
> > > > > > 3RD
> > > > > > > > A) <
> > > > > > > > > > hgerald...@bloomberg.net> wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi,
> > > > > > > > > > >
> > > > > > > > > > > What is the difference between this KIP and KIP-975:
> > Docker
> > > > > > Image for
> > > > > > > > > > > Apache Kafka?
> > > > > > > > > > >
> > > > > > > > > > > From: dev@kafka.apache.org At: 03/21/24 07:30:07
> > > UTC-4:00To:
> > > > > > > > > > > dev@kafka.apache.org
> > > > > > > > > > > Subject: [DISCUSS] KIP-1028: Docker Official Image for
> > > Apache
> > > > > > Kafka
> > > > > > > > > > >
> > > > > > > > > > > Hi everyone,
> > > > > > > > > > >
> > > > > > > > > > > I would like to start the discussion on
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1028%3A+Docker+Official+Im
> > > > > > > > > > > age+for+Apache+Kafka
> > > > > > > > > > > .
> > > > > > > > > > >
> > > > > > > > > > > This KIP aims to introduce JVM based Docker Official
> > Image
> > > > (DOI)
> > > > > > for
> > > > > > > > > > Apache
> > > > > > > > > > > Kafka.
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > > Krish.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > >
> > >
> >
>
>
> --
>
> [image: Confluent] <https://www.confluent.io>
> Prabha Manepalli
> Product Manager for Confluent Platform Security, Docker
> linkedin.com/in/prabhamanepalli
>
