Hi Mickael, Thanks for going through the KIP and providing valuable feedback.
1. We will support the latest LTS version of Java supported by Apache Kafka. 2. We will provide support for the last three releases. We've added a detailed example of this in the KIP under our EOL policy. 3. We can establish a nightly cron job using GitHub Actions and leverage an open-source vulnerability scanning tool like trivy ( https://github.com/aquasecurity/trivy), to get vulnerability reports on all supported images. This tool offers a straightforward way to integrate vulnerability checks directly into our GitHub Actions workflow. 4. That's a good suggestion to have a GitHub Actions workflow. We will implement a GitHub Actions workflow to automate the build and testing process. 5. Regarding the release process, we observed that there isn't an existing CI/CD pipeline. We can consider the addition of a GitHub workflow to facilitate the release process. Please let us know your thoughts on the above. Thanks and regards, Vedarth On Fri, Oct 20, 2023 at 7:34 PM Mickael Maison <mickael.mai...@gmail.com> wrote: > Hi Krishna, > > Overall I'm supportive of having an official docker image. > I have a few questions: > - Can you clarify the process of selecting the Java version? Is the > proposal to only pick LTS versions? or to pick the highest version > supported by Kafka? > - Once a new Kafka version is released, what happens to the image > containing the previous release? Do we expect to still update it in > case of CVEs? If so for how long? > - How will we get notified that the base image has a CVE? > - Rather than having scripts PMC members have to run from their > machines, would it e possible to have a Jenkins job or GitHub action? > > Thanks, > Mickael > > > > On Fri, Oct 20, 2023 at 12:51 PM Vedarth Sharma > <vedarth.sha...@gmail.com> wrote: > > > > Hi Manikumar, > > > > Thanks for the feedback! > > > > 1. We propose the addition of a new directory named "docker" at the root > of > > the repository, where all Docker-related code will be stored. A detailed > > directory structure has been added in the KIP. > > 2. We request the creation of an Apache Kafka repository (apache/kafka) > on > > DockerHub, to be administered under the The Apache Software Foundation > > <https://hub.docker.com/u/apache>. The PMC members should have the > > necessary permissions for pushing updates to the docker repo. > > > > Thanks and regards, > > Vedarth > > > > > > On Fri, Oct 20, 2023 at 2:44 PM Manikumar <manikumar.re...@gmail.com> > wrote: > > > > > Hi Krishna, Vedarth, > > > > > > Thanks for the KIP. > > > > > > 1. Can we add directory structure of Docker Image related files in > Kafka > > > repo. > > > > > > 2. > Steps for the Docker image release will be included in the Release > > > Process doc of Apache Kafka > > > > > > Can we list down the requirements (repos, accounts) for releasing > images to > > > docker hub. I am mainly asking because PMC needs to request docker hub > > > access/repos. > > > I can help in getting required repos/accounts. > > > https://infra.apache.org/docker-hub-policy.html > > > > > > > > > Thanks, > > > Manikumar > > > > > > On Thu, Oct 19, 2023 at 8:22 PM Krishna Agarwal < > > > krishna0608agar...@gmail.com> wrote: > > > > > > > Hi Viktor, > > > > > > > > I've noticed there are two types of custom jar configurations: > > > > > > > > 1. *Type 1*: In this case, only the class name is required(e.g > > > > *authorizer.class.name > > > > <http://authorizer.class.name>**)* This can be configured by the > > > > following steps: > > > > - Mount the jar in the container. > > > > - Configure the *CLASSPATH* environment variable (used by > > > > *kafka-run-class.sh*) by providing the mounted path to it. > This can > > > > be passed as an environment variable to the docker container. > > > > 2. *Type 2*: Here, in addition to the class name, classpath can > also > > > be > > > > configured (eg *remote.log.metadata.manager.class.name > > > > <http://remote.log.metadata.manager.class.name> *and > > > > *remote.log.metadata.manager.class.path*). This can be configured > by > > > the > > > > following steps: > > > > - Mount the jar in the container. > > > > - Configure the respective *class.path* property. > > > > > > > > Regards, > > > > Krishna > > > > > > > > On Mon, Sep 25, 2023 at 11:41 PM Krishna Agarwal < > > > > krishna0608agar...@gmail.com> wrote: > > > > > > > > > Hi Viktor, > > > > > Thanks for the questions. > > > > > > > > > > 1. While the docker image outlined in KIP-975 is designed for > > > > > production environments, it is equally suitable for development > and > > > > testing > > > > > purposes. We will furnish the docker image, allowing users the > > > > flexibility > > > > > to employ it according to their specific needs. > > > > > 2. The configs will be injected into the docker container > through > > > > > environment variables. These environment variables will have a > > > prefix > > > > > allowing for efficient parsing to extract the relevant > > > > properties.(Will add > > > > > this implementation in the KIP as well once we converge on > this.) > > > > > 3. Regarding this question, I'll conduct a test on my end after > > > > > gaining a better understanding, and then provide you with a > > > response. > > > > > > > > > > Regards, > > > > > Krishna > > > > > > > > > > > > > > > On Tue, Sep 19, 2023 at 3:42 PM Viktor Somogyi-Vass > > > > > <viktor.somo...@cloudera.com.invalid> wrote: > > > > > > > > > >> Hi Ismael, > > > > >> > > > > >> I'm not trying to advocate against the docker image, I just > pointed > > > out > > > > >> that the current scoping of the KIP may be a bit too generic and > > > thought > > > > >> that KIP-974 and KIP-975 were aiming for mostly the same thing > and can > > > > be > > > > >> discussed under one umbrella. Apologies if this was rooted in a > > > > >> misunderstanding. > > > > >> > > > > >> Kirshna, > > > > >> > > > > >> I think we need to refine the KIP a bit more. I think there are > some > > > > >> interfaces that we need to include in the KIP as Kafka has > plugins in > > > > >> certain cases where users are expected to provide implementation > and I > > > > >> think it's worth discussing this in the KIP as they're kind of > > > > interfaces > > > > >> for users. Here are my questions in order: > > > > >> 1. In what environments do you want the image to be used? As I > > > > understand > > > > >> it would replace the current testing image and serve as a basis > for > > > > >> development, but would it aim at production use cases too > > > > (docker-compose, > > > > >> Kubernetes, etc.)? > > > > >> 2. How do you plan to forward configs to the broker? Do we expect > a > > > > >> populated server.properties file placed in a certain location or > > > should > > > > >> the > > > > >> docker image create this file based on some input (like env vars)? > > > > >> 3. Certain parts can be pluggable, like metric reporters or > remote log > > > > >> implementations that were just introduced by KIP-405. These > manifest > > > in > > > > >> jar > > > > >> files that must be put on the classpath of Kafka while certain > > > > classnames > > > > >> have to be configured. How do you plan to implement this, how do > we > > > > >> allow users to configure such things? > > > > >> > > > > >> Thanks, > > > > >> Viktor > > > > >> > > > > >> > > > > >> > > > > >> > > > > >> On Thu, Sep 14, 2023 at 4:59 PM Kenneth Eversole > > > > >> <kevers...@cloudflare.com.invalid> wrote: > > > > >> > > > > >> > Hello, > > > > >> > > > > > >> > I think this would be a wonderful improvement to the ecosystem. > > > While > > > > >> > Viktor is correct that most Docker pipelines eventually lead to > a > > > > >> > kubernetes deployment, that should not stop us from creating an > > > > >> > Official Docker Image. Creating a Docker image would allow us to > > > > ensure > > > > >> a > > > > >> > level of quality and support for people who want to deploy > Kafka as > > > a > > > > >> > container on baremetal machines, it could allow us to create > > > > >> > a sandbox/developer environment for new contributors and > developers > > > to > > > > >> test > > > > >> > and have a single agreed upon environment that kafka works in > for > > > > future > > > > >> > KIPs and would most likely spawn more contributions from people > > > > wanting > > > > >> to > > > > >> > optimize kafka for k8s. > > > > >> > > > > > >> > > > > > >> > I am 100% for this and will gladly help if approved. > > > > >> > > > > > >> > Kenneth > > > > >> > > > > > >> > On Thu, Sep 14, 2023 at 5:47 AM Ismael Juma <m...@ismaeljuma.com> > > > > wrote: > > > > >> > > > > > >> > > Hi Viktor, > > > > >> > > > > > > >> > > I disagree. Docker is a very popular deployment tool and it's > not > > > > only > > > > >> > used > > > > >> > > with Kubernetes. > > > > >> > > > > > > >> > > Ismael > > > > >> > > > > > > >> > > On Thu, Sep 14, 2023, 1:14 AM Viktor Somogyi-Vass > > > > >> > > <viktor.somo...@cloudera.com.invalid> wrote: > > > > >> > > > > > > >> > > > Hi Krishna, > > > > >> > > > > > > > >> > > > I think you should merge this KIP and KIP-974 > > > > >> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-974> as > > > there > > > > >> are > > > > >> > overlaps as > > > > >> > > > Federico pointed out on KIP-974 > > > > >> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-974>. I > > > think > > > > >> you > > > > >> > should keep that one as it > > > > >> > > > has well defined goals (improve tests) while I feel this > one is > > > > too > > > > >> > > > generic. Docker is usually just a tool for either testing or > > > > >> > Kubernetes, > > > > >> > > so > > > > >> > > > they have very well defined use-cases. In the case of Flink > for > > > > >> > instance > > > > >> > > > the image is used for its kubernetes operator. The use case > > > would > > > > >> > > determine > > > > >> > > > a lot of things and I think a generic image would likely > not fit > > > > the > > > > >> > > needs > > > > >> > > > of all use-cases. > > > > >> > > > > > > > >> > > > Best, > > > > >> > > > Viktor > > > > >> > > > > > > > >> > > > On Fri, Sep 8, 2023 at 9:58 AM Krishna Agarwal < > > > > >> > > > krishna0608agar...@gmail.com> > > > > >> > > > wrote: > > > > >> > > > > > > > >> > > > > Hi, > > > > >> > > > > Apache Kafka does not have an official docker image > currently. > > > > >> > > > > I want to submit a KIP to publish a docker image for > Apache > > > > Kafka. > > > > >> > > > > > > > > >> > > > > KIP-975 < > > > > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-975>: > > > > >> > Docker Image for Apache Kafka > > > > >> > > > > < > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > Regards, > > > > >> > > > > Krishna > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > >
