Hi Krishna, Vedarth, Thanks for the KIP.
1. Can we add directory structure of Docker Image related files in Kafka repo. 2. > Steps for the Docker image release will be included in the Release Process doc of Apache Kafka Can we list down the requirements (repos, accounts) for releasing images to docker hub. I am mainly asking because PMC needs to request docker hub access/repos. I can help in getting required repos/accounts. https://infra.apache.org/docker-hub-policy.html Thanks, Manikumar On Thu, Oct 19, 2023 at 8:22 PM Krishna Agarwal < krishna0608agar...@gmail.com> wrote: > Hi Viktor, > > I've noticed there are two types of custom jar configurations: > > 1. *Type 1*: In this case, only the class name is required(e.g > *authorizer.class.name > <http://authorizer.class.name>**)* This can be configured by the > following steps: > - Mount the jar in the container. > - Configure the *CLASSPATH* environment variable (used by > *kafka-run-class.sh*) by providing the mounted path to it. This can > be passed as an environment variable to the docker container. > 2. *Type 2*: Here, in addition to the class name, classpath can also be > configured (eg *remote.log.metadata.manager.class.name > <http://remote.log.metadata.manager.class.name> *and > *remote.log.metadata.manager.class.path*). This can be configured by the > following steps: > - Mount the jar in the container. > - Configure the respective *class.path* property. > > Regards, > Krishna > > On Mon, Sep 25, 2023 at 11:41 PM Krishna Agarwal < > krishna0608agar...@gmail.com> wrote: > > > Hi Viktor, > > Thanks for the questions. > > > > 1. While the docker image outlined in KIP-975 is designed for > > production environments, it is equally suitable for development and > testing > > purposes. We will furnish the docker image, allowing users the > flexibility > > to employ it according to their specific needs. > > 2. The configs will be injected into the docker container through > > environment variables. These environment variables will have a prefix > > allowing for efficient parsing to extract the relevant > properties.(Will add > > this implementation in the KIP as well once we converge on this.) > > 3. Regarding this question, I'll conduct a test on my end after > > gaining a better understanding, and then provide you with a response. > > > > Regards, > > Krishna > > > > > > On Tue, Sep 19, 2023 at 3:42 PM Viktor Somogyi-Vass > > <viktor.somo...@cloudera.com.invalid> wrote: > > > >> Hi Ismael, > >> > >> I'm not trying to advocate against the docker image, I just pointed out > >> that the current scoping of the KIP may be a bit too generic and thought > >> that KIP-974 and KIP-975 were aiming for mostly the same thing and can > be > >> discussed under one umbrella. Apologies if this was rooted in a > >> misunderstanding. > >> > >> Kirshna, > >> > >> I think we need to refine the KIP a bit more. I think there are some > >> interfaces that we need to include in the KIP as Kafka has plugins in > >> certain cases where users are expected to provide implementation and I > >> think it's worth discussing this in the KIP as they're kind of > interfaces > >> for users. Here are my questions in order: > >> 1. In what environments do you want the image to be used? As I > understand > >> it would replace the current testing image and serve as a basis for > >> development, but would it aim at production use cases too > (docker-compose, > >> Kubernetes, etc.)? > >> 2. How do you plan to forward configs to the broker? Do we expect a > >> populated server.properties file placed in a certain location or should > >> the > >> docker image create this file based on some input (like env vars)? > >> 3. Certain parts can be pluggable, like metric reporters or remote log > >> implementations that were just introduced by KIP-405. These manifest in > >> jar > >> files that must be put on the classpath of Kafka while certain > classnames > >> have to be configured. How do you plan to implement this, how do we > >> allow users to configure such things? > >> > >> Thanks, > >> Viktor > >> > >> > >> > >> > >> On Thu, Sep 14, 2023 at 4:59 PM Kenneth Eversole > >> <kevers...@cloudflare.com.invalid> wrote: > >> > >> > Hello, > >> > > >> > I think this would be a wonderful improvement to the ecosystem. While > >> > Viktor is correct that most Docker pipelines eventually lead to a > >> > kubernetes deployment, that should not stop us from creating an > >> > Official Docker Image. Creating a Docker image would allow us to > ensure > >> a > >> > level of quality and support for people who want to deploy Kafka as a > >> > container on baremetal machines, it could allow us to create > >> > a sandbox/developer environment for new contributors and developers to > >> test > >> > and have a single agreed upon environment that kafka works in for > future > >> > KIPs and would most likely spawn more contributions from people > wanting > >> to > >> > optimize kafka for k8s. > >> > > >> > > >> > I am 100% for this and will gladly help if approved. > >> > > >> > Kenneth > >> > > >> > On Thu, Sep 14, 2023 at 5:47 AM Ismael Juma <m...@ismaeljuma.com> > wrote: > >> > > >> > > Hi Viktor, > >> > > > >> > > I disagree. Docker is a very popular deployment tool and it's not > only > >> > used > >> > > with Kubernetes. > >> > > > >> > > Ismael > >> > > > >> > > On Thu, Sep 14, 2023, 1:14 AM Viktor Somogyi-Vass > >> > > <viktor.somo...@cloudera.com.invalid> wrote: > >> > > > >> > > > Hi Krishna, > >> > > > > >> > > > I think you should merge this KIP and KIP-974 > >> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-974> as there > >> are > >> > overlaps as > >> > > > Federico pointed out on KIP-974 > >> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-974>. I think > >> you > >> > should keep that one as it > >> > > > has well defined goals (improve tests) while I feel this one is > too > >> > > > generic. Docker is usually just a tool for either testing or > >> > Kubernetes, > >> > > so > >> > > > they have very well defined use-cases. In the case of Flink for > >> > instance > >> > > > the image is used for its kubernetes operator. The use case would > >> > > determine > >> > > > a lot of things and I think a generic image would likely not fit > the > >> > > needs > >> > > > of all use-cases. > >> > > > > >> > > > Best, > >> > > > Viktor > >> > > > > >> > > > On Fri, Sep 8, 2023 at 9:58 AM Krishna Agarwal < > >> > > > krishna0608agar...@gmail.com> > >> > > > wrote: > >> > > > > >> > > > > Hi, > >> > > > > Apache Kafka does not have an official docker image currently. > >> > > > > I want to submit a KIP to publish a docker image for Apache > Kafka. > >> > > > > > >> > > > > KIP-975 < > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-975>: > >> > Docker Image for Apache Kafka > >> > > > > < > >> > > > > > >> > > > > >> > > > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka > >> > > > > > > >> > > > > > >> > > > > Regards, > >> > > > > Krishna > >> > > > > > >> > > > > >> > > > >> > > >> > > >
