Hi Viktor, Thanks for the questions. 1. While the docker image outlined in KIP-975 is designed for production environments, it is equally suitable for development and testing purposes. We will furnish the docker image, allowing users the flexibility to employ it according to their specific needs. 2. The configs will be injected into the docker container through environment variables. These environment variables will have a prefix allowing for efficient parsing to extract the relevant properties.(Will add this implementation in the KIP as well once we converge on this.) 3. Regarding this question, I'll conduct a test on my end after gaining a better understanding, and then provide you with a response.
Regards, Krishna On Tue, Sep 19, 2023 at 3:42 PM Viktor Somogyi-Vass <viktor.somo...@cloudera.com.invalid> wrote: > Hi Ismael, > > I'm not trying to advocate against the docker image, I just pointed out > that the current scoping of the KIP may be a bit too generic and thought > that KIP-974 and KIP-975 were aiming for mostly the same thing and can be > discussed under one umbrella. Apologies if this was rooted in a > misunderstanding. > > Kirshna, > > I think we need to refine the KIP a bit more. I think there are some > interfaces that we need to include in the KIP as Kafka has plugins in > certain cases where users are expected to provide implementation and I > think it's worth discussing this in the KIP as they're kind of interfaces > for users. Here are my questions in order: > 1. In what environments do you want the image to be used? As I understand > it would replace the current testing image and serve as a basis for > development, but would it aim at production use cases too (docker-compose, > Kubernetes, etc.)? > 2. How do you plan to forward configs to the broker? Do we expect a > populated server.properties file placed in a certain location or should the > docker image create this file based on some input (like env vars)? > 3. Certain parts can be pluggable, like metric reporters or remote log > implementations that were just introduced by KIP-405. These manifest in jar > files that must be put on the classpath of Kafka while certain classnames > have to be configured. How do you plan to implement this, how do we > allow users to configure such things? > > Thanks, > Viktor > > > > > On Thu, Sep 14, 2023 at 4:59 PM Kenneth Eversole > <kevers...@cloudflare.com.invalid> wrote: > > > Hello, > > > > I think this would be a wonderful improvement to the ecosystem. While > > Viktor is correct that most Docker pipelines eventually lead to a > > kubernetes deployment, that should not stop us from creating an > > Official Docker Image. Creating a Docker image would allow us to ensure a > > level of quality and support for people who want to deploy Kafka as a > > container on baremetal machines, it could allow us to create > > a sandbox/developer environment for new contributors and developers to > test > > and have a single agreed upon environment that kafka works in for future > > KIPs and would most likely spawn more contributions from people wanting > to > > optimize kafka for k8s. > > > > > > I am 100% for this and will gladly help if approved. > > > > Kenneth > > > > On Thu, Sep 14, 2023 at 5:47 AM Ismael Juma <m...@ismaeljuma.com> wrote: > > > > > Hi Viktor, > > > > > > I disagree. Docker is a very popular deployment tool and it's not only > > used > > > with Kubernetes. > > > > > > Ismael > > > > > > On Thu, Sep 14, 2023, 1:14 AM Viktor Somogyi-Vass > > > <viktor.somo...@cloudera.com.invalid> wrote: > > > > > > > Hi Krishna, > > > > > > > > I think you should merge this KIP and KIP-974 > > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-974> as there are > > overlaps as > > > > Federico pointed out on KIP-974 > > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-974>. I think you > > should keep that one as it > > > > has well defined goals (improve tests) while I feel this one is too > > > > generic. Docker is usually just a tool for either testing or > > Kubernetes, > > > so > > > > they have very well defined use-cases. In the case of Flink for > > instance > > > > the image is used for its kubernetes operator. The use case would > > > determine > > > > a lot of things and I think a generic image would likely not fit the > > > needs > > > > of all use-cases. > > > > > > > > Best, > > > > Viktor > > > > > > > > On Fri, Sep 8, 2023 at 9:58 AM Krishna Agarwal < > > > > krishna0608agar...@gmail.com> > > > > wrote: > > > > > > > > > Hi, > > > > > Apache Kafka does not have an official docker image currently. > > > > > I want to submit a KIP to publish a docker image for Apache Kafka. > > > > > > > > > > KIP-975 <https://cwiki.apache.org/confluence/display/KAFKA/KIP-975 > >: > > Docker Image for Apache Kafka > > > > > < > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka > > > > > > > > > > > > > > > > Regards, > > > > > Krishna > > > > > > > > > > > > > > >
