Hey Viktor! Thanks for bringing up this use case. I think we can take advantage of Docker volume for this. We can allow users to mount a folder containing the secret files. This folder can then be used to pass secrets to the container.
Thanks and regards, Vedarth On Wed, Sep 27, 2023 at 4:39 PM Viktor Somogyi-Vass <viktor.somo...@cloudera.com.invalid> wrote: > Hi Krishna, > > Thanks for the answer. I've seen multiple such images where environment > variables are used and I think they are generally good but it's unsafe for > passing around secrets, jaas configs and so on. Perhaps for secrets we > could recommend using the file config provider. Then users can create and > mount secured properties file(s) with configs that are considered as > secrets. What do you think? Did you already have something in your mind > regarding this? > > Thanks, > Viktor > > On Tue, Sep 26, 2023 at 3:05 PM Krishna Agarwal < > krishna0608agar...@gmail.com> wrote: > > > Hi Ismael, > > Apologies for missing the mailing list in the last reply. > > > > Thank you for the suggestions. > > Just to clarify, the sizes mentioned in the previous email are of the > > uncompressed base images, not the resulting Apache Kafka docker images: > > > > 1. eclipse-temurin:17-jre -- 263MB (They should release JRE 21 images > > soon) > > 2. > registry.access.redhat.com/ubi8/openjdk-17-runtime:1.17-1.1693366274 > > -- 375MB > > > > Regards, > > Krishna > > > > > > On Tue, Sep 26, 2023 at 9:24 AM Ismael Juma <m...@ismaeljuma.com> wrote: > > > > > Hi Krishna, > > > > > > Looks like you sent the response to me and not the mailing list, > > > please include the mailing list in the replies. Comments below. > > > > > > On Mon, Sep 25, 2023 at 11:45 AM Krishna Agarwal < > > > krishna0608agar...@gmail.com> wrote: > > > > > >> Hi Ismael, > > >> Thanks for the questions. > > >> > > >> 1. We intend to support only the latest Java supported by Apache > > >> Kafka(As per this documentation > > >> <https://kafka.apache.org/documentation/#java> Apache Kafka > > currently > > >> supports Java 8, Java 11, and Java 17) which currently is Java 17. > If > > >> Apache Kafka supports Java 21 in the future, we will align with it. > > >> > > >> We are already building and testing with Java 21 ( > > > https://github.com/apache/kafka/pull/14451 updates `README.md` to > > > indicate that). By 3.7.0 (the next release), we'll have Java 21 as one > of > > > the officially supported versions. I think we should start with that > > > version for both docker image KIPs. > > > > > >> > > >> 1. For users seeking a Docker image with an alternative Java > version, > > >> they will have the flexibility to build their own Docker image > > utilising > > >> the Dockerfiles we provide. In our documentation, we will provide > > clear > > >> guidance on the designated base images for various Java versions. > > >> > > >> This sounds good to me. We should include these details as part of the > > > KIP and also the documentation for the docker images. More > specifically, > > we > > > should state that we will update the Java major version as part of > minor > > > Apache Kafka releases. The implication is that users who include broker > > > plugins alongside the broker should use custom images to ensure their > > > custom code is not broken by Java upgrades. > > > > > >> > > >> 1. Apache Kafka only requires JRE, not JDK, for operation. > Utilizing > > >> a base image with only JRE, rather than JDK, is a logical choice as > > it > > >> significantly reduces the size of the docker image. > > >> Upon further investigation, I discovered the eclipse-temurin > > >> < > > > https://hub.docker.com/layers/library/eclipse-temurin/17-jre/images/sha256-d1dfb065ae433fe1b43ac7e50a1ed03660f487c73ec256c686b126c37fd4d086?context=explore > > > > > >> docker image, which is notably smaller than Redhat’s ubi8 docker > > image (263 > > >> MB vs 375 MB). Additionally, the fact that Apache Flink relies on > > >> eclipse-temurin base images > > >> < > > > https://github.com/apache/flink-docker/blob/master/1.17/scala_2.12-java11-ubuntu/Dockerfile#L19 > > > > > >> further increases our confidence in their dependability(Will make > > this > > >> change in the KIP). > > >> > > >> Yes, eclipse-temurin looks like a good choice to me. Nice size > > reduction! > > > > > >> > > >> 1. I'll conduct comparisons between our docker image and existing > > >> ones, and incorporate the findings into the KIP. I'll keep you > > posted on > > >> the same. > > >> > > >> Excellent, thanks! > > > > > > Ismael > > > > > > > > >> On Wed, Sep 20, 2023 at 11:26 PM Ismael Juma <m...@ismaeljuma.com> > wrote: > > >> > > >>> Hi Krishna, > > >>> > > >>> Thanks for the KIP. A few quick questions: > > >>> > > >>> 1. Since this will only be available for Kafka 3.7 in the best case, > I > > >>> suggest we go with Java 21 instead of Java 17. Also, we should be > clear > > >>> about Java version expectations. Are we allowed to change the Java > > >>> version > > >>> whenever we want? Or can it only be done in major versions? I prefer > > the > > >>> former option by default, but a subset of users may have their own > > >>> plugins > > >>> running within the broker. Such users may need to have control over > the > > >>> Java version they use - how would they achieve tha > > >>> 2. We state that ubi-minimal would be the base image - it would be > good > > >>> to > > >>> provide some details on why this was the choice. > > >>> 3. How does this docker image compare to the existing popular images? > > >>> Perhaps we can compare with the 3 most popular images for Apache > Kafka > > >>> these days. > > >>> > > >>> Ismael > > >>> > > >>> On Fri, Sep 8, 2023 at 12:57 AM Krishna Agarwal < > > >>> krishna0608agar...@gmail.com> wrote: > > >>> > > >>> > Hi, > > >>> > Apache Kafka does not have an official docker image currently. > > >>> > I want to submit a KIP to publish a docker image for Apache Kafka. > > >>> > > > >>> > KIP-975: Docker Image for Apache Kafka > > >>> > < > > >>> > > > >>> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-975%3A+Docker+Image+for+Apache+Kafka > > >>> > > > > >>> > > > >>> > Regards, > > >>> > Krishna > > >>> > > > >>> > > >> > > >
