[jira] [Commented] (KAFKA-9774) Create official Docker image for Kafka Connect
[
https://issues.apache.org/jira/browse/KAFKA-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070641#comment-17070641
]
Jordan Moore commented on KAFKA-9774:
-
Hi [~ewencp]. Thanks for the response.
{quote}
it's a significant public support, compatibility, and maintenance commitment
for the project
{quote}
1. Public support and maintenance can be tracked in JIRA, mailing list, and
other mediums as currently done, no?
2. Compatibility is locked to the versions of dependencies locked in Gradle.
The existing Kafka Compatibility Matrix page is good enough for all clients,
including Connect. If you mean compatibility of various Java base images or
OS's, then there should be a vote on that, perhaps?
{quote}
what's the commitment to publication during releases
{quote}
Well, the jib-gradle-plugin can be added to release as part of a standard AK
release, so sure, the release process will have one new build artifact that
would push into DockerHub. The RM would maintain a DockerHub credential, maybe
initialized by a PMC, I assume.
{quote}
periodic updates to get base image CVE updates
{quote}
I'd say this would be done per AK release, as needed. Base images I can think
of would be openjdk , adoptopenjdk, or the new RedHat UBI images.
{quote}
verification, testing
{quote}
What kinds of testing would be needed outside of the existing Connect test
framework? A Docker image is just a build artifact like a JAR, no? After JARs
are built, what extra testing is being done?
> Create official Docker image for Kafka Connect
> --
>
> Key: KAFKA-9774
> URL: https://issues.apache.org/jira/browse/KAFKA-9774
> Project: Kafka
> Issue Type: Task
> Components: build, KafkaConnect, packaging
>Affects Versions: 2.4.1
>Reporter: Jordan Moore
>Priority: Major
> Labels: build, features
> Attachments: image-2020-03-27-05-04-46-792.png,
> image-2020-03-27-05-05-59-024.png
>
>
> This is a ticket for creating an *official* apache/kafka-connect Docker
> image.
> Does this need a KIP? - I don't think so. This would be a new feature, not
> any API change.
> Why is this needed?
> # Kafka Connect is stateless. I believe this is why a Kafka image is not
> created?
> # It scales much more easily with Docker and orchestrators. It operates much
> like any other serverless / "microservice" web application
> # People struggle with deploying it because it is packaged _with Kafka_ ,
> which leads some to believe it needs to _*run* with Kafka_ on the same
> machine.
> I think there is separate ticket for creating an official Docker image for
> Kafka but clearly none exist. I reached out to Confluent about this, but
> heard nothing yet.
> !image-2020-03-27-05-05-59-024.png|width=740,height=196!
>
> Zookeeper already has one , btw
> !image-2020-03-27-05-04-46-792.png|width=739,height=288!
> *References*:
> [Docs for Official
> Images|[https://docs.docker.com/docker-hub/official_images/]]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-9774) Create official Docker image for Kafka Connect
[ https://issues.apache.org/jira/browse/KAFKA-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070633#comment-17070633 ] Ewen Cheslack-Postava commented on KAFKA-9774: -- My first reaction to this was that is should have a KIP because it's a significant public support, compatibility, and maintenance commitment for the project... Maybe if it's very clearly highlighted as unofficial, no compatibility guarantees, etc, though I'm not sure how many people want that type of image being published from the project. As simple off-the-top-of-my-head examples, what's the commitment to publication during releases (is this being added to [https://cwiki.apache.org/confluence/display/KAFKA/Release+Process]?), periodic updates to get base image CVE updates, verification, testing, etc? > Create official Docker image for Kafka Connect > -- > > Key: KAFKA-9774 > URL: https://issues.apache.org/jira/browse/KAFKA-9774 > Project: Kafka > Issue Type: Task > Components: build, KafkaConnect, packaging >Affects Versions: 2.4.1 >Reporter: Jordan Moore >Priority: Major > Labels: build, features > Attachments: image-2020-03-27-05-04-46-792.png, > image-2020-03-27-05-05-59-024.png > > > This is a ticket for creating an *official* apache/kafka-connect Docker > image. > Does this need a KIP? - I don't think so. This would be a new feature, not > any API change. > Why is this needed? > # Kafka Connect is stateless. I believe this is why a Kafka image is not > created? > # It scales much more easily with Docker and orchestrators. It operates much > like any other serverless / "microservice" web application > # People struggle with deploying it because it is packaged _with Kafka_ , > which leads some to believe it needs to _*run* with Kafka_ on the same > machine. > I think there is separate ticket for creating an official Docker image for > Kafka but clearly none exist. I reached out to Confluent about this, but > heard nothing yet. > !image-2020-03-27-05-05-59-024.png|width=740,height=196! > > Zookeeper already has one , btw > !image-2020-03-27-05-04-46-792.png|width=739,height=288! > *References*: > [Docs for Official > Images|[https://docs.docker.com/docker-hub/official_images/]] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-9347) Detect deleted log directory before becoming leader
[ https://issues.apache.org/jira/browse/KAFKA-9347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070549#comment-17070549 ] Richard Yu commented on KAFKA-9347: --- [~hachikuji] Can I pick this one up? > Detect deleted log directory before becoming leader > --- > > Key: KAFKA-9347 > URL: https://issues.apache.org/jira/browse/KAFKA-9347 > Project: Kafka > Issue Type: Improvement >Reporter: Jason Gustafson >Priority: Major > Labels: needs-discussion > > There is no protection currently if a broker has had its log directory > deleted to prevent it from becoming the leader of a partition that it still > remains in the ISR of. This scenario can happen when the last remaining > replica in the ISR is shutdown. It will remain in the ISR and be eligible for > leadership as soon as it starts up. It would be useful to either detect this > case situation dynamically in order to force the user to do an unclean > election or recover another broker. One option might be just to pass a flag > on startup to specify that a broker should not be eligible for leadership. -- This message was sent by Atlassian Jira (v8.3.4#803005)
