[
https://issues.apache.org/jira/browse/KAFKA-9216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17127789#comment-17127789
]
Randall Hauch commented on KAFKA-9216:
--------------------------------------
[~kkonstantine], thanks for reviewing and merging [~EeveeB]'s PR to check the
partition count of the connector configs topic. I've created a followup PR
(https://github.com/apache/kafka/pull/8828) that verifies that each internal
topic has the `cleanup.policy=compact`; the worker fails to start if the
cleanup policy is any other value (e.g., `delete`, `delete,compact` or
`compact,delete`). The logic will avoid the check if the worker created the
topic, since the topic is always created with `cleanup.policy=compact`.
Note that I chose to avoid checking retention, since that only applies if the
`delete` cleanup policy is used. Therefore, the check described in the
preceding paragraph should be sufficient.
> Enforce connect internal topic configuration at startup
> -------------------------------------------------------
>
> Key: KAFKA-9216
> URL: https://issues.apache.org/jira/browse/KAFKA-9216
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Affects Versions: 0.11.0.0
> Reporter: Randall Hauch
> Assignee: Evelyn Bayes
> Priority: Major
> Fix For: 2.3.2, 2.6.0, 2.4.2, 2.5.1
>
>
> Users sometimes configure Connect's internal topic for configurations with
> more than one partition. One partition is expected, however, and using more
> than one leads to weird behavior that is sometimes not easy to spot.
> Here's one example of a log message:
> {noformat}
> "textPayload": "[2019-11-20 11:12:14,049] INFO [Worker clientId=connect-1,
> groupId=td-connect-server] Current config state offset 284 does not match
> group assignment 274. Forcing rebalance.
> (org.apache.kafka.connect.runtime.distributed.DistributedHerder:942)\n"
> {noformat}
> Would it be possible to add a check in the KafkaConfigBackingStore and
> prevent the worker from starting if connect config partition count !=1 ?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
