[
https://issues.apache.org/jira/browse/KAFKA-13338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816800#comment-17816800
]
Greg Harris commented on KAFKA-13338:
-------------------------------------
Apache Kafka does not have 5.x or 6.x releases, so you must be using a fork of
Kafka.
The log message and your remediation are as-expected; Connect must use
compacted internal topics, and enforcement of this was added after-the-fact.
The delay between the upgrade and the connectors failing also indicates that
the two are unrelated, and compaction shouldn't be able to cause a
InvalidRecordException.
As this issue is so stale, and there isn't any information to proceed on, I
have closed it. Feel free to raise a new issue if this problem reappears.
> kafka upgrade 6.1.1 asking to change log.cleanup.policy from delete to compact
> ------------------------------------------------------------------------------
>
> Key: KAFKA-13338
> URL: https://issues.apache.org/jira/browse/KAFKA-13338
> Project: Kafka
> Issue Type: Bug
> Components: log
> Reporter: Fátima Galera
> Priority: Major
> Attachments: image003.png
>
>
> Hi all,
>
> During kafka upgrade from 5.3.1 to 6.1.1.1 we get below error starting
> kafka-connector services
>
> ERROR [Worker clientId=connect-1, groupId=connect-cluster] Uncaught exception
> in herder work thread, exiting:
> (org.apache.kafka.connect.runtime.distributed.DistributedHerder)
> 87738-org.apache.kafka.common.config.ConfigException: Topic 'connect-offsets'
> supplied via the 'offset.storage.topic' property is required to have
> 'cleanup.policy=compact' to guarantee consistency and durability of source
> connector offsets, but found the topic currently has 'cleanup.policy=delete'.
> Continuing would likely result in eventually losing source connector offsets
> and problems restarting this Connect cluster in the future. Change the
> 'offset.storage.topic' property in the Connect worker configurations to use a
> topic with 'cleanup.policy=compact'.
>
> After that error we added log.cleanup.policy=compact parameter on
> /etc/kafka/server.properties and kafka-connector service started but a few
> days later all the connectors are down with attached error.
>
> Could you please let us know how we can resolved the issue?
>
> Best regards and thanks
> Fátima
> NGA -Alight
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
