[
https://issues.apache.org/jira/browse/KAFKA-17375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai resolved KAFKA-17375.
------------------------------------
Fix Version/s: 4.5.0
Resolution: Fixed
> Partition reassignment cannot proceed after segment.bytes change
> ----------------------------------------------------------------
>
> Key: KAFKA-17375
> URL: https://issues.apache.org/jira/browse/KAFKA-17375
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 3.7.1
> Reporter: Tommi Vainikainen
> Priority: Major
> Fix For: 4.5.0
>
>
> If a topic's dynamic configuration `segment.bytes` is changed to smaller
> value than existing messages, Kafka cannot anymore handle partition
> reassignment between brokers for given topic.
> Current behaviour causes Kafka to throw `{_}RecordBatchTooLargeException{_}`
> if during partition reassignment the topic contains messages larger than
> current `segment.bytes`.
> Expected behaviour: Kafka should proceed with partition reassignment for all
> existing messages. `{_}segment.bytes{_}` should only limit producing new
> messages from clients.
> Steps to reproduce:
> 1. Start multi node cluster.
> 2. Create a single partition topic and produce a message of size 512 kb into
> it
> 3. Alter dynamic topic configuration with `segment.bytes`value less than 512
> kb.
> _./bin/kafka-configs.sh --bootstrap-server localhost:9092 --alter
> --entity-type topics --entity-name my-test-topic --add-config
> segment.bytes=500000_
> (Optionally verify that one cannot anymore produce 512 kb messages due
> "{_}Delivery failed for message: Broker: Message batch larger than configured
> server segment size{_}".)
> 4. Reassign this partition to another broker with kafka-reassign-partitions.sh
> 5. Observe error messages on broker node and ongoing reassignment getting
> stuck
> {noformat}
> ERROR [ReplicaFetcher replicaId=3, leaderId=2, fetcherId=0] Unexpected error
> occurred while processing data for partition my-test-topic-0 at offset 0
> (kafka.server.ReplicaFetcherThread)
> org.apache.kafka.common.errors.RecordBatchTooLargeException: Message batch
> size is 524360 bytes in append to partition my-test-topic-0, which exceeds
> the maximum configured segment size of 500000.
> WARN [ReplicaFetcher replicaId=3, leaderId=2, fetcherId=0] Partition
> my-test-topic-0 marked as failed (kafka.server.ReplicaFetcherThread)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
