[
https://issues.apache.org/jira/browse/KAFKA-15328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dgiri updated KAFKA-15328:
--------------------------
Description:
When running the intra-broker partition reassignments from one disk to another,
kafka triggers offline log dir. This happens randomly and can be reproduced
only after running the intra-broker reassignments multiple times.
Observations made:
# After running the partition reassignment from (/data1 to /data2), kafka
creates a *_topic-partition.uniqueId-future_* location on /data2 and starts
copying the data.
# Partition reassignment completes as expected by renaming the
_*topic-partition.uniqueId-future*_ location to _*topic-partition*_ on /data2.
# Within few milliseconds kafka still looks for
_*topic-partition.uniqueId-future*_ location{_},{_} since it won't find that
location anymore it marks the /data2 log dir as offline
Broker logs filtered with particular partition and error has been attached for
more details.
was:
When running the intra-broker partition reassignments from one disk to another,
kafka triggers offline log dir. This happens randomly and can be reproduced
only after running the intra-broker reassignments multiple times.
Observations made:
# After running the partition reassignment from (/data1 to /data2), kafka
creates a *_topic-partition.uniqueId-future_* location on /data2 and starts
copying the data.
# Partition reassignment completes as expected by renaming the
_*topic-partition.uniqueId-future*_ location __ to _*topic-partition*_ on
/data2.
# Within few milliseconds kafka still looks for
_*topic-partition.uniqueId-future*_ location{_},{_} since it won't find that
location anymore it marks the /data2 log dir as offline
Broker logs filtered with particular partition and error has been attached for
more details.
> Inter broker partition reassignment triggers offline log dirs randomly
> ----------------------------------------------------------------------
>
> Key: KAFKA-15328
> URL: https://issues.apache.org/jira/browse/KAFKA-15328
> Project: Kafka
> Issue Type: Bug
> Components: core, tools
> Affects Versions: 3.3.2
> Environment: RHEL 8.8 (Ootpa)
> Reporter: dgiri
> Priority: Major
> Labels: kafka-reassign-partitions
> Attachments: kafka_partition_reassignment_offline_lod_dir_errors.txt
>
>
> When running the intra-broker partition reassignments from one disk to
> another, kafka triggers offline log dir. This happens randomly and can be
> reproduced only after running the intra-broker reassignments multiple times.
>
> Observations made:
> # After running the partition reassignment from (/data1 to /data2), kafka
> creates a *_topic-partition.uniqueId-future_* location on /data2 and starts
> copying the data.
> # Partition reassignment completes as expected by renaming the
> _*topic-partition.uniqueId-future*_ location to _*topic-partition*_ on /data2.
> # Within few milliseconds kafka still looks for
> _*topic-partition.uniqueId-future*_ location{_},{_} since it won't find that
> location anymore it marks the /data2 log dir as offline
>
> Broker logs filtered with particular partition and error has been attached
> for more details.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
