[ https://issues.apache.org/jira/browse/KAFKA-13700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17569115#comment-17569115 ]
Divij Vaidya commented on KAFKA-13700: -------------------------------------- cc: [~junrao] for visibility since he is answering another corruption issue in https://issues.apache.org/jira/browse/KAFKA-13953 > Kafka reporting CorruptRecordException exception > ------------------------------------------------ > > Key: KAFKA-13700 > URL: https://issues.apache.org/jira/browse/KAFKA-13700 > Project: Kafka > Issue Type: Bug > Components: core > Affects Versions: 2.4.1 > Environment: ubuntu 16.04 > kafka 2.4 > Reporter: Uday Bhaskar > Priority: Critical > > In our kafka cluster a couple of partitions in __consumer_offsets and 1 > regular topic getting data corruption issue while replicas trying to read > from leader. Similar messages for other partitions as well . > > [2022-02-28 21:57:29,941] ERROR [ReplicaFetcher replicaId=6, leaderId=1, > fetcherId=2] Found invalid messages during fetch for partition > __consumer_offsets-10 offset 108845487 (kafka.server.ReplicaFetcherThread) > org.apache.kafka.common.errors.CorruptRecordException: Record is corrupt > (stored crc = 1524235439) in topic partition __consumer_offsets-10 > > another topic partitions with same errors > [2022-02-28 22:17:00,235] ERROR [ReplicaFetcher replicaId=6, leaderId=1, > fetcherId=0] Found invalid messages during fetch for partition > px-11351-xxxxxx-a56c642-0 offset 11746872 (kafka.server.ReplicaFetcherThread) > org.apache.kafka.common.errors.CorruptRecordException: Record is corrupt > (stored crc = 475179617) in topic partition px-11351-xxxxxx-a56c642-0. > > I have verified all infrastructure, dish network and system for any errors > found and nothing found. I am not sure why it is happening or how to > troubleshoot. > > Bellow is output of the message from DumpLogSegments , > > $ /opt/ns/kafka/bin/kafka-run-class.sh kafka.tools.DumpLogSegments > --verify-index-only --deep-iteration --files ./00000000000011324034.log | > grep 11746872 > baseOffset: 11746872 lastOffset: 11746872 count: 1 baseSequence: 50278 > lastSequence: 50278 producerId: 17035 producerEpoch: 0 partitionLeaderEpoch: > 8 isTransactional: false isControl: false position: 252530345 CreateTime: > 1645886348240 size: 647 magic: 2 compresscodec: SNAPPY crc: 475179617 > isvalid: true > | offset: 11746872 CreateTime: 1645886348240 keysize: 54 valuesize: 637 > sequence: 50278 headerKeys: [] -- This message was sent by Atlassian Jira (v8.20.10#820010)