[
https://issues.apache.org/jira/browse/HUDI-1309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar updated HUDI-1309:
---------------------------------
Fix Version/s: 0.9.0
> Listing Metadata unreadable in S3 as the log block is deemed corrupted
> ----------------------------------------------------------------------
>
> Key: HUDI-1309
> URL: https://issues.apache.org/jira/browse/HUDI-1309
> Project: Apache Hudi
> Issue Type: Sub-task
> Components: Writer Core
> Reporter: Balaji Varadarajan
> Priority: Blocker
> Fix For: 0.9.0
>
>
> When running metadata list-partitions CLI command, I am seeing the below
> messages and the partition list is empty. Was expecting 10K partitions.
>
> {code:java}
> 36589 [Spring Shell] INFO
> org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner - Scanning
> log file
> HoodieLogFile{pathStr='s3a://robinhood-encrypted-hudi-data-cove/dev_hudi_tables/balaji_varadarajan/benchmark_1M_10K_partitions/.hoodie/metadata/metadata_partition/.f02585bd-bb02-43f6-8bc8-cec71df87d1e-0_00000000000000.log.1_0-23-206045',
> fileLen=0}
> 36590 [Spring Shell] INFO
> org.apache.hudi.common.table.log.HoodieLogFileReader - Found corrupted block
> in file
> HoodieLogFile{pathStr='s3a://robinhood-encrypted-hudi-data-cove/dev_hudi_tables/balaji_varadarajan/benchmark_1M_10K_partitions/.hoodie/metadata/metadata_partition/.f02585bd-bb02-43f6-8bc8-cec71df87d1e-0_00000000000000.log.1_0-23-206045',
> fileLen=0} with block size(3723305) running past EOF
> 36684 [Spring Shell] INFO
> org.apache.hudi.common.table.log.HoodieLogFileReader - Log
> HoodieLogFile{pathStr='s3a://xxxx/dev_hudi_tables/balaji_varadarajan/benchmark_1M_10K_partitions/.hoodie/metadata/metadata_partition/.f02585bd-bb02-43f6-8bc8-cec71df87d1e-0_00000000000000.log.1_0-23-206045',
> fileLen=0} has a corrupted block at 14
> 44515 [Spring Shell] INFO
> org.apache.hudi.common.table.log.HoodieLogFileReader - Next available block
> in
> HoodieLogFile{pathStr='s3a://xxxx/dev_hudi_tables/balaji_varadarajan/benchmark_1M_10K_partitions/.hoodie/metadata/metadata_partition/.f02585bd-bb02-43f6-8bc8-cec71df87d1e-0_00000000000000.log.1_0-23-206045',
> fileLen=0} starts at 3723319
> 44566 [Spring Shell] INFO
> org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner - Found a
> corrupt block in
> s3a://xxxx/dev_hudi_tables/balaji_varadarajan/benchmark_1M_10K_partitions/.hoodie/metadata/metadata_partition/.f02585bd-bb02-43f6-8bc8-cec71df87d1e-0_00000000000000.log.1_0-23-206045
> 44567 [Spring Shell] INFO
> org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner - M{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
