Gatsby-Lee commented on issue #10235:
URL: https://github.com/apache/hudi/issues/10235#issuecomment-2264980204
Any updates on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific co
zyclove commented on issue #10235:
URL: https://github.com/apache/hudi/issues/10235#issuecomment-1838182121
SparkMetadataTableRecordIndex
fileGroupSize =
hoodieTable.getMetadataTable().getNumFileGroupsForPartition(MetadataPartitionType.RECORD_INDEX);
Why not 512 fileGroupSi
zyclove commented on issue #10235:
URL: https://github.com/apache/hudi/issues/10235#issuecomment-1838138200
@danny0405 With set hoodie.metadata.enable=true, now is RECORD_INDEX.
But the follow stage is very very slow too.
23/12/04 14:39:29 WARN SparkMetadataTa
zyclove commented on issue #10235:
URL: https://github.com/apache/hudi/issues/10235#issuecomment-1837946117
@danny0405
why is back to GLOBAL_SIMPLE?
https://github.com/apache/hudi/assets/15028279/9cddf011-e25c-4c0f-9b40-c2d7fdd17cf9";>
23/12/04 14:39:29 WARN SparkMetadataTableRe
zyclove opened a new issue, #10235:
URL: https://github.com/apache/hudi/issues/10235
**Describe the problem you faced**
The spark job is too slow in follow stage. Adjusting CPU, memory, and
concurrency has no effect.
Which stage can be optimized or skipped?
