cshuo opened a new issue, #19517: URL: https://github.com/apache/hudi/issues/19517
### Problem When Flink bulk insert uses the bucket index with non-blocking concurrency control (NBCC), fixed bucket file IDs can repeat across partitions because the bucket ID is only unique within a partition. The bulk-insert sort path currently orders records by file ID only, and `BucketBulkInsertWriterHelper` caches write handles by file ID only. Records from different partitions that share the same bucket ID can therefore be grouped together or reuse the wrong handle. This affects both the regular bucket layout and the LSM bucket layout. The non-NBCC paths should retain their existing schema and sorting behavior to avoid unnecessary overhead. ### Expected behavior In NBCC mode, bulk-insert records should be sorted and cached by the partition-aware file group identity, while non-NBCC behavior remains unchanged. ### Reproduction Use a partitioned Flink table configured with: - `operation = bulk_insert` - a bucket index where the same bucket ID is present in multiple partitions - `write.concurrency.mode = NON_BLOCKING_CONCURRENCY_CONTROL` - bulk-insert input sorting enabled or disabled Each partition should produce its own write status and file group even when the generated file IDs are identical. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
