hudi-agent commented on code in PR #19844:
URL: https://github.com/apache/hudi/pull/19844#discussion_r3946679560
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/sink/utils/Pipelines.java:
##########
@@ -410,11 +410,22 @@ public static DataStream<HoodieFlinkInternalRow>
bootstrap(
boolean overwrite) {
final boolean globalIndex = conf.get(FlinkOptions.INDEX_GLOBAL_ENABLED);
if (overwrite || OptionsResolver.isBucketIndexType(conf)) {
+ // overwrite and bucket index do not rely on a bootstrapped key index.
return rowDataToHoodieRecord(conf, rowType, dataStream);
}
- if (bounded && !globalIndex && OptionsResolver.isPartitionedTable(conf)) {
+ boolean isRLI = OptionsResolver.isGlobalRecordLevelIndex(conf) ||
OptionsResolver.isRecordLevelIndex(conf);
+ if (bounded && isRLI && !conf.get(FlinkOptions.INDEX_BOOTSTRAP_ENABLED)) {
+ // RLI bucket assignment (global or partitioned) looks up the record
index directly,
+ // so the generic bootstrap step can be skipped unless it is explicitly
requested.
+ return rowDataToHoodieRecord(conf, rowType, dataStream);
+ }
+ if (bounded && !globalIndex && !isRLI &&
OptionsResolver.isPartitionedTable(conf)) {
+ // non-RLI partitioned tables shuffle by partition path and bootstrap
per-partition
+ // during batch execution to support batch UPSERT.
return boundedBootstrap(conf, rowType, dataStream);
}
+ // streaming execution, or bounded execution for a
non-partitioned/global-index/RLI-with-
Review Comment:
🤖 With this reordering, a bounded + partitioned + non-global table using
RECORD_LEVEL_INDEX with INDEX_BOOTSTRAP_ENABLED=true no longer reaches
`boundedBootstrap()` — it falls through to `streamBootstrap()`. When the RLI
backend isn't rocksdb, `BootstrapOperatorFactory` then returns the plain
`BootstrapOperator`, whose emitted index records land in
`DynamicBucketAssignFunction.processElement`, which (unlike
`BucketAssignFunction`/`MinibatchBucketAssignFunction`) never checks
`isIndexRecord()` and would forward them downstream as data. Is that
combination expected to be guarded (e.g. still route to `boundedBootstrap`, or
require the rocksdb backend)?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
