cshuo commented on code in PR #18897:
URL: https://github.com/apache/hudi/pull/18897#discussion_r3345467741
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java:
##########
@@ -638,6 +638,14 @@ public class FlinkOptions extends HoodieConfig {
.withDescription("Users can use this parameter to specify expression and
the corresponding bucket "
+ "numbers (separated by commas).Multiple rules are separated by
semicolons like "
+
"hoodie.bucket.index.partition.expressions=expression1,bucket-number1;expression2,bucket-number2");
+
+ @AdvancedConfig
+ public static final ConfigOption<Boolean>
BUCKET_INDEX_REMOTE_PARTITIONER_ENABLE = ConfigOptions
Review Comment:
it's unnecessary to introduce this config as a flink option, users can
configure $BUCKET_PARTITIONER directly in DDL options.
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/OptionsResolver.java:
##########
@@ -258,6 +258,24 @@ public static boolean
isSimpleBucketIndexType(Configuration conf) {
return isBucketIndexType(conf) &&
getBucketEngineType(conf).equals(HoodieIndex.BucketIndexEngineType.SIMPLE);
}
+ /**
+ * Returns whether the simple bucket index should use remote partitioner.
+ */
+ public static boolean enableBucketRemotePartitioner(Configuration conf) {
Review Comment:
+1
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/embedded/EmbeddedTimelineService.java:
##########
@@ -280,17 +283,16 @@ public boolean equals(Object o) {
return false;
}
TimelineServiceIdentifier that = (TimelineServiceIdentifier) o;
- if (this.hostAddr != null && that.hostAddr != null) {
- return isMetadataEnabled == that.isMetadataEnabled &&
isEarlyConflictDetectionEnable == that.isEarlyConflictDetectionEnable
- && hostAddr.equals(that.hostAddr) && markerType == that.markerType;
- } else {
Review Comment:
can you explain a bit more about why the if else branch is removed.
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/OptionsResolver.java:
##########
@@ -258,6 +258,24 @@ public static boolean
isSimpleBucketIndexType(Configuration conf) {
return isBucketIndexType(conf) &&
getBucketEngineType(conf).equals(HoodieIndex.BucketIndexEngineType.SIMPLE);
}
+ /**
+ * Returns whether the simple bucket index should use remote partitioner.
+ */
+ public static boolean enableBucketRemotePartitioner(Configuration conf) {
+ return isSimpleBucketIndexType(conf)
+ && isBucketRemotePartitionerEnabled(conf)
+ && Boolean.parseBoolean(conf.getString(
+ HoodieWriteConfig.EMBEDDED_TIMELINE_SERVER_ENABLE.key(),
Review Comment:
The check for `EMBEDDED_TIMELINE_SERVER_ENABLE` is not needed, since
embedded timeline server is always started in the writer coordinator and always
disabled on client side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
