[GitHub] [hudi] yanghua commented on a change in pull request #3238: [HUDI-2143] Tweak the default compaction target IO to 500GB when flin…
yanghua commented on a change in pull request #3238:
URL: https://github.com/apache/hudi/pull/3238#discussion_r666787064
##
File path:
hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java
##
@@ -199,6 +199,12 @@ private static void setupCleaningOptions(Configuration
conf) {
conf.setInteger(FlinkOptions.ARCHIVE_MIN_COMMITS, commitsToRetain + 10);
conf.setInteger(FlinkOptions.ARCHIVE_MAX_COMMITS, commitsToRetain + 20);
}
+if (conf.getBoolean(FlinkOptions.COMPACTION_SCHEDULE_ENABLED)
Review comment:
The rewritten value is a constant value. Why not provide an
`SYNC_COMPACTION_TARGET_IO` option and make it to be `500GB`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] yanghua commented on a change in pull request #3238: [HUDI-2143] Tweak the default compaction target IO to 500GB when flin…
yanghua commented on a change in pull request #3238:
URL: https://github.com/apache/hudi/pull/3238#discussion_r666768715
##
File path:
hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java
##
@@ -199,6 +199,12 @@ private static void setupCleaningOptions(Configuration
conf) {
conf.setInteger(FlinkOptions.ARCHIVE_MIN_COMMITS, commitsToRetain + 10);
conf.setInteger(FlinkOptions.ARCHIVE_MAX_COMMITS, commitsToRetain + 20);
}
+if (conf.getBoolean(FlinkOptions.COMPACTION_SCHEDULE_ENABLED)
Review comment:
Yes, but the user may not know it would be rewritten under some
condition like this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] yanghua commented on a change in pull request #3238: [HUDI-2143] Tweak the default compaction target IO to 500GB when flin…
yanghua commented on a change in pull request #3238:
URL: https://github.com/apache/hudi/pull/3238#discussion_r10891
##
File path:
hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java
##
@@ -199,6 +199,12 @@ private static void setupCleaningOptions(Configuration
conf) {
conf.setInteger(FlinkOptions.ARCHIVE_MIN_COMMITS, commitsToRetain + 10);
conf.setInteger(FlinkOptions.ARCHIVE_MAX_COMMITS, commitsToRetain + 20);
}
+if (conf.getBoolean(FlinkOptions.COMPACTION_SCHEDULE_ENABLED)
Review comment:
IMO, @garyli1019 's opinion is reasonable. @danny0405 I know what's your
purpose. Can we define an explicit config option for this? So that users can
know this detailed information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hudi] yanghua commented on a change in pull request #3238: [HUDI-2143] Tweak the default compaction target IO to 500GB when flin…
yanghua commented on a change in pull request #3238:
URL: https://github.com/apache/hudi/pull/3238#discussion_r10891
##
File path:
hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableFactory.java
##
@@ -199,6 +199,12 @@ private static void setupCleaningOptions(Configuration
conf) {
conf.setInteger(FlinkOptions.ARCHIVE_MIN_COMMITS, commitsToRetain + 10);
conf.setInteger(FlinkOptions.ARCHIVE_MAX_COMMITS, commitsToRetain + 20);
}
+if (conf.getBoolean(FlinkOptions.COMPACTION_SCHEDULE_ENABLED)
Review comment:
IMO, @garyli1019 's opinion is reasonable. @danny0405 I know what's your
purpose. Can we define an explicit config option for this? So that users can
know this detail information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
