Guosmilesmile commented on code in PR #15566:
URL: https://github.com/apache/iceberg/pull/15566#discussion_r2909377891
##########
flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java:
##########
@@ -682,6 +698,14 @@ IcebergSink build() {
FlinkMaintenanceConfig flinkMaintenanceConfig =
new FlinkMaintenanceConfig(table, writeOptions, readableConfig);
+ // If the "compactMode" option is configured,
+ // add it as a separate maintenance task
+ if (flinkWriteConf.compactMode()) {
+ RewriteDataFilesConfig rewriteDataFilesConfig =
+ flinkMaintenanceConfig.createRewriteDataFilesConfig();
+
maintenanceTasks.add(RewriteDataFiles.builder().config(rewriteDataFilesConfig));
+ }
Review Comment:
Out of curiosity, is the scenario we’re trying to solve here to avoid having
to maintain a separate maintenance job per table Or what scenario is this
intended to address?
From my perspective, I have a few (still rough) questions would like to ask:
1. Do we allow multiple ways to configure the same maintenance task, and
also allow multiple instances of the same type of task? For example, conflicts
between the `compactMode` option and the `maintenance()` method, or adding
multiple rewrite-data tasks with different configurations via `maintenance()`.
2. Regarding config injection, we can inject configs via the existing
`flinkConf` approach, or bypass `flinkConf` and have `maintenance()` generate
task configs directly. That could get confusing—can we unify this behind a
single entry point?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
