cshuo commented on code in PR #20776:
URL: https://github.com/apache/hudi/pull/20776#discussion_r4640171314
##########
hudi-examples/hudi-examples-flink/src/main/java/org/apache/hudi/examples/quickstart/HoodieFlinkQuickstart.java:
##########
@@ -114,6 +114,9 @@ public void initEnv() {
// configure not to retry after failure
execConf.setString("restart-strategy", "fixed-delay");
execConf.setString("restart-strategy.fixed-delay.attempts", "0");
+ // Flink 2.3 rejects an INSERT whose upsert key differs from the primary
key of the sink table
+ // unless it has an ON CONFLICT clause (FLINK-38927), restore the
behavior of earlier versions.
+ execConf.setString("table.exec.sink.require-on-conflict", "false");
Review Comment:
[HoodieFlinkQuickstart.java:119](https://github.com/apache/hudi/blob/0f2874fed38d9da44659635993059d75fc7fbb11/hudi-examples/hudi-examples-flink/src/main/java/org/apache/hudi/examples/quickstart/HoodieFlinkQuickstart.java#L119)
·
[HoodieTableSink.java:148–153](https://github.com/apache/hudi/blob/0f2874fed38d9da44659635993059d75fc7fbb11/hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSink.java#L148-L153)
I’d prefer the default UPSERT branch to preserve the row kinds in
`requestedMode` and filter out only `UPDATE_BEFORE`:
```java
ChangelogMode.Builder builder = ChangelogMode.newBuilder();
for (RowKind kind : requestedMode.getContainedKinds()) {
if (kind != RowKind.UPDATE_BEFORE) {
builder.addContainedKind(kind);
}
}
return builder.build();
```
With the current change, an insert-only streaming job that worked on 2.2 can
fail at planning time on 2.3 when Flink cannot infer an upsert key for the
keyed Hudi sink. Users must explicitly set
`table.exec.sink.require-on-conflict=false` to keep the same SQL. Filtering the
requested kinds keeps this input insert-only, so the job runs with default
settings and avoids the extra `SinkMaterializer` on 2.3.0. Flink 2.4 also
changed `require-on-conflict` to default to `false` to restore compatibility.
[Upstream change](https://github.com/apache/flink/pull/29935)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]