pvary commented on code in PR #10526:
URL: https://github.com/apache/iceberg/pull/10526#discussion_r1647501589
##########
flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergFilesCommitter.java:
##########
@@ -448,8 +452,8 @@ private byte[] writeToManifest(long checkpointId) throws
IOException {
if (writeResultsOfCurrentCkpt.isEmpty()) {
return EMPTY_MANIFEST_DATA;
}
-
- WriteResult result =
WriteResult.builder().addAll(writeResultsOfCurrentCkpt).build();
+ List<WriteResult> writeResults =
writeResultsOfCurrentCkpt.get(checkpointId);
Review Comment:
What happens with the `WriteResult` of the previous checkpoints?
The situation which lead you this case would cause data loss. The data from
the first (unfinished) checkpoint needs to be committed as well, so if some
rows are updated only in that given checkpoint it is stored here as well.
We should write a test which covers this scenario
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
