sungwy commented on issue #15900: URL: https://github.com/apache/iceberg/issues/15900#issuecomment-4195806676
Hi @tomer90 - thanks for raising this issue. I took a look at the second code reference you shared and I suspect one of the two are true about the invariant for partition source IDs to be a part of equality fields for upserts: 1. The invariant is only needed for **hash distribution mode**. Then the implementation is wrong and should gate this check on the actual distribution mode like you suggest. 2. The invariant is correct **independent of distribution mode**. Then the error message is simply wrong and should be updated. We may benefit from pulling in Flink Iceberg experts here, but based on the [Javadocs](https://iceberg.apache.org/javadoc/1.9.2/org/apache/iceberg/flink/sink/FlinkSink.Builder.html#upsert(boolean)) I think the second conclusion is likely to be true. Let me know if the explanation in the javadocs aligns with your understanding of how upserts should work as well -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
