rainerschamm commented on PR #14797: URL: https://github.com/apache/iceberg/pull/14797#issuecomment-4135495858
Hi all, I have been trying to follow the message thread and looked at some of the other PR's mentioned. They seem like valid concerns in their own right but relate at best tangentially to the original purpose of the PR. The original PR introduces at its core: > The BaseDeltaWriter (in very similar way to Flinks BaseDeltaTaskWriter) > And its two derived classes PartitionedDeltaWriter and UnpartitionedDeltaWriter > Some config changes and some basic utility changes Unless I missed something it seems to me that none of the concerns (in the Coordinator/Commit buffer) are related to any of this code; and should be addressed on their own, since they probably cause similar issues with the current append writer. We have been testing both versions of the delta writer, partitioned and partitioned with both v2 and v3 (e.g. with DV enabled) formats for weeks now with no issues. I am just worried that we are conflating issues a bit here. Let's find any remaining problems with the code in the PR and then merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
