Hi all, A question regarding OverwriteFiles API in comparison to AppendFiles. I noticed that AppendFiles allows for adding manifest files, but OverwriteFiles does not. Is this intentional?
The use case that we have is distributed manifest file writes as part of a table update. We'd like to be able to add these manifest files, along with removing some set of data files that have been overwritten by data files contained in those manifests, as part of a single snapshot. My understanding is that the current API supports either of the following: - Don't do distributed manifest file writes - add data files directly via the OverwriteFiles operation - Do distrubuted manifest file writes, use AppendFiles op to add the manifest files, then use DeleteFiles to remove any data files. But we found that each operation generates another snapshot even if used in the same transaction which is undesirable. Thoughts on the best way to enable this scenario? Would extending OverwriteFiles to support adding manifest files be a workable solution? Thanks Scott
