Hi everyone, Long time ago, Anton introduced *FileWriterFactory* [1] as a replacement for *FileAppenderFactory* for creating Iceberg data and delete files. While Spark has been migrated to use the new factory, both Flink and the Kafka connector still rely on *FileAppenderFactory*.
As part of the ongoing discussions [2] around the File Format API proposal, the topic of these factories came up [3]. The current plan is to migrate only *FileWriterFactory* to the new API. To support this, the suggestion is to deprecate *FileAppenderFactory* and migrate Flink and Kafka to use FileWriterFactory. I’ve opened a PR to migrate Flink to use *FlinkFileWriterFactory* [4] The change is relatively straightforward, and we can create a similar PR for Kafka as well. *Questions for the community:* - Are you aware of any users currently relying on *FileAppenderFactory*? - Are there any blockers preventing them from switching to *FileWriterFactory*? *Deprecation timeline:* If we decide to deprecate *FileAppenderFactory*, we have a couple of options: - Deprecate in 1.11.0 and remove in *1.12.0* - Or keep it until *2.0* What are your thoughts on the timeline? We’ll also discuss this topic during the next *File Format API sync *on* Monday, October 13 at 9:00 AM PST*. Join the meeting here: https://meet.google.com/fow-bauj-mmg Looking forward to your feedback! Thanks, Peter [1] - PR: Core: Add WriterFactory - https://github.com/apache/iceberg/pull/2873 [2] - Dev list discussion - https://lists.apache.org/thread/ovyh52m2b6c1hrg4fhw3rx92bzr793n2 [3] - Iceberg DataFile reader and writer API proposal document / Open questions - https://docs.google.com/document/d/1sF_d4tFxJsZWsZFCyCL9ZE7YuI7-P3VrzMLIrrTIxds/edit?pli=1&tab=t.0#heading=h.fntuqbcl74w1 [4] - PR: Flink: Move write from AppenderFactory to FileWriterFactory - https://github.com/apache/iceberg/pull/14271
