Hi Folks, I've been working on fixing the behavior of PURGE_VIEW_METADATA_ON_DROP, which is defined here [1].
My understanding is that when this configuration is set to true, dropping a view should also clean up its associated metadata. I'm trying to implement/fix this behavior as part of [2]. The challenge is that ViewMetadata in Iceberg differs from TableMetadata. Unlike tables, view metadata does not maintain a list of previous metadata JSON files; it only tracks the current metadata file. As a result, during a view drop operation we can only reliably delete the current metadata file. This is also the behavior followed by Iceberg itself today [3]. To move forward, I see a few possible options: 1. Match Iceberg's current behavior Delete only the current metadata file. If Iceberg later adds tracking of previous metadata files for views (similar to tables), we can align our implementation accordingly. 2. Do nothing Leave the current behavior unchanged, effectively making PURGE_VIEW_METADATA_ON_DROP a no-op for views. 3. Use a heuristic approach Derive the metadata directory from the current metadata JSON path and delete all metadata files within that directory. While this would work in most cases, it becomes unsafe if multiple views share the same metadata directory, as it could result in accidental data loss. Even if this is only a corner case, I don't think the risk is acceptable. 4. Alternative ideas Open to other suggestions that I may have missed. I'd appreciate any feedback or thoughts on the best approach here. -Ayush [1] https://github.com/apache/polaris/blob/958f483c24ac213b062104b7841bb714c51db187/polaris-core/src/main/java/org/apache/polaris/core/config/FeatureConfiguration.java#L352-L359 [2] https://github.com/apache/polaris/pull/4730 [3] https://github.com/apache/iceberg/blob/6976e020b894f6a6777704df2b8c4458cb291ae9/core/src/main/java/org/apache/iceberg/CatalogUtil.java#L149-L163
