MonkeyCanCode commented on issue #11820: URL: https://github.com/apache/iceberg/issues/11820#issuecomment-2568763829
@lordk911 that is expected. Drop table only remove metadata refs and not the actual data files since 0.14. For actual data file removal, you will need to add `purge` at the end. This is documented in https://iceberg.apache.org/docs/latest/spark-ddl/#drop-table. You can find the same from the code as well: Entry point for drop table in spark: https://github.com/apache/iceberg/blob/main/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkSessionCatalog.java#L284 Then it goes to https://github.com/apache/iceberg/blob/ab6365d42d6c3bfa7054ecd98dde8407e5ba1201/spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java#L340 which will tell us the following: ``` @Override public boolean dropTable(Identifier ident) { return dropTableWithoutPurging(ident); } ``` and the content of `dropTableWithoutPurging` is following which also shows by default `drop table xxx` in spark won't remove data files but only metadata refs: ``` private boolean dropTableWithoutPurging(Identifier ident) { if (isPathIdentifier(ident)) { return tables.dropTable(((PathIdentifier) ident).location(), false /* don't purge data */); } else { return icebergCatalog.dropTable(buildIdentifier(ident), false /* don't purge data */); } } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
