danny0405 commented on issue #19428: URL: https://github.com/apache/hudi/issues/19428#issuecomment-5173375635
Spark `DROP PARTITION` has the purge syntax. As of Apache Hudi 1.2.0 and current `master` (August 2026), Spark SQL `PURGE` does **not** immediately delete a Hudi partition’s physical files. ```sql ALTER TABLE hudi_table DROP PARTITION (dt = '2026-08-01') PURGE; ``` Spark accepts the `[PURGE]` syntax, but Hudi’s implementation receives the `purge` flag and never uses it. Instead, it records a `DELETE_PARTITION` operation and explicitly relies on **lazy cleaning**. The partition disappears from the current Hudi snapshot immediately, while its files remain until the cleaner removes them. See the [[current Hudi command implementation](https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableDropPartitionCommand.scala)](https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/command/AlterHoodieTableDropPartitionCommand.scala), [[Hudi SQL DDL](https://hudi.apache.org/docs/sql_ddl/)](https://hudi.apache.org/docs/sql_ddl/), and [[Spark’s generic syntax](https://spark.apache.org/docs/4.0.0/sql-ref-syntax-ddl-alter-table.html)](https://spark.apache.org/docs/4.0.0/sql-ref-syntax-ddl-alter-tab le.html). To reclaim storage promptly, run cleaning explicitly—for example: ```sql CALL run_clean( table => 'database.hudi_table', retain_commits => 1 ); ``` Lowering retention reduces rollback/time-travel protection, so use it carefully. Don’t manually remove the partition directory, because that can leave Hudi’s timeline and metadata inconsistent. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
