Tonylin1998 opened a new issue, #9956:
URL: https://github.com/apache/iceberg/issues/9956
### Query engine
spark
### Question
I was using Iceberg with PySpark, and using JDBC catalog, and set warehouse
to GCS
I creata a table using `date` as partition key.
I write some data into table, and decide to delete date=20240220, so I
```
spark.sql(f"DELETE FROM {iceberg_table} WHERE date = '20241220'")
spark.sql(f"CALL {catalog_name}.system.expire_snapshots('{iceberg_table}')")
```
I find that the parquet file under `date=20240220` is deleted, but the
folder `date=20240220` still remain
Also the same, when I drop the table using
```
spark.sql(f"DROP TABLE {iceberg_table} PURGE")
```
the data will be deleted, but all the partition folders will still remain
These behavior cause many empty folders in my gcs, I wonder if there is any
way I can do in iceberg to prevent this from happening?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
