JingsongLi commented on PR #10464: URL: https://github.com/apache/paimon/pull/10464#issuecomment-6079984233
[P1] Complete DATE expiration through the real catalog's `.done` cleanup The DATE normalization is useful, but the newly enabled `values-time` path still fails when `metastore.partitioned-table=true` and the sole/final partition key is DATE. After the actual expired partition is removed, `NormalPartitionExpire.doBatchExpire` unconditionally calls `dropPartitions` again with a value such as `2023-01-01.done`. `HiveCatalog.dropPartitions` ignores a missing HMS partition but still calls `super.dropPartitions`; that reaches `FileStoreCommitImpl.dropPartitions` and tries to cast the synthetic value to DATE, throwing `DateTimeException`. This propagates into commit maintenance, so the newly supported DATE expiration can fail the writer after deletion. The new test helper at [PartitionExpireTest.java:141-143](https://github.com/apache/paimon/blob/243bfc595ba1522f18b9696b501904d3475e0ade/paimon-core/src/test/java/org/apache/paimon/operation/PartitionExpireTest.java#L141-L143) skips nonexistent `.done` specs, which differs from the real catalog and masks the failure. I ran the original 29 focused tests successfully on JDK 8, then replaced only that helper guard with unconditional delegation to the real `FileStoreCommit`. Running `PartitionExpireTest#testExpireDatePartition` gives 4 cases / 2 errors: both metastore-enabled cases fail with `For input string: '2023-01-01.done'`, while both non-metastore cases pass. The update-time failure corroborates the shared cleanup defect; the newly enabled values-time failure is the regression relevant to this PR. Please make marker cleanup avoid casting unsupported synthetic DATE values (for example, metadata-only marker removal or skipping marker specs that cannot exist), and cover the real catalog delegation behavior instead of suppressing it in the test double. Reviewed head: `243bfc595ba1522f18b9696b501904d3475e0ade`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
