LuciferYang opened a new issue, #10611: URL: https://github.com/apache/paimon/issues/10611
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version master (5d019f5a6d) ### Compute Engine Flink ### Minimal reproduce step Run a batch compaction with `partition_idle_time`, for example `CALL sys.compact(`table` => 'db.T', partition_idle_time => '1 d')`, on a partitioned table that keeps receiving writes, and let a new partition get its first files after the compaction job is submitted but before it starts scanning. ### What doesn't meet your expectations? The compaction job fails with a `NullPointerException` in the idle filter (`AppendTableCompact` for unaware-bucket tables, `CompactorSourceBuilder` for bucketed tables). The filter compares each planned partition with a map of partition → last file creation time. The map is read when the job is built, on the client, while the partitions to compact are planned when the job runs. A partition first written in between has no entry in the map, and `partitionInfo.get(partition) <= historyMilli` unboxes null. Restarting the job does not help, because the map is part of the job graph. The job has to be submitted again. ### Anything else? _No response_ ### Are you willing to submit a PR? - [X] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
