LuciferYang opened a new pull request, #10612:
URL: https://github.com/apache/paimon/pull/10612

   ### Purpose
   
   close #10611
   
   With `partition_idle_time`, batch compaction filters the planned partitions 
through a map of partition → last file creation time. The map is read when the 
job is built, while the partitions are planned when the job runs, so a 
partition first written in between has no entry and the filter fails with a 
`NullPointerException`. Restarting the job runs into the same map.
   
   Such a partition only has files written after the job was built, so it is 
not idle. Both filters now skip it instead of failing: `AppendTableCompact` for 
unaware-bucket tables and `CompactorSourceBuilder` for bucketed tables.
   
   ### Tests
   
   
`CompactActionITCase#testHistoryPartitionCompactWithPartitionAddedAfterBuild`, 
for a primary-key table and an unaware-bucket append table: builds the compact 
action on an idle partition, writes a new partition, then runs the job. The 
idle partition is compacted and the new one is left alone. On master both cases 
fail with the NPE.
   
   ### API and Format
   
   no
   
   ### Documentation
   
   no
   
   Generated-by: Claude Opus
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to