JingsongLi commented on code in PR #10662:
URL: https://github.com/apache/paimon/pull/10662#discussion_r4346233346
##########
paimon-core/src/main/java/org/apache/paimon/operation/BucketSelector.java:
##########
@@ -136,7 +160,9 @@ private Optional<PartitionSelector>
createPartitionSelector(Predicate bucketPred
@SuppressWarnings("unchecked")
List<Object>[] bucketValues = new List[bucketKeyType.getFieldCount()];
- BucketFunction bucketFunction =
BucketFunction.create(bucketFunctionType, bucketKeyType);
+ // Factory discovery is cached by ClassLoader; functions need not be
safe to share
+ // across concurrent partition selectors.
+ BucketFunction bucketFunction = BucketFunction.create(coreOptions,
bucketKeyType);
Review Comment:
[P2] Keep the provider function confined during bucket-cache population
The new factory contract explicitly allows functions that are not thread
safe, but each PartitionSelector retains this single function while its
ConcurrentHashMap.computeIfAbsent can build sets for different numBuckets
concurrently. A valid sequence of withOverwrite commits at bucket counts 2 -> 3
-> 4 leaves old DELETE manifests; a filtered real-table scan processes their
entries in parallel and invokes the same function instance concurrently for
counts 4 and 3. My overlap-detecting deterministic provider fails that scan,
while scan.manifest.parallelism=1 and a synchronized-provider control both read
the expected row. Create a separate function for each bucket-count computation
or serialize access to the retained function, and retain this overwrite-history
regression test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
