github-actions[bot] commented on code in PR #61518:
URL: https://github.com/apache/doris/pull/61518#discussion_r3409185254
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/source/IcebergScanNode.java:
##########
@@ -884,9 +886,11 @@ private Split createIcebergSplit(FileScanTask
fileScanTask) {
split.setPartitionDataJson(IcebergUtils.getPartitionDataJson(
partitionData, partitionSpec,
sessionVariable.getTimeZone()));
Map<String, String> partitionInfoMap =
partitionMapInfos.computeIfAbsent(
- partitionData, k ->
IcebergUtils.getIdentityPartitionInfoMap(
- partitionData, partitionSpec, icebergTable,
sessionVariable.getTimeZone()));
- if (!partitionInfoMap.isEmpty()) {
+ partitionData, k ->
IcebergUtils.getIdentityPartitionInfoMapForCache(
Review Comment:
This helper is now used to populate `partitionMapInfos`, but it returns
`null` for any non-identity Iceberg spec. `Map.computeIfAbsent` does not record
a mapping when the function returns `null`, so transform-partitioned tables,
for example `PARTITIONED BY day(ts)`, no longer add their `PartitionData` to
`partitionMapInfos`. Later `selectedPartitionNum = partitionMapInfos.size()`
and `numApproximateSplits()` use that same map, so those scans report zero
selected partitions and batch mode estimates only one split regardless of the
number of scanned partitions.
This is separate from whether the cache context should be omitted for
non-identity specs. Please keep partition accounting independent from the
cache-context map, or store a sentinel/empty map for non-cacheable partition
specs while only skipping `setIcebergPartitionValues`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
