github-actions[bot] commented on code in PR #65782:
URL: https://github.com/apache/doris/pull/65782#discussion_r3608485291
##########
fe/fe-core/src/main/java/org/apache/doris/statistics/util/StatisticsUtil.java:
##########
@@ -605,9 +605,17 @@ public static Optional<ColumnStatistic>
getIcebergColumnStats(String colName, or
try (CloseableIterable<FileScanTask> fileScanTasks =
tableScan.planFiles()) {
for (FileScanTask task : fileScanTasks) {
int colId = getColId(task.spec(), colName);
- totalDataSize += task.file().columnSizes().get(colId);
+ Map<Integer, Long> columnSizes = task.file().columnSizes();
+ Map<Integer, Long> nullValueCounts =
task.file().nullValueCounts();
+ Long columnSize = columnSizes == null ? null :
columnSizes.get(colId);
+ Long nullValueCount = nullValueCounts == null ? null :
nullValueCounts.get(colId);
+ // Iceberg can omit maps or entries for mode=none; partial
aggregation would fabricate zero stats.
+ if (columnSize == null || nullValueCount == null) {
+ return Optional.empty();
Review Comment:
[P2] Preserve unknown stats when closing the scan fails
This return runs inside try-with-resources. If `CloseableIterable.close()`
throws `IOException`, Java abandons the pending `Optional.empty()` return,
enters the catch below, and then builds a statistic from the current
accumulators. For example, a complete file followed by a metrics-disabled file
will expose partial totals if close also fails; if the disabled file is first,
it exposes a fabricated zero statistic. Please return `Optional.empty()` from
the IOException catch (or otherwise preserve the unavailable state) and cover a
throwing-close iterable. This is distinct from the prior missing-map thread:
normal close is fixed here, while the close-error path still escapes partial
data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
