github-actions[bot] commented on code in PR #65548:
URL: https://github.com/apache/doris/pull/65548#discussion_r3584900800
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/FileScanNode.java:
##########
@@ -88,6 +88,8 @@ public FileScanNode(PlanNodeId id, TupleDescriptor desc,
String planNodeName,
@Override
protected void toThrift(TPlanNode planNode) {
planNode.setPushDownAggTypeOpt(pushDownAggNoGroupingOp);
+ planNode.setPushDownCountSlotIds(
+ pushDownCountSlotIds.stream().map(id ->
id.asInt()).collect(Collectors.toList()));
planNode.setNodeType(TPlanNodeType.FILE_SCAN_NODE);
Review Comment:
This still leaves a pre-BE table-format COUNT path using the raw `COUNT`
flag. For `COUNT(required_col)`, the planner now serializes a non-empty
`push_down_count_slot_ids`, and BE correctly falls back instead of consuming
table-level row counts. But Iceberg and Paimon decide their table-level count
split planning before BE sees this field:
`IcebergScanNode.isBatchMode()`/`doGetSplits()` and `PaimonScanNode` check only
`getPushDownAggNoGroupingOp() == COUNT`, then keep just one or `parallel *
numBackends` count splits and assign the total row count to them. Since BE
ignores that assigned count for non-empty arguments, it scans only the retained
subset of real splits, so `COUNT(non_null_col)` can undercount any table with
more data splits than the retained representatives. Please gate the
Iceberg/Paimon table-level count split reduction on the explicit COUNT(*)
marker, or otherwise avoid dropping real splits whenever `pushDownCountSlotIds`
is non-empty.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
