github-actions[bot] commented on code in PR #66818:
URL: https://github.com/apache/doris/pull/66818#discussion_r3793817404
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonScanPlanProvider.java:
##########
@@ -664,6 +671,15 @@ private List<ConnectorScanRange> planScanInternal(
if (projected.length > 0) {
readBuilder.withProjection(projected);
}
+ if (limit > 0 && limit <= Integer.MAX_VALUE
Review Comment:
[P1] Require trustworthy output-row counts before pruning splits
This gate assumes Paimon's limit is measured in Doris output rows, but
pinned 1.3.1 has reachable mismatches:
- [FormatTableScan caps the number of
files](https://github.com/apache/paimon/blob/release-1.3.1/paimon-core/src/main/java/org/apache/paimon/table/format/FormatTableScan.java#L153-L159),
so an empty first CSV/JSON file followed by a nonempty file makes LIMIT 1 plan
only the empty file and return zero rows.
- Even an ordinary primary-key table can have legacy metadata where
[`rowCount` includes adds and deletes but `deleteRowCount` is
absent](https://github.com/apache/paimon/blob/release-1.3.1/paimon-core/src/main/java/org/apache/paimon/io/PojoDataFileMeta.java#L45-L66).
With the supported `force_jni_scanner=true` path, the split generator [treats
that absence as
raw-convertible](https://github.com/apache/paimon/blob/release-1.3.1/paimon-core/src/main/java/org/apache/paimon/table/source/MergeTreeSplitGenerator.java#L108-L153),
so planning counts tombstones, while the execution reader [rejects the raw
path because deletes are
unknown](https://github.com/apache/paimon/blob/release-1.3.1/paimon-core/src/main/java/org/apache/paimon/table/source/splitread/PrimaryKeyTableRawFileSplitReadProvider.java#L35-L49)
and the merge reader drops them. A tombstone-only first split can satisfy
LIMIT 1, emit no row, and prune a later live split.
A `FileStoreTable` check is therefore insufficient. Apply this only when
split accounting is proven to equal output cardinality (or disable it for
unsafe table/metadata modes), and add result-bearing format and
legacy-tombstone regressions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
