Gabriel39 commented on code in PR #65965:
URL: https://github.com/apache/doris/pull/65965#discussion_r3639546373
##########
be/src/format_v2/parquet/parquet_scan.cpp:
##########
@@ -1875,15 +1986,45 @@ Status
ParquetScanScheduler::read_filter_columns(int64_t batch_rows,
};
RETURN_IF_ERROR(read_round_by_round());
- // Single-column expressions only touch the just-read column, so earlier
columns can retain
- // their own row mappings. Compact only when a later expression needs a
shared coordinate
- // space; otherwise the final boundary can discard hidden predicate
payloads without scanning
- // them again.
- if (!schedule.remaining_conjuncts.empty()) {
+ if (*selected_rows == 0) {
+ RETURN_IF_ERROR(skip_unmaterialized_predicate_columns());
+ return compact_predicate_columns_with_profile(true);
+ }
+
+ // Complex residuals keep their original conjunct order. Materialize only
the columns needed
+ // by the next reachable expression, then compact previously read columns
into the same row
+ // space before evaluating it. This is the scanner-side equivalent of
expression-triggered
+ // lazy columns: a conjunct that rejects the batch prevents later-only
columns from decoding.
+ for (const auto& stage : schedule.remaining_stages) {
Review Comment:
Fixed in 8eb8cdb0d88. Alignment now tracks whether SelectionVector actually
changed and skips prior-column walks after all-pass stages. Added
PredicateAlignmentColumns plus AllPassResidualChainAlignsEachColumnOnce; a
three-stage/six-column all-pass chain visits 6 mappings total instead of the
former 18.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
