github-actions[bot] commented on code in PR #65498:
URL: https://github.com/apache/doris/pull/65498#discussion_r3566445402
##########
be/src/format_v2/table_reader.h:
##########
@@ -142,9 +142,18 @@ struct TableReadOptions {
struct SplitReadOptions {
// Split-level information for reader initialization, which may include
file path, partition values, delete file info, etc. The content is table format
specific and opaque to table reader base class; it's the responsibility of the
concrete table reader implementation to parse necessary information for reader
initialization and filter pushdown.
std::map<std::string, Field> partition_values;
- // Latest scanner conjuncts rewritten to table/global column indices.
Runtime filters can
- // arrive after TableReader::init(), so split preparation must receive a
fresh snapshot.
+ // Latest scanner conjuncts rewritten to table/global column indices.
Runtime filters may
+ // arrive after TableReader::init(), so scanner-driven splits replace the
initial snapshot.
+ // nullopt preserves the initial snapshot for standalone TableReader
callers.
+ std::optional<VExprContextSPtrs> conjuncts;
+ // Independent clones used for partition pruning because evaluation
prepares and opens them
+ // against a synthetic partition block before the file reader opens its
row-level conjuncts.
VExprContextSPtrs partition_prune_conjuncts;
+ // Table-level COUNT may emit one metadata-derived batch and resume on a
later scheduler turn.
+ // It is safe only after every runtime filter assigned to the scanner has
arrived; otherwise a
+ // filter could arrive after synthetic rows have already been returned and
those rows cannot be
+ // retracted. Standalone TableReader callers have no scanner
runtime-filter lifecycle.
+ bool all_runtime_filters_applied = true;
Review Comment:
The pending-runtime-filter guard here only disables the table-level
row-count shortcut in `prepare_split()`. When `all_runtime_filters_applied` is
false and no RF has arrived yet, `_conjuncts` is still empty, so the split
falls through to the normal reader path. On the first `get_block()`,
`_try_materialize_aggregate_pushdown_rows()` can still call
`get_aggregate_result()` because `_supports_aggregate_pushdown()` has no
pending-RF check and allows `COUNT` and `MINMAX` when `_table_filters` is
empty. Parquet/ORC can then emit metadata-derived COUNT or MIN/MAX rows, and
text can emit COUNT rows, before closing the reader; if the RF arrives on a
later scheduler turn, those synthetic rows can no longer be evaluated against
real file rows. Please carry this pending-RF state into the aggregate-pushdown
check, or otherwise make aggregate pushdown ineligible until every scanner RF
has arrived, and add coverage for `all_runtime_filters_applied = false` with
aggregate-capable COUNT/MINMAX r
eaders.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
