github-actions[bot] commented on code in PR #65674:
URL: https://github.com/apache/doris/pull/65674#discussion_r3626890018
##########
be/src/format_v2/table_reader.h:
##########
@@ -602,7 +636,15 @@ class TableReader {
DORIS_CHECK(block->columns() > 0 || rows == 0);
for (size_t column_idx = 0; column_idx < block->columns();
++column_idx) {
auto column =
block->get_by_position(column_idx).type->create_column();
- column->resize(rows);
+ if (auto* nullable =
check_and_get_column<ColumnNullable>(*column)) {
+ // Metadata COUNT emits synthetic input rows for the unchanged
upper aggregate.
+ // They must be non-NULL for COUNT(nullable_col), and
constructing them explicitly
+ // also keeps every nullable null map boolean-valid in
debug/ASAN block checks.
Review Comment:
[P1] Bound synthetic COUNT materialization to the runtime batch
The newly enabled nullable `COUNT(col)` path passes the file's entire exact
`FileAggregateResult.count` here as `rows`. This branch then allocates that
many nested defaults plus a byte per row for the null map in one block,
although the unchanged upper COUNT immediately reduces those synthetic rows
again. A valid nullable BIGINT file with 500 million non-NULL rows can
therefore request roughly 4.5 GB before returning one scanner block. Keep a
remaining file-count state and emit at most one runtime batch per `get_block()`
call (as `_read_table_level_count()` already does), or disable this pushdown
until its result can be materialized boundedly; add a large synthetic-count
test that asserts the block cap.
##########
be/src/format_v2/table_reader.h:
##########
@@ -602,7 +636,15 @@ class TableReader {
DORIS_CHECK(block->columns() > 0 || rows == 0);
for (size_t column_idx = 0; column_idx < block->columns();
++column_idx) {
auto column =
block->get_by_position(column_idx).type->create_column();
- column->resize(rows);
+ if (auto* nullable =
check_and_get_column<ColumnNullable>(*column)) {
+ // Metadata COUNT emits synthetic input rows for the unchanged
upper aggregate.
+ // They must be non-NULL for COUNT(nullable_col), and
constructing them explicitly
Review Comment:
[P1] Bound synthetic COUNT materialization to the runtime batch
The newly enabled nullable `COUNT(col)` path passes the file's entire exact
`FileAggregateResult.count` here as `rows`. This branch then allocates that
many nested defaults plus a byte per row for the null map in one block,
although the unchanged upper COUNT immediately reduces those synthetic rows
again. A valid nullable BIGINT file with 500 million non-NULL rows can
therefore request roughly 4.5 GB before returning one scanner block. Keep a
remaining file-count state and emit at most one runtime batch per `get_block()`
call (as `_read_table_level_count()` already does), or disable this pushdown
until its result can be materialized boundedly; add a large synthetic-count
test that asserts the block cap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
