github-actions[bot] commented on code in PR #68610:
URL: https://github.com/apache/doris/pull/68610#discussion_r4227165026
##########
be/src/exec/scan/scanner.cpp:
##########
@@ -101,6 +101,16 @@ Status Scanner::get_block_after_projects(RuntimeState*
state, Block* block, bool
RETURN_IF_ERROR(get_block(state, &_origin_block, eos));
return _do_projections(&_origin_block, block);
}
+ // get_block() charges the shared LIMIT as soon as rows pass the
filters, but rows that stay
+ // in _padding_block when this call returns are not returned by it.
Peer scanners may
+ // exhaust the LIMIT before this scanner runs again, and the context
then finishes without
+ // running it, so those rows would be charged but never returned. Keep
the counter equal
+ // to "LIMIT minus returned rows" between calls: charge the held rows
again on entry, so
+ // every row this scanner holds during the call is charged and the
read loop below stops
+ // as soon as the LIMIT is reached, and refund whatever is still held
on exit.
+ if (_shared_scan_limit && !_padding_block.empty()) {
+ _shared_scan_limit->fetch_sub(_padding_block.rows(),
std::memory_order_acq_rel);
+ }
const auto min_batch_size = std::max(state->batch_size() / 2, 1);
Review Comment:
[P2] Keep the shared LIMIT pointer when padding recharge makes the counter
negative. `Scanner::init()` caches `shared_scan_limit_ptr()` once, but that
accessor returns null whenever the live counter is negative. With 8 rows
remaining, A can recharge 2 held rows, read 4, and B can charge 3, briefly
making the counter -1; a parallel local state opening then caches null before A
refunds 2 and restores the counter to 1. That scanner never checks or charges
the shared LIMIT, so it may read a long selective tail after peers satisfy the
query. Determine pointer presence from the immutable SQL LIMIT instead of the
mutable counter, and cover a late scanner initialization during this
negative-to-positive transition.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
