github-actions[bot] commented on code in PR #66602:
URL: https://github.com/apache/doris/pull/66602#discussion_r3759518004
##########
be/src/exec/scan/file_scanner_v2.cpp:
##########
@@ -584,12 +648,34 @@ Status FileScannerV2::_prepare_next_split(bool* eos) {
// get_block() follows the symmetric branch in _get_block_impl().
Both paths must
// advance exactly one scan range and preserve later files in the
same scan.
RETURN_IF_ERROR(_table_reader->abort_split());
+ if (is_file_parent) {
+ RETURN_IF_ERROR(finish_file_parent({}));
+ }
COUNTER_UPDATE(_empty_file_counter, 1);
_state->update_num_finished_scan_range(1);
continue;
}
- RETURN_IF_ERROR(status);
+ if (!status.ok()) {
+ if (is_file_parent) {
+ RETURN_IF_ERROR(finish_file_parent({}));
+ }
+ return status;
+ }
if (_table_reader->current_split_pruned()) {
+ if (is_file_parent) {
+ RETURN_IF_ERROR(finish_file_parent({}));
+ }
+ _state->update_num_finished_scan_range(1);
+ continue;
+ }
+ if (is_file_parent) {
+ std::vector<FileScanSplitTask> children;
+ const auto build_status =
_table_reader->build_file_split_tasks(&children);
+ if (!build_status.ok()) {
+ RETURN_IF_ERROR(finish_file_parent({}));
+ return build_status;
+ }
+ RETURN_IF_ERROR(finish_file_parent(std::move(children)));
_state->update_num_finished_scan_range(1);
Review Comment:
This marks the original FE parent range finished, but every generated Row
Group child later reaches the ordinary EOF path at line 560 and increments the
same counter again. The FE progress denominator still contains only the
original planned range (`scanRangeNum`), and `ProgressManager` does not clamp
`finished / total`; an insert-select from one N-Row-Group Iceberg/Paimon file
can therefore report N+1 finished ranges against a total of one. Please make
generated children non-reporting (or count the parent only after all of its
children finish) and cover one parent with multiple children through the
fragment progress report.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
