sollhui opened a new pull request, #67728: URL: https://github.com/apache/doris/pull/67728
### What problem does this PR solve? Backport #66847 to branch-4.1. For S3-backed cloud loads, a memtable flush can finish after submitting asynchronous uploads, so flush-running count does not represent downstream upload pressure. This change detects S3 storage and uses the S3 upload queue for backpressure, falling back to the existing flush-running limit when the upload pool is unavailable. HDFS retains flush-count backpressure. ### Branch-4.1 adaptation branch-4.1 does not contain WriteRequestType::GROUP or CloudGroupRowsetBuilder. The row-binlog GROUP-specific changes from the original PR are therefore omitted. CloudRowsetBuilder is final in this branch, so is_s3_storage is non-virtual. Local-storage flush limits remain unchanged. ### Release note Improve load backpressure for S3-backed cloud loads. Higher concurrency can increase memory use. ### Check List (For Author) - Test: Compilation and tests skipped as requested. clang-format 16 and git diff --check passed. - Behavior changed: Yes; S3-backed loads use upload queue pressure. HDFS and local-storage limits remain unchanged. - Does this need documentation: No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
