AshinGau opened a new pull request, #26923: URL: https://github.com/apache/doris/pull/26923
backport #26741 `MergeRangeFileReader` is used to merge small IOs, and `max_amplified_read_ratio` controls the proportion of read amplification. However, in some extreme cases(eg. `orc strip size`/`parquet row group size` is less than 3MB), the control effect of `max_amplified_read_ratio` is not good, resulting in a large amount of small IOs. After testing, the return time of a single IO for IO size smaller than 4kb in hdfs(512kb in oss) remains basically unchanged. Therefore, equivalent IO size is used to measure merge effectiveness: ``` EquivalentIOSize = MergeSize / Request IOs ``` When `EquivalentIOSize` is greater than 4kb in hdfs, or 512kb in oss, we believe that this kind of merge is effective. ## Proposed changes Issue Number: close #xxx <!--Describe your changes.--> ## Further comments If this is a relatively large or complex change, kick off the discussion at [[email protected]](mailto:[email protected]) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
