Yuxin Tan created FLINK-32870: --------------------------------- Summary: Merge multiple small buffers into one large buffer when loading data for tiered storage Key: FLINK-32870 URL: https://issues.apache.org/jira/browse/FLINK-32870 Project: Flink Issue Type: Bug Components: Runtime / Network Affects Versions: 1.18.0 Reporter: Yuxin Tan
Currently, when the file reader of tiered storage loads data from the disk file, it reads data in buffer granularity. Before compression, each buffer is 32K by default, after compression the size will become smaller (may less than 5K), which is pretty small for the network buffer and the file IO. We should merge the multiple small buffers into a larger one to decrease the buffer competition and the file IO, leading to better performance. -- This message was sent by Atlassian Jira (v8.20.10#820010)