[ https://issues.apache.org/jira/browse/FLINK-33879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jiang Xin updated FLINK-33879: ------------------------------ Description: Currently, the Hybrid Shuffle can work with Memory Tier and Disk Tier together, however, in the following scenirio the result partition would stop working. Suppose we have a shuffle task with 2 sub partitions. The LocalBufferPool has 15 buffers, the memory tier can use at most 15-(2*(2+1)+1) = 8 buffers. If the memory tier used up all 8 buffers and the input channel doesn't consume them because of some problem, the disk tier can still work with 1 reserved buffer. However, if a redistribution happens now and the pool size is decrease to less than 8, then the BufferAccumulator can not request buffers any more, so was:Currently, the Hybrid Shuffle can work with Memory Tier and Disk Tier together, if the memory tier however, in the following scenirio the result partition would stop > Hybrid Shuffle may hang during redistribution > --------------------------------------------- > > Key: FLINK-33879 > URL: https://issues.apache.org/jira/browse/FLINK-33879 > Project: Flink > Issue Type: Bug > Components: Runtime / Network > Reporter: Jiang Xin > Priority: Major > Labels: pull-request-available > Fix For: 1.19.0 > > > Currently, the Hybrid Shuffle can work with Memory Tier and Disk Tier > together, however, in the following scenirio the result partition would stop > working. > Suppose we have a shuffle task with 2 sub partitions. The LocalBufferPool has > 15 buffers, the memory tier can use at most 15-(2*(2+1)+1) = 8 buffers. If > the memory tier used up all 8 buffers and the input channel doesn't consume > them because of some problem, the disk tier can still work with 1 reserved > buffer. However, if a redistribution happens now and the pool size is > decrease to less than 8, then the BufferAccumulator can not request buffers > any more, so -- This message was sent by Atlassian Jira (v8.20.10#820010)