[ https://issues.apache.org/jira/browse/FLINK-31610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706904#comment-17706904 ]
Piotr Nowojski commented on FLINK-31610: ---------------------------------------- {{LocalBufferPool}} size can change everytime a new subtask is starting/finishing on a TaskManager, so it's pretty common. Especially for session clusters, but not only. > Refactoring of LocalBufferPool > ------------------------------ > > Key: FLINK-31610 > URL: https://issues.apache.org/jira/browse/FLINK-31610 > Project: Flink > Issue Type: Improvement > Components: Runtime / Network > Affects Versions: 1.17.0 > Reporter: Anton Kalashnikov > Priority: Major > > FLINK-31293 bug highlighted the issue with the internal mutual consistency of > different fields in LocalBufferPool. ex.: > - `numberOfRequestedOverdraftMemorySegments` > - `numberOfRequestedMemorySegments` > - `availableMemorySegment` > - `currentPoolSize` > Most of the problem was fixed already(I hope) but it is a good idea to > reorganize the code in such a way that all invariants between all fields > inside will be clearly determined and difficult to break. > As one example I can propose getting rid of > numberOfRequestedOverdraftMemorySegments and using existing > numberOfRequestedMemorySegments instead. That means: > - the pool will be available when `!availableMemorySegments.isEmpty() && > unavailableSubpartitionsCount == 0` > - we don't request a new `ordinary` buffer when > `numberOfRequestedMemorySegments >= currentPoolSize` but we request the > overdraft buffer instead > - `setNumBuffers` should work automatically without any changes > I think we can come up with a couple of such improvements to simplify the > code. -- This message was sent by Atlassian Jira (v8.20.10#820010)