[ https://issues.apache.org/jira/browse/FLINK-34430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17818838#comment-17818838 ]
Piotr Nowojski commented on FLINK-34430: ---------------------------------------- [~roman] is already working on that under FLINK-26050 ticket. However the akka frame size is still an issue. The 10MB limit is dangerously small, especially when the {{ByteStreamHandle}} size limit is bumped to 1MB (which is a pretty reasonable value all things consider). In that case the limit would exceeded with only 10+ files. > Akka frame size exceeded with many ByteStreamStateHandle being used > ------------------------------------------------------------------- > > Key: FLINK-34430 > URL: https://issues.apache.org/jira/browse/FLINK-34430 > Project: Flink > Issue Type: Bug > Components: Runtime / Coordination > Affects Versions: 1.16.3, 1.17.2, 1.19.0, 1.18.1 > Reporter: Piotr Nowojski > Assignee: Chesnay Schepler > Priority: Major > > The following error can happen > {noformat} > Discarding oversized payload sent to > Actor[akka.tcp://flink@xxxx/user/rpc/taskmanager_0#-yyyy]: max allowed size > 10485760 bytes, actual size of encoded class > org.apache.flink.runtime.rpc.messages.RemoteRpcInvocation was 11212046 bytes. > > error.stack_trace > akka.remote.OversizedPayloadException: Discarding oversized payload sent to > Actor[akka.tcp://flink@xxxx/user/rpc/taskmanager_0#-yyyy]: max allowed size > 10485760 bytes, actual size of encoded class > org.apache.flink.runtime.rpc.messages.RemoteRpcInvocation was 11212046 bytes. > {noformat} > when https://issues.apache.org/jira/browse/FLINK-26050 is causing large > amount of small sst files to be created and never deleted. If those files are > small enough to be handled by {{ByteStreamStateHandle}} akka frame size can > be exceeded. -- This message was sent by Atlassian Jira (v8.20.10#820010)