On Flink job re-submission, observing error, "the rpc invocation size exceeds the maximum akka framesize"

2020-09-18 Thread shravan
Hi, This is in continuation to an already raised request, (had replied to the same thread but couldn't get any response yet, hence posting a new request) http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/The-rpc-invocation-size-exceeds-the-maximum-akka-framesize-when-the-job-was-

Re: On Flink job re-submission, observing error, "the rpc invocation size exceeds the maximum akka framesize"

2020-09-18 Thread Chesnay Schepler
how can we know the expected size for which it is failing? If you did not configure akka.framesize yourself then it is set to the documented default value. See the configuration documentation for the release you are using. > Does the operator state have any impact on the expected Akka frame

Re: On Flink job re-submission, observing error, "the rpc invocation size exceeds the maximum akka framesize"

2020-09-18 Thread shravan
Thanks for the quick response. I might have wrongly phrased one of the questions. /"> how can we know the expected size for which it is failing? If you did not configure akka.framesize yourself then it is set to the documented default value. See the configuration documentation for the release y

Re: On Flink job re-submission, observing error, "the rpc invocation size exceeds the maximum akka framesize"

2020-09-18 Thread Chesnay Schepler
If you use 1.10.0 or above the framesize for which it failed is part of the exception message, see FLINK-14618. If you are using older version, then I'm afraid there is no way to tell. On 9/18/2020 12:11 PM, shravan wrote: Thanks for the quick response. I might have wrongly phrased one of the

Re: On Flink job re-submission, observing error, "the rpc invocation size exceeds the maximum akka framesize"

2020-09-18 Thread shravan
Thanks again for the quick response. In that case, could you tell me what are the possible factors that warrant a framesize increase? I see the official documentation and it simply states "If Flink fails because messages exceed this limit, then you should increase it", which isn't very convincing.

Re: On Flink job re-submission, observing error, "the rpc invocation size exceeds the maximum akka framesize"

2020-09-18 Thread Chesnay Schepler
There are quite a few reason why the framesize could be exceeded. The most common one we see is due to the parallelism being so high that tasks can't be deployed in the first place. When a task is deployed the RPC payload also contains information about all downstream tasks this task sends dat