GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/5571
[FLINK-8759][network] preparations for the update of netty to version 4.0.56 ## What is the purpose of the change Based on the changes from #5570, this PR prepares a netty version bump by circumventing a bug we had with the change from Netty 4.0.27 to 4.0.28 with the old non-credit based flow control paths: Versions >= 4.0.28 contain an improvement by Netty, which slices a Netty buffer instead of doing a memory copy (https://github.com/netty/netty/issues/3704) in the `LengthFieldBasedFrameDecoder`. In some situations, this interacts badly with our Netty pipeline leading to `OutOfMemory` errors. With credit-based flow control this problem should not exist anymore which is why we can use the original netty code there. ## Brief change log - override `LengthFieldBasedFrameDecoder#extractFrame()` and implement two different code paths depending on whether credit based flow control is on or not ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): **no** - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **no** - The serializers: **no** - The runtime per-record code paths (performance sensitive): **no** (only per buffer) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: **no** - The S3 file system connector: **no** ## Documentation - Does this pull request introduce a new feature? **no** - If yes, how is the feature documented? **JavaDocs** You can merge this pull request into a Git repository by running: $ git pull https://github.com/NicoK/flink flink-8759 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/5571.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5571 ---- commit 272899e01c2836a2a5b47958db7d9d9f6cbf471d Author: Nico Kruber <nico@...> Date: 2018-02-23T12:56:29Z [FLINK-8768][network] Let NettyMessageDecoder inherit from LengthFieldBasedFrameDecoder This replaces one additional step from the pipeline and does not only remove overhead there but also allows use to override the #extractFrame() method to restore the old Netty 4.0.27 behaviour for non-credit based code paths which had a bug with Netty >= 4.0.28 there (see FLINK-8759). commit 11f673b415bce310cc2195fb6778051c095083fa Author: Nico Kruber <nico@...> Date: 2018-02-23T13:06:00Z [FLINK-8759][network] preparations for the update of netty to version 4.0.56 ---- ---