Hi Problem not directly in flink, but it you use flink with beam then in classpath you have original netty 4.0.27 from flink and netty 4.1.x from beam (grpc use netty 4.1.x for communication).
Other interest (specific for me now): netty have custom wrapper for openssl library which have more productivity versus default jdk version, when you have enabled ssl cluster communication it's will be very usefull give users select implementation of ssl (default,openssl,boringssl). But netty have a lot of fixes for openssl/boringssl in latest versions and more preferable do update netty as first step and enable select sslengine as second step, not all in one step. > However, if we do the change at the beginning of the next release cycle, we > might have enough exposure time to verify whether things work or not. We just start 1.4 iteration and have time for testing. Thank, Alexey Demin 2017-05-18 11:48 GMT+04:00 Till Rohrmann <trohrm...@apache.org>: > Hi Alexey, > > thanks for looking into it. Are we currently facing any problems with Netty > 4.0.27 (bugs or performance)? I agree that in general we should try to use > the latest bug fix release. However, in the past we have seen that they > might entail some slight behaviour changes which breaks things on the Flink > side. Since Netty is quite crucial for Flink, I would be extra careful here > when bumping versions, especially if there is no strong need for it. > > However, if we do the change at the beginning of the next release cycle, we > might have enough exposure time to verify whether things work or not. > > Cheers, > Till > > On Thu, May 18, 2017 at 8:51 AM, Alexey Demin <diomi...@gmail.com> wrote: > > > Hi > > > > For now we use very old netty version. > > > > Netty 4.0.27.Final released on 02-Apr-15 > > > > <!-- Don't upgrade for now. Netty versions >= 4.0.28.Final > > contain an improvement by Netty, which slices a Netty buffer > > instead of doing a memory copy [1] in the > > LengthFieldBasedFrameDecoder. In some situations, this > > interacts badly with our Netty pipeline leading to OutOfMemory > > errors. > > > > [1] https://github.com/netty/netty/issues/3704 --> > > > > If we so worry about slice in LengthFieldBasedFrameDecoder we can add > > custom > > LengthFieldBasedCopyFrameDecoder which extend original > > LengthFieldBasedFrameDecoder and override extractFrame for keep current > > behavior. > > > > With this small changes we can update for last 4.0.x. > > > > For now LengthFieldBasedFrameDecoder also used in KvStateClient and > > KvStateServer. > > Can we keep use original LengthFieldBasedFrameDecoder or must also change > > on LengthFieldBasedCopyFrameDecoder? > > > > If we want we can migrate on 4.1. > > I already did tests and all work correctly, small changes in > > NettyBufferPool.java and ChunkedByteBuf.java is required (implement new > > method which added to interface). > > > > > > Thanks > > Alexey Diomin > > >
