[
https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17167650#comment-17167650
]
Zhijiang edited comment on FLINK-18695 at 7/30/20, 5:55 AM:
------------------------------------------------------------
>Do we actually also cover the receiving side and use our own memory segments
>deep in the Netty receiver stack?
Yes, we resolved it in release-1.11 by
[FLINK-10742|https://issues.apache.org/jira/browse/FLINK-10742], [~NicoK]
>Allowing netty to use heap buffers is quite easy (just remove the overridden
>methods in or NettyBufferPool, but what other changes would be required? Would
>we also have to increase the number of heap arenas (from 0)?
Yes, we should adjust the number of heap arenas as the same with direct arenas,
because the heap buffer allocation relies on the heap arena.
> How difficult is it adjust the memory model to account for this?
After further checking the codes, I find that our previous improvement of netty
memory in FLINK-10742 did not cover the case of SSL. The pipeline handler in
receiver side is SslHandler -> NettyMessageDecoder. The FLINK-10742 only
reduced the netty memory overhead in message coder, but the SslHandler still
allocates memory via internal netty allocator. As we never used the SSL mode in
our production before, so not quite sure the effect of this portion memory
overhead in practice.
Another concern is that since we config separate heap arenas for SSL, then the
total memory overhead might be enlarged compared with reusing the direct arena
as before. Maybe it need some framework heap memory supplementing for this
overhead if upgrading. Anyway I think it would be safer to make some
experiments to get some insights if possible before upgrading directly.
[~chesnay]
was (Author: zjwang):
> Do we actually also cover the receiving side and use our own memory segments
> deep in the Netty receiver stack?
Yes, we resolved it in release-1.11 by
[FLINK-10742|https://issues.apache.org/jira/browse/FLINK-10742], [~NicoK]
> Allowing netty to use heap buffers is quite easy (just remove the overridden
> methods in or NettyBufferPool, but what other changes would be required?
> Would we also have to increase the number of heap arenas (from 0)?
Yes, we should adjust the number of heap arenas as the same with direct arenas,
because the heap buffer allocation relies on the heap arena.
> How difficult is it adjust the memory model to account for this?
After further checking the codes, I find that our previous improvement of netty
memory in FLINK-10742 did not cover the case of SSL. The pipeline handler in
receiver side is SslHandler -> NettyMessageDecoder. The FLINK-10742 only
reduced the netty memory overhead in message coder, but the SslHandler still
allocates memory via internal netty allocator. As we never used the SSL mode in
our production before, so not quite sure the effect of this portion memory
overhead in practice.
Another concern is that since we config separate heap arenas for SSL, then the
total memory overhead might be enlarged compared with reusing the direct arena
as before. Maybe it need some framework heap memory supplementing for this
overhead if upgrading. Anyway I think it would be safer to make some
experiments to get some insights if possible before upgrading directly.
> Allow NettyBufferPool to allocate heap buffers
> ----------------------------------------------
>
> Key: FLINK-18695
> URL: https://issues.apache.org/jira/browse/FLINK-18695
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Network
> Reporter: Chesnay Schepler
> Priority: Major
> Fix For: 1.12.0
>
>
> in 4.1.43 netty made a change to their SslHandler to always use heap buffers
> for JDK SSLEngine implementations, to avoid an additional memory copy.
> However, our {{NettyBufferPool}} forbids heap buffer allocations.
> We will either have to allow heap buffer allocations, or create a custom
> SslHandler implementation that does not use heap buffers (although this seems
> ill-adviced?).
> /cc [~sewen] [~uce] [~NicoK] [~zjwang] [~pnowojski]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
