[
https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166914#comment-17166914
]
Zhijiang commented on FLINK-18695:
----------------------------------
I agree with [~sewen]'s above analysis.
Regarding the proposed two options by [~chesnay], I prefer the first one to
allow heap buffer allocation in NettyBufferPool for two reasons:
* The current unsupport heap buffer indeed brings some limitations to upgrade
netty version. Not only for the current SSL concern, maybe there are also other
required heap memory improvements in future.
* The main direct memory overhead was already resolved after FLINK 1.11. I
guess the heap memory overhead should be equivalent to the direct memory, if so
the impact should be tiny even ignored.
* If we are not quite sure of the heap memory overhead, we can also take some
ways to monitor and statistic the heap usages inside NettyBufferPool as did for
direct memory before.
> Allow NettyBufferPool to allocate heap buffers
> ----------------------------------------------
>
> Key: FLINK-18695
> URL: https://issues.apache.org/jira/browse/FLINK-18695
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Network
> Reporter: Chesnay Schepler
> Priority: Major
> Fix For: 1.12.0
>
>
> in 4.1.43 netty made a change to their SslHandler to always use heap buffers
> for JDK SSLEngine implementations, to avoid an additional memory copy.
> However, our {{NettyBufferPool}} forbids heap buffer allocations.
> We will either have to allow heap buffer allocations, or create a custom
> SslHandler implementation that does not use heap buffers (although this seems
> ill-adviced?).
> /cc [~sewen] [~uce] [~NicoK] [~zjwang] [~pnowojski]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
