[
https://issues.apache.org/jira/browse/FLINK-18695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185656#comment-17185656
]
Yun Gao commented on FLINK-18695:
---------------------------------
I future tested the same job with 1) The current version with netty
4.1.39-FINAL 2) the version using 4.1.50-FINAL and use Unpool heap memory 3)
the version using 4.1.50-FINAL and use direct memory to fake heap memory, the
average heap consumption of a single TaskManager (acquired via _jstat -gc_ and
sum each heap region usage together) is
|| ||4.1.39 ||4.1.50
UnPooled Heap||4.1.50
Faked Heap||
|1500 x 500|181KB|221.4KB|185KB|
|500 x 1500|204KB|209KB|204.8KB|
It seems that
# The overall heap consumption is not very high.
# Using un-pooled heap memory might cause some additional overhead.
Besides, using direct buffer to fake the heap buffer did not cause failures.
> Allow NettyBufferPool to allocate heap buffers
> ----------------------------------------------
>
> Key: FLINK-18695
> URL: https://issues.apache.org/jira/browse/FLINK-18695
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Network
> Reporter: Chesnay Schepler
> Assignee: Yun Gao
> Priority: Major
> Fix For: 1.12.0
>
>
> in 4.1.43 netty made a change to their SslHandler to always use heap buffers
> for JDK SSLEngine implementations, to avoid an additional memory copy.
> However, our {{NettyBufferPool}} forbids heap buffer allocations.
> We will either have to allow heap buffer allocations, or create a custom
> SslHandler implementation that does not use heap buffers (although this seems
> ill-adviced?).
> /cc [~sewen] [~uce] [~NicoK] [~zjwang] [~pnowojski]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
