[
https://issues.apache.org/jira/browse/FLINK-10339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16628295#comment-16628295
]
ASF GitHub Bot commented on FLINK-10339:
----------------------------------------
zhijiangW opened a new pull request #6762: [FLINK-10339][network] Use off-heap
memory for SpillReadBufferPool
URL: https://github.com/apache/flink/pull/6762
## What is the purpose of the change
Currently the `NetworkBufferPool` already uses off-heap memory directly. But
the `BufferPool` for `SpilledSubpartitionView` still uses heap memory by
default. It should keep the same behavior with `NetworkBufferPool` and it may
get benefit for reusing this off-heap memory in netty stack during transporting.
## Brief change log
- *Use off-heap memory directly for `SpillReadBufferPool` in
`SpilledSubpartitionView`.*
## Verifying this change
This change is already covered by existing tests, such as
*SpillableSubpartitionTest*.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (yes)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> SpillReadBufferPool cannot use off-heap memory
> ----------------------------------------------
>
> Key: FLINK-10339
> URL: https://issues.apache.org/jira/browse/FLINK-10339
> Project: Flink
> Issue Type: Improvement
> Components: Network
> Affects Versions: 1.5.0, 1.5.1, 1.5.2, 1.5.3, 1.6.0, 1.7.0
> Reporter: zhijiang
> Assignee: zhijiang
> Priority: Minor
> Labels: pull-request-available
>
> Currently, the {{NetworkBufferPool}} always uses off-heap memory to reduce
> memory copy from flink {{Buffer}} to netty internal {{ByteBuf}} during
> transporting on sender side.
>
> But for {{SpillReadBufferPool}} in {{SpilledSubpartitionView}}, it still uses
> heap memory for caching. We can make it as off-heap by default similar with
> {{NetworkBufferPool}} or decide the type by the current parameter
> {{taskmanager.memory.off-heap.}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
