Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/1769#issuecomment-194250033
I think the right way to solve that would actually not be in the
MemoryManager (to cache), but in the operators that know that they will need
the memory again and again and will hold onto it.
For another feature in the code, I am currently changing the access to
memory such that it goes through "allocators". These are used by for example
the sorter, to get their memory segments from the memory manager. It would be
simple to create them in the batch operators such that they don't immediately
release to the mem manager, but only after the operator is disposed. That means
memory segments will be held onto through all iterations.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
