[
https://issues.apache.org/jira/browse/FLINK-178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabian Hueske closed FLINK-178.
-------------------------------
Resolution: Duplicate
Is subsumed by issues around FLINK-986 to cache intermediate results.
> Memory-Temping Data Sink / Source
> ---------------------------------
>
> Key: FLINK-178
> URL: https://issues.apache.org/jira/browse/FLINK-178
> Project: Flink
> Issue Type: Improvement
> Reporter: GitHub Import
> Labels: github-import
> Fix For: pre-apache
>
>
> One of Spark's most popular features is caching the result of computations in
> memory. This enables much more interactive data analysis.
> We could add a similar feature by proving a memory-temping data sink and data
> source. The Pact runtime already offers a ResettableReader which writes data
> into memory and eventually to disk. Currently, all data structures are
> destroyed as soon as a job finishes. Keeping such a ResettableReader, would
> enable to read data from memory that was computed from a previous job.
> However, there are also some challenges with this approach:
> - How much memory should be assigned to temping data sinks
> - How is working memory and temping memory split up
> - What is done if we run out of memory (eviction strategies)
> This feature would benefit a lot from dynamic memory assignment / trading.
> ---------------- Imported from GitHub ----------------
> Url: https://github.com/stratosphere/stratosphere/issues/178
> Created by: [fhueske|https://github.com/fhueske]
> Labels: core, enhancement,
> Created at: Fri Oct 18 15:39:39 CEST 2013
> State: open
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
