Re: Interactive Programming in Flink (Cache operation)

]. Best, Matthias [1] https://issues.apache.org/jira/browse/FLINK-19343 <https://issues.apache.org/jira/browse/FLINK-19343> [2] https://flink.apache.org/news/2021/05/03/release-1.13.0.html <https://flink.apache.org/news/2021/05/03/release-1.13.0.html> On Mon, May 3, 2021 at 8:12 PM Ja

Interactive Programming in Flink (Cache operation)

Hello, Does the new release of Flink 1.13.0 includes the cache operation feature (https://cwiki.apache.org/confluence/display/FLINK/FLIP-36%3A+Support+Interactive+Programming+in+Flink). Thank you, Iacovos

Re: Caching Mechanism in Flink

t 12:00 PM Jack Kolokasis mailto:koloka...@ics.forth.gr>> wrote: Hi Matthias, Thank you for your reply and useful information. I find that the off-heap is used when Flink uses HybridMemorySegments. Well, how the Flink knows when to use these HybridMemorySegments and in

Re: Caching Mechanism in Flink

/memory/mem_setup_tm.html#detailed-memory-model On Tue, Nov 10, 2020 at 3:57 AM Jack Kolokasis <mailto:koloka...@ics.forth.gr>> wrote: Thank you Xuannan for the reply. Also I want to ask about how Flink uses the off-heap memory. If I set taskmanager.memory.task.off-heap.size t

Re: Caching Mechanism in Flink

, At the moment, Flink doesn't support caching the intermediate result. However, there is some ongoing effort to support caching in Flink. FLIP-36[1] propose to add the caching mechanism at the Table API. And it is planned for 1.13. Best, Xuannan On Nov 10, 2020, 4:29 AM +0800, Jack Kolokasis

Caching Mechanism in Flink

Hello all, I am new to Flink and I want to ask if the Flink supports a caching mechanism to store intermediate results in memory for machine learning workloads. If yes, how can I enable it and how can I use it? Thank you, Iacovos