Stephan Ewen created FLINK-2084:
-----------------------------------
Summary: Create a dedicated streaming mode
Key: FLINK-2084
URL: https://issues.apache.org/jira/browse/FLINK-2084
Project: Flink
Issue Type: Improvement
Components: Distributed Runtime
Affects Versions: 0.9
Reporter: Stephan Ewen
Assignee: Stephan Ewen
Fix For: 0.9
As per discussion on the mailing list
http://mail-archives.apache.org/mod_mbox/flink-dev/201505.mbox/browser
- We add a dedicated streaming mode for now. The streaming mode supersedes the
batch mode, so it can run both type of programs.
- The streaming mode sets the memory manager to "lazy allocation".
-> So long as it runs pure streaming jobs, the full heap will be
available to window buffers and UDFs.
-> Batch programs can still run, so mixed workloads are not prevented.
Batch programs are a bit less robust there, because the memory manager does not
pre-allocate memory. UDFs can eat into Flink's memory portion.
- The streaming mode starts the necessary configured components/services for
state backups
Over the next versions, we want to bring these things together:
- use the managed memory for window buffers
- on-demand starting of the state backend
Then, we deprecate the streaming mode, let both modes start the cluster in the
same way.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
