[jira] [Updated] (FLINK-1671) Add execution modes for programs
[
https://issues.apache.org/jira/browse/FLINK-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephan Ewen updated FLINK-1671:
Labels: starter (was: )
Add execution modes for programs
Key: FLINK-1671
URL: https://issues.apache.org/jira/browse/FLINK-1671
Project: Flink
Issue Type: Bug
Affects Versions: 0.9
Reporter: Stephan Ewen
Assignee: Stephan Ewen
Fix For: 0.9
Currently, there is a single way that programs get executed: Pipelined. With
the new code for batch shuffles (https://github.com/apache/flink/pull/471),
we have much more flexibility and I would like to expose that.
I suggest to add more execution modes that can be chosen on the
`ExecutionEnvironment`:
- {{BATCH}} A mode where every shuffle is executed in a batch way, meaning
preceding operators must be done before successors start. Only for the batch
programs (d'oh).
- {{PIPELINED}} This is the mode corresponding to the current execution
mode. It pipelines where possible and batches, where deadlocks would
otherwise happen. Initially, I would make this the default (be close to the
current behavior). Only available for batch programs.
- {{PIPELINED_WITH_BATCH_FALLBACK}} This would start out with pipelining
shuffles and fall back to batch shuffles upon failure and recovery, or once
it sees that not enough slots are available to bring up all operators at once
(requirement for pipelining).
- {{STREAMING}} This is the default and only way for streaming programs.
All communication is pipelined, and the special streaming checkpointing code
is activated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLINK-1671) Add execution modes for programs
[
https://issues.apache.org/jira/browse/FLINK-1671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephan Ewen updated FLINK-1671:
Labels: (was: easyfix starter)
Add execution modes for programs
Key: FLINK-1671
URL: https://issues.apache.org/jira/browse/FLINK-1671
Project: Flink
Issue Type: Bug
Affects Versions: 0.9
Reporter: Stephan Ewen
Assignee: Stephan Ewen
Fix For: 0.9
Currently, there is a single way that programs get executed: Pipelined. With
the new code for batch shuffles (https://github.com/apache/flink/pull/471),
we have much more flexibility and I would like to expose that.
I suggest to add more execution modes that can be chosen on the
`ExecutionEnvironment`:
- {{BATCH}} A mode where every shuffle is executed in a batch way, meaning
preceding operators must be done before successors start. Only for the batch
programs (d'oh).
- {{PIPELINED}} This is the mode corresponding to the current execution
mode. It pipelines where possible and batches, where deadlocks would
otherwise happen. Initially, I would make this the default (be close to the
current behavior). Only available for batch programs.
- {{PIPELINED_WITH_BATCH_FALLBACK}} This would start out with pipelining
shuffles and fall back to batch shuffles upon failure and recovery, or once
it sees that not enough slots are available to bring up all operators at once
(requirement for pipelining).
- {{STREAMING}} This is the default and only way for streaming programs.
All communication is pipelined, and the special streaming checkpointing code
is activated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
