Greg Hogan created FLINK-2655:
---------------------------------
Summary: Minimize intermediate merging of spilled buffers
Key: FLINK-2655
URL: https://issues.apache.org/jira/browse/FLINK-2655
Project: Flink
Issue Type: Improvement
Components: Distributed Runtime
Affects Versions: master
Reporter: Greg Hogan
If the number of spilled buffers exceeds taskmanager.runtime.max-fan then the
number of files must reduced with an intermediate merge by reading, merging,
and spilling into a single, larger file.
The current implementation performs an intermediate merge on all files. An
optimal implementation minimizes the amount of merged data by performing
partial merges first.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
