[
https://issues.apache.org/jira/browse/BEAM-10164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maximilian Michels closed BEAM-10164.
-------------------------------------
Fix Version/s: 2.22.0
Resolution: Fixed
> Flink: Memory efficient combine implementation for batch runner
> ---------------------------------------------------------------
>
> Key: BEAM-10164
> URL: https://issues.apache.org/jira/browse/BEAM-10164
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: David Morávek
> Assignee: David Morávek
> Priority: P2
> Fix For: 2.22.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Current Combine implementation assumes that all input values for a single key
> (on both map and reduce side) fit in memory as it needs to sort them by
> window before combining.
> We can easily optimize this for non-merging windows by pre-grouping elements
> by (K, W) tuples.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
