[
https://issues.apache.org/jira/browse/BEAM-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862325#comment-15862325
]
Jingsong Lee edited comment on BEAM-1393 at 2/11/17 9:08 AM:
-------------------------------------------------------------
BTW, I think SPLIT_DISTRIBUTE state maybe need a new Repartitioner, not only
round-robin. Let each element in ListState have the opportunity to select a
KeyGroupIndex. {{CheckpointCoordinator}} use the KeyGroupIndex to redistribute
state.
I understood it. {{CheckpointCoordinator}} is run in JobManager. That is too
heavy to read every elements.
Maybe Flink can abstract KeyGroup state, provide split(snapshot to several
KeyGroups) and merge(restore by several KeyGroups) methods to manage state with
KeyGroups.
was (Author: lzljs3620320):
BTW, I think SPLIT_DISTRIBUTE state maybe need a new Repartitioner, not only
round-robin. Let each element in ListState have the opportunity to select a
KeyGroupIndex. {{CheckpointCoordinator}} use the KeyGroupIndex to redistribute
state.
> Update Flink Runner to Flink 1.2.0
> ----------------------------------
>
> Key: BEAM-1393
> URL: https://issues.apache.org/jira/browse/BEAM-1393
> Project: Beam
> Issue Type: Improvement
> Components: runner-flink
> Reporter: Aljoscha Krettek
> Assignee: Jingsong Lee
>
> When we update to 1.2.0 we can use the new internal Timer API that is
> available to Flink operators: {{InternalTimerService}} and also use broadcast
> state to store side-input data.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
