[
https://issues.apache.org/jira/browse/FLINK-5991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15929756#comment-15929756
]
ASF GitHub Bot commented on FLINK-5991:
---------------------------------------
Github user StefanRRichter commented on the issue:
https://github.com/apache/flink/pull/3508
I wonder if there could also exist a case for broadcasting operator state
(non-keyed), where only one operator instance is selected as sender and all
others receive on restore. Furthermore, the union aspect may (or may not)
happen at restore time, but not at the time that a user requests this state.
For what this currently does, I think `ReplicatingState` describes it pretty
well. Broadcast would be a good description from the operator's perspective: it
broadcasts the generated data to all peers on restore.
> Expose Broadcast Operator State through public APIs
> ---------------------------------------------------
>
> Key: FLINK-5991
> URL: https://issues.apache.org/jira/browse/FLINK-5991
> Project: Flink
> Issue Type: New Feature
> Components: DataStream API, State Backends, Checkpointing
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Tzu-Li (Gordon) Tai
> Fix For: 1.3.0
>
>
> The broadcast operator state functionality was added in FLINK-5265, it just
> hasn't been exposed through any public APIs yet.
> Currently, we have 2 streaming connector features for 1.3 that are pending on
> broadcast state: rescalable Kinesis / Kafka consumers with shard / partition
> discovery (FLINK-4821 & FLINK-4022). We should consider exposing broadcast
> state for the 1.3 release also.
> This JIRA also serves the purpose to discuss how we want to expose it.
> To initiate the discussion, I propose:
> 1. For the more powerful {{CheckpointedFunction}}, add the following to the
> {{OperatorStateStore}} interface:
> {code}
> <S> ListState<S> getBroadcastOperatorState(ListStateDescriptor<S>
> stateDescriptor);
> <T extends Serializable> ListState<T>
> getBroadcastSerializableListState(String stateName);
> {code}
> 2. For a simpler {{ListCheckpointed}} variant, we probably should have a
> separate {{BroadcastListCheckpointed}} interface.
> Extending {{ListCheckpointed}} to let the user define either the list state
> type of either {{PARTITIONABLE}} or {{BROADCAST}} might also be possible, if
> we can rely on a contract that the value doesn't change. Or we expose a
> defining method (e.g. {{getListStateType()}}) that is called only once in the
> operator. This would break user code, but can be considered because it is
> marked as {{PublicEvolving}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
