[
https://issues.apache.org/jira/browse/FLINK-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xu Pingyong updated FLINK-7015:
-------------------------------
Description:
Motivation:
A Task contains one or more operators with chainning, however
configs of operator and task are all put in StreamConfig. For example, when a
opeator sets up with the StreamConfig, it can see the interface about
physicalEdges or chained.task.configs that are confused. Similarly a
streamTask should not see the interface aboule chain.index.
So we need to separate OperatorConfig from StreamConfig. A streamTask
builds execution enviroment with the streamConfig, and extract operatorConfigs
from it, then build streamOperators with every operatorConfig.
OperatorConfig: for the streamOperator to setup with, it constains
informations that only belong to the streamOperator. It contains:
1) operator information: name, id
2) Serialized StreamOperator
3) input serializer.
4) output edges and serializers.
5) chain.index
6) state.key.serializer
StreamConfig: for the streamTask to use:
1) in.physical.edges
2) out.physical.edges
3) chained OperatorConfigs
4) execution environment: checkpoint, state.backend and so on...
was:
Now stream config contains configs not only the batch task needs, but also the
operator needs, so stream config can see configs of the operator, and operator
can see configs of the batch task.
We need to separate operator config from stream config, and they can only see
configs of themselves.
> Separate OperatorConfig from StreamConfig
> -----------------------------------------
>
> Key: FLINK-7015
> URL: https://issues.apache.org/jira/browse/FLINK-7015
> Project: Flink
> Issue Type: Improvement
> Components: DataStream API
> Reporter: Xu Pingyong
> Assignee: Xu Pingyong
>
> Motivation:
> A Task contains one or more operators with chainning, however
> configs of operator and task are all put in StreamConfig. For example, when a
> opeator sets up with the StreamConfig, it can see the interface about
> physicalEdges or chained.task.configs that are confused. Similarly a
> streamTask should not see the interface aboule chain.index.
> So we need to separate OperatorConfig from StreamConfig. A
> streamTask builds execution enviroment with the streamConfig, and extract
> operatorConfigs from it, then build streamOperators with every
> operatorConfig.
>
> OperatorConfig: for the streamOperator to setup with, it constains
> informations that only belong to the streamOperator. It contains:
> 1) operator information: name, id
> 2) Serialized StreamOperator
> 3) input serializer.
> 4) output edges and serializers.
> 5) chain.index
> 6) state.key.serializer
> StreamConfig: for the streamTask to use:
> 1) in.physical.edges
> 2) out.physical.edges
> 3) chained OperatorConfigs
> 4) execution environment: checkpoint, state.backend and so on...
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
