Aljoscha Krettek created FLINK-2398:
---------------------------------------
Summary: Decouple StreamGraph Building from the API
Key: FLINK-2398
URL: https://issues.apache.org/jira/browse/FLINK-2398
Project: Flink
Issue Type: Improvement
Components: Streaming
Reporter: Aljoscha Krettek
Assignee: Aljoscha Krettek
Currently, the building of the StreamGraph is very intertwined with the API
methods. DataStream knows about the StreamGraph and keeps track of splitting,
selected names, unions and so on. This leads to the problem that is is very
hard to understand how the StreamGraph is built because the code that does it
is all over the place. This also makes it hard to extend/change parts of the
Streaming system.
I propose to introduce "Transformations". A transformation hold information
about one operation: The input streams, types, names, operator and so on. An
API method creates a transformation instead of fiddling with the StreamGraph
directly. A new component, the StreamGraphGenerator creates a StreamGraph from
the tree of transformations that result from program specification using the
API methods. This would relieve DataStream from knowing about the StreamGraph
and makes unions, splitting, selection visible transformations instead of being
scattered across the different API classes as fields.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
