[
https://issues.apache.org/jira/browse/FLINK-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15748469#comment-15748469
]
Fabian Hueske commented on FLINK-5226:
--------------------------------------
[~tonycox], costs make sense for streaming operators as well. Otherwise the
optimizer can not compare plans (all have the same cost) and will not push
filters or projections close to the source of a stream.
For cases with a single stream, we can work with a static input cardinality. If
we have multiple streams, the input cardinality of a stream should somehow
reflect the rate of incoming events (events / second).
> Eagerly project unused attributes
> ---------------------------------
>
> Key: FLINK-5226
> URL: https://issues.apache.org/jira/browse/FLINK-5226
> Project: Flink
> Issue Type: Improvement
> Components: Table API & SQL
> Affects Versions: 1.2.0
> Reporter: Fabian Hueske
> Assignee: Fabian Hueske
> Fix For: 1.2.0
>
>
> The optimizer does currently not eagerly remove unused attributes.
> For example given a table {{tab5}} with five attributes {{a, b, c, d, e}},
> the following query
> {code}
> SELECT x.a, y.b FROM tab5 AS x, tab5 AS y WHERE x.a = y.a
> {code}
> would result in the non-optimized plan
> {code}
> LogicalProject(a=[$0], b=[$6])
> LogicalFilter(condition=[=($0, $5)])
> LogicalJoin(condition=[true], joinType=[inner])
> LogicalTableScan(table=[[tab5]])
> LogicalTableScan(table=[[tab5]])
> {code}
> and the optimized plan:
> {code}
> DataSetCalc(select=[a, b0 AS b])
> DataSetJoin(where=[=(a, a0)], join=[a, b, c, d, e, a0, b0, c0, d0, e0],
> joinType=[InnerJoin])
> DataSetScan(table=[[_DataSetTable_0]])
> DataSetScan(table=[[_DataSetTable_0]])
> {code}
> This plan is inefficient because it joins all ten attributes of both tables
> instead of eagerly projecting out all unused fields ({{x.b, x.c, x.d, x.e,
> y.c, y.d, y.e}}).
> Since this is one of the most common optimizations, I would assume that
> Calcite provides some rules to extract eager projections. If this is the
> case, the issue can be solved by adding such rules to {{FlinkRuleSets}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
