Fabian Hueske created FLINK-7696:
------------------------------------
Summary: Add projection push-down support for TableSources with
time attributes
Key: FLINK-7696
URL: https://issues.apache.org/jira/browse/FLINK-7696
Project: Flink
Issue Type: Improvement
Components: Table API & SQL
Affects Versions: 1.4.0
Reporter: Fabian Hueske
Table sources that implement the {{DefinedProctimeAttribute}} or
{{DefinedRowtimeAttribute}} do not support projection push-down even if they
also implement {{ProjectableTableSource}}.
There are several problems:
- the schema of a {{TableSource}} that implements {{DefinedRowtimeAttribute}}
or {{DefinedProctimeAttribute}} is constructed in the catalog not in the
{{TableSource}} (proctime fields are always appended at the end).
- the {{ProjectableTableSource.projectFields()}} method returns the projected
fields as int indicies. In order to handle the indicies correctly, the
TableSource would need to know about the internals of the Table API.
- {{ProjectableTableSource.projectFields()}} might reorder fields and move a
proctime field into the middle of the schema. However, the TableSource has no
control over that.
- A {{TableSource}} that implements {{DefinedRowtimeAttribute}} or
{{DefinedProctimeAttribute}} would need to change the return values of
{{getRowtimeAttribute()}} or {{getProctimeAttribute()}} depending on whether
the attribute is kept or not.
Adjusting the schema of table sources inside of the Table API makes all of this
quite messy. Maybe we need to refine the interfaces. For instance, we could ask
users to explicitly add time indicator fields in the {{TypeInformation}}
returned by {{TableSource.getReturnType()}}. However, that might collide with
plans to add computable time attributes as proposed in FLINK-7548.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
