[
https://issues.apache.org/jira/browse/FLINK-18508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingsong Lee updated FLINK-18508:
---------------------------------
Description:
Add SupportsStatisticsReport and SupportsParallelismReport to dynamic source,
source can return some information to help table optimizer.
This information can be more accurate from the source rather than the catalog.
* First, the information is computed base on real data, for iceberg /
filesystem connector, it can be calculated from real files. Although it is
related to physical/runtime, it is real and exact.
* Second, For example, for iceberg / filesystem connector, after filter and
partition pushdown, the statistics have been greatly adjusted, and many files
may have been filtered out.
was:
Add SupportsStatisticsReport and SupportsParallelismReport to dynamic source,
source can return some information to help table optimizer.
This information can be more accurate from the source rather than the catalog.
For example, for iceberg / filesystem connector, after filter and partition
pushdown, the statistics have been greatly adjusted, and many files may have
been filtered out.
> Dynamic source supports statistics and parallelism report
> ---------------------------------------------------------
>
> Key: FLINK-18508
> URL: https://issues.apache.org/jira/browse/FLINK-18508
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / API
> Reporter: Jingsong Lee
> Priority: Major
> Fix For: 1.12.0
>
>
> Add SupportsStatisticsReport and SupportsParallelismReport to dynamic source,
> source can return some information to help table optimizer.
> This information can be more accurate from the source rather than the
> catalog.
> * First, the information is computed base on real data, for iceberg /
> filesystem connector, it can be calculated from real files. Although it is
> related to physical/runtime, it is real and exact.
> * Second, For example, for iceberg / filesystem connector, after filter and
> partition pushdown, the statistics have been greatly adjusted, and many files
> may have been filtered out.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
