Hi,
I remember the projection only works with SupportsProjectionPushDown.
You can take a look at
`PushProjectIntoTableSourceScanRuleTest.testNestProjectWithMetadata`.
Will applyReadableMetadata again in the PushProjectIntoTableSourceScanRule.
But there may be bug in
PushProjectIntoTableSourceScanRule.applyPhysicalAndMetadataPushDown:
if (!usedMetadataNames.isEmpty()) {
sourceAbilitySpecs.add(new ReadingMetadataSpec(usedMetadataNames,
newProducedType));
}
If there is no meta column left, we should apply again, We should tell
the source that there is no meta column left after projection.
Best,
Jingsong
On Fri, Aug 20, 2021 at 7:56 PM Ingo Bürk <i...@ververica.com> wrote:
>
> Hi everyone,
>
> according to the SupportsReadableMetadata interface, the planner is
> supposed to project required metadata columns prior to applying them:
>
> > The planner will select required metadata columns (i.e. perform
> projection push down) and will call applyReadableMetadata(List, DataType)
> with a list of metadata keys.
>
> However, from my experiments it seems that this is not true: regardless of
> what columns I select from a table, #applyReadableMetadata always seems to
> be called with all metadata declared in the schema of the table. Metadata
> columns are also excluded from SupportsProjectionPushDown#applyProjection,
> so the source cannot perform the projection either.
>
> This is in Flink 1.13.2. Am I misreading the docs here or is this not
> working as intended?
>
>
> Best
> Ingo
--
Best, Jingsong Lee
