[
https://issues.apache.org/jira/browse/FLINK-39421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18086408#comment-18086408
]
Sergey Nuyanzin commented on FLINK-39421:
-----------------------------------------
Merged as
[a62a144a5e99ef1d66007f2caf409a62fdbcb5ad|https://github.com/apache/flink/commit/a62a144a5e99ef1d66007f2caf409a62fdbcb5ad]
> Metadata filter push-down for table sources
> -------------------------------------------
>
> Key: FLINK-39421
> URL: https://issues.apache.org/jira/browse/FLINK-39421
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Reporter: Jim Hughes
> Assignee: Jim Hughes
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.0
>
>
> Predicates on metadata columns (e.g., Kafka offset, timestamp, partition) are
> excluded from the existing SupportsFilterPushDown mechanism.
> FilterPushDownSpec serializes RexInputRef indices relative to the scan's
> produced row type, but metadata columns are appended after physical columns.
> When ProjectPushDownSpec narrows the row type during compiled plan
> restoration, metadata indices become out-of-bounds, causing
> ArrayIndexOutOfBoundsException.
> This ticket introduces a dedicated metadata filter push-down path:
> - New supportsMetadataFilterPushDown() and applyMetadataFilters() default
> methods on SupportsReadingMetadata (opt-in; existing sources unaffected)
> - New MetadataFilterPushDownSpec that snapshots the row type at optimization
> time, solving the index mismatch
> - Extension to PushFilterIntoTableSourceScanRule to classify predicates as
> physical, metadata, or mixed
> See FLIP-574
> (https://cwiki.apache.org/confluence/display/FLINK/FLIP-574+Metadata+Filter+Push-Down+for+Table+Sources)
> for full design.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
