[
https://issues.apache.org/jira/browse/FLINK-24008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nico Kruber updated FLINK-24008:
--------------------------------
Description:
In a join of two tables where we join on unique columns, e.g. from primary
keys, we could clean up join state more pro-actively since we now whether
future joins with this row are still possible (assuming uniqueness of that
key). While this may not solve all issues of growing state in non-time-based
joins it may still considerably reduce state size, depending on the involved
columns.
This would add one more way of expiring state that the operator stores;
currently there are only these
* time-based joins, e.g. interval join
* idle state retention via {{TableConfig#setIdleStateRetention()}}
was:
In a join of two tables where we join on unique columns, e.g. from primary
keys, we could clean up join state more pro-actively since we now whether
future joins with this row are still possible (assuming uniqueness of that
key). While this may not solve all issues of growing state in non-time-based
joins it may still considerably reduce state size, depending on the involved
columns.
This would add one more way of expiring state that the operator stores;
currently there are only these
* time-based joins, e.g. interval join
* idle state retention via \{{TableConfig#setIdleStateRetention()}}
> Support state cleanup based on unique keys
> ------------------------------------------
>
> Key: FLINK-24008
> URL: https://issues.apache.org/jira/browse/FLINK-24008
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / Runtime
> Affects Versions: 1.14.0
> Reporter: Nico Kruber
> Priority: Major
>
> In a join of two tables where we join on unique columns, e.g. from primary
> keys, we could clean up join state more pro-actively since we now whether
> future joins with this row are still possible (assuming uniqueness of that
> key). While this may not solve all issues of growing state in non-time-based
> joins it may still considerably reduce state size, depending on the involved
> columns.
> This would add one more way of expiring state that the operator stores;
> currently there are only these
> * time-based joins, e.g. interval join
> * idle state retention via {{TableConfig#setIdleStateRetention()}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
