[
https://issues.apache.org/jira/browse/FLINK-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17020136#comment-17020136
]
Shi Feng commented on FLINK-15393:
----------------------------------
Hi [~hailong wang] , "regular joins use rocksdb filter to clean expired data"
——— Was this feature introduced into flink1.10 by
https://issues.apache.org/jira/browse/FLINK-14898 ? I did not see any clean
strategy related to JoinRecordStateViews#createTtlConfig in flink1.9(In my test
case, regular join state keep growing because data rows of one join key appear
only once ).
> Change 'ReturnExpiredIfNotCleanedUp' to 'NeverReturnExpired' in
> JoinRecordStateViews#createTtlConfig
> ------------------------------------------------------------------------------------------------------
>
> Key: FLINK-15393
> URL: https://issues.apache.org/jira/browse/FLINK-15393
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Runtime
> Affects Versions: 1.10.0
> Reporter: hailong wang
> Priority: Major
>
> In Blink planner, we use ttl state to clean expired data by rocksdb for
> regular joins. The StateTtlConfig is:
> {code:java}
> StateTtlConfig
> .newBuilder(Time.milliseconds(retentionTime))
> .setUpdateType(StateTtlConfig.UpdateType.OnCreateAndWrite)
>
> .setStateVisibility(StateTtlConfig.StateVisibility.ReturnExpiredIfNotCleanedUp)
> .build();
> {code}
> I think we should use StateTtlConfig.StateVisibility.NeverReturnExpired to
> prevent to join expired data.
> BTW, Only regular joins use rocksdb filter to clean expired data, should we
> change timer to rocksdb filter in other sql operator such as
> TemporalTableJoin?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
