[jira] [Commented] (FLINK-15393) Change 'ReturnExpiredIfNotCleanedUp' to 'NeverReturnExpired' in JoinRecordStateViews#createTtlConfig

2020-01-30 Thread Shi Feng (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17027170#comment-17027170
 ] 

Shi Feng commented on FLINK-15393:
--

Hi [~hailong wang], I knew. But it does not work when using blink planner :)

> Change  'ReturnExpiredIfNotCleanedUp' to 'NeverReturnExpired' in 
> JoinRecordStateViews#createTtlConfig 
> --
>
> Key: FLINK-15393
> URL: https://issues.apache.org/jira/browse/FLINK-15393
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / Runtime
>Affects Versions: 1.10.0
>Reporter: hailong wang
>Priority: Major
>
> In Blink planner, we use ttl state to clean expired data by rocksdb for 
> regular joins. The StateTtlConfig is:
> {code:java}
>  StateTtlConfig
>.newBuilder(Time.milliseconds(retentionTime))
>.setUpdateType(StateTtlConfig.UpdateType.OnCreateAndWrite)
>
> .setStateVisibility(StateTtlConfig.StateVisibility.ReturnExpiredIfNotCleanedUp)
>.build();
> {code}
> I think we should use StateTtlConfig.StateVisibility.NeverReturnExpired to 
> prevent to join expired data.
> BTW, Only regular joins use rocksdb filter to clean expired data, should we 
> change timer to rocksdb filter in other sql operator such as 
> TemporalTableJoin?
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (FLINK-15393) Change 'ReturnExpiredIfNotCleanedUp' to 'NeverReturnExpired' in JoinRecordStateViews#createTtlConfig

2020-01-21 Thread Shi Feng (Jira) 


[ 
https://issues.apache.org/jira/browse/FLINK-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17020136#comment-17020136
 ] 

Shi Feng commented on FLINK-15393:
--

Hi [~hailong wang] , "regular joins use rocksdb filter to clean expired data" 
——— Was this feature introduced into flink1.10 by  
https://issues.apache.org/jira/browse/FLINK-14898 ? I did not see any clean 
strategy related to JoinRecordStateViews#createTtlConfig in flink1.9(In my test 
case, regular join state keep growing because data rows of one join key appear 
only once ).

> Change  'ReturnExpiredIfNotCleanedUp' to 'NeverReturnExpired' in 
> JoinRecordStateViews#createTtlConfig 
> --
>
> Key: FLINK-15393
> URL: https://issues.apache.org/jira/browse/FLINK-15393
> Project: Flink
>  Issue Type: Improvement
>  Components: Table SQL / Runtime
>Affects Versions: 1.10.0
>Reporter: hailong wang
>Priority: Major
>
> In Blink planner, we use ttl state to clean expired data by rocksdb for 
> regular joins. The StateTtlConfig is:
> {code:java}
>  StateTtlConfig
>.newBuilder(Time.milliseconds(retentionTime))
>.setUpdateType(StateTtlConfig.UpdateType.OnCreateAndWrite)
>
> .setStateVisibility(StateTtlConfig.StateVisibility.ReturnExpiredIfNotCleanedUp)
>.build();
> {code}
> I think we should use StateTtlConfig.StateVisibility.NeverReturnExpired to 
> prevent to join expired data.
> BTW, Only regular joins use rocksdb filter to clean expired data, should we 
> change timer to rocksdb filter in other sql operator such as 
> TemporalTableJoin?
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)