[
https://issues.apache.org/jira/browse/FLINK-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Zagrebin updated FLINK-12476:
------------------------------------
Description:
At the moment we have two efficient background cleanup strategies: incremental
for heap and compaction filter for RocksDB. *StateTtlConfig* has 2 methods to
activate them: *cleanupIncrementally* and *cleanupInRocksdbCompactFilter*. Each
is activated only for certain backend type and inactive for other. They have
different tuning parameters.
The idea is to add method *cleanupInBackground* which would activate default
background cleanup. User does not need to think then about details or used
backend if not needed. Depending on actually used backend, the corresponding
cleanup will kick in. Backends should check whether default background cleanup
is activated
We can also deprecated the parameterless *cleanupInRocksdbCompactFilter()* in
favour of this new method.
was:
At the moment we have two efficient background cleanup strategies: incremental
for heap and compaction filter for RocksDB. *StateTtlConfig* has 2 methods to
activate them: *cleanupIncrementally* and *cleanupInRocksdbCompactFilter*. Each
is activated only for certain backend type and inactive for other. They have
different tuning parameters.
The idea is to add method *cleanupInBackground* which would activate both of
them with default parameters. User does not need to think then about details or
used backend if not needed. Depending on actually used backend, the
corresponding cleanup will kick in. Sample implementation:
{code:java}
public Builder cleanupInBackground() {
return cleanupIncrementally(10, false).cleanupInRocksdbCompactFilter(1000L);
}
{code}
We can also deprecated the parameterless *cleanupInRocksdbCompactFilter()* in
favour of this new method.
> [State TTL] Consider setting a default background cleanup strategy in
> StateTtlConfig
> ------------------------------------------------------------------------------------
>
> Key: FLINK-12476
> URL: https://issues.apache.org/jira/browse/FLINK-12476
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Affects Versions: 1.8.0
> Reporter: Andrey Zagrebin
> Assignee: vinoyang
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.8.0, 1.9.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> At the moment we have two efficient background cleanup strategies:
> incremental for heap and compaction filter for RocksDB. *StateTtlConfig* has
> 2 methods to activate them: *cleanupIncrementally* and
> *cleanupInRocksdbCompactFilter*. Each is activated only for certain backend
> type and inactive for other. They have different tuning parameters.
> The idea is to add method *cleanupInBackground* which would activate default
> background cleanup. User does not need to think then about details or used
> backend if not needed. Depending on actually used backend, the corresponding
> cleanup will kick in. Backends should check whether default background
> cleanup is activated
> We can also deprecated the parameterless *cleanupInRocksdbCompactFilter()* in
> favour of this new method.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
