Re: State TTL in Flink 1.6.0

2018-08-24 Thread Andrey Zagrebin
Hi Juho, As Aljoscha mentioned the current TTL implementation was mostly targeted to data privacy applications where only processing time matters. I think the event time can be also useful for TTL and should address your concerns. The event time extension is on the road map for the future

Re: State TTL in Flink 1.6.0

2018-08-22 Thread Aljoscha Krettek
Hi Juho, The main motivation for the initial implementation of TTL was compliance with new GDPR rules. I.e. data cannot be accessible and must be dropped according to time in the real world, i.e. processing time. The behaviour you describe, with data being dropped if you keep a savepoint for

Re: State TTL in Flink 1.6.0

2018-08-22 Thread Chesnay Schepler
Just a quick note for the docs: https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/stream/state/state.html#state-time-to-live-ttl On 22.08.2018 10:53, Juho Autio wrote: First, I couldn't find anything about State TTL in Flink docs, is there anything like that? I can manage based

State TTL in Flink 1.6.0

2018-08-22 Thread Juho Autio
First, I couldn't find anything about State TTL in Flink docs, is there anything like that? I can manage based on Javadocs & source code, but just wondering. Then to main main question, why doesn't the TTL support event time, and is there any sensible use case for the TTL if the streaming