[ https://issues.apache.org/jira/browse/FLINK-9513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16519369#comment-16519369 ]
ASF GitHub Bot commented on FLINK-9513: --------------------------------------- GitHub user azagrebin opened a pull request: https://github.com/apache/flink/pull/6196 [FLINK-9513] Implement TTL state wrappers factory and serializer for value with TTL ## What is the purpose of the change This PR introduces a state factory for wrapping state objects with TTL logic and serialiser of user value with expiration timestamp. NOTE: This PR is based on #6186 and only last commit makes difference with it and needs review. ## Brief change log - abstract state creation in backends with `KeyedStateFactory` interface - add `TtlStateFactory` - add `CompositeSerializer` ## Verifying this change This change is a trivial addition without any test coverage in this PR and should be covered together with TTL feature activation by final integration and e2e tests. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (yes) - The runtime per-record code paths (performance sensitive): (not yet) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (not yet) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (yes) - If yes, how is the feature documented? (not applicable at the moment) You can merge this pull request into a Git repository by running: $ git pull https://github.com/azagrebin/flink FLINK-9513 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/6196.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #6196 ---- commit 62faa8ee220c21fa824fec690073c27a0a994be5 Author: Andrey Zagrebin <azagrebin@...> Date: 2018-06-04T15:28:40Z [FLINK-9514,FLINK-9515,FLINK-9516] State ttl wrappers commit 74c689e1660d40176b3c131fb0f3f9dcafa33889 Author: Andrey Zagrebin <azagrebin@...> Date: 2018-06-20T15:05:28Z Check overflow in expiration timestamp, allow only non-negative TTL commit 1164aa2a9c4298461eaa44322ef9cefa00b4f0fe Author: Andrey Zagrebin <azagrebin@...> Date: 2018-06-21T12:24:04Z small fixes commit 1d19d4ac2b73ac83290b4b117b82895c99b51865 Author: Andrey Zagrebin <azagrebin@...> Date: 2018-06-21T13:13:42Z Make AbstractTtlState.getSerializedValue() unsupported for now in case of queryable state commit 99994dedb9a20244a2addd337617778b17fe8349 Author: Andrey Zagrebin <azagrebin@...> Date: 2018-06-11T17:34:47Z [FLINK-9513] Implement TTL state wrappers factory and serializer for value with TTL ---- > Wrap state binder with TTL logic > -------------------------------- > > Key: FLINK-9513 > URL: https://issues.apache.org/jira/browse/FLINK-9513 > Project: Flink > Issue Type: Sub-task > Components: State Backends, Checkpointing > Affects Versions: 1.6.0 > Reporter: Andrey Zagrebin > Assignee: Andrey Zagrebin > Priority: Major > Labels: pull-request-available > Fix For: 1.6.0 > > > The main idea is to wrap user state value with a class holding the value and > the expiration timestamp (maybe meta data in future) and use the new object > as a value in the existing implementations: > {code:java} > class TtlValue<V> { > V value; > long expirationTimestamp; > } > {code} > The original state binder factory is wrapped with TtlStateBinder if TTL is > enabled: > {code:java} > state = ttlConfig.updateType == DISABLED ? > bind(binder) : bind(new TtlStateBinder(binder, timerService)); > {code} > TtlStateBinder decorates the states produced by the original binder with TTL > logic wrappers and adds TtlValue serialisation logic: > {code:java} > TtlStateBinder { > StateBinder binder; > ProcessingTimeProvier timeProvider; // System.currentTimeMillis() > <V> TtlValueState<V> createValueState(valueDesc) { > serializer = new TtlValueSerializer(valueDesc.getSerializer); > ttlValueDesc = new ValueDesc(serializer, ...); > // or implement custom TypeInfo > originalStateWithTtl = binder.createValueState(valueDesc); > return new TtlValueState(originalStateWithTtl, timeProvider); > } > // List, Map, ... > } > {code} > TTL serializer should add expiration timestamp -- This message was sent by Atlassian JIRA (v7.6.3#76005)