rkhachatryan commented on a change in pull request #15420:
URL: https://github.com/apache/flink/pull/15420#discussion_r658799112
##########
File path:
flink-state-backends/flink-statebackend-changelog/src/main/java/org/apache/flink/state/changelog/ChangelogKeyedStateBackend.java
##########
@@ -240,8 +275,38 @@ public boolean
deregisterKeySelectionListener(KeySelectionListener<K> listener)
@Nonnull CheckpointStreamFactory streamFactory,
@Nonnull CheckpointOptions checkpointOptions)
throws Exception {
- return keyedStateBackend.snapshot(
- checkpointId, timestamp, streamFactory, checkpointOptions);
+ // The range to upload may overlap with the previous one(s). To reuse
them, we could store
+ // the previous results either here in the backend or in the writer.
However,
+ // materialization may truncate only a part of the previous result and
the backend would
+ // have to split it somehow for the former option, so the latter is
used.
Review comment:
Some component has to maintain a data structure allowing range queries
for changes (SqN >= ...).
Currently (in #14839/#15322), this is done in Writer with a sorting tree.
But essentially this is an index over the log and can be implemented
differently by different stores; and might need some maintanence which is not
the backend responsibility.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
