rkhachatryan commented on code in PR #22669:
URL: https://github.com/apache/flink/pull/22669#discussion_r1220434655
##########
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/snapshot/RocksDBSnapshotStrategyBase.java:
##########
@@ -395,18 +394,16 @@ public void release() {
/** Previous snapshot with uploaded sst files. */
protected static class PreviousSnapshot {
- @Nullable private final Map<StateHandleID, Long> confirmedSstFiles;
+ @Nullable private final Map<StateHandleID, StreamStateHandle>
confirmedSstFiles;
- protected PreviousSnapshot(@Nullable Map<StateHandleID, Long>
confirmedSstFiles) {
+ protected PreviousSnapshot(
+ @Nullable Map<StateHandleID, StreamStateHandle>
confirmedSstFiles) {
this.confirmedSstFiles = confirmedSstFiles;
}
protected Optional<StreamStateHandle> getUploaded(StateHandleID
stateHandleID) {
if (confirmedSstFiles != null &&
confirmedSstFiles.containsKey(stateHandleID)) {
- // we introduce a placeholder state handle, that is replaced
with the
- // original from the shared state registry (created from a
previous checkpoint)
- return Optional.of(
- new
PlaceholderStreamStateHandle(confirmedSstFiles.get(stateHandleID)));
Review Comment:
Thanks @zoltar9264 ,
> For example, the registry in this issue is wrongly de-duplicated.
> If we only perform the replacement when the placeholder is used, and
reduce the scope of the placeholder, it is possible to avoid the problem caused
by the registry returning the wrong handle.
But this issue is not related to the `PlaceholderStreamStateHandle`, is it?
It happens when both new and old handles are NOT
`PlaceholderStreamStateHandle`s.
> Furthermore, if we check that the handle returned by the registry is equal
to the registered handle when the placeholder is not used, we can detect the
problem earlier.
I think the problem is actually when the handles under the same key are NOT
equal - and we need to resolve this conflict somehow. Note that's not related
`PlaceholderStreamStateHandle`.
But since in this PR we entirely eliminate the possibility of collision by
using unique keys, such a collision would mean a bug; so we can remove that
resolution logic and raise an error instead:
- remove `SharedStateRegistryImpl.SharedStateEntry.confirmed` field
- remove any calls to `scheduleAsyncDelete()` from `registerReference()`
- (but keep `PlaceholderStreamStateHandle`)
WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
