Xushaohong commented on PR #4052:
URL: https://github.com/apache/ozone/pull/4052#issuecomment-1341938783
Hi @errose28
> * Can you site specific examples where it does not work in its current
state, and why we cannot use a similar mechanism for OM incremental snapshot.
GetUpdatesSince is mainly for streaming updates, and is limited by the WAL
ttl,
https://github.com/facebook/rocksdb/blob/5f52dbc5054ab0524563580fd064aa87f611afdc/java/src/main/java/org/rocksdb/RocksDB.java#L3947-L3963.
Recon currently tries getting the delta updates, if that fails then will ask
for the full snapshot. The WAL at leader starting from the snapshot index might
already got purged since the snapshot would be big enough in the future.The
incremental snapshot could be effective since it is a common solution for the
RDB checkpoint mechianism. Recon can just download the incremental part of
snapshot/checkpoint instead of asking the full snapshot every time until it can
use delta updates.
> * It seems there may be some duplication between the snapshot diff feature
being implemented on the snapshot branch and this feature, due to the sst file
comparisons between checkpoints. I've only looked at each at a high level so
someone like @smengcl could weigh in more on this if we decide to go this route.
This should be fine, the IC logic is not changing the original snapshot
logic, which should be compatible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
