Github user StefanRRichter commented on a diff in the pull request:
https://github.com/apache/flink/pull/5582#discussion_r192052899
--- Diff:
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeySerializationUtils.java
---
@@ -138,4 +138,12 @@ private static void writeVariableIntBytes(
value >>>= 8;
} while (value != 0);
}
+
+ public static byte[] serializeKeyGroup(int keyGroup, int
keyGroupPrefixBytes) {
+ byte[] startKeyGroupPrefixBytes = new byte[keyGroupPrefixBytes];
--- End diff --
Cool, no problem, I am also completing this review soon. One general thing
that I was wondering about: did you ever see the sstable ingestion feature? It
would be super nice for the rescaling of incremental checkpoints if we could
simply ingest the sstables from multiple checkpoints into one database and the
just clip the range boundaries. Unfortunately, from what I have seen this only
works for external sstables written bei the sstable writer (see here:
https://github.com/facebook/rocksdb/wiki/Creating-and-Ingesting-SST-files). I
wonder if there is any way to modify the sstables if incremental checkpoints to
make them usable for ingestion, but maybe it is just completely impossible. I
also found this interesting discussion that outlines another potential
approach: https://github.com/facebook/rocksdb/issues/499. Any thoughts?
---
