smiroslav opened a new pull request, #3274: URL: https://github.com/apache/jackrabbit-oak/pull/3274
https://issues.apache.org/jira/browse/OAK-12516 ## Problem For remote segment stores the binary references (`.brf`) of an archive are written only when the archive is closed. They are missing while the archive is still open, or when the Oak process terminated before closing it. `SegmentStoreMigrator` skipped the missing binary references silently, so the migrated TAR archive had no `.brf` entry and `SegmentTarReader.getBinaryReferences()` failed with `InvalidBinaryReferencesIndexException: Unrecognized magic number`. ## Change - `SegmentStoreMigrator` (Azure) reads the binary references of the source archive first. If they are missing, it rebuilds them from the migrated segments (with a WARN) and writes them after the segments, so the TAR layout stays `[segments][.brf][.gph][.idx]`. - New `BinaryReferencesCollector` (`oak-segment-tar`, package `file.tar.binaries`, not exported) rebuilds the index from the segments of an archive. Long blob ids whose value is stored in another archive are read from a fallback store. The migrator opens the source as a `ReadOnlyFileStore` for that, lazily and only when needed, so the source is never recovered or modified. - `BinaryReferencesCollector.readBinaryReferences(Segment, Map)` is now the single place that reads the blob ids of a segment; `FileStore.writeSegment` and TAR recovery (`AbstractFileStore.populateTarBinaryReferences`) use it as well. No behaviour change there. - The error log of a failed archive migration now names the archive. ## Tests - `SegmentStoreMigratorTest` (Azurite): closed archive (binary references copied unchanged), open archive, archive not closed before termination, and a long blob id whose value is in a previous archive (also asserts the source archive was not recovered). - `BinaryReferencesCollectorTest`: rebuilt index equals the written index for short ids, long ids, ids stored in bulk segments and ids whose value is in a previous segment; behaviour with and without fallback store. - `LongIdMappingBlobStore` accepts an id length to produce these cases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
