[
https://issues.apache.org/jira/browse/OAK-2294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Parvulescu updated OAK-2294:
---------------------------------
Attachment: OAK-2294-v7.patch
attaching V7. as it turns out keeping both V10 and V11 read&write support is a
bit tricky so this version is a bit more complex. I had to introduce a
SegmentVersion (heavily inpired by lucene's Version class).
I managed to add some backwards compatibility tests (write as V10 and then read
from a V11 store), and a good thing that was, I actually uncovered a bug while
writing them. the question remains if they are enough or if some more node
structures need to be added.
this patch also fixes the record analyzer tests, but I think those parts may
need closer inspection.
For _SegmentReferenceLimitTestIT_ I changed the dynamic a bit, lowered the
number of writes and increased the number of threads. test still fails on the
old version, passes on this one but at least there's no more OOME during the IT
tests.
Not the same can be said about _LargeNumberOfPropertiesTestIT_. I ignored the
tests so it doesn't run during ITs, I couldn't find any slimmer version of it
other than increasing the heap size during the tests. I'm not sure what to do
here.
as usual [~mduerig] feedback is kindly requested :)
> Corrupt repository after concurrent version operations
> ------------------------------------------------------
>
> Key: OAK-2294
> URL: https://issues.apache.org/jira/browse/OAK-2294
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segmentmk
> Reporter: Michael Dürig
> Assignee: Alex Parvulescu
> Labels: corruption
> Fix For: 1.0.12, 1.1.8
>
> Attachments: OAK-2294-2.patch, OAK-2294-v3.patch, OAK-2294-v4.patch,
> OAK-2294-v5.patch, OAK-2294-v6.patch, OAK-2294-v7.patch, OAK-2294.patch
>
>
> Performing version operations (checkin / checkout / addVersionLabel)
> concurrently can corrupt the repository.
> Executing the following code in parallel from multiple threads demonstrates
> this:
> {code}
> Version version = versionManager.checkin(vPath);
> versionManager.checkout(vPath);
> String label = version.getName() + " " + Thread.currentThread().getName();
> version.getContainingHistory()
> .addVersionLabel(version.getName(), label, true);
> {code}
> In my tests this eventually lead to all sorts of exceptions:
> {noformat}
> java.lang.IllegalStateException: RefId '85' doesn't exist in data segment
> 0c5c0814-902c-429c-ad41-cd82aea276a2
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.getRefId(Segment.java:196)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.internalReadRecordId(Segment.java:307)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.readRecordId(Segment.java:303)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.getBucketList(MapRecord.java:134)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntries(MapRecord.java:347)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.getEntries(MapRecord.java:325)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:474)
> at
> org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:394)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:544)
> ...
> {noformat}
> {noformat}
> java.lang.IllegalStateException: String is too long: 2159501163930351661
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.loadString(Segment.java:352)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.readString(Segment.java:319)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.readString(Segment.java:313)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.loadTemplate(Segment.java:418)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.readTemplate(Segment.java:367)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.readTemplate(Segment.java:361)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getTemplate(SegmentNodeState.java:78)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:408)
> ...
> {noformat}
> {noformat}
> java.lang.IllegalStateException
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:134)
> at
> org.apache.jackrabbit.oak.plugins.segment.file.TarWriter.writeEntry(TarWriter.java:206)
> at
> org.apache.jackrabbit.oak.plugins.segment.file.TarWriter.writeEntry(TarWriter.java:200)
> at
> org.apache.jackrabbit.oak.plugins.segment.file.FileStore.writeSegment(FileStore.java:682)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.flush(SegmentWriter.java:228)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.prepare(SegmentWriter.java:329)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeTemplate(SegmentWriter.java:969)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1039)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1062)
> at
> org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:395)
> ...
> {noformat}
> {noformat}
> Caused by: java.lang.IllegalArgumentException: Invalid type tag: 81
> at org.apache.jackrabbit.oak.api.Type.fromTag(Type.java:202)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.loadTemplate(Segment.java:418)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.readTemplate(Segment.java:367)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.readTemplate(Segment.java:361)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getTemplate(SegmentNodeState.java:78)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.getProperty(SegmentNodeState.java:122)
> ...
> {noformat}
> {noformat}
> Caused by: java.lang.IllegalStateException
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:134)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.pos(Segment.java:178)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.loadString(Segment.java:326)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.readString(Segment.java:319)
> at
> org.apache.jackrabbit.oak.plugins.segment.Segment.readString(Segment.java:313)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentPropertyState.getValue(SegmentPropertyState.java:174)
> at
> org.apache.jackrabbit.oak.plugins.segment.SegmentPropertyState.getValue(SegmentPropertyState.java:147)
> at
> org.apache.jackrabbit.oak.plugins.memory.AbstractPropertyState.equal(AbstractPropertyState.java:53)
> ...
> {noformat}
> Will attach a patch with a test case shortly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
