[ https://issues.apache.org/jira/browse/OAK-4675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Davide Giannella closed OAK-4675. --------------------------------- Bulk close for 1.4.7 > SNFE thrown while testing FileStore.cleanup() running concurrently with writes > ------------------------------------------------------------------------------ > > Key: OAK-4675 > URL: https://issues.apache.org/jira/browse/OAK-4675 > Project: Jackrabbit Oak > Issue Type: Bug > Components: segment-tar, segmentmk > Affects Versions: 1.0.32, 1.4.6, 1.2.18, Segment Tar 0.0.8 > Reporter: Andrei Dulceanu > Assignee: Andrei Dulceanu > Labels: cleanup, gc > Fix For: Segment Tar 0.0.10, 1.4.7, 1.2.19, 1.0.34 > > Attachments: OAK-4675-01.patch, OAK-4675-02.patch, > OAK-4675-oak-segment-03.patch, test-case.patch > > > {{SegmentNotFoundException}} is thrown from time to time in the following > scenario: plenty of concurrent writes (each creating a {{625 bytes}} blob) > interrupted by a cleanup. > Stack trace (including some debugging statements added by me): > {code:java} > Pre cleanup readers: [] > Before cleanup readers: [/Users/dulceanu/work/test-repo/data00000a.tar] > Initial size: 357.4 kB > After cleanup readers: [/Users/dulceanu/work/test-repo/data00000a.tar] > After cleanup size: 357.4 kB > Final size: 361.0 kB > Exception in thread "pool-5-thread-74" > org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Cannot copy > record from a generation that has been gc'ed already > at > org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.isOldGeneration(SegmentWriter.java:1207) > at > org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNodeUncached(SegmentWriter.java:1096) > at > org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNode(SegmentWriter.java:1013) > at > org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNodeUncached(SegmentWriter.java:1074) > at > org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNode(SegmentWriter.java:1013) > at > org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.writeNode(SegmentWriter.java:987) > at > org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.access$700(SegmentWriter.java:379) > at > org.apache.jackrabbit.oak.segment.SegmentWriter$8.execute(SegmentWriter.java:337) > at > org.apache.jackrabbit.oak.segment.SegmentBufferWriterPool.execute(SegmentBufferWriterPool.java:105) > at > org.apache.jackrabbit.oak.segment.SegmentWriter.writeNode(SegmentWriter.java:334) > at > org.apache.jackrabbit.oak.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:111) > at > org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.prepare(SegmentNodeStore.java:550) > at > org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.optimisticMerge(SegmentNodeStore.java:571) > at > org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.execute(SegmentNodeStore.java:627) > at > org.apache.jackrabbit.oak.segment.SegmentNodeStore.merge(SegmentNodeStore.java:287) > at > org.apache.jackrabbit.oak.segment.CompactionAndCleanupIT$1.run(CompactionAndCleanupIT.java:961) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.jackrabbit.oak.segment.SegmentNotFoundException: > Segment 4fb637cc-5013-4925-ab13-0629c4406481 not found > at > org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1341) > at > org.apache.jackrabbit.oak.segment.SegmentId.getSegment(SegmentId.java:123) > at > org.apache.jackrabbit.oak.segment.RecordId.getSegment(RecordId.java:94) > at > org.apache.jackrabbit.oak.segment.SegmentWriter$SegmentWriteOperation.isOldGeneration(SegmentWriter.java:1199) > ... 18 more > Caused by: java.util.concurrent.ExecutionException: > java.lang.IllegalStateException: Invalid segment format. Dumping segment > 4fb637cc-5013-4925-ab13-0629c4406481 > 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000000A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000000B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000000C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000000D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000000E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000000F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000140 39 37 39 31 31 36 30 38 2D 63 31 63 65 2D 34 62 97911608-c1ce-4b > 00000150 35 63 2D 61 36 33 37 2D 39 36 61 65 39 34 38 38 5c-a637-96ae9488 > 00000160 61 37 65 38 2E 30 61 62 34 30 36 38 36 00 00 00 a7e8.0ab40686... > 00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000001A0 00 00 00 00 30 30 30 30 34 30 30 00 30 30 30 30 ....0000400.0000 > 000001B0 30 30 30 00 30 30 30 30 30 30 30 00 30 30 30 30 000.0000000.0000 > 000001C0 30 30 30 31 33 30 30 00 31 32 37 35 34 36 30 33 0001300.12754603 > 000001D0 37 32 32 00 30 31 32 33 30 37 00 20 30 00 00 00 722.012307. 0... > 000001E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000001F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000210 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000220 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000240 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000260 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000270 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000280 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000290 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000002A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000002B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > at > org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1015) > at > org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:972) > at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:283) > at > org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:92) > at > org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1275) > ... 21 more > Caused by: java.lang.IllegalStateException: Invalid segment format. Dumping > segment 4fb637cc-5013-4925-ab13-0629c4406481 > 00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000080 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000090 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000000A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000000B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000000C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000000D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000000E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000000F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000140 39 37 39 31 31 36 30 38 2D 63 31 63 65 2D 34 62 97911608-c1ce-4b > 00000150 35 63 2D 61 36 33 37 2D 39 36 61 65 39 34 38 38 5c-a637-96ae9488 > 00000160 61 37 65 38 2E 30 61 62 34 30 36 38 36 00 00 00 a7e8.0ab40686... > 00000170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000001A0 00 00 00 00 30 30 30 30 34 30 30 00 30 30 30 30 ....0000400.0000 > 000001B0 30 30 30 00 30 30 30 30 30 30 30 00 30 30 30 30 000.0000000.0000 > 000001C0 30 30 30 31 33 30 30 00 31 32 37 35 34 36 30 33 0001300.12754603 > 000001D0 37 32 32 00 30 31 32 33 30 37 00 20 30 00 00 00 722.012307. 0... > 000001E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000001F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000200 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000210 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000220 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000230 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000240 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000250 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000260 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000270 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000280 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 00000290 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000002A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > 000002B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > at > com.google.common.base.Preconditions.checkState(Preconditions.java:150) > at org.apache.jackrabbit.oak.segment.Segment.<init>(Segment.java:185) > at > org.apache.jackrabbit.oak.segment.file.FileStore$15.call(FileStore.java:1292) > at > org.apache.jackrabbit.oak.segment.file.FileStore$15.call(FileStore.java:1) > at > org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1011) > ... 25 more > 0 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)