[ https://issues.apache.org/jira/browse/OAK-4917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15562576#comment-15562576 ]
Timothee Maret commented on OAK-4917: ------------------------------------- Discussing this with [~alexparvulescu], it seems this is expected and in fact the cold stdby code deal with it by re-trying to load in a loop with a delay. This looping behaviour is already implemented in the {{DefaultStandbySegmentReader}} [0]. However the {{DefaultStandbyReferencesReader}} does not contains the loop, causing the issue. [0] https://github.com/apache/jackrabbit-oak/blob/066edc9eb5ef500339de0ebf7859e7ee9dd3ab4b/oak-segment-tar/src/main/java/org/apache/jackrabbit/oak/segment/standby/server/DefaultStandbySegmentReader.java > DefaultStandbyReferencesReader does not re-attempt to read segment upon > failure > -------------------------------------------------------------------------------- > > Key: OAK-4917 > URL: https://issues.apache.org/jira/browse/OAK-4917 > Project: Jackrabbit Oak > Issue Type: Bug > Components: segment-tar > Affects Versions: Segment Tar 0.0.14 > Reporter: Timothee Maret > Fix For: Segment Tar 0.0.16 > > > This following logs occurred while running integration testing. > {code} > 07.10.2016 16:50:30.882 *DEBUG* [nioEventLoopGroup-3-2] > org.apache.jackrabbit.oak.segment.standby.codec.GetHeadResponseEncoder > Sending head 9d91bcd2-c4e0-4411-a0e0-d358fff02afc.00000020 to client > qastandby1 > 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2] > org.apache.jackrabbit.oak.segment.standby.codec.RequestDecoder Parsed 'get > references' message > 07.10.2016 16:50:30.885 *DEBUG* [nioEventLoopGroup-3-2] > org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler > Reading references of segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc for client > qastandby1 > 07.10.2016 16:50:30.886 *WARN* [nioEventLoopGroup-3-2] > org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader > Unable to read segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc > org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment > 9d91bcd2-c4e0-4411-a0e0-d358fff02afc not found > at > org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1364) > at > org.apache.jackrabbit.oak.segment.file.FileStore$18.call(FileStore.java:1304) > at > org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1013) > at > org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:974) > at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:285) > at > org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:92) > at > org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1304) > at > org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readSegment(DefaultStandbyReferencesReader.java:66) > at > org.apache.jackrabbit.oak.segment.standby.server.DefaultStandbyReferencesReader.readReferences(DefaultStandbyReferencesReader.java:49) > at > org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:41) > at > org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler.channelRead0(GetReferencesRequestHandler.java:27) > at > io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345) > at > io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345) > at > io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345) > at > io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:108) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345) > at > org.apache.jackrabbit.oak.segment.standby.server.RequestObserverHandler.channelRead(RequestObserverHandler.java:53) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345) > at > io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) > at > org.apache.jackrabbit.oak.segment.standby.server.StateHandler.channelRead(StateHandler.java:62) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345) > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345) > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345) > at > io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293) > at > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345) > at > io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:345) > at > io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:366) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:352) > at > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911) > at > io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) > at > io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:611) > at > io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:552) > at > io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:466) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:438) > at > io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140) > at > io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144) > at java.lang.Thread.run(Thread.java:745) > 07.10.2016 16:50:30.887 *DEBUG* [nioEventLoopGroup-3-2] > org.apache.jackrabbit.oak.segment.standby.server.GetReferencesRequestHandler > References for segment 9d91bcd2-c4e0-4411-a0e0-d358fff02afc not found, > discarding request from client qastandby1 > 07.10.2016 16:50:32.216 *DEBUG* [TarMK flush > [/Users/tmaret/Desktop/qa/quick-provider-scenarios/py/coldstandby/target/primary/crx-quickstart/repository/segmentstore]] > org.apache.jackrabbit.oak.segment.SegmentBufferWriter Writing data segment: > id=9d91bcd2-c4e0-4411-a0e0-d358fff02afc,size=6192,segmentIdCount=28,recordIdCount=126,recordCount=33 > 07.10.2016 16:50:32.216 *DEBUG* [TarMK flush > [/Users/tmaret/Desktop/qa/quick-provider-scenarios/py/coldstandby/target/primary/crx-quickstart/repository/segmentstore]] > org.apache.jackrabbit.oak.segment.file.TarWriter Writing segment > 9d91bcd2-c4e0-4411-a0e0-d358fff02afc to > /Users/tmaret/Desktop/qa/quick-provider-scenarios/py/coldstandby/target/primary/crx-quickstart/repository/segmentstore/data00007a.tar > 07.10.2016 16:50:32.217 *DEBUG* [TarMK flush > [/Users/tmaret/Desktop/qa/quick-provider-scenarios/py/coldstandby/target/primary/crx-quickstart/repository/segmentstore]] > org.apache.jackrabbit.oak.segment.file.TarRevisions TarMK journal update > 36c81550-4988-4658-abba-d27b744832c7.00000033 -> > 9d91bcd2-c4e0-4411-a0e0-d358fff02afc.00000020 > {code} > The IT test includes a primary instance and a standby instance. > The standby instance fetches the HEAD from the primary instance which returns > the segment {{9d91bcd2-c4e0-4411-a0e0-d358fff02afc}}. When the secondary > attempts to get the segment {{9d91bcd2-c4e0-4411-a0e0-d358fff02afc}} from the > primary instance, the primary instance fails to load the segment. -- This message was sent by Atlassian JIRA (v6.3.4#6332)