[
https://issues.apache.org/jira/browse/OAK-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545205#comment-15545205
]
Francesco Mari commented on OAK-4884:
-------------------------------------
In r1763269 I made bind, connect and close synchronous for the standby client,
server and proxy. I also added a bunch of debug messages along the way and
reorganised some code. I moved NetworkErrorProxy to the
org.apache.jackrabbit.oak.segment.standby package, so that its debug messages
are logged as part of our tests. I also refined the useProxy method in
BrokenNetworkTest regarding the acquisition and releasing of the standby
client, server and proxy. I am already aware that this change will increase the
build time of one or two minutes, but I am going to keep it this way as long as
we are seeing intermittent failures in the cold standby tests.
> Test failure:
> org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT.testProxyFlippedIntermediateByteChange2
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: OAK-4884
> URL: https://issues.apache.org/jira/browse/OAK-4884
> Project: Jackrabbit Oak
> Issue Type: Bug
> Components: segment-tar
> Reporter: Francesco Mari
> Assignee: Francesco Mari
> Fix For: Segment Tar 0.0.14
>
>
> The test
> {{org.apache.jackrabbit.oak.segment.standby.ExternalSharedStoreIT.testProxyFlippedIntermediateByteChange2}}
> fails intermittently with the following error:
> {noformat}
> expected: org.apache.jackrabbit.oak.segment.SegmentNodeState<{ root = { ... }
> }> but was: org.apache.jackrabbit.oak.segment.SegmentNodeState<{ root = { ...
> } }>
> {noformat}
> The cause of the issue seems to be the proxy holding to a port after it's
> been asked to disconnect:
> {noformat}
> java.net.BindException: Address already in use
> at sun.nio.ch.Net.bind0(Native Method) ~[na:1.7.0_80]
> at sun.nio.ch.Net.bind(Net.java:463) ~[na:1.7.0_80]
> at sun.nio.ch.Net.bind(Net.java:455) ~[na:1.7.0_80]
> at
> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
> ~[na:1.7.0_80]
> at
> io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:127)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:501)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1218)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:505)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:490)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:965)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:210)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:353)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:408)
> ~[netty-common-4.0.41.Final.jar:4.0.41.Final]
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
> ~[netty-common-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
> ~[netty-common-4.0.41.Final.jar:4.0.41.Final]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_80]
> {noformat}
> After that, the client fails when trying to connect to the server through the
> proxy:
> {noformat}
> 09:39:11.795 ERROR [main] StandbyClientSync.java:160 Failed
> synchronizing state.
> io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection
> refused: /127.0.0.1:41866
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> ~[na:1.7.0_80]
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
> ~[na:1.7.0_80]
> at
> io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:257)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:291)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:628)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:552)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:466)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:438)
> ~[netty-transport-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:140)
> ~[netty-common-4.0.41.Final.jar:4.0.41.Final]
> at
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
> ~[netty-common-4.0.41.Final.jar:4.0.41.Final]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_80]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
