Ksenia Rybakova created IGNITE-4187: ---------------------------------------
Summary: "Remote node ID is not as expected" when client SSL certificate is signed by untrusted CA Key: IGNITE-4187 URL: https://issues.apache.org/jira/browse/IGNITE-4187 Project: Ignite Issue Type: Bug Affects Versions: 1.6 Reporter: Ksenia Rybakova Test config: - 1 client node, 1 server node - SSL is enabled {noformat} <property name="sslContextFactory"> <bean class="org.apache.ignite.ssl.SslContextFactory"> <property name="protocol" value="TLSv1.2"/> <property name="keyStoreFilePath" value="/home/keystore/server.jks"/> <property name="keyStorePassword" value="123456"/> <property name="trustStoreFilePath" value="/home/keystore/trust.jks"/> <property name="trustStorePassword" value="123456"/> </bean> </property> {noformat} trust.jks on server side has one CA certificate and this is NOT the one that was used to sign the client certificate (so the server doesn't trust to the client) trust.jks on client side has one CA certificate and this is the one that was used to sign the server certificate (so the client does trust to the server) - Yardstick is used to run simple load test (configs and property file are attached) Result: client connects to server, but there are errors in log: client: {noformat} [16:05:21,751][ERROR][exchange-worker-#22%null%][GridDhtAssignmentFetchFuture] Failed to request affinity assignment from remote node (will continue to another node): TcpDiscoveryNode [id=c02cdaa3-80de-4b81-884f-ca9ba830dba5, addrs=[127.0.0.1, 172.25.1.32], sockAddrs=[/172.25.1.32:47500, /127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1478178315859, loc=false, ver=1.7.0#20161031-sha1:6b78ad0c, isClient=false] class org.apache.ignite.IgniteCheckedException: Failed to send message (node may have left the grid or TCP connection cannot be established due to firewall issues) [node=TcpDiscoveryNode [id=c02cdaa3-80de-4b81-884f-ca9ba830dba5, addrs=[127.0.0.1, 172.25.1.32], sockAddrs=[/172.25.1.32:47500, /127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1478178315859, loc=false, ver=1.7.0#20161031-sha1:6b78ad0c, isClient=false], topic=TOPIC_CACHE, msg=GridDhtAffinityAssignmentRequest [topVer=AffinityTopologyVersion [topVer=2, minorTopVer=0], super=GridCacheMessage [msgId=2, depInfo=null, err=null, skipPrepare=false, cacheId=1489451830, cacheId=1489451830]], policy=4] at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1151) at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1215) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.send(GridCacheIoManager.java:836) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtAssignmentFetchFuture.requestFromNextNode(GridDhtAssignmentFetchFuture.java:185) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtAssignmentFetchFuture.init(GridDhtAssignmentFetchFuture.java:107) at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.fetchAffinityOnJoin(CacheAffinitySharedManager.java:953) at org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onClientEvent(CacheAffinitySharedManager.java:639) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onClientNodeEvent(GridDhtPartitionsExchangeFuture.java:619) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:464) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:1453) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:745) Caused by: class org.apache.ignite.spi.IgniteSpiException: Failed to send message to remote node: TcpDiscoveryNode [id=c02cdaa3-80de-4b81-884f-ca9ba830dba5, addrs=[127.0.0.1, 172.25.1.32], sockAddrs=[/172.25.1.32:47500, /127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1478178315859, loc=false, ver=1.7.0#20161031-sha1:6b78ad0c, isClient=false] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:2017) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage(TcpCommunicationSpi.java:1955) at org.apache.ignite.internal.managers.communication.GridIoManager.send(GridIoManager.java:1146) ... 11 more Caused by: class org.apache.ignite.IgniteCheckedException: Failed to connect to node (is node still alive?). Make sure that each ComputeTask and GridCacheTransaction has a timeout set in order to prevent parties from waiting forever in case of network issues [nodeId=c02cdaa3-80de-4b81-884f-ca9ba830dba5, addrs=[/172.25.1.32:47100, /127.0.0.1:47100]] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2521) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2161) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2055) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.sendMessage0(TcpCommunicationSpi.java:1989) ... 13 more Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to connect to address: /172.25.1.32:47100 at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2526) ... 16 more Caused by: class org.apache.ignite.IgniteCheckedException: Failed to read remote node response (connection closed). at org.apache.ignite.internal.util.nio.ssl.BlockingSslHandler.readFromNet(BlockingSslHandler.java:496) at org.apache.ignite.internal.util.nio.ssl.BlockingSslHandler.unwrapHandshake(BlockingSslHandler.java:377) at org.apache.ignite.internal.util.nio.ssl.BlockingSslHandler.handshake(BlockingSslHandler.java:160) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeHandshake(TcpCommunicationSpi.java:2602) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2398) ... 16 more Suppressed: class org.apache.ignite.IgniteCheckedException: Failed to connect to address: /127.0.0.1:47100 at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2526) ... 16 more Caused by: class org.apache.ignite.IgniteCheckedException: Remote node ID is not as expected [expected=c02cdaa3-80de-4b81-884f-ca9ba830dba5, rcvd=a90809f8-b7f0-44ea-b78b-b8eb6c642f8f] at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.safeHandshake(TcpCommunicationSpi.java:2638) at org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:2398) ... 16 more {noformat} server: {noformat} [16:05:19,037][WARN ][grid-nio-worker-3-#12%null%][TcpCommunicationSpi] Closing NIO session because of unhandled exception [cls=class o.a.i.i.util.nio.GridNioException, msg=Failed to decode SSL data: GridSelectorNioSessionImpl [selectorIdx=3, queueSize=0, writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], readBuf=java.nio.DirectByteBuffer[pos=82 lim=82 cap=32768], recovery=null, super=GridNioSessionImpl [locAddr=/172.25.1.32:47100, rmtAddr=/172.25.1.31:41986, createTime=1478178318962, closeTime=0, bytesSent=3049, bytesRcvd=280, sndSchedTime=1478178318962, lastSndTime=1478178319022, lastRcvTime=1478178319032, readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter [parser=o.a.i.i.util.nio.GridDirectParser@b9e19da, directMode=true], GridConnectionBytesVerifyFilter, SSL filter], accepted=true]]] {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)