[
https://issues.apache.org/jira/browse/ARTEMIS-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liviu Citu updated ARTEMIS-4884:
--------------------------------
Description:
We are currently in process of migrating our broker from Classic 5.x to
Artemis. We are currently using CMS C++ client for connecting to the broker
{*}but the issue replicates also with the OpenWire JMS client{*}. Everything
works fine when using non-SSL setup (on both Windows and Linux) but we have
some issues when using SSL on Linux (SSL on Windows is OK).
The initial problem started with the following exceptions on the client side:
{noformat}
024-02-22 09:54:37.377 [ERROR] [activemq_connection.cc:336] CMS exception:
Channel was inactive for too long:
FILE: activemq/core/ActiveMQConnection.cpp, LINE: 1293
FILE: activemq/core/ActiveMQConnection.cpp, LINE: 1371
FILE: activemq/core/ActiveMQConnection.cpp, LINE: 573{noformat}
while on the broker side we had:
{noformat}
2024-03-20 12:29:08,700 ERROR [org.apache.activemq.artemis.core.server]
AMQ224088: Timeout (10 seconds) on acceptor "netty-ssl-acceptor" during
protocol handshake with /10.21.70.53:33053 has occurred.{noformat}
To bypass these we have added the following setting to the *broker.xml*
*netty-ssl-acceptor* acceptor: *handshake-timeout=0*
However now the exceptions we are receiving are:
*+CMS client+*
{noformat}
2024-05-22 09:26:40.842 [ERROR] [activemq_connection.cc:348] CMS exception:
OpenWireFormatNegotiator::onewayWire format negotiation timeout: peer did not
send his wire format.
FILE: activemq/core/ActiveMQConnection.cpp, LINE: 1293
FILE: activemq/core/ActiveMQConnection.cpp, LINE: 1371
FILE: activemq/core/ActiveMQConnection.cpp, LINE: 573{noformat}
+*Java client*+
{noformat}
jakarta.jms.JMSException: Could not connect to broker URL:
ssl://linux_host:61617?keepAlive=true&wireFormat.maxInactivityDuration=0.
Reason: java.net.SocketException: Broken pipe
at
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:423)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:353)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:245)
~[activemq-client-6.1.2.jar!/:6.1.2]
.........................................................................
Caused by: java.net.SocketException: Broken pipe
at java.base/sun.nio.ch.NioSocketImpl.implWrite(NioSocketImpl.java:425) ~[?:?]
at java.base/sun.nio.ch.NioSocketImpl.write(NioSocketImpl.java:445) ~[?:?]
at java.base/sun.nio.ch.NioSocketImpl$2.write(NioSocketImpl.java:831) ~[?:?]
at java.base/java.net.Socket$SocketOutputStream.write(Socket.java:1035) ~[?:?]
at
java.base/sun.security.ssl.SSLSocketOutputRecord.deliver(SSLSocketOutputRecord.java:345)
~[?:?]
at
java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1308)
~[?:?]
at
org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:115)
~[activemq-client-6.1.2.jar!/:6.1.2]
at java.base/java.io.DataOutputStream.flush(DataOutputStream.java:128) ~[?:?]
at
org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:194)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(AbstractInactivityMonitor.java:336)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.AbstractInactivityMonitor.oneway(AbstractInactivityMonitor.java:318)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.WireFormatNegotiator.sendWireFormat(WireFormatNegotiator.java:181)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.WireFormatNegotiator.sendWireFormat(WireFormatNegotiator.java:84)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.WireFormatNegotiator.start(WireFormatNegotiator.java:74)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64)
~[activemq-client-6.1.2.jar!/:6.1.2]
at org.apache.{noformat}
The problem replicates with the following:
* SSL on Linux. Problem does not replicate if non-SSL configuration is used.
Also does not replicate on Windows (regardless if SSL or non-SSL is used)
* *problem does not replicate with Classic Brokers. So it is specific to
Artemis broker*
* when testing with both Classic Broker and Artemis Broker, the client
connections using the Classic Broker were fine. Only those using Artemis Broker
were failing
* Artemis clients are also running on the same same host with the Broker.
Basically both client and server are running on the same host
* there are many connections done in the same time to the broker (25+). If
there are only few then the problem does not happen
* example of connection URL used by the client (the other instance just uses
a different port)
*ssl://linux_host:61617?keepAlive=true&wireFormat.MaxInactivityDuration=0*
* Broker configuration file attached (just mangled the SSL stuff and name of
the host). The other one is similar (different ports)
When monitoring the successful connections I found out that usual connections
took less than 0.5 seconds to succeed. I was unable to find any successful
connection that took more than this.
Looking to the broker logs we are unable to find any relevant message when the
connection fails.
was:
We are currently in process of migrating our broker from Classic 5.x to
Artemis. We are currently using CMS C++ client for connecting to the broker
{*}but the issue replicates also with the OpenWire JMS client{*}. Everything
works fine when using non-SSL setup (on both Windows and Linux) but we have
some issues when using SSL on Linux (SSL on Windows is OK).
The initial problem started with the following exceptions on the client side:
{noformat}
024-02-22 09:54:37.377 [ERROR] [activemq_connection.cc:336] CMS exception:
Channel was inactive for too long:
FILE: activemq/core/ActiveMQConnection.cpp, LINE: 1293
FILE: activemq/core/ActiveMQConnection.cpp, LINE: 1371
FILE: activemq/core/ActiveMQConnection.cpp, LINE: 573{noformat}
while on the broker side we had:
{noformat}
2024-03-20 12:29:08,700 ERROR [org.apache.activemq.artemis.core.server]
AMQ224088: Timeout (10 seconds) on acceptor "netty-ssl-acceptor" during
protocol handshake with /10.21.70.53:33053 has occurred.{noformat}
To bypass these we have added the following setting to the *broker.xml*
*netty-ssl-acceptor* acceptor: *handshake-timeout=0*
However now the exceptions we are receiving are:
*+CMS client+*
{noformat}
2024-05-22 09:26:40.842 [ERROR] [activemq_connection.cc:348] CMS exception:
OpenWireFormatNegotiator::onewayWire format negotiation timeout: peer did not
send his wire format.
FILE: activemq/core/ActiveMQConnection.cpp, LINE: 1293
FILE: activemq/core/ActiveMQConnection.cpp, LINE: 1371
FILE: activemq/core/ActiveMQConnection.cpp, LINE: 573{noformat}
+*Java client*+
{noformat}
jakarta.jms.JMSException: Could not connect to broker URL:
ssl://linux_host:61617?keepAlive=true&wireFormat.maxInactivityDuration=0.
Reason: java.net.SocketException: Broken pipe
at
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:423)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:353)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:245)
~[activemq-client-6.1.2.jar!/:6.1.2]
.........................................................................
Caused by: java.net.SocketException: Broken pipe
at java.base/sun.nio.ch.NioSocketImpl.implWrite(NioSocketImpl.java:425) ~[?:?]
at java.base/sun.nio.ch.NioSocketImpl.write(NioSocketImpl.java:445) ~[?:?]
at java.base/sun.nio.ch.NioSocketImpl$2.write(NioSocketImpl.java:831) ~[?:?]
at java.base/java.net.Socket$SocketOutputStream.write(Socket.java:1035) ~[?:?]
at
java.base/sun.security.ssl.SSLSocketOutputRecord.deliver(SSLSocketOutputRecord.java:345)
~[?:?]
at
java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1308)
~[?:?]
at
org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:115)
~[activemq-client-6.1.2.jar!/:6.1.2]
at java.base/java.io.DataOutputStream.flush(DataOutputStream.java:128) ~[?:?]
at
org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:194)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(AbstractInactivityMonitor.java:336)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.AbstractInactivityMonitor.oneway(AbstractInactivityMonitor.java:318)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.WireFormatNegotiator.sendWireFormat(WireFormatNegotiator.java:181)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.WireFormatNegotiator.sendWireFormat(WireFormatNegotiator.java:84)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.WireFormatNegotiator.start(WireFormatNegotiator.java:74)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64)
~[activemq-client-6.1.2.jar!/:6.1.2]
at
org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64)
~[activemq-client-6.1.2.jar!/:6.1.2]
at org.apache.{noformat}
The problem replicates with the following:
* SSL on Linux. Problem does not replicate if non-SSL configuration is used.
Also does not replicate on Windows (regardless if SSL or non-SSL is used)
* two Artemis instances running on the same Linux host (problem does not
replicate if there is only one Artemis instance running)
* problem also replicates if there is one Artemis Broker and one Classic
Broker instance running on the same host
* *problem does not replicate with two instances of Classic Brokers. So it is
specific to Artemis broker*
* when testing with both Classic Broker and Artemis Broker, the client
connections using the Classic Broker were fine. Only those using Artemis Broker
were failing
* Artemis clients are also running on the same same host with the Broker.
Basically both client and server are running on the same host
* there are many connections done in the same time to the broker (25+). If
there are only few then the problem does not happen
* example of connection URL used by the client (the other instance just uses
a different port)
*ssl://linux_host:61617?keepAlive=true&wireFormat.MaxInactivityDuration=0*
* Broker configuration file attached (just mangled the SSL stuff and name of
the host). The other one is similar (different ports)
When monitoring the successful connections I found out that usual connections
took less than 0.5 seconds to succeed. I was unable to find any successful
connection that took more than this.
Looking to the broker logs we are unable to find any relevant message when the
connection fails.
> [OpenWire] WireFormatNegotiator timeout during multiple parallel SSL
> connections
> --------------------------------------------------------------------------------
>
> Key: ARTEMIS-4884
> URL: https://issues.apache.org/jira/browse/ARTEMIS-4884
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Affects Versions: 2.35.0
> Reporter: Liviu Citu
> Assignee: Justin Bertram
> Priority: Major
> Attachments: activemq-clients.zip, broker.xml, classic-client_1.log,
> classic-client_2.log
>
>
> We are currently in process of migrating our broker from Classic 5.x to
> Artemis. We are currently using CMS C++ client for connecting to the broker
> {*}but the issue replicates also with the OpenWire JMS client{*}. Everything
> works fine when using non-SSL setup (on both Windows and Linux) but we have
> some issues when using SSL on Linux (SSL on Windows is OK).
> The initial problem started with the following exceptions on the client side:
> {noformat}
> 024-02-22 09:54:37.377 [ERROR] [activemq_connection.cc:336] CMS exception:
> Channel was inactive for too long:
> FILE: activemq/core/ActiveMQConnection.cpp, LINE: 1293
> FILE: activemq/core/ActiveMQConnection.cpp, LINE: 1371
> FILE: activemq/core/ActiveMQConnection.cpp, LINE:
> 573{noformat}
> while on the broker side we had:
> {noformat}
> 2024-03-20 12:29:08,700 ERROR [org.apache.activemq.artemis.core.server]
> AMQ224088: Timeout (10 seconds) on acceptor "netty-ssl-acceptor" during
> protocol handshake with /10.21.70.53:33053 has occurred.{noformat}
> To bypass these we have added the following setting to the *broker.xml*
> *netty-ssl-acceptor* acceptor: *handshake-timeout=0*
> However now the exceptions we are receiving are:
> *+CMS client+*
> {noformat}
> 2024-05-22 09:26:40.842 [ERROR] [activemq_connection.cc:348] CMS exception:
> OpenWireFormatNegotiator::onewayWire format negotiation timeout: peer did not
> send his wire format.
> FILE: activemq/core/ActiveMQConnection.cpp, LINE: 1293
> FILE: activemq/core/ActiveMQConnection.cpp, LINE: 1371
> FILE: activemq/core/ActiveMQConnection.cpp, LINE: 573{noformat}
> +*Java client*+
> {noformat}
> jakarta.jms.JMSException: Could not connect to broker URL:
> ssl://linux_host:61617?keepAlive=true&wireFormat.maxInactivityDuration=0.
> Reason: java.net.SocketException: Broken pipe
> at
> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
> ~[activemq-client-6.1.2.jar!/:6.1.2]
> at
> org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:423)
> ~[activemq-client-6.1.2.jar!/:6.1.2]
> at
> org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:353)
> ~[activemq-client-6.1.2.jar!/:6.1.2]
> at
> org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:245)
> ~[activemq-client-6.1.2.jar!/:6.1.2]
> .........................................................................
> Caused by: java.net.SocketException: Broken pipe
> at java.base/sun.nio.ch.NioSocketImpl.implWrite(NioSocketImpl.java:425)
> ~[?:?]
> at java.base/sun.nio.ch.NioSocketImpl.write(NioSocketImpl.java:445) ~[?:?]
> at java.base/sun.nio.ch.NioSocketImpl$2.write(NioSocketImpl.java:831) ~[?:?]
> at java.base/java.net.Socket$SocketOutputStream.write(Socket.java:1035)
> ~[?:?]
> at
> java.base/sun.security.ssl.SSLSocketOutputRecord.deliver(SSLSocketOutputRecord.java:345)
> ~[?:?]
> at
> java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:1308)
> ~[?:?]
> at
> org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:115)
> ~[activemq-client-6.1.2.jar!/:6.1.2]
> at java.base/java.io.DataOutputStream.flush(DataOutputStream.java:128)
> ~[?:?]
> at
> org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:194)
> ~[activemq-client-6.1.2.jar!/:6.1.2]
> at
> org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(AbstractInactivityMonitor.java:336)
> ~[activemq-client-6.1.2.jar!/:6.1.2]
> at
> org.apache.activemq.transport.AbstractInactivityMonitor.oneway(AbstractInactivityMonitor.java:318)
> ~[activemq-client-6.1.2.jar!/:6.1.2]
> at
> org.apache.activemq.transport.WireFormatNegotiator.sendWireFormat(WireFormatNegotiator.java:181)
> ~[activemq-client-6.1.2.jar!/:6.1.2]
> at
> org.apache.activemq.transport.WireFormatNegotiator.sendWireFormat(WireFormatNegotiator.java:84)
> ~[activemq-client-6.1.2.jar!/:6.1.2]
> at
> org.apache.activemq.transport.WireFormatNegotiator.start(WireFormatNegotiator.java:74)
> ~[activemq-client-6.1.2.jar!/:6.1.2]
> at
> org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64)
> ~[activemq-client-6.1.2.jar!/:6.1.2]
> at
> org.apache.activemq.transport.TransportFilter.start(TransportFilter.java:64)
> ~[activemq-client-6.1.2.jar!/:6.1.2]
> at org.apache.{noformat}
> The problem replicates with the following:
> * SSL on Linux. Problem does not replicate if non-SSL configuration is used.
> Also does not replicate on Windows (regardless if SSL or non-SSL is used)
> * *problem does not replicate with Classic Brokers. So it is specific to
> Artemis broker*
> * when testing with both Classic Broker and Artemis Broker, the client
> connections using the Classic Broker were fine. Only those using Artemis
> Broker were failing
> * Artemis clients are also running on the same same host with the Broker.
> Basically both client and server are running on the same host
> * there are many connections done in the same time to the broker (25+). If
> there are only few then the problem does not happen
> * example of connection URL used by the client (the other instance just
> uses a different port)
> *ssl://linux_host:61617?keepAlive=true&wireFormat.MaxInactivityDuration=0*
> * Broker configuration file attached (just mangled the SSL stuff and name of
> the host). The other one is similar (different ports)
> When monitoring the successful connections I found out that usual connections
> took less than 0.5 seconds to succeed. I was unable to find any successful
> connection that took more than this.
> Looking to the broker logs we are unable to find any relevant message when
> the connection fails.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@activemq.apache.org
For additional commands, e-mail: issues-h...@activemq.apache.org
For further information, visit: https://activemq.apache.org/contact
