Hi,
I am facing a big problem in our application. It uses a cluster of 3 brokers
configured with a network connector. We have a huge number of clients around
10000 connected to theses brokers. Each client opens 2 connections to send
and receive messages. We use a powerful machines running on Solaris 10 64
bits. The issue is well known: Too many open files
We checked the current number of open files in some cases and it is usually
around 58000 in some machines. Bellow, we provide information about tcp
parameters and open connections in Solaris.
Broker1
Broker2 Broker3
tcp_fin_wait_2_flush_interval 16000 16000
16000
tcp_keepalive_interval 15000 15000
15000
tcp_conn_req_max_q 8192
8192 8192
tcp_conn_req_max_q0 8192
8192 8192
tcp_time_wait_interval 6000 6000
6000
Conexiones ESTABLISHED 6188 6791
6178
Conexiones CLOSE_WAIT 162
51 20
As you can see, the number of connections in CLOSE_WAIT is low and I think
that the most influential tcp parameters have a reasonable value. Our
application also uses a cluster of Hazelcast in-memory data grid although I
don’t think that it has to do with our issue.
I am disconcerted with the huge number of open files and I don’t know how to
reduce it despite the adjustments we already did in broker configuration.
The clients that cannot connect to a given broker reattempt every 2 minutes.
client.uri =
failover:(ssl://broker1:61616?soTimeout=10000&wireFormat.maxInactivityDuration=600000,ssl://broker2:61616?soTimeout=10000&wireFormat.maxInactivityDuration=600000,ssl://broker3:61616?soTimeout=10000&wireFormat.maxInactivityDuration=600000)?jms.useCompression=true&jms.alwaysSyncSend=true&startupMaxReconnectAttempts=5&maxReconnectAttempts=5&randomize=true&timeout=4000
For broker1 for example:
1 queue and 1topic are used.
dedicatedTaskRunner = false
persistent = false
networkConnector.URI=
static:(tcp://${broker2}:61617?soTimeout=10000&soWriteTimeout=15000,tcp://${broker3}:61617?soTimeout=10000&soWriteTimeout=15000)
networkConnector.dynamicOnly =false
networkConnector.networkTTL =3
networkConnector.decreaseNetworkConsumerPriority =true;
networkConnector.suppressDuplicateQueueSubscriptions = true;
networkConnector.suppressDuplicateTopicSubscriptions = true;
networkConnector.staticBridge = true;
transportConnector.URI=
nio+ssl://${broker1}:61616?transport.soTimeout=10000&transport.soWriteTimeout=15000&wireFormat.maxInactivityDuration=600000&useQueueForAccept=false&transport.maximumConnections=10000&transport.closeAsync=false
Hope someone can help me to solver this problem, at least to know how to
reduce the number of open files.
Thank you
--
View this message in context:
http://activemq.2283324.n4.nabble.com/Too-many-open-files-with-thousands-connections-in-Solaris-tp4695069.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
