Hi Martyn,
after a while of stress-test sessions with the new 2.1.0-SNAPSHOT of
Artemis, I'm fairly confident to say that ram consumption now seems ok.
Currently, instead, we're facing some problems with cpu usage during clients
connections.
Our stress test program, at the beginning, initializes 1500 subscriber
instances: then try to connects them all, at a rate of one every 500 ms.
Clients connect using tls (only server side, not mutual auth) with user,
passwd and clientid.
For every client, our Artemis authenticates it by matching user and passwd
on a redis backend and loading the user's role in session.
After the connection succeeds, each client subscribes 50 topics. Here
Artemis authorizes the subscription, by checking the user role and related
permissions on topics: this information are loaded from redis at the startup
and refreshed every time there's a change on redis (e.g. if a new user or
role is added).
The server we're using (where we installed Artemis) is an aws ec2 instance
(m4.xlarge) with the following properties:
vCPU = 4 (Intel Xeon 2.40GHz)
RAM = 16GB
Disk = SSD
Bandwidth = 750 Mbps
OS = CentOS Linux release 7.3.1611
Kernel = 3.10.0-514.2.2.el7.x86_64
What we notice is that during clients' connection the CPU of our machine
goes to 100%. 400% considering 4 vCPU (almost 380% used by Artemis):
top - 10:49:32 up 6 days, 1:03, 1 user, load average: 12.16, 11.93, 11.12
Tasks: 105 total, 4 running, 101 sleeping, 0 stopped, 0 zombie
%Cpu0 : 99.0 us, 0.0 sy, 0.0 ni, 1.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0
st
%Cpu1 : 99.0 us, 1.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0
st
%Cpu2 : 99.0 us, 0.0 sy, 0.0 ni, 1.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0
st
%Cpu3 : 97.0 us, 1.0 sy, 0.0 ni, 1.0 id, 0.0 wa, 0.0 hi, 1.0 si, 0.0
st
KiB Mem : 16004820 total, 8214776 free, 7183864 used, 606180 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 8439988 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16015 mqtt 20 0 12.637g 6.641g 15784 S 378.2 43.5 221:00.54 java
19229 mqtt 20 0 138964 3832 1212 R 15.8 0.0 14:49.52
redis-server
This results in a lot of "Connection timeOut" at client side. Usually are
needed about 40 minutes before going to the regime, hence before that all
1500 subscribers have been all able to get their connections. And even after
reaching the regime, for a certain time the cpu usage remain at 100%, before
to decrease.
Is this normal? There are no errors in artemis.log.
I've the suspect to have misconfigured something at Artemis side. Attached
there's my broker.xml. Maybe you can help me to understand where I'm wrong.
Thanks in advance for your help as usual,
Francesco broker.xml
<http://activemq.2283324.n4.nabble.com/file/n4724552/broker.xml>
--
View this message in context:
http://activemq.2283324.n4.nabble.com/Apache-Artemis-Stress-test-time-tp4723999p4724552.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
