Hi Team, I am new bee to ssl/openssl. Need some help to resolve the below problem.
I have server process that talks to client on end-host. It supports 1000/3000 or more client connection. I can do 6000+ (Non-SSL) client connection and communication without issue. The client connecting rate is 4~5 clients per second. I use simulated client to do testing of bulk client (1000/3000 or more) connection and communication. This simulated client run on linux (test) host. Each client is separate process running on linux test host. I have introduced SSL communication between this client and server connection. And I have done 1000 (SSL) connection and communication without issue. The client connecting rate was 4~5 per second. #1 I tried to do 3000 (SSL) client connection. Around 2500+ client are successfull and there SSL communication goes through. I tried to debug why remaining client connection didnot go through. After debugging why the remaining client didnot go through, I see the below issue. There was no (client) process running on linux test host. And server was not accepting new SSL connection. So I tried the below, #2 Do single client connection. Client was doing TCP connection and getting stuck in SSL_connect call. In the Pcap I could see client sending TLSv1 Client Hello message and server was not responding for the same. In the same packet capture I could see server sending TLSv1 Server Hello message for client connection (IP) which didnot exists (Please note: this client connection (IP) were used by me in 3000 users test done #1 above). After looking at the /var/log/messages I could see server was trying to do SSL connection (SSL_accept) with this non-existing clients. I did netstat for the IP's I have used for my testing and this IP's were seen in netstat output. Later in the packet capture I could see server was sending TLSv1 Server Hello message to each client which I have seen in netstat output. Server was failing for this clients with error=5 (error number return by SSL_get_error API). And close call came out with errno=110 which is connection timed out. Now please I need help to fix this. 1. Why server is holding on this client connections (Even after I did interface down/up) ? 2. How can make the server not do this? 3. Can I set some kind of timeout in server? 4. Any other option I need to set to avoid this situation? And also I would like to know why this is happening. Please let me know any other information is needed. Like Pcap, SSL Log etc. Regards ~Nazir