When the server receives requests from the clients, does it really reply in parallell, or does it handle the requests sequencially?
If the latter, improve the performance by dynamically creating connection handlers that process requests from each client individually. Compression is another thing that may improve the performance.