I've gotten complaints from Gerrit users that the SSH port is horribly slow. An operation that normally takes 30 seconds through OpenSSH could take 30 minutes on Gerrit, which uses MINA SSHD for its SSH port.
More specifically, this operation is opening and closing over 120 unique SSH connections in rapid succession. One of the large bottlenecks in SSHD was the use of SecureRandom.generateSeed() on every request. I suspect the Sun JRE is reading /dev/random on each call, and the kernel just doesn't have enough entropy on hand so it blocks and waits for more. https://issues.apache.org/jira/browse/SSHD-13 has a patch which obtains a new seed every 100 requests, and in between uses a PRNG instead. It cuts quite a bit off the wall-clock time. With this patch in place, SSHD is now about 2x slower than OpenSSH. The remaining bottleneck appears to be Sun's BigInteger.modPow() function, used by BouncyCastle during the DH key exchange. That single method is 60% of the running time for a setup-exec-close done over 600 times as quickly as possible.
