czldb2 commented on issue #524: URL: https://github.com/apache/mina-sshd/issues/524#issuecomment-2235000412
> I found a couple of things that could be improved; for details see PR #530. > > While comparing out-of-the-box performance of JSch and sshd is legitimate, one needs to be aware that they are using different encryption algorithms by default. JSch uses the AES implementation from the built-in SunJCE provider (that's native code), sshd uses a Java implementation of the ChaCha20-Poly1305 cipher (if the server supports it, as does OpenSSH). > > So, three findings: > > 1. It turned out that our ChaCha20-Poly1305 implementation was very clean, but not speed-optimized. PR [Performance optimizations #530](https://github.com/apache/mina-sshd/pull/530) optimizes this, and it is now faster than the BC implementation, and nearly as fast as the native code AES. > 2. A second area improved in PR [Performance optimizations #530](https://github.com/apache/mina-sshd/pull/530) is general buffer handling in sshd. Sometimes data was copied needlessly from one buffer into another. However, in the grand scheme of things these buffer optimizations are only a (vanishingly) small factor. > 3. Thirdly, I discovered that if Bouncy Castle was available, and an AES encryption was used for the connection, sshd would use the Bouncy Castle AES, not the one from SunJCE. The AES in BouncyCastle 1.78.1 is a Java implementation and thus much slower. That is also fixed in PR [Performance optimizations #530](https://github.com/apache/mina-sshd/pull/530); sshd now prefers the SunJCE AES over the one from Bouncy Castle. > > I think we basically have two kinds of reports: > > * Some that see sshd as about 10-50% slower than JSch. Most likely that's points (1) and (2), and comparing AES vs. ChaCha20-Poly1305. > * Some that see sshd as 200-300% (or even more) slower than JSch. Most likely that's point (3): sshd using the Bouncy Castle AES. > > Performance testing SFTP uploads is difficult. I used JMH benchmarks measuring the performance of only the upload itself (excluding SSH session setup, authentication, and setting up the SFTP channel), with a local JSch or sshd client uploading a 20MB random file to an OpenSSH server. To really benchmark this, one would need a lab setup with a controlled environment: a second machine running the SFTP server, and a local network connection with stable and known throughput and latency. > > I do not have such a setup. I used mostly a local container running the SFTP server, which of course introduced uncontrollable variation in timings. After all, the same machine then also runs the server, in a container, so there's unpredictable virtualization overhead. The timings I got from this were still surprisingly consistent and enabled me to find the above three problems. With PR #530 applied, JSch and sshd perform equally well in these tests for me. (Uploading 20MBs takes generally about 270-290ms on the machine I tested.) > > Just to double-check I also ran the same benchmarks with an AWS EC2 t4g.small instance for the OpenSSH server. Again, JSch and sshd with PR #530 applied performed equal -- uploading 20MB took 4-5 seconds, but there were some outliers with both JSch or sshd when it took 6-8, or 12, or once even 20 seconds. Of course, these uploads went over the general Internet (and had cross at least two firewalls, both beyond my control), so I cannot possibly say what happened in these outlier cases. There were about 3 such outlier per 15 upload attempts. > > In both setups I still have the feeling that JSch is a teeny-weeny bit faster (if both are using AES), maybe 2-3%, but that's a feeling only; the statistics don't really support that. Variability and standard deviation in the timings is too high. If the feeling is true, though, it might be due to the multi-threaded nature of sshd. Multi-threading makes a lot of sense for an SSH server, but for a client, the single-threaded approach of JSch has perhaps slightly less overhead. In any case I do not see any significant differences anymore. > > Feel free to run your own benchmarks on the code from PR #530 and to report the setups and results. Thank you very much for your help. I will try my best to use the code you provide to conduct some experiments and tests in the future, and then give you feedback -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@mina.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@mina.apache.org For additional commands, e-mail: dev-h...@mina.apache.org
