Under certain load, MemoryCache operations take a substantial fraction of the time needed to complete SSL handshakes. This series of patches improves performance characteristics of MemoryCache, at the cost of a functional change: expired entries are no longer guaranteed to be removed before live ones. Unused entries are still removed before used ones, and cache performance no longer depends on its capacity.
First patch in the series contains a benchmark that can be run with `make test TEST="micro:CacheBench"`. Baseline results before any MemoryCache changes: Benchmark (size) (timeout) Mode Cnt Score Error Units CacheBench.put 20480 86400 avgt 25 83.653 ? 6.269 us/op CacheBench.put 20480 0 avgt 25 0.107 ? 0.001 us/op CacheBench.put 204800 86400 avgt 25 2057.781 ? 35.942 us/op CacheBench.put 204800 0 avgt 25 0.108 ? 0.001 us/op there's a nonlinear performance drop between 20480 and 204800 entries, probably attributable to CPU cache thrashing. Beyond 204800 entries the cache scales more linearly. Benchmark results after the 2nd and 3rd patches are pretty similar, so I'll only copy one: Benchmark (size) (timeout) Mode Cnt Score Error Units CacheBench.put 20480 86400 avgt 25 0.146 ? 0.002 us/op CacheBench.put 20480 0 avgt 25 0.108 ? 0.002 us/op CacheBench.put 204800 86400 avgt 25 0.150 ? 0.001 us/op CacheBench.put 204800 0 avgt 25 0.106 ? 0.001 us/op The third patch improves worst-case times on a mostly idle cache by scattering removal of expired entries over multiple `put` calls. It does not affect performance of an overloaded cache. The 4th patch removes all code that clears cached values before handing them over to the GC. [This comment](https://github.com/openjdk/jdk/commit/5859a0320334bfb6b46b62eb16b4c387641f4a2a#diff-c6bd583a97fbc4f471621fee7eab37c63718cdb6932ce357fa403cfda4b32b6fL346) stated that clearing values was supposed to be a GC performance optimization. It wasn't. Benchmark results after that commit: Benchmark (size) (timeout) Mode Cnt Score Error Units CacheBench.put 20480 86400 avgt 25 0.113 ? 0.001 us/op CacheBench.put 20480 0 avgt 25 0.075 ? 0.002 us/op CacheBench.put 204800 86400 avgt 25 0.116 ? 0.001 us/op CacheBench.put 204800 0 avgt 25 0.072 ? 0.001 us/op I wasn't expecting that much of an improvement, and don't know how to explain it. The 40ns difference between cache with and without a timeout can be attributed to 2 `System.currentTimeMillis()` calls; they were pretty slow on my VM. ------------- Commit messages: - Do not invalidate objects before GC - Always expunge on put - Stop scanning expired entries after first non-expired one - Add cache microbenchmark Changes: https://git.openjdk.java.net/jdk/pull/2255/files Webrev: https://webrevs.openjdk.java.net/?repo=jdk&pr=2255&range=00 Issue: https://bugs.openjdk.java.net/browse/JDK-8259886 Stats: 138 lines in 3 files changed: 85 ins; 40 del; 13 mod Patch: https://git.openjdk.java.net/jdk/pull/2255.diff Fetch: git fetch https://git.openjdk.java.net/jdk pull/2255/head:pull/2255 PR: https://git.openjdk.java.net/jdk/pull/2255
