> ### Summary > This PR adds a non-temporal store path to the x86_64 AVX2 forward primitive > arraycopy stubs, extending the existing AVX-512 streaming-store approach to > the AVX2 implementation and improving throughput for large streaming copies. > > ### Problem > The existing AVX2 arraycopy implementation suffers from two major performance > issues: > > - Unnecessary write-allocate traffic: temporal destination stores for > streaming copies introduce write-allocate traffic and cache pollution > - Missing streaming path in AVX2: the AVX-512 implementation already provides > a streaming store path for large copies, but the AVX2 implementation does not > > ### Changes > Added a non-temporal store path for large primitive array copies in the AVX2 > forward stubs > > The new loop aligns the destination to 32 bytes and copies 64 bytes per > iteration using two 256-bit loads and two VMOVNTDQ stores, followed by an > SFENCE. Alignment and trailing elements use temporal stores, with each source > element copied once to preserve correctness for forward overlapping copies. > > `ArrayCopyNonTemporalThreshold` specifies the selection threshold in bytes. > When unspecified, it is initialized from the detected L3 cache size: standard > CPUID leaf 4 on Intel and other applicable x86 processors, and extended leaf > 0x80000006 on AMD/Hygon. An explicit nonzero value overrides detection; > explicit zero disables the new path; failed detection leaves the default at > zero. The threshold is included in AOT code-cache configuration checks to > prevent reuse of stubs generated with a different setting. > > The path requires AVX2 and `UseUnalignedLoadStores`. Reference array copies, > backward copies, and the existing AVX-512 paths retain their current > implementations. > > ### Testing > JMH benchmarks > > Measurements were taken on a two-socket Intel Xeon Platinum 8558 (Sapphire > Rapids) system with a reported 260 MiB L3 cache per socket, running CentOS 8. > The benchmark repeatedly copies a source byte array into two alternating > destination arrays and measures throughput in operations per second at 0.5x, > 1x, 2x, and 4x the reported per-socket L3 size. > > The machine supports AVX-512 and defaults to `UseAVX=3`; these runs > explicitly use `-XX:UseAVX=2` to exercise the AVX2 implementation, with > `ArrayCopyNonTemporalThreshold` automatically detected at 272629760 bytes > (260 MiB). > > | Copy size | Baseline, ops/s | Patched, ops/s | Speedup | > |-----------|----------------:|---------------:|--------:| > | 130 MiB (0.5x L3) | 47.8 | 53.8 | 1.13x | > | 260 MiB (1...
Larry has updated the pull request incrementally with one additional commit since the last revision: Update test cases ------------- Changes: - all: https://git.openjdk.org/jdk/pull/33437/files - new: https://git.openjdk.org/jdk/pull/33437/files/05d27449..c0868a67 Webrevs: - full: https://webrevs.openjdk.org/?repo=jdk&pr=33437&range=03 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=33437&range=02-03 Stats: 27 lines in 1 file changed: 21 ins; 0 del; 6 mod Patch: https://git.openjdk.org/jdk/pull/33437.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/33437/head:pull/33437 PR: https://git.openjdk.org/jdk/pull/33437
