On Tue, 17 Nov 2026 02:23:15 GMT, Larry <[email protected]> wrote:
>> ### Summary
>> This PR adds a non-temporal store path to the x86_64 AVX2 forward primitive
>> arraycopy stubs, extending the existing AVX-512 streaming-store approach to
>> the AVX2 implementation and improving throughput for large streaming copies.
>>
>> ### Problem
>> The existing AVX2 arraycopy implementation suffers from two major
>> performance issues:
>>
>> - Unnecessary write-allocate traffic: temporal destination stores for
>> streaming copies introduce write-allocate traffic and cache pollution
>> - Missing streaming path in AVX2: the AVX-512 implementation already
>> provides a streaming store path for large copies, but the AVX2
>> implementation does not
>>
>> ### Changes
>> Added a non-temporal store path for large primitive array copies in the AVX2
>> forward stubs
>>
>> The new loop aligns the destination to 32 bytes and copies 64 bytes per
>> iteration using two 256-bit loads and two VMOVNTDQ stores, followed by an
>> SFENCE. Alignment and trailing elements use temporal stores, with each
>> source element copied once to preserve correctness for forward overlapping
>> copies.
>>
>> `ArrayCopyNonTemporalThreshold` specifies the selection threshold in bytes.
>> When unspecified, it is initialized from the detected L3 cache size:
>> standard CPUID leaf 4 on Intel and other applicable x86 processors, and
>> extended leaf 0x80000006 on AMD/Hygon. An explicit nonzero value overrides
>> detection; explicit zero disables the new path; failed detection leaves the
>> default at zero. The threshold is included in AOT code-cache configuration
>> checks to prevent reuse of stubs generated with a different setting.
>>
>> The path requires AVX2 and `UseUnalignedLoadStores`. Reference array copies,
>> backward copies, and the existing AVX-512 paths retain their current
>> implementations.
>>
>> ### Testing
>> JMH benchmarks
>>
>> Measurements were taken on a two-socket Intel Xeon Platinum 8558 (Sapphire
>> Rapids) system with a reported 260 MiB L3 cache per socket, running CentOS
>> 8. The benchmark repeatedly copies a source byte array into two alternating
>> destination arrays and measures throughput in operations per second at 0.5x,
>> 1x, 2x, and 4x the reported per-socket L3 size.
>>
>> The machine supports AVX-512 and defaults to `UseAVX=3`; these runs
>> explicitly use `-XX:UseAVX=2` to exercise the AVX2 implementation, with
>> `ArrayCopyNonTemporalThreshold` automatically detected at 272629760 bytes
>> (260 MiB).
>>
>> | Copy size | Baseline, ops/s | Patched, ops/s | Speedup |
>> |-----------|----------------:|---------------:|--------:|
>> ...
>
> Larry has updated the pull request incrementally with one additional commit
> since the last revision:
>
> 8394825: Add jtreg test for non-temporal arraycopy
Changes requested by mhaessig (Reviewer).
test/hotspot/jtreg/compiler/arraycopy/TestArrayCopyNonTemporal.java line 46:
> 44: * -XX:ArrayCopyNonTemporalThreshold=65536
> 45: * compiler.arraycopy.TestArrayCopyNonTemporal
> 46: */
Ideally, we'd like all of our tests to run on all possible platforms. Further,
this could also serve as an additional test for the AVX-512 path. I would
propose that this tests has multiple run statements with different requirements:
1. An x64 AVX2 test run
2. An x64 AVX3 test run
3. An non-x64 test run
Lastly, please use the jtreg variable `${test.main.class}` instead of writing
the class manually. This avoids typos.
-------------
PR Review: https://git.openjdk.org/jdk/pull/33437#pullrequestreview-5973934330
PR Review Comment: https://git.openjdk.org/jdk/pull/33437#discussion_r4627887591