NEON non-strictly ordered floating-point reduction instruction rules are updated to avoid materialising a redundant `fadd`, and additionally now use an immediate operand instead of a register.
Vector API and auto-vectorisation share the AddReductionV node which has to support the needs of both - it models a scalar value plus a vector operand. Since `reduceLanes()` has no scalar source the Vector API intrinsic generates an identity scalar value which is passed as the first input to AddReductionVF/VD. Because addition with the identity value is unnecessary when implementing a Vector API reduction, NEON non-strict floating-point reductions no longer generate an extra instruction. Furthermore, the matcher has been updated to use zero-constrained immediate operands to guarantee that the scalar value is zero. Update the predicate of `reduce_non_strict_order_add2D_neon` to restrict it to 128-bit vectors, ensuring vectors wider than 128 bits select `reduce_addD_sve` for correct code generation on all SVE-capable systems. Before: faddp v26.4s, v18.4s, v18.4s faddp s19, v26.2s fadd s19, s19, s17 // accum(s19) += s17(value 0.0) After: faddp v26.4s, v18.4s, v18.4s faddp s19, v26.2s Throughput benchmarks are split into 2 different types, based on the reduction benchmarks in `test/micro/org/openjdk/bench/vm/compiler/VectorAlgorithmsImpl.java` 1. A naive version which performs lane reductions inside the loop, e.g. `vectorDoubleAddReduce_naive` 2. 'reduction_after_loop' version which performs element-wise addition inside the loop, followed by a single `reduceLanes` after the loop, e.g. `vectorDoubleAddReduce_reduction_after_loop` Benchmarking has been conducted on Arm Neoverse N1, Neoverse V2, as well as Arm Cortex-A725, Cortex-X925 CPUs. Mode: thrpt Units: ops/us ObjectAlignmentInBytes: 64 Number is speed-up, higher number is better | Benchmark | Type | Count | A725 | X925 | N1 | V2 | | naive | Double | 128 | 1.0 | 1.0 | 1.0 | 1.0 | | naive | Double | 512 | 1.0 | 1.0 | 1.0 | 1.0 | | naive | Double | 2048 | 1.0 | 1.0 | 1.0 | 1.0 | | naive | Double | 4096 | 1.0 | 1.01 | 1.0 | 1.0 | | reduction_after_loop | Double | 128 | 1.02 | 1.02 | 1.03 | 1.01 | | reduction_after_loop | Double | 512 | 1.0 | 0.99 | 1.0 | 1.0 | | reduction_after_loop | Double | 2048 | 1.0 | 1.0 | 1.0 | 1.0 | | reduction_after_loop | Double | 4096 | 1.0 | 1.0 | 1.0 | 1.0 | | naive | Float | 128 | 1.02 | 1.01 | 1.21 | 1.0 | | naive | Float | 512 | 1.02 | 1.0 | 1.16 | 1.0 | | naive | Float | 2048 | 1.02 | 1.0 | 1.14 | 1.0 | | naive | Float | 4096 | 1.06 | 1.01 | 1.14 | 1.0 | | reduction_after_loop | Float | 128 | 1.02 | 1.02 | 1.03 | 1.01 | | reduction_after_loop | Float | 512 | 1.01 | 1.01 | 1.0 | 1.01 | | reduction_after_loop | Float | 2048 | 1.0 | 1.0 | 1.0 | 1.0 | | reduction_after_loop | Float | 4096 | 1.0 | 1.0 | 1.0 | 1.0 | Speed-up is observed on Neoverse N1 for the naive Float benchmark, as well as a slight general improvement for low trip counts of reduction_after_loop. As well as, a latency benchmark which uses reduction results in subsequent reductions, which forms a latency-sensitive instruction sequence. Mode: avgt Units: ns/op ObjectAlignmentInBytes: 64 Number is reduction in time, lower number is better | Count | Type | A725 | X925 | N1 | V2 | | 128 | Double | 0.73 | 0.64 | 0.74 | 0.73 | | 512 | Double | 0.75 | 0.73 | 0.75 | 0.75 | | 2048 | Double | 0.75 | 0.74 | 0.75 | 0.75 | | 4096 | Double | 0.75 | 0.75 | 0.75 | 0.75 | | 128 | Float | 0.8 | 0.76 | 0.81 | 0.8 | | 512 | Float | 0.82 | 0.8 | 0.82 | 0.82 | | 2048 | Float | 0.82 | 0.82 | 0.82 | 0.82 | | 4096 | Float | 0.82 | 0.82 | 0.82 | 0.82 | --------- - [x] I confirm that I make this contribution in accordance with the [OpenJDK Interim AI Policy](https://openjdk.org/legal/ai). ------------- Commit messages: - 8392077: AArch64: Remove redundant FADD in floating-point add reductions on NEON Changes: https://git.openjdk.org/jdk/pull/33319/files Webrev: https://webrevs.openjdk.org/?repo=jdk&pr=33319&range=00 Issue: https://bugs.openjdk.org/browse/JDK-8392077 Stats: 192 lines in 3 files changed: 154 ins; 8 del; 30 mod Patch: https://git.openjdk.org/jdk/pull/33319.diff Fetch: git fetch https://git.openjdk.org/jdk.git pull/33319/head:pull/33319 PR: https://git.openjdk.org/jdk/pull/33319
