anton-vinogradov commented on PR #13422:
URL: https://github.com/apache/ignite/pull/13422#issuecomment-5169184012
### Measured: is there a performance drop from `GridUnsafe` to `VarHandle`?
Short answer: **yes on x86, no on aarch64 — and on aarch64 it is a large
win.** The absolute cost on x86 is about 1.5 ns per message, so it does not
show up next to a transaction commit.
JMH, JDK 17 (Corretto), Apple aarch64, 1 thread, `AverageTime`, ns/op, lower
is better. `items` is the number of partitions in one message.
**The x86 code path** (`-DIGNITE_MEMORY_UNALIGNED_ACCESS=true`, so
`GridUnsafe` takes the raw `UNSAFE.getInt`/`getLong` branch):
| benchmark | items | GridUnsafe | VarHandle | change |
|---|---|---|---|---|
| read | 4 | 2.426 | 3.553 | +46% |
| read | 32 | 18.698 | 25.881 | +38% |
| read | 1024 | 1073.1 | 1118.5 | +4% |
| write | 4 | 2.006 | 3.500 | +74% |
| write | 32 | 12.960 | 19.668 | +52% |
| write | 1024 | 420.0 | 605.6 | +44% |
The percentages look big, the absolute numbers do not: a transaction touches
a handful of partitions, so the whole message write goes from 2.0 ns to 3.5 ns.
The extra work is the bounds check that `Unsafe` skips - the same check that
turns the out-of-bounds write described in the description from a silently lost
value into an exception.
**The aarch64 path as it ships today** (default, no property):
| benchmark | items | GridUnsafe | VarHandle | change |
|---|---|---|---|---|
| read | 4 | 9.270 | 3.444 | −63% |
| read | 32 | 71.793 | 25.490 | −65% |
| read | 1024 | 2366.0 | 1097.4 | −54% |
| write | 4 | 10.486 | 3.642 | −65% |
| write | 32 | 84.026 | 19.274 | −77% |
| write | 1024 | 2669.7 | 593.3 | −78% |
The reason is `GridUnsafe#unaligned()`: it returns `true` only for `i386`,
`x86`, `amd64` and `x86_64`, and otherwise falls back to the
`IGNITE_MEMORY_UNALIGNED_ACCESS` property, which is `false` by default. So on
aarch64 every `getInt`/`getLong`/`putInt`/`putLong` over a `byte[]` is
assembled byte by byte. The `VarHandle` view does not have that split and is 2
to 4.5 times faster there.
Two caveats on the numbers. Both runs are on aarch64 hardware; the first one
forces the same code branch x86 takes, but it is not a substitute for a run on
x86 silicon. And a microbenchmark of three field accesses is easy to over-read
- the point of the tables is the order of magnitude, not the exact percent.
<details>
<summary>Benchmark source (not part of this PR)</summary>
Put it in
`modules/benchmarks/src/main/java/org/apache/ignite/internal/benchmarks/jmh/misc/`,
build with `-Pbenchmarks`, and run with the project's `--add-opens` list,
otherwise `GridUnsafe` fails to initialize and only the `VarHandle` benchmarks
run.
```java
@State(Scope.Thread)
@Fork(1)
@Threads(1)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@Warmup(iterations = 3, time = 1)
@Measurement(iterations = 5, time = 1)
public class JmhCounterPackingBenchmark {
private static final int ITEM_SIZE = 4 + 8 + 8;
private static final VarHandle INT_VIEW =
MethodHandles.byteArrayViewVarHandle(int[].class, ByteOrder.LITTLE_ENDIAN);
private static final VarHandle LONG_VIEW =
MethodHandles.byteArrayViewVarHandle(long[].class, ByteOrder.LITTLE_ENDIAN);
@Param({"4", "32", "1024"})
private int items;
private byte[] data;
@Setup
public void setup() {
data = new byte[items * ITEM_SIZE];
for (int i = 0; i < items; i++)
addVarHandle(i, i, i * 10L, i * 100L);
}
private void addUnsafe(int idx, int part, long init, long updatesCnt) {
long off = GridUnsafe.BYTE_ARR_OFF + (long)idx * ITEM_SIZE;
GridUnsafe.putInt(data, off, part); off += 4;
GridUnsafe.putLong(data, off, init); off += 8;
GridUnsafe.putLong(data, off, updatesCnt);
}
private void addVarHandle(int idx, int part, long init, long updatesCnt)
{
int off = idx * ITEM_SIZE;
INT_VIEW.set(data, off, part);
LONG_VIEW.set(data, off + 4, init);
LONG_VIEW.set(data, off + 12, updatesCnt);
}
@Benchmark
public byte[] writeUnsafe() {
for (int i = 0; i < items; i++)
addUnsafe(i, i, i * 10L, i * 100L);
return data;
}
@Benchmark
public byte[] writeVarHandle() {
for (int i = 0; i < items; i++)
addVarHandle(i, i, i * 10L, i * 100L);
return data;
}
@Benchmark
public long readUnsafe() {
long res = 0;
for (int i = 0; i < items; i++) {
long off = GridUnsafe.BYTE_ARR_OFF + (long)i * ITEM_SIZE;
res += GridUnsafe.getInt(data, off);
res += GridUnsafe.getLong(data, off + 4);
res += GridUnsafe.getLong(data, off + 12);
}
return res;
}
@Benchmark
public long readVarHandle() {
long res = 0;
for (int i = 0; i < items; i++) {
int off = i * ITEM_SIZE;
res += (int)INT_VIEW.get(data, off);
res += (long)LONG_VIEW.get(data, off + 4);
res += (long)LONG_VIEW.get(data, off + 12);
}
return res;
}
}
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
