This is an automated email from the ASF dual-hosted git repository. krickert pushed a commit to branch feat/embeddings-timings-tests in repository https://gitbox.apache.org/repos/asf/opennlp.git
commit b26898cacaf6e40be9e807971d36cb21525bb849 Author: Kristian Rickert <[email protected]> AuthorDate: Sun Oct 11 02:18:44 2026 -0400 Document the SentenceVectorsDL benchmark and allocation numbers Signed-off-by: Kristian Rickert <[email protected]> --- opennlp-core/opennlp-ml/opennlp-dl/BENCHMARKS.md | 64 ++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/opennlp-core/opennlp-ml/opennlp-dl/BENCHMARKS.md b/opennlp-core/opennlp-ml/opennlp-dl/BENCHMARKS.md new file mode 100644 index 000000000..82f0e79fa --- /dev/null +++ b/opennlp-core/opennlp-ml/opennlp-dl/BENCHMARKS.md @@ -0,0 +1,64 @@ +<!-- +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to You under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +--> + +# Embedding Benchmarks + +`SentenceVectorsDLBenchmark` times `SentenceVectorsDL` on a real model: `embedEach` calls +`embed` once per text, `embedAll` makes one call. A call holds `callSize` texts of 4 to 40 +words, drawn with a fixed seed, so tokenized lengths are mixed as in natural text. + +The model directory holds `model.onnx` and `vocab.txt` (for example all-MiniLM-L6-v2) and +defaults to `~/.opennlp/onnx/sentence-transformers`; set it with `-p modelDir=<dir>`. + +## Building and running + +```bash +mvn test-compile -Pjmh -pl opennlp-core/opennlp-ml/opennlp-dl -am -Drat.skip=true + +mvn dependency:build-classpath -pl opennlp-core/opennlp-ml/opennlp-dl \ + -Pjmh -DincludeScope=test -Dmdep.outputFile=/tmp/cp.txt + +CP="opennlp-core/opennlp-ml/opennlp-dl/target/classes:opennlp-core/opennlp-ml/opennlp-dl/target/test-classes:$(cat /tmp/cp.txt)" + +# CPU, with the bytes allocated per call +java -cp "$CP" org.openjdk.jmh.Main SentenceVectorsDLBenchmark -prof gc +``` + +For CUDA, replace the `onnxruntime` jar in `CP` with `onnxruntime_gpu` of the same version, +put the CUDA and cuDNN libraries that release needs on `LD_LIBRARY_PATH`, and add +`-p device=gpu`. Build only `opennlp-dl` with `-Pjmh`: the generated `*_jmhTest` classes land +in its test-jar, which `opennlp-dl-gpu` scans for tests. + +## Allocation per call (all-MiniLM-L6-v2, JDK 21, RTX 4080 SUPER, 1 fork x 5 iterations) + +`gc.alloc.rate.norm` before and after the inputs and outputs moved to direct buffers. + +| Benchmark | callSize | GPU before | GPU after | CPU before | CPU after | +|-----------|---------:|-----------:|----------:|-----------:|----------:| +| `embedAll` | 1 | 50 KB | 21 KB | 54 KB | 17 KB | +| `embedAll` | 8 | 384 KB | 136 KB | 410 KB | 161 KB | +| `embedAll` | 32 | 2037 KB | 767 KB | 2039 KB | 769 KB | +| `embedAll` | 128 | 7904 KB | 2424 KB | 7908 KB | 2862 KB | +| `embedEach` | 1 | 53 KB | 17 KB | 53 KB | 21 KB | +| `embedEach` | 8 | 411 KB | 162 KB | 410 KB | 162 KB | +| `embedEach` | 32 | 1930 KB | 656 KB | 2062 KB | 787 KB | +| `embedEach` | 128 | 7591 KB | 2597 KB | 8121 KB | 3035 KB | + +Average time per call on the GPU did not change (it is bound by inference). `embedAll` at +callSize 128 takes 23.8 ms against 67.4 ms for `embedEach`. CPU times vary by more than 50 +percent between iterations with ONNX Runtime's default thread settings, so they are not +listed.
