This is an automated email from the ASF dual-hosted git repository.
tisonkun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datasketches-rust.git
The following commit(s) were added to refs/heads/main by this push:
new 33ec5c9 refactor(benchmarks): measure fixed workloads in milliseconds
(#323)
33ec5c9 is described below
commit 33ec5c9c1df0796b0b4d93e05231b256b73ab53e
Author: tison <[email protected]>
AuthorDate: Wed Nov 4 22:22:42 2026 +0800
refactor(benchmarks): measure fixed workloads in milliseconds (#323)
---
.github/workflows/ci.yml | 2 +
CONTRIBUTING.md | 34 ++++-
Cargo.lock | 50 +------
Cargo.toml | 1 -
benchmarks/Cargo.toml | 8 +-
benchmarks/bloom/mod.rs | 4 +
benchmarks/bloom/update.rs | 40 ++---
benchmarks/countmin/decay.rs | 22 ++-
benchmarks/countmin/estimate.rs | 18 ++-
benchmarks/countmin/mod.rs | 6 +
benchmarks/countmin/update.rs | 18 ++-
benchmarks/cpc/mod.rs | 5 +
benchmarks/cpc/serde.rs | 37 ++---
benchmarks/cpc/update.rs | 18 ++-
benchmarks/frequencies/mod.rs | 4 +
benchmarks/frequencies/update.rs | 18 ++-
benchmarks/harness.rs | 307 ++++++++++++++++++++++++++++++++++++++
benchmarks/hll/mod.rs | 6 +
benchmarks/hll/serde.rs | 31 +++-
benchmarks/hll/union.rs | 57 ++++++-
benchmarks/hll/update.rs | 18 ++-
benchmarks/kll/merge.rs | 24 +--
benchmarks/kll/mod.rs | 7 +
benchmarks/kll/query.rs | 25 +++-
benchmarks/kll/serde.rs | 23 ++-
benchmarks/kll/update.rs | 20 ++-
benchmarks/main.rs | 19 ++-
benchmarks/req/mod.rs | 5 +
benchmarks/req/query.rs | 23 ++-
benchmarks/req/update.rs | 20 ++-
benchmarks/tdigest/aggregation.rs | 161 ++++++++++++--------
benchmarks/tdigest/compress.rs | 44 +++---
benchmarks/tdigest/merge.rs | 257 +++++++++++++++++--------------
benchmarks/tdigest/mod.rs | 9 ++
benchmarks/tdigest/query.rs | 111 ++++++++++----
benchmarks/tdigest/serde.rs | 172 +++++++++++----------
benchmarks/tdigest/update.rs | 148 +++++++++---------
benchmarks/theta/mod.rs | 4 +
benchmarks/theta/update.rs | 18 ++-
benchmarks/tuple/mod.rs | 4 +
benchmarks/tuple/update.rs | 18 ++-
benchmarks/xor/build.rs | 174 +++++++++++----------
benchmarks/xor/matched_build.rs | 159 ++++++++++++--------
benchmarks/xor/mod.rs | 6 +
benchmarks/xor/query.rs | 227 ++++++++++++++++------------
xtask/src/main.rs | 22 ++-
46 files changed, 1586 insertions(+), 818 deletions(-)
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index a83d451..ecd4d8f 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -54,6 +54,8 @@ jobs:
run: cargo x lint
- name: Check feature matrix
run: cargo x check
+ - name: Exercise benchmark workloads
+ run: cargo x bench -- --test
msrv:
name: Resolve MSRV
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 2798be7..05089af 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -67,14 +67,42 @@ Benchmark:
cargo x bench
```
-Benchmarks live in the standalone `benchmarks` crate. Files are grouped first
by sketch and then by workload, for example `benchmarks/cpc/serde.rs` and
`benchmarks/tdigest/update.rs`. Keep distinct workloads in separate modules so
Divan reports stable names such as `cpc::serde::serialize`.
+Benchmarks live in the standalone `benchmarks` crate. Files are grouped first
by sketch and then by workload, for example `benchmarks/cpc/serde.rs` and
`benchmarks/tdigest/update.rs`. Each workload module registers its scenarios
and fixed round counts with the [runner](benchmarks/harness.rs). Keep distinct
workloads in separate modules and use names such as `cpc::serde::serialize/200`
that identify the sketch, operation, and input.
-To run only one sketch or workload, pass a Divan filter directly:
+Compile, list, or smoke-test the benchmarks:
```shell
-cargo bench --package benchmarks --bench benchmarks -- cpc::serde
+cargo x bench --no-run
+cargo x bench -- --list
+cargo x bench -- --test
```
+Smoke mode executes every selected implementation for one outer round,
preserving input sizes, inner loops, and correctness checks. CI runs this mode
without asserting elapsed times. The runner's tests are included in `cargo x
test`.
+
+To select workloads and inspect repeated measurements:
+
+```shell
+cargo x bench -- cpc::serde --repeat 10 --raw
+cargo x bench -- xor::matched_build::8 --repeat 5
+cargo x bench -- 'kll::query::quantile/0.5' --exact --rounds 100000
+```
+
+Filters are substrings; a scenario must match all supplied filters. Use
`--exact` with one complete name to select only that scenario. A filter that
matches nothing fails. Runner options can also be passed directly with `cargo
bench --package benchmarks --bench benchmarks -- ...`.
+
+### Fixed workloads and timing
+
+[Benchmark In
Milliseconds](https://matklad.github.io/2026/10/05/benchmark-milliseconds.html)
recommends sizing useful work to take roughly 300 ms. Choose that work size on
a reference revision, then preserve it across comparisons. The runner never
calibrates rounds automatically: an optimization should reduce elapsed time for
the same work. The initial counts were sized against the library at `5f080f9`
using the timing boundaries below. Hardware and toolchains affect duration, so
300 ms [...]
+
+Each repetition starts the clock before fixture construction and stops after
cleanup. A fixture is built once per repetition and may be reused across its
fixed rounds. Updates rebuild the sketch each round; read-only queries reuse a
prepared sketch; destructive operations clone their inputs each round. Input
generation, cloning, allocations, output destruction, and fixture validation
all count toward elapsed time. No allocation profiler is installed. These
measurements establish a new ba [...]
+
+A round is one invocation of the workload closure, not necessarily one item.
For example, hash-based updates process 10,000 inputs, parameterized updates
process the named input size, XOR queries visit every key in the selected
present or absent set, and aggregation rounds process the entire specified
group of partial states. Keep inner batch sizes and sketch parameters stable
when adjusting outer rounds; changing cardinality can change the algorithmic
path.
+
+The default is five repetitions. Output reports the median and full
minimum-to-maximum range in integer milliseconds. `--raw` includes every
elapsed time in measurement order, including the first run and outliers. XOR
tables interleave implementations and rotate the first implementation across
repetitions, using identical rounds and input parameters. `vs ref` divides each
median by the first implementation's median before rounding; `0.80x` means 20%
less elapsed time for that workload. ` [...]
+
+XOR build tables compare DataSketches' distinct-hash and deduplicating entry
points with xorf on distinct inputs. The `matched_build` scenarios additionally
align the seed sequence and check each resulting seed against the expected
first or second attempt, including a case requiring a second attempt. Query
tables use the same key sets and fingerprint widths. These comparisons include
each implementation's fixture construction and cleanup. Different scenario rows
may have different round [...]
+
+To add or resize a scenario, select a reference revision and run it with an
explicit `--rounds` value and several repetitions. Choose a convenient fixed
count that puts the reference in the hundreds of milliseconds, then record it
in the workload registration. Give all implementations of that scenario the
same count. Keep it fixed for later revisions and faster peers. When inputs,
rounds, or timing boundaries change, rerun both sides under the new definition.
Inspect the measured range b [...]
+
## Public API documentation
- Describe types with noun phrases and API behavior with third-person
present-tense verbs such as `Creates`, `Updates`, and `Returns`.
diff --git a/Cargo.lock b/Cargo.lock
index 2421b06..ccc8f1a 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -83,8 +83,8 @@ checksum =
"72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
name = "benchmarks"
version = "0.0.0"
dependencies = [
+ "clap",
"datasketches",
- "divan",
"rand",
"xorf",
]
@@ -186,7 +186,6 @@ dependencies = [
"anstyle",
"clap_lex",
"strsim",
- "terminal_size",
]
[[package]]
@@ -213,12 +212,6 @@ version = "1.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570"
-[[package]]
-name = "condtype"
-version = "1.3.0"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "baf0a07a401f374238ab8e2f11a104d2851bf9ce711ec69804834de8af45c7af"
-
[[package]]
name = "console"
version = "0.16.4"
@@ -247,31 +240,6 @@ dependencies = [
"insta",
]
-[[package]]
-name = "divan"
-version = "0.1.21"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "a405457ec78b8fe08b0e32b4a3570ab5dff6dd16eb9e76a5ee0a9d9cbd898933"
-dependencies = [
- "cfg-if",
- "clap",
- "condtype",
- "divan-macros",
- "libc",
- "regex-lite",
-]
-
-[[package]]
-name = "divan-macros"
-version = "0.1.21"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9556bc800956545d6420a640173e5ba7dfa82f38d3ea5a167eb555bc69ac3323"
-dependencies = [
- "proc-macro2",
- "quote",
- "syn 2.0.119",
-]
-
[[package]]
name = "encode_unicode"
version = "1.0.0"
@@ -546,12 +514,6 @@ dependencies = [
"regex-syntax",
]
-[[package]]
-name = "regex-lite"
-version = "0.1.9"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "cab834c73d247e67f4fae452806d17d3c7501756d98c8808d7c9c7aa7d18f973"
-
[[package]]
name = "regex-syntax"
version = "0.8.11"
@@ -765,16 +727,6 @@ dependencies = [
"windows-sys 0.61.2",
]
-[[package]]
-name = "terminal_size"
-version = "0.4.4"
-source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "230a1b821ccbd75b185820a1f1ff7b14d21da1e442e22c0863ea5f08771a8874"
-dependencies = [
- "rustix",
- "windows-sys 0.61.2",
-]
-
[[package]]
name = "tests-integration"
version = "0.0.0"
diff --git a/Cargo.toml b/Cargo.toml
index 92c4775..bb36c89 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -33,7 +33,6 @@ datasketches = { path = "datasketches" }
# Crates.io dependencies
clap = { version = "4.6.5", features = ["derive"] }
-divan = { version = "0.1.21" }
googletest = { version = "0.14.3" }
insta = { version = "1.48.0" }
cargo_metadata = { version = "0.23.1" }
diff --git a/benchmarks/Cargo.toml b/benchmarks/Cargo.toml
index 49c3b9b..98e3523 100644
--- a/benchmarks/Cargo.toml
+++ b/benchmarks/Cargo.toml
@@ -22,6 +22,9 @@ publish = false
edition.workspace = true
rust-version.workspace = true
+[dependencies]
+clap = { workspace = true }
+
[dev-dependencies]
xorf = { version = "=0.13.0", default-features = false }
datasketches = { workspace = true, features = [
@@ -37,9 +40,12 @@ datasketches = { workspace = true, features = [
"tuple",
"xor",
] }
-divan = { workspace = true }
rand = { workspace = true }
+[lib]
+path = "harness.rs"
+bench = false
+
[[bench]]
harness = false
name = "benchmarks"
diff --git a/benchmarks/bloom/mod.rs b/benchmarks/bloom/mod.rs
index 1dea33b..4839176 100644
--- a/benchmarks/bloom/mod.rs
+++ b/benchmarks/bloom/mod.rs
@@ -16,3 +16,7 @@
// under the License.
mod update;
+
+pub fn register(runner: &mut benchmarks::Runner) {
+ update::register(runner);
+}
diff --git a/benchmarks/bloom/update.rs b/benchmarks/bloom/update.rs
index 7227b21..a71a86a 100644
--- a/benchmarks/bloom/update.rs
+++ b/benchmarks/bloom/update.rs
@@ -15,18 +15,18 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::bloom::BloomFilterBuilder;
use datasketches::hash::value::raw_bytes;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use crate::hash_inputs::ITEMS;
use crate::hash_inputs::bytes_32_values;
-#[divan::bench]
fn u64(bencher: Bencher) {
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut filter = BloomFilterBuilder::with_accuracy(ITEMS as u64, 0.01)
.build()
.unwrap();
@@ -37,11 +37,10 @@ fn u64(bencher: Bencher) {
});
}
-#[divan::bench]
fn bytes_32(bencher: Bencher) {
let values = bytes_32_values();
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut filter = BloomFilterBuilder::with_accuracy(ITEMS as u64, 0.01)
.build()
.unwrap();
@@ -52,7 +51,6 @@ fn bytes_32(bencher: Bencher) {
});
}
-#[divan::bench]
fn contains_and_insert_present_u64(bencher: Bencher) {
let mut populated = BloomFilterBuilder::with_accuracy(ITEMS as u64, 0.01)
.build()
@@ -61,13 +59,21 @@ fn contains_and_insert_present_u64(bencher: Bencher) {
populated.insert(value);
}
- bencher
- .counter(ItemsCount::new(ITEMS))
- .with_inputs(|| populated.clone())
- .bench_local_values(|mut filter| {
- for value in 0..ITEMS as u64 {
- black_box(filter.contains_and_insert(black_box(&value)));
- }
- filter
- });
+ bencher.bench_local(|| {
+ let mut filter = populated.clone();
+ for value in 0..ITEMS as u64 {
+ black_box(filter.contains_and_insert(black_box(&value)));
+ }
+ filter
+ });
+}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("bloom::update::u64", 2_000, u64);
+ runner.case("bloom::update::bytes_32", 1_000, bytes_32);
+ runner.case(
+ "bloom::update::contains_and_insert_present_u64",
+ 2_000,
+ contains_and_insert_present_u64,
+ );
}
diff --git a/benchmarks/countmin/decay.rs b/benchmarks/countmin/decay.rs
index 82de772..4fa0ed9 100644
--- a/benchmarks/countmin/decay.rs
+++ b/benchmarks/countmin/decay.rs
@@ -15,18 +15,28 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::countmin::CountMinSketch;
-use divan::Bencher;
-use divan::black_box;
-#[divan::bench(args = [0.5, 0.99, 1.0])]
fn u64(bencher: Bencher, factor: f64) {
let mut sketch = CountMinSketch::<u64>::new(4, 16_384).unwrap();
for item in 0..128_u64 {
sketch.update_with_weight(item, (1 << 53) + item);
}
- bencher
- .with_inputs(|| sketch.clone())
- .bench_local_refs(|sketch| black_box(sketch).decay(black_box(factor)));
+ bencher.bench_local(|| {
+ let mut sketch = sketch.clone();
+ black_box(&mut sketch).decay(black_box(factor))
+ });
+}
+
+pub fn register(runner: &mut Runner) {
+ for &(factor, rounds) in &[(0.5, 20_000), (0.99, 10_000), (1.0, 50_000)] {
+ runner.case(format!("countmin::decay::u64/{factor:?}"), rounds, |b| {
+ u64(b, factor)
+ });
+ }
}
diff --git a/benchmarks/countmin/estimate.rs b/benchmarks/countmin/estimate.rs
index 4a2777f..aae7206 100644
--- a/benchmarks/countmin/estimate.rs
+++ b/benchmarks/countmin/estimate.rs
@@ -15,23 +15,23 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::countmin::CountMinSketch;
use datasketches::hash::value::raw_bytes;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use crate::hash_inputs::ITEMS;
use crate::hash_inputs::bytes_32_values;
-#[divan::bench]
fn u64(bencher: Bencher) {
let mut sketch = CountMinSketch::<u64>::new(4, 16_384).unwrap();
for value in 0..ITEMS as u64 {
sketch.update(value);
}
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut total = 0u64;
for value in 0..ITEMS as u64 {
total = total.wrapping_add(sketch.estimate(black_box(value)));
@@ -40,7 +40,6 @@ fn u64(bencher: Bencher) {
});
}
-#[divan::bench]
fn bytes_32(bencher: Bencher) {
let values = bytes_32_values();
let mut sketch = CountMinSketch::<u64>::new(4, 16_384).unwrap();
@@ -48,7 +47,7 @@ fn bytes_32(bencher: Bencher) {
sketch.update(raw_bytes::from_slice(value));
}
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut total = 0u64;
for value in &values {
total =
total.wrapping_add(sketch.estimate(raw_bytes::from_slice(black_box(value))));
@@ -56,3 +55,8 @@ fn bytes_32(bencher: Bencher) {
black_box(total)
});
}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("countmin::estimate::u64", 2_000, u64);
+ runner.case("countmin::estimate::bytes_32", 1_000, bytes_32);
+}
diff --git a/benchmarks/countmin/mod.rs b/benchmarks/countmin/mod.rs
index 39f1404..cc19dcf 100644
--- a/benchmarks/countmin/mod.rs
+++ b/benchmarks/countmin/mod.rs
@@ -18,3 +18,9 @@
mod decay;
mod estimate;
mod update;
+
+pub fn register(runner: &mut benchmarks::Runner) {
+ decay::register(runner);
+ estimate::register(runner);
+ update::register(runner);
+}
diff --git a/benchmarks/countmin/update.rs b/benchmarks/countmin/update.rs
index f25af1e..dd6db39 100644
--- a/benchmarks/countmin/update.rs
+++ b/benchmarks/countmin/update.rs
@@ -15,18 +15,18 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::countmin::CountMinSketch;
use datasketches::hash::value::raw_bytes;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use crate::hash_inputs::ITEMS;
use crate::hash_inputs::bytes_32_values;
-#[divan::bench]
fn u64(bencher: Bencher) {
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut sketch = CountMinSketch::<u64>::new(4, 16_384).unwrap();
for value in 0..ITEMS as u64 {
sketch.update(black_box(value));
@@ -35,11 +35,10 @@ fn u64(bencher: Bencher) {
});
}
-#[divan::bench]
fn bytes_32(bencher: Bencher) {
let values = bytes_32_values();
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut sketch = CountMinSketch::<u64>::new(4, 16_384).unwrap();
for value in &values {
sketch.update(raw_bytes::from_slice(black_box(value)));
@@ -47,3 +46,8 @@ fn bytes_32(bencher: Bencher) {
black_box(sketch)
});
}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("countmin::update::u64", 2_000, u64);
+ runner.case("countmin::update::bytes_32", 1_000, bytes_32);
+}
diff --git a/benchmarks/cpc/mod.rs b/benchmarks/cpc/mod.rs
index 13c2486..32c8b7f 100644
--- a/benchmarks/cpc/mod.rs
+++ b/benchmarks/cpc/mod.rs
@@ -17,3 +17,8 @@
mod serde;
mod update;
+
+pub fn register(runner: &mut benchmarks::Runner) {
+ serde::register(runner);
+ update::register(runner);
+}
diff --git a/benchmarks/cpc/serde.rs b/benchmarks/cpc/serde.rs
index dc3ce09..5ea55ae 100644
--- a/benchmarks/cpc/serde.rs
+++ b/benchmarks/cpc/serde.rs
@@ -15,36 +15,24 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::cpc::CpcSketch;
-use divan::Bencher;
-use divan::black_box;
-use divan::black_box_drop;
-use divan::counter::BytesCount;
-use divan::counter::ItemsCount;
const LG_K: u8 = 10;
-#[divan::bench(args = [200_u64, 8_000_u64])]
fn serialize(bencher: Bencher, items: u64) {
let sketch = sketch(items);
- let serialized_bytes = sketch.serialize().len();
- bencher
- .counter(ItemsCount::new(items))
- .counter(BytesCount::new(serialized_bytes))
- .bench_local(|| black_box_drop(black_box(&sketch).serialize()));
+ bencher.bench_local(|| black_box(&sketch).serialize());
}
-#[divan::bench(args = [200_u64, 8_000_u64])]
fn deserialize(bencher: Bencher, items: u64) {
let bytes = sketch(items).serialize();
- bencher
- .counter(ItemsCount::new(items))
- .counter(BytesCount::new(bytes.len()))
- .bench_local(|| {
- black_box_drop(CpcSketch::deserialize(black_box(&bytes)).unwrap());
- });
+ bencher.bench_local(|| CpcSketch::deserialize(black_box(&bytes)).unwrap());
}
fn sketch(items: u64) -> CpcSketch {
@@ -54,3 +42,16 @@ fn sketch(items: u64) -> CpcSketch {
}
sketch
}
+
+pub fn register(runner: &mut Runner) {
+ for &(items, rounds) in &[(200_u64, 400_000), (8_000_u64, 500_000)] {
+ runner.case(format!("cpc::serde::serialize/{items:?}"), rounds, |b| {
+ serialize(b, items)
+ });
+ }
+ for &(items, rounds) in &[(200_u64, 400_000), (8_000_u64, 100_000)] {
+ runner.case(format!("cpc::serde::deserialize/{items:?}"), rounds, |b| {
+ deserialize(b, items)
+ });
+ }
+}
diff --git a/benchmarks/cpc/update.rs b/benchmarks/cpc/update.rs
index ba532b8..80fe7bd 100644
--- a/benchmarks/cpc/update.rs
+++ b/benchmarks/cpc/update.rs
@@ -15,18 +15,18 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::cpc::CpcSketch;
use datasketches::hash::value::raw_bytes;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use crate::hash_inputs::ITEMS;
use crate::hash_inputs::bytes_32_values;
-#[divan::bench]
fn u64(bencher: Bencher) {
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut sketch = CpcSketch::new(11).unwrap();
for value in 0..ITEMS as u64 {
sketch.update(black_box(value));
@@ -35,11 +35,10 @@ fn u64(bencher: Bencher) {
});
}
-#[divan::bench]
fn bytes_32(bencher: Bencher) {
let values = bytes_32_values();
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut sketch = CpcSketch::new(11).unwrap();
for value in &values {
sketch.update(raw_bytes::from_slice(black_box(value)));
@@ -47,3 +46,8 @@ fn bytes_32(bencher: Bencher) {
black_box(sketch)
});
}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("cpc::update::u64", 3_000, u64);
+ runner.case("cpc::update::bytes_32", 2_000, bytes_32);
+}
diff --git a/benchmarks/frequencies/mod.rs b/benchmarks/frequencies/mod.rs
index 1dea33b..4839176 100644
--- a/benchmarks/frequencies/mod.rs
+++ b/benchmarks/frequencies/mod.rs
@@ -16,3 +16,7 @@
// under the License.
mod update;
+
+pub fn register(runner: &mut benchmarks::Runner) {
+ update::register(runner);
+}
diff --git a/benchmarks/frequencies/update.rs b/benchmarks/frequencies/update.rs
index 9026b29..32f7cf3 100644
--- a/benchmarks/frequencies/update.rs
+++ b/benchmarks/frequencies/update.rs
@@ -15,19 +15,19 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::frequencies::FrequentItemsSketch;
use datasketches::hash::value::raw_bytes;
use datasketches::hash::value::raw_bytes::RawBytes;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use crate::hash_inputs::ITEMS;
use crate::hash_inputs::bytes_32_values;
-#[divan::bench]
fn u64(bencher: Bencher) {
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut sketch = FrequentItemsSketch::<u64>::new(16_384).unwrap();
for value in 0..ITEMS as u64 {
sketch.update(black_box(value));
@@ -36,11 +36,10 @@ fn u64(bencher: Bencher) {
});
}
-#[divan::bench]
fn bytes_32(bencher: Bencher) {
let values = bytes_32_values();
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut sketch =
FrequentItemsSketch::<RawBytes<&[u8]>>::new(16_384).unwrap();
for value in &values {
sketch.update(raw_bytes::from_slice(black_box(value)));
@@ -48,3 +47,8 @@ fn bytes_32(bencher: Bencher) {
black_box(sketch)
});
}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("frequencies::update::u64", 2_000, u64);
+ runner.case("frequencies::update::bytes_32", 900, bytes_32);
+}
diff --git a/benchmarks/harness.rs b/benchmarks/harness.rs
new file mode 100644
index 0000000..a5668f2
--- /dev/null
+++ b/benchmarks/harness.rs
@@ -0,0 +1,307 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::hint::black_box;
+use std::num::NonZeroUsize;
+use std::time::Duration;
+use std::time::Instant;
+
+use clap::Parser;
+
+type Implementation<'a> = (&'a str, &'a dyn Fn(Bencher));
+
+#[derive(Parser)]
+#[command(about = "Compare implementations at a fixed amount of work.")]
+struct Options {
+ /// Select scenario names containing all these substrings.
+ filters: Vec<String>,
+ /// Match a complete scenario name instead of substrings.
+ #[arg(long)]
+ exact: bool,
+ /// List matching scenarios, rounds and implementations without running
them.
+ #[arg(long, conflicts_with = "test")]
+ list: bool,
+ /// Exercise each workload once without timing.
+ #[arg(long)]
+ test: bool,
+ /// Number of measured repetitions per implementation.
+ #[arg(long, default_value = "5")]
+ repeat: NonZeroUsize,
+ /// Override the scenario's fixed rounds (equally for all implementations).
+ #[arg(long)]
+ rounds: Option<NonZeroUsize>,
+ /// Run only the scenario's DataSketches reference, e.g. to choose its
fixed work size.
+ #[arg(long)]
+ baseline_only: bool,
+ /// Include individual elapsed times in their original order.
+ #[arg(long)]
+ raw: bool,
+ // Cargo passes this flag to harness-free executables.
+ #[arg(long = "bench", hide = true)]
+ _bench: bool,
+}
+
+pub struct Runner {
+ options: Options,
+ matched: usize,
+}
+
+impl Runner {
+ pub fn from_args() -> Self {
+ Self {
+ options: Options::parse(),
+ matched: 0,
+ }
+ }
+
+ /// Runs a workload with a fixed number of rounds.
+ pub fn case(&mut self, name: impl AsRef<str>, rounds: usize, workload:
impl Fn(Bencher)) {
+ self.run(name.as_ref(), rounds, &[("DataSketches", &workload)]);
+ }
+
+ /// All adapters for one scenario use the same rounds. The first is its
DataSketches reference.
+ pub fn compare(&mut self, name: &str, rounds: usize, implementations:
&[Implementation<'_>]) {
+ self.run(name, rounds, implementations);
+ }
+
+ fn run(&mut self, name: &str, rounds: usize, implementations:
&[Implementation<'_>]) {
+ if !self.options.filters.iter().all(|filter| {
+ if self.options.exact {
+ name == filter
+ } else {
+ name.contains(filter)
+ }
+ }) {
+ return;
+ }
+ self.matched += 1;
+ let rounds = if self.options.test {
+ 1
+ } else {
+ self.options.rounds.map_or(rounds, NonZeroUsize::get)
+ };
+ assert!(rounds > 0, "{name}: need at least one round");
+ assert!(
+ !implementations.is_empty(),
+ "{name}: missing reference implementation"
+ );
+ let implementations = if self.options.baseline_only {
+ &implementations[..1]
+ } else {
+ implementations
+ };
+ if self.options.list {
+ let names = implementations
+ .iter()
+ .map(|(label, _)| *label)
+ .collect::<Vec<_>>();
+ println!("{name} {rounds} rounds [{}]", names.join(", "));
+ return;
+ }
+ if self.options.test {
+ for (label, workload) in implementations {
+ workload(Bencher { rounds });
+ println!("ok {name} / {label}");
+ }
+ return;
+ }
+
+ let repeat = self.options.repeat.get();
+ let mut samples = vec![Vec::with_capacity(repeat);
implementations.len()];
+ for repetition in 0..repeat {
+ // Interleave peers and rotate the first one so any one adapter
does not always run
+ // cold.
+ for offset in 0..implementations.len() {
+ let index = (repetition + offset) % implementations.len();
+ let workload = implementations[index].1;
+ let start = Instant::now();
+ workload(Bencher { rounds });
+ samples[index].push(start.elapsed());
+ }
+ }
+ println!(
+ "\n{name} ({rounds} rounds, {repeat} runs; reference: {})",
+ implementations[0].0
+ );
+ println!(
+ " {:<16} {:>10} {:>15} {:>9}",
+ "implementation", "median ms", "min..max ms", "vs ref"
+ );
+ let reference = summary(&samples[0]).1;
+ for ((label, _), times) in implementations.iter().zip(&samples) {
+ let (min, median, max) = summary(times);
+ let range = format!("{}..{}", min.as_millis(), max.as_millis());
+ let ratio = if reference.is_zero() {
+ String::from("n/a")
+ } else {
+ format!("{:.2}x", median.as_secs_f64() /
reference.as_secs_f64())
+ };
+ print!(
+ " {label:<16} {:>10} {range:>15} {ratio:>9}",
+ median.as_millis()
+ );
+ if self.options.raw {
+ print!(
+ " {:?}",
+ times.iter().map(Duration::as_millis).collect::<Vec<_>>()
+ );
+ }
+ println!();
+ }
+ }
+
+ pub fn finish(self) {
+ if self.matched == 0 {
+ eprintln!("No matching scenarios. Use --list for this target.");
+ std::process::exit(1);
+ }
+ }
+}
+
+fn summary(samples: &[Duration]) -> (Duration, Duration, Duration) {
+ let mut sorted = samples.to_vec();
+ sorted.sort_unstable();
+ let lo = sorted[(sorted.len() - 1) / 2];
+ let hi = sorted[sorted.len() / 2];
+ (sorted[0], lo + (hi - lo) / 2, sorted[sorted.len() - 1])
+}
+
+/// Executes fixed rounds, including output destruction, inside the timed
workload.
+pub struct Bencher {
+ rounds: usize,
+}
+
+impl Bencher {
+ pub fn bench_local<O>(self, mut operation: impl FnMut() -> O) {
+ for _ in 0..self.rounds {
+ black_box(operation());
+ }
+ }
+}
+
+#[cfg(test)]
+mod tests {
+ use std::cell::Cell;
+ use std::cell::RefCell;
+
+ use super::*;
+
+ #[test]
+ fn repeated_workloads_recreate_state_and_finish_cleanup() {
+ let options = Options::try_parse_from(["bench", "--repeat", "3",
"--rounds", "7"]).unwrap();
+ let live = Cell::new(0);
+ let completed = Cell::new(0);
+ struct Output<'a>(&'a Cell<usize>);
+ impl Drop for Output<'_> {
+ fn drop(&mut self) {
+ self.0.set(self.0.get() - 1);
+ }
+ }
+ let mut runner = Runner {
+ options,
+ matched: 0,
+ };
+ runner.case("lifecycle", 100, |bencher| {
+ assert_eq!(live.get(), 0);
+ let mut rounds = 0;
+ bencher.bench_local(|| {
+ assert_eq!(live.get(), 0);
+ live.set(1);
+ rounds += 1;
+ Output(&live)
+ });
+ assert_eq!(rounds, 7);
+ assert_eq!(live.get(), 0);
+ completed.set(completed.get() + 1);
+ });
+ assert_eq!(completed.get(), 3);
+ }
+
+ #[test]
+ fn peers_share_work_and_rotate_across_repetitions() {
+ let options = Options::try_parse_from(["bench", "--repeat", "3",
"--rounds", "7"]).unwrap();
+ let mut runner = Runner {
+ options,
+ matched: 0,
+ };
+ let order = RefCell::new(Vec::new());
+ let workload = |label, b: Bencher| {
+ let mut n = 0;
+ b.bench_local(|| n += 1);
+ order.borrow_mut().push((label, n));
+ };
+ runner.compare(
+ "roundtrip",
+ 100,
+ &[
+ ("ref", &|b| workload('A', b)),
+ ("peer", &|b| workload('B', b)),
+ ],
+ );
+ assert_eq!(
+ order.into_inner(),
+ [('A', 7), ('B', 7), ('B', 7), ('A', 7), ('A', 7), ('B', 7)]
+ );
+ }
+
+ #[test]
+ fn median_keeps_timer_precision_and_range_keeps_the_outlier() {
+ let times = [99, 1, 6, 3].map(Duration::from_millis);
+ assert_eq!(
+ summary(×),
+ (times[1], Duration::from_micros(4500), times[0])
+ );
+ }
+
+ #[test]
+ fn smoke_mode_filters_cases_and_runs_each_peer_once() {
+ let options = Options::try_parse_from([
+ "bench", "xor", "present", "--test", "--rounds", "7", "--repeat",
"3",
+ ])
+ .unwrap();
+ let mut runner = Runner {
+ options,
+ matched: 0,
+ };
+ let calls = RefCell::new(Vec::new());
+ runner.case("xor/absent", 100, |_| panic!("filtered out"));
+ runner.case("bloom/present", 100, |_| panic!("filtered out"));
+ runner.compare(
+ "xor/present",
+ 100,
+ &[
+ ("ref", &|b| b.bench_local(|| calls.borrow_mut().push('A'))),
+ ("peer", &|b| b.bench_local(|| calls.borrow_mut().push('B'))),
+ ],
+ );
+ assert_eq!(runner.matched, 1);
+ assert_eq!(calls.into_inner(), ['A', 'B']);
+ }
+
+ #[test]
+ fn listing_matches_exact_names_without_constructing_fixtures() {
+ let options = Options::try_parse_from(["bench", "query/1", "--exact",
"--list"]).unwrap();
+ let mut runner = Runner {
+ options,
+ matched: 0,
+ };
+ for name in ["query/1", "query/10", "other/query/1"] {
+ runner.case(name, 100, |_| panic!("listing must not execute
workloads"));
+ }
+ assert_eq!(runner.matched, 1);
+ }
+}
diff --git a/benchmarks/hll/mod.rs b/benchmarks/hll/mod.rs
index 9e440b2..7c950ee 100644
--- a/benchmarks/hll/mod.rs
+++ b/benchmarks/hll/mod.rs
@@ -18,3 +18,9 @@
mod serde;
mod union;
mod update;
+
+pub fn register(runner: &mut benchmarks::Runner) {
+ serde::register(runner);
+ union::register(runner);
+ update::register(runner);
+}
diff --git a/benchmarks/hll/serde.rs b/benchmarks/hll/serde.rs
index 8bafba4..de9fa35 100644
--- a/benchmarks/hll/serde.rs
+++ b/benchmarks/hll/serde.rs
@@ -15,10 +15,12 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::hll::HllType;
use datasketches::hll::HllUnion;
-use divan::Bencher;
-use divan::black_box;
fn union(rows: u64) -> HllUnion {
let mut union = HllUnion::new(18).unwrap();
@@ -28,14 +30,35 @@ fn union(rows: u64) -> HllUnion {
union
}
-#[divan::bench(args = [0, 4, 100, 80_000])]
fn serialize_union(bencher: Bencher, rows: u64) {
let union = union(rows);
bencher.bench_local(|| black_box(&union).serialize());
}
-#[divan::bench(args = [0, 4, 100, 80_000])]
fn serialize_union_via_sketch(bencher: Bencher, rows: u64) {
let union = union(rows);
bencher.bench_local(||
black_box(&union).to_sketch(HllType::Hll8).serialize());
}
+
+pub fn register(runner: &mut Runner) {
+ let cases = &[
+ (0, 15_000_000),
+ (4, 12_000_000),
+ (100, 2_000_000),
+ (80_000, 90_000),
+ ];
+ for &(rows, rounds) in cases {
+ runner.case(
+ format!("hll::serde::serialize_union/{rows:?}"),
+ rounds,
+ |b| serialize_union(b, rows),
+ );
+ }
+ for &(rows, rounds) in cases {
+ runner.case(
+ format!("hll::serde::serialize_union_via_sketch/{rows:?}"),
+ rounds,
+ |b| serialize_union_via_sketch(b, rows),
+ );
+ }
+}
diff --git a/benchmarks/hll/union.rs b/benchmarks/hll/union.rs
index c6be59c..c365ddc 100644
--- a/benchmarks/hll/union.rs
+++ b/benchmarks/hll/union.rs
@@ -15,11 +15,13 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::hll::HllSketch;
use datasketches::hll::HllType;
use datasketches::hll::HllUnion;
-use divan::Bencher;
-use divan::black_box;
fn partials(lg_k: u8, count: usize, rows: usize) -> Vec<HllSketch> {
(0..count)
@@ -33,7 +35,6 @@ fn partials(lg_k: u8, count: usize, rows: usize) ->
Vec<HllSketch> {
.collect()
}
-#[divan::bench(args = [12, 18])]
fn empty_from_dense(bencher: Bencher, lg_k: u8) {
let source = partials(lg_k, 1, 1 << lg_k).pop().unwrap();
bencher.bench_local(|| {
@@ -43,7 +44,6 @@ fn empty_from_dense(bencher: Bencher, lg_k: u8) {
});
}
-#[divan::bench(args = [12, 18])]
fn recreated_small_partials(bencher: Bencher, lg_k: u8) {
let partials = partials(lg_k, 400, 200);
bencher.bench_local(|| {
@@ -59,7 +59,6 @@ fn recreated_small_partials(bencher: Bencher, lg_k: u8) {
});
}
-#[divan::bench(args = [12, 18])]
fn persistent_small_partials(bencher: Bencher, lg_k: u8) {
let partials = partials(lg_k, 400, 200);
bencher.bench_local(|| {
@@ -72,7 +71,6 @@ fn persistent_small_partials(bencher: Bencher, lg_k: u8) {
});
}
-#[divan::bench(args = [12, 18])]
fn dense_partials(bencher: Bencher, lg_k: u8) {
let partials = partials(lg_k, 16, (1 << lg_k) / 4);
bencher.bench_local(|| {
@@ -85,7 +83,6 @@ fn dense_partials(bencher: Bencher, lg_k: u8) {
});
}
-#[divan::bench(args = [12, 18])]
fn dense_partials_with_queries(bencher: Bencher, lg_k: u8) {
let partials = partials(lg_k, 16, (1 << lg_k) / 4);
bencher.bench_local(|| {
@@ -98,7 +95,6 @@ fn dense_partials_with_queries(bencher: Bencher, lg_k: u8) {
});
}
-#[divan::bench(args = [12, 18])]
fn downsampled_dense_partials(bencher: Bencher, lg_k: u8) {
let partials = partials(lg_k + 1, 16, (1 << lg_k) / 2);
bencher.bench_local(|| {
@@ -110,3 +106,48 @@ fn downsampled_dense_partials(bencher: Bencher, lg_k: u8) {
black_box(union.to_sketch(HllType::Hll8))
});
}
+
+pub fn register(runner: &mut Runner) {
+ for &(lg_k, rounds) in &[(12, 2_000_000), (18, 70_000)] {
+ runner.case(
+ format!("hll::union::empty_from_dense/{lg_k:?}"),
+ rounds,
+ |b| empty_from_dense(b, lg_k),
+ );
+ }
+ for &(lg_k, rounds) in &[(12, 500), (18, 90)] {
+ runner.case(
+ format!("hll::union::recreated_small_partials/{lg_k:?}"),
+ rounds,
+ |b| recreated_small_partials(b, lg_k),
+ );
+ }
+ for &(lg_k, rounds) in &[(12, 500), (18, 300)] {
+ runner.case(
+ format!("hll::union::persistent_small_partials/{lg_k:?}"),
+ rounds,
+ |b| persistent_small_partials(b, lg_k),
+ );
+ }
+ for &(lg_k, rounds) in &[(12, 80_000), (18, 700)] {
+ runner.case(
+ format!("hll::union::dense_partials/{lg_k:?}"),
+ rounds,
+ |b| dense_partials(b, lg_k),
+ );
+ }
+ for &(lg_k, rounds) in &[(12, 7_000), (18, 60)] {
+ runner.case(
+ format!("hll::union::dense_partials_with_queries/{lg_k:?}"),
+ rounds,
+ |b| dense_partials_with_queries(b, lg_k),
+ );
+ }
+ for &(lg_k, rounds) in &[(12, 6_000), (18, 80)] {
+ runner.case(
+ format!("hll::union::downsampled_dense_partials/{lg_k:?}"),
+ rounds,
+ |b| downsampled_dense_partials(b, lg_k),
+ );
+ }
+}
diff --git a/benchmarks/hll/update.rs b/benchmarks/hll/update.rs
index 88a335c..73f60e0 100644
--- a/benchmarks/hll/update.rs
+++ b/benchmarks/hll/update.rs
@@ -15,19 +15,19 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::hash::value::raw_bytes;
use datasketches::hll::HllSketch;
use datasketches::hll::HllType;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use crate::hash_inputs::ITEMS;
use crate::hash_inputs::bytes_32_values;
-#[divan::bench]
fn u64(bencher: Bencher) {
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut sketch = HllSketch::new(12, HllType::Hll8).unwrap();
for value in 0..ITEMS as u64 {
sketch.update(black_box(value));
@@ -36,11 +36,10 @@ fn u64(bencher: Bencher) {
});
}
-#[divan::bench]
fn bytes_32(bencher: Bencher) {
let values = bytes_32_values();
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut sketch = HllSketch::new(12, HllType::Hll8).unwrap();
for value in &values {
sketch.update(raw_bytes::from_slice(black_box(value)));
@@ -48,3 +47,8 @@ fn bytes_32(bencher: Bencher) {
black_box(sketch)
});
}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("hll::update::u64", 6_000, u64);
+ runner.case("hll::update::bytes_32", 2_000, bytes_32);
+}
diff --git a/benchmarks/kll/merge.rs b/benchmarks/kll/merge.rs
index c35a1af..829b731 100644
--- a/benchmarks/kll/merge.rs
+++ b/benchmarks/kll/merge.rs
@@ -15,24 +15,26 @@
// specific language governing permissions and limitations
// under the License.
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use super::support::build_sketch;
use super::support::values;
-#[divan::bench]
fn merge(bencher: Bencher) {
let values = values(200_000);
let left = build_sketch(&values[..100_000]);
let right = build_sketch(&values[100_000..]);
- bencher
- .counter(ItemsCount::new(values.len()))
- .with_inputs(|| left.clone())
- .bench_local_values(|mut left| {
- left.merge(black_box(&right)).unwrap();
- black_box(left)
- });
+ bencher.bench_local(|| {
+ let mut left = left.clone();
+ left.merge(black_box(&right)).unwrap();
+ black_box(left)
+ });
+}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("kll::merge::merge", 70_000, merge);
}
diff --git a/benchmarks/kll/mod.rs b/benchmarks/kll/mod.rs
index 4ff11ff..eb0538b 100644
--- a/benchmarks/kll/mod.rs
+++ b/benchmarks/kll/mod.rs
@@ -20,3 +20,10 @@ mod query;
mod serde;
mod support;
mod update;
+
+pub fn register(runner: &mut benchmarks::Runner) {
+ merge::register(runner);
+ query::register(runner);
+ serde::register(runner);
+ update::register(runner);
+}
diff --git a/benchmarks/kll/query.rs b/benchmarks/kll/query.rs
index 508a54b..35a6376 100644
--- a/benchmarks/kll/query.rs
+++ b/benchmarks/kll/query.rs
@@ -15,36 +15,49 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::common::SearchCriteria;
use datasketches::kll::KllFloat;
-use divan::Bencher;
-use divan::black_box;
use super::support::prepared_sketch;
-#[divan::bench]
fn rank(bencher: Bencher) {
let sketch = prepared_sketch();
let item = KllFloat::<f64>::new(500_000.0).unwrap();
bencher.bench_local(|| black_box(&sketch).rank(black_box(&item),
SearchCriteria::Inclusive));
}
-#[divan::bench(args = [0.0, 0.5, 1.0])]
fn quantile(bencher: Bencher, rank: f64) {
let sketch = prepared_sketch();
bencher.bench_local(|| black_box(&sketch).quantile(black_box(rank),
SearchCriteria::Inclusive));
}
-#[divan::bench]
fn sorted_view_quantile(bencher: Bencher) {
let view = prepared_sketch().sorted_view();
bencher.bench_local(|| black_box(&view).quantile(black_box(0.5),
SearchCriteria::Inclusive));
}
-#[divan::bench]
fn batch_quantiles(bencher: Bencher) {
let sketch = prepared_sketch();
let ranks = [0.01, 0.1, 0.25, 0.5, 0.75, 0.9, 0.99];
bencher
.bench_local(|| black_box(&sketch).quantiles(black_box(&ranks),
SearchCriteria::Inclusive));
}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("kll::query::rank", 7_000_000, rank);
+ for &(rank, rounds) in &[(0.0, 100_000_000), (0.5, 100_000), (1.0,
100_000_000)] {
+ runner.case(format!("kll::query::quantile/{rank:?}"), rounds, |b| {
+ quantile(b, rank)
+ });
+ }
+ runner.case(
+ "kll::query::sorted_view_quantile",
+ 40_000_000,
+ sorted_view_quantile,
+ );
+ runner.case("kll::query::batch_quantiles", 100_000, batch_quantiles);
+}
diff --git a/benchmarks/kll/serde.rs b/benchmarks/kll/serde.rs
index 3882f23..697de3b 100644
--- a/benchmarks/kll/serde.rs
+++ b/benchmarks/kll/serde.rs
@@ -15,27 +15,26 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::kll::KllFloat;
use datasketches::kll::KllSketch;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::BytesCount;
use super::support::prepared_sketch;
-#[divan::bench]
fn serialize(bencher: Bencher) {
let sketch = prepared_sketch();
- let bytes = sketch.serialize();
- bencher
- .counter(BytesCount::new(bytes.len()))
- .bench_local(|| black_box(&sketch).serialize());
+ bencher.bench_local(|| black_box(&sketch).serialize());
}
-#[divan::bench]
fn deserialize(bencher: Bencher) {
let bytes = prepared_sketch().serialize();
- bencher
- .counter(BytesCount::new(bytes.len()))
- .bench_local(||
KllSketch::<KllFloat<f64>>::deserialize(black_box(&bytes)).unwrap());
+ bencher.bench_local(||
KllSketch::<KllFloat<f64>>::deserialize(black_box(&bytes)).unwrap());
+}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("kll::serde::serialize", 200_000, serialize);
+ runner.case("kll::serde::deserialize", 100_000, deserialize);
}
diff --git a/benchmarks/kll/update.rs b/benchmarks/kll/update.rs
index 6a3c865..ed718a7 100644
--- a/benchmarks/kll/update.rs
+++ b/benchmarks/kll/update.rs
@@ -15,17 +15,23 @@
// specific language governing permissions and limitations
// under the License.
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use super::support::build_sketch;
use super::support::values;
-#[divan::bench(args = [1_000, 10_000, 100_000])]
fn update(bencher: Bencher, len: usize) {
let values = values(len);
- bencher
- .counter(ItemsCount::new(len))
- .bench_local(|| build_sketch(black_box(&values)));
+ bencher.bench_local(|| build_sketch(black_box(&values)));
+}
+
+pub fn register(runner: &mut Runner) {
+ for &(len, rounds) in &[(1_000, 30_000), (10_000, 2_000), (100_000, 100)] {
+ runner.case(format!("kll::update::update/{len:?}"), rounds, |b| {
+ update(b, len)
+ });
+ }
}
diff --git a/benchmarks/main.rs b/benchmarks/main.rs
index 202f8e0..53bcf14 100644
--- a/benchmarks/main.rs
+++ b/benchmarks/main.rs
@@ -15,11 +15,6 @@
// specific language governing permissions and limitations
// under the License.
-use divan::AllocProfiler;
-
-#[global_allocator]
-static ALLOC: AllocProfiler = AllocProfiler::system();
-
mod bloom;
mod countmin;
mod cpc;
@@ -34,5 +29,17 @@ mod tuple;
mod xor;
fn main() {
- divan::main();
+ let mut runner = benchmarks::Runner::from_args();
+ bloom::register(&mut runner);
+ countmin::register(&mut runner);
+ cpc::register(&mut runner);
+ frequencies::register(&mut runner);
+ hll::register(&mut runner);
+ kll::register(&mut runner);
+ req::register(&mut runner);
+ tdigest::register(&mut runner);
+ theta::register(&mut runner);
+ tuple::register(&mut runner);
+ xor::register(&mut runner);
+ runner.finish();
}
diff --git a/benchmarks/req/mod.rs b/benchmarks/req/mod.rs
index 900ddb9..1b1897f 100644
--- a/benchmarks/req/mod.rs
+++ b/benchmarks/req/mod.rs
@@ -18,3 +18,8 @@
mod query;
mod support;
mod update;
+
+pub fn register(runner: &mut benchmarks::Runner) {
+ query::register(runner);
+ update::register(runner);
+}
diff --git a/benchmarks/req/query.rs b/benchmarks/req/query.rs
index 57f4eae..0e05650 100644
--- a/benchmarks/req/query.rs
+++ b/benchmarks/req/query.rs
@@ -15,14 +15,15 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::common::SearchCriteria;
use datasketches::req::ReqFloat;
-use divan::Bencher;
-use divan::black_box;
use super::support::prepared_sketch;
-#[divan::bench]
fn rank(bencher: Bencher) {
let sketch = prepared_sketch();
let item = ReqFloat::<f64>::new(500_000.0).unwrap();
@@ -30,7 +31,6 @@ fn rank(bencher: Bencher) {
bencher.bench_local(|| black_box(&sketch).rank(black_box(&item),
SearchCriteria::Inclusive));
}
-#[divan::bench(args = [0.0, 0.5, 1.0])]
fn quantile(bencher: Bencher, rank: f64) {
let sketch = prepared_sketch();
@@ -39,10 +39,23 @@ fn quantile(bencher: Bencher, rank: f64) {
// Intended pattern for repeated queries: build the view once, query it many
// times. Contrast against `quantile`, which rebuilds the view on every call.
-#[divan::bench]
fn sorted_view_quantile(bencher: Bencher) {
let sketch = prepared_sketch();
let view = sketch.sorted_view();
bencher.bench_local(|| black_box(&view).quantile(black_box(0.5),
SearchCriteria::Inclusive));
}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("req::query::rank", 6_000_000, rank);
+ for &(rank, rounds) in &[(0.0, 100_000_000), (0.5, 30_000), (1.0,
100_000_000)] {
+ runner.case(format!("req::query::quantile/{rank:?}"), rounds, |b| {
+ quantile(b, rank)
+ });
+ }
+ runner.case(
+ "req::query::sorted_view_quantile",
+ 30_000_000,
+ sorted_view_quantile,
+ );
+}
diff --git a/benchmarks/req/update.rs b/benchmarks/req/update.rs
index 5d8c62a..192d00b 100644
--- a/benchmarks/req/update.rs
+++ b/benchmarks/req/update.rs
@@ -15,18 +15,24 @@
// specific language governing permissions and limitations
// under the License.
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use super::support::build_sketch;
use super::support::values;
-#[divan::bench(args = [1_000, 10_000, 100_000])]
fn update(bencher: Bencher, len: usize) {
let values = values(len);
- bencher
- .counter(ItemsCount::new(len))
- .bench_local(|| build_sketch(black_box(&values)));
+ bencher.bench_local(|| build_sketch(black_box(&values)));
+}
+
+pub fn register(runner: &mut Runner) {
+ for &(len, rounds) in &[(1_000, 20_000), (10_000, 2_000), (100_000, 200)] {
+ runner.case(format!("req::update::update/{len:?}"), rounds, |b| {
+ update(b, len)
+ });
+ }
}
diff --git a/benchmarks/tdigest/aggregation.rs
b/benchmarks/tdigest/aggregation.rs
index dd41823..a59bbeb 100644
--- a/benchmarks/tdigest/aggregation.rs
+++ b/benchmarks/tdigest/aggregation.rs
@@ -15,103 +15,95 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::tdigest::TDigestMut;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use super::support::build_mut_digest;
use super::support::values;
-// (number of partial states, values per state). Each partial samples the same
distribution.
-const SERIALIZED_INPUTS: &[(usize, usize)] =
- &[(1, 8), (64, 8), (512, 8), (64, 64), (512, 64), (64, 4_096)];
+// ((partial states, values per state), rounds). Keep the work equal across
merge strategies.
+const SERIALIZED_INPUTS: &[((usize, usize), usize)] = &[
+ ((1, 8), 2_000_000),
+ ((64, 8), 9_000),
+ ((512, 8), 800),
+ ((64, 64), 5_000),
+ ((512, 64), 300),
+ ((64, 4_096), 2_000),
+];
-#[divan::bench(args = SERIALIZED_INPUTS, sample_size = 1)]
fn serialized_borrowed(bencher: Bencher, shape: (usize, usize)) {
let (num_partials, rows_per_partial) = shape;
let partials = serialize_partials(&values(num_partials *
rows_per_partial), rows_per_partial);
- bencher
- .counter(ItemsCount::new(num_partials * rows_per_partial))
- .bench_local(|| {
- let mut merged = TDigestMut::default();
- for bytes in black_box(&partials) {
- merged.merge(&TDigestMut::deserialize(bytes).unwrap());
- }
- final_quantile(merged)
- });
+ bencher.bench_local(|| {
+ let mut merged = TDigestMut::default();
+ for bytes in black_box(&partials) {
+ merged.merge(&TDigestMut::deserialize(bytes).unwrap());
+ }
+ final_quantile(merged)
+ });
}
-#[divan::bench(args = SERIALIZED_INPUTS, sample_size = 1)]
fn serialized_owned(bencher: Bencher, shape: (usize, usize)) {
let (num_partials, rows_per_partial) = shape;
let partials = serialize_partials(&values(num_partials *
rows_per_partial), rows_per_partial);
- bencher
- .counter(ItemsCount::new(num_partials * rows_per_partial))
- .bench_local(|| {
- let merged = black_box(&partials)
- .iter()
- .map(|bytes| TDigestMut::deserialize(bytes))
- .collect::<Result<TDigestMut, _>>()
- .unwrap();
- final_quantile(merged)
- });
+ bencher.bench_local(|| {
+ let merged = black_box(&partials)
+ .iter()
+ .map(|bytes| TDigestMut::deserialize(bytes))
+ .collect::<Result<TDigestMut, _>>()
+ .unwrap();
+ final_quantile(merged)
+ });
}
-#[divan::bench(args = SERIALIZED_INPUTS, sample_size = 1)]
fn serialized_in_batches(bencher: Bencher, shape: (usize, usize)) {
let (num_partials, rows_per_partial) = shape;
let partials = serialize_partials(&values(num_partials *
rows_per_partial), rows_per_partial);
- bencher
- .counter(ItemsCount::new(num_partials * rows_per_partial))
- .bench_local(|| {
- let mut merged = TDigestMut::default();
- for batch in black_box(&partials).chunks(16) {
- let batch = batch
- .iter()
- .map(|bytes| TDigestMut::deserialize(bytes))
- .collect::<Result<TDigestMut, _>>()
- .unwrap();
- merged.merge(&batch);
- }
- final_quantile(merged)
- });
+ bencher.bench_local(|| {
+ let mut merged = TDigestMut::default();
+ for batch in black_box(&partials).chunks(16) {
+ let batch = batch
+ .iter()
+ .map(|bytes| TDigestMut::deserialize(bytes))
+ .collect::<Result<TDigestMut, _>>()
+ .unwrap();
+ merged.merge(&batch);
+ }
+ final_quantile(merged)
+ });
}
-#[divan::bench(args = [8, 64], sample_size = 1)]
fn end_to_end_borrowed(bencher: Bencher, rows_per_partial: usize) {
let inputs = values(64 * rows_per_partial);
- bencher
- .counter(ItemsCount::new(inputs.len()))
- .bench_local(|| {
- let partials = serialize_partials(black_box(&inputs),
rows_per_partial);
- let mut merged = TDigestMut::default();
- for bytes in &partials {
- merged.merge(&TDigestMut::deserialize(bytes).unwrap());
- }
- final_quantile(merged)
- });
+ bencher.bench_local(|| {
+ let partials = serialize_partials(black_box(&inputs),
rows_per_partial);
+ let mut merged = TDigestMut::default();
+ for bytes in &partials {
+ merged.merge(&TDigestMut::deserialize(bytes).unwrap());
+ }
+ final_quantile(merged)
+ });
}
-#[divan::bench(args = [8, 64], sample_size = 1)]
fn end_to_end_owned(bencher: Bencher, rows_per_partial: usize) {
let inputs = values(64 * rows_per_partial);
- bencher
- .counter(ItemsCount::new(inputs.len()))
- .bench_local(|| {
- let partials = serialize_partials(black_box(&inputs),
rows_per_partial);
- let merged = partials
- .iter()
- .map(|bytes| TDigestMut::deserialize(bytes))
- .collect::<Result<TDigestMut, _>>()
- .unwrap();
- final_quantile(merged)
- });
+ bencher.bench_local(|| {
+ let partials = serialize_partials(black_box(&inputs),
rows_per_partial);
+ let merged = partials
+ .iter()
+ .map(|bytes| TDigestMut::deserialize(bytes))
+ .collect::<Result<TDigestMut, _>>()
+ .unwrap();
+ final_quantile(merged)
+ });
}
fn serialize_partials(inputs: &[f64], rows_per_partial: usize) -> Vec<Vec<u8>>
{
@@ -129,3 +121,42 @@ fn final_quantile(mut merged: TDigestMut) -> Option<f64> {
.quantile(black_box(0.9))
.unwrap()
}
+
+pub fn register(runner: &mut Runner) {
+ let end_to_end_cases = &[(8, 8_000), (64, 3_000)];
+ for &(shape, rounds) in SERIALIZED_INPUTS {
+ runner.case(
+ format!("tdigest::aggregation::serialized_borrowed/{shape:?}"),
+ rounds,
+ |b| serialized_borrowed(b, shape),
+ );
+ }
+ for &(shape, rounds) in SERIALIZED_INPUTS {
+ runner.case(
+ format!("tdigest::aggregation::serialized_owned/{shape:?}"),
+ rounds,
+ |b| serialized_owned(b, shape),
+ );
+ }
+ for &(shape, rounds) in SERIALIZED_INPUTS {
+ runner.case(
+ format!("tdigest::aggregation::serialized_in_batches/{shape:?}"),
+ rounds,
+ |b| serialized_in_batches(b, shape),
+ );
+ }
+ for &(rows_per_partial, rounds) in end_to_end_cases {
+ runner.case(
+
format!("tdigest::aggregation::end_to_end_borrowed/{rows_per_partial:?}"),
+ rounds,
+ |b| end_to_end_borrowed(b, rows_per_partial),
+ );
+ }
+ for &(rows_per_partial, rounds) in end_to_end_cases {
+ runner.case(
+
format!("tdigest::aggregation::end_to_end_owned/{rows_per_partial:?}"),
+ rounds,
+ |b| end_to_end_owned(b, rows_per_partial),
+ );
+ }
+}
diff --git a/benchmarks/tdigest/compress.rs b/benchmarks/tdigest/compress.rs
index ead1900..7508565 100644
--- a/benchmarks/tdigest/compress.rs
+++ b/benchmarks/tdigest/compress.rs
@@ -15,26 +15,25 @@
// specific language governing permissions and limitations
// under the License.
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use super::support::build_mut_digest;
use super::support::values;
-#[divan::bench]
fn initial_buffer(bencher: Bencher) {
// The default k=200 digest buffers 1,640 values before automatic
compression.
let values = values(1_640);
let digest = build_mut_digest(&values);
- bencher
- .counter(ItemsCount::new(values.len()))
- .with_inputs(|| digest.clone())
- .bench_local_values(|mut digest| black_box(digest.rank(0.5).unwrap()));
+ bencher.bench_local(|| {
+ let mut digest = digest.clone();
+ black_box(digest.rank(0.5).unwrap())
+ });
}
-#[divan::bench]
fn unmerged_tail(bencher: Bencher) {
let values = values(3_280);
let mut digest = build_mut_digest(&values[..1_640]);
@@ -43,19 +42,28 @@ fn unmerged_tail(bencher: Bencher) {
digest.update(value);
}
- bencher
- .counter(ItemsCount::new(1_640_usize))
- .with_inputs(|| digest.clone())
- .bench_local_values(|mut digest| black_box(digest.rank(0.5).unwrap()));
+ bencher.bench_local(|| {
+ let mut digest = digest.clone();
+ black_box(digest.rank(0.5).unwrap())
+ });
}
-#[divan::bench]
fn freeze_initial_buffer(bencher: Bencher) {
let values = values(1_640);
let digest = build_mut_digest(&values);
- bencher
- .counter(ItemsCount::new(values.len()))
- .with_inputs(|| digest.clone())
- .bench_local_values(|digest| black_box(digest.freeze()));
+ bencher.bench_local(|| {
+ let digest = digest.clone();
+ black_box(digest.freeze())
+ });
+}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("tdigest::compress::initial_buffer", 20_000, initial_buffer);
+ runner.case("tdigest::compress::unmerged_tail", 10_000, unmerged_tail);
+ runner.case(
+ "tdigest::compress::freeze_initial_buffer",
+ 20_000,
+ freeze_initial_buffer,
+ );
}
diff --git a/benchmarks/tdigest/merge.rs b/benchmarks/tdigest/merge.rs
index d8ae273..affdc26 100644
--- a/benchmarks/tdigest/merge.rs
+++ b/benchmarks/tdigest/merge.rs
@@ -15,10 +15,11 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::tdigest::TDigestMut;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use super::support::DEFAULT_DIGEST_K;
use super::support::ROWS_PER_PARTIAL;
@@ -29,7 +30,6 @@ use super::support::partial_digests_with;
use super::support::serialized_partial_digests;
use super::support::values;
-#[divan::bench]
fn merge(bencher: Bencher) {
let values = values(200_000);
let mut left = build_mut_digest(&values[..100_000]);
@@ -37,30 +37,24 @@ fn merge(bencher: Bencher) {
black_box(left.rank(0.5).unwrap());
black_box(right.rank(0.5).unwrap());
- bencher
- .counter(ItemsCount::new(values.len()))
- .with_inputs(|| left.clone())
- .bench_local_values(|mut left| {
- left.merge(black_box(&right));
- black_box(left)
- });
+ bencher.bench_local(|| {
+ let mut left = left.clone();
+ left.merge(black_box(&right));
+ black_box(left)
+ });
}
-#[divan::bench(args = [8, 1_640])]
fn unmerged(bencher: Bencher, left_rows: usize) {
let left = build_mut_digest(&values(left_rows));
let right = build_mut_digest(&values(1_640));
- bencher
- .counter(ItemsCount::new(left_rows + 1_640))
- .with_inputs(|| left.clone())
- .bench_local_values(|mut left| {
- left.merge(black_box(&right));
- black_box(left)
- });
+ bencher.bench_local(|| {
+ let mut left = left.clone();
+ left.merge(black_box(&right));
+ black_box(left)
+ });
}
-#[divan::bench]
fn small_partials(bencher: Bencher) {
let partials = partial_digests(64, SMALL_ROWS_PER_PARTIAL)
.into_iter()
@@ -70,18 +64,15 @@ fn small_partials(bencher: Bencher) {
})
.collect::<Vec<_>>();
- bencher
- .counter(ItemsCount::new(64 * SMALL_ROWS_PER_PARTIAL))
- .bench_local(|| {
- let mut merged = TDigestMut::default();
- for partial in &partials {
- merged.merge(black_box(partial));
- }
- black_box(merged)
- });
+ bencher.bench_local(|| {
+ let mut merged = TDigestMut::default();
+ for partial in &partials {
+ merged.merge(black_box(partial));
+ }
+ black_box(merged)
+ });
}
-#[divan::bench]
fn partials(bencher: Bencher) {
let partials = partial_digests_with(DEFAULT_DIGEST_K, 64, ROWS_PER_PARTIAL)
.into_iter()
@@ -91,18 +82,15 @@ fn partials(bencher: Bencher) {
})
.collect::<Vec<_>>();
- bencher
- .counter(ItemsCount::new(64 * ROWS_PER_PARTIAL))
- .bench_local(|| {
- let mut merged = TDigestMut::new(DEFAULT_DIGEST_K).unwrap();
- for partial in &partials {
- merged.merge(black_box(partial));
- }
- black_box(merged)
- });
+ bencher.bench_local(|| {
+ let mut merged = TDigestMut::new(DEFAULT_DIGEST_K).unwrap();
+ for partial in &partials {
+ merged.merge(black_box(partial));
+ }
+ black_box(merged)
+ });
}
-#[divan::bench]
fn partials_from_iter(bencher: Bencher) {
let partials = partial_digests_with(DEFAULT_DIGEST_K, 64, ROWS_PER_PARTIAL)
.into_iter()
@@ -112,15 +100,12 @@ fn partials_from_iter(bencher: Bencher) {
})
.collect::<Vec<_>>();
- bencher
- .counter(ItemsCount::new(64 * ROWS_PER_PARTIAL))
- // Construct fresh owned inputs outside the measurement, as a real
caller transfers
- // ownership rather than cloning solely for `FromIterator`.
- .with_inputs(|| partials.clone())
- .bench_local_values(|partials|
black_box(partials).into_iter().collect::<TDigestMut>());
+ bencher.bench_local(|| {
+ let partials = partials.clone();
+ black_box(partials).into_iter().collect::<TDigestMut>()
+ });
}
-#[divan::bench]
fn partials_from_iter_cloned(bencher: Bencher) {
let partials = partial_digests_with(DEFAULT_DIGEST_K, 64, ROWS_PER_PARTIAL)
.into_iter()
@@ -131,12 +116,10 @@ fn partials_from_iter_cloned(bencher: Bencher) {
.collect::<Vec<_>>();
bencher
- .counter(ItemsCount::new(64 * ROWS_PER_PARTIAL))
- // Include cloning when the caller needs to retain the original inputs.
+ // Cloning through the iterator avoids a temporary vector of owned
inputs.
.bench_local(||
black_box(&partials).iter().cloned().collect::<TDigestMut>());
}
-#[divan::bench(args = [0, 64, 4_096])]
fn partials_from_iter_finalize(bencher: Bencher, additional_rows: usize) {
let partials = partial_digests_with(DEFAULT_DIGEST_K, 64, ROWS_PER_PARTIAL)
.into_iter()
@@ -150,19 +133,16 @@ fn partials_from_iter_finalize(bencher: Bencher,
additional_rows: usize) {
.map(|value| value * (64 * ROWS_PER_PARTIAL) as f64)
.collect::<Vec<_>>();
- bencher
- .counter(ItemsCount::new(64 * ROWS_PER_PARTIAL + additional_rows))
- .with_inputs(|| partials.clone())
- .bench_local_values(|partials| {
- let mut merged =
black_box(partials).into_iter().collect::<TDigestMut>();
- for &value in black_box(&updates) {
- merged.update(value);
- }
- black_box(merged.freeze())
- });
+ bencher.bench_local(|| {
+ let partials = partials.clone();
+ let mut merged =
black_box(partials).into_iter().collect::<TDigestMut>();
+ for &value in black_box(&updates) {
+ merged.update(value);
+ }
+ black_box(merged.freeze())
+ });
}
-#[divan::bench]
fn uncompressed_partials_from_iter(bencher: Bencher) {
let values = values(64 * ROWS_PER_PARTIAL);
let partials = values
@@ -170,29 +150,25 @@ fn uncompressed_partials_from_iter(bencher: Bencher) {
.map(build_mut_digest)
.collect::<Vec<_>>();
- bencher
- .counter(ItemsCount::new(values.len()))
- .with_inputs(|| partials.clone())
- .bench_local_values(|partials|
black_box(partials).into_iter().collect::<TDigestMut>());
+ bencher.bench_local(|| {
+ let partials = partials.clone();
+ black_box(partials).into_iter().collect::<TDigestMut>()
+ });
}
-#[divan::bench(args = [SMALL_ROWS_PER_PARTIAL, ROWS_PER_PARTIAL])]
fn serialized_partials(bencher: Bencher, rows_per_partial: usize) {
let partials = serialized_partial_digests(64, rows_per_partial);
- bencher
- .counter(ItemsCount::new(64 * rows_per_partial))
- .bench_local(|| {
- let mut merged = TDigestMut::default();
- for partial in black_box(&partials) {
- let partial = TDigestMut::deserialize(partial).unwrap();
- merged.merge(&partial);
- }
- black_box(merged.quantile(0.5).unwrap())
- });
+ bencher.bench_local(|| {
+ let mut merged = TDigestMut::default();
+ for partial in black_box(&partials) {
+ let partial = TDigestMut::deserialize(partial).unwrap();
+ merged.merge(&partial);
+ }
+ black_box(merged.quantile(0.5).unwrap())
+ });
}
-#[divan::bench]
fn serialized_overlapping_partials(bencher: Bencher) {
let values = values(64 * ROWS_PER_PARTIAL);
let partials = values
@@ -200,52 +176,44 @@ fn serialized_overlapping_partials(bencher: Bencher) {
.map(|values| build_mut_digest(values).serialize())
.collect::<Vec<_>>();
- bencher
- .counter(ItemsCount::new(values.len()))
- .bench_local(|| {
- let mut merged = TDigestMut::default();
- for partial in black_box(&partials) {
- let partial = TDigestMut::deserialize(partial).unwrap();
- merged.merge(&partial);
- }
- black_box(merged.quantile(0.5).unwrap())
- });
+ bencher.bench_local(|| {
+ let mut merged = TDigestMut::default();
+ for partial in black_box(&partials) {
+ let partial = TDigestMut::deserialize(partial).unwrap();
+ merged.merge(&partial);
+ }
+ black_box(merged.quantile(0.5).unwrap())
+ });
}
-#[divan::bench(args = [false, true])]
fn serialized_partials_from_iter(bencher: Bencher, overlapping: bool) {
let partials = serialized_merge_inputs(overlapping);
- bencher
- .counter(ItemsCount::new(64 * ROWS_PER_PARTIAL))
- .bench_local(|| {
- let mut merged = black_box(&partials)
- .iter()
- .map(|partial| TDigestMut::deserialize(partial))
- .collect::<Result<TDigestMut, _>>()
- .unwrap();
- black_box(merged.quantile(0.5).unwrap())
- });
+ bencher.bench_local(|| {
+ let mut merged = black_box(&partials)
+ .iter()
+ .map(|partial| TDigestMut::deserialize(partial))
+ .collect::<Result<TDigestMut, _>>()
+ .unwrap();
+ black_box(merged.quantile(0.5).unwrap())
+ });
}
-#[divan::bench(args = [false, true])]
fn serialized_partials_in_batches(bencher: Bencher, overlapping: bool) {
let partials = serialized_merge_inputs(overlapping);
- bencher
- .counter(ItemsCount::new(64 * ROWS_PER_PARTIAL))
- .bench_local(|| {
- let mut merged = TDigestMut::new(DEFAULT_DIGEST_K).unwrap();
- for batch in black_box(&partials).chunks(16) {
- let batch = batch
- .iter()
- .map(|partial| TDigestMut::deserialize(partial))
- .collect::<Result<TDigestMut, _>>()
- .unwrap();
- merged.merge(&batch);
- }
- black_box(merged.quantile(0.5).unwrap())
- });
+ bencher.bench_local(|| {
+ let mut merged = TDigestMut::new(DEFAULT_DIGEST_K).unwrap();
+ for batch in black_box(&partials).chunks(16) {
+ let batch = batch
+ .iter()
+ .map(|partial| TDigestMut::deserialize(partial))
+ .collect::<Result<TDigestMut, _>>()
+ .unwrap();
+ merged.merge(&batch);
+ }
+ black_box(merged.quantile(0.5).unwrap())
+ });
}
fn serialized_merge_inputs(overlapping: bool) -> Vec<Vec<u8>> {
@@ -258,3 +226,66 @@ fn serialized_merge_inputs(overlapping: bool) ->
Vec<Vec<u8>> {
serialized_partial_digests(64, ROWS_PER_PARTIAL)
}
}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("tdigest::merge::merge", 150_000, merge);
+ for &(left_rows, rounds) in &[(8, 20_000), (1_640, 8_000)] {
+ runner.case(
+ format!("tdigest::merge::unmerged/{left_rows:?}"),
+ rounds,
+ |b| unmerged(b, left_rows),
+ );
+ }
+ runner.case("tdigest::merge::small_partials", 10_000, small_partials);
+ runner.case("tdigest::merge::partials", 7_000, partials);
+ runner.case(
+ "tdigest::merge::partials_from_iter",
+ 20_000,
+ partials_from_iter,
+ );
+ runner.case(
+ "tdigest::merge::partials_from_iter_cloned",
+ 20_000,
+ partials_from_iter_cloned,
+ );
+ for &(additional_rows, rounds) in &[(0, 10_000), (64, 10_000), (4_096,
3_000)] {
+ runner.case(
+
format!("tdigest::merge::partials_from_iter_finalize/{additional_rows:?}"),
+ rounds,
+ |b| partials_from_iter_finalize(b, additional_rows),
+ );
+ }
+ runner.case(
+ "tdigest::merge::uncompressed_partials_from_iter",
+ 6_000,
+ uncompressed_partials_from_iter,
+ );
+ for &(rows_per_partial, rounds) in
+ &[(SMALL_ROWS_PER_PARTIAL, 10_000), (ROWS_PER_PARTIAL, 6_000)]
+ {
+ runner.case(
+
format!("tdigest::merge::serialized_partials/{rows_per_partial:?}"),
+ rounds,
+ |b| serialized_partials(b, rows_per_partial),
+ );
+ }
+ runner.case(
+ "tdigest::merge::serialized_overlapping_partials",
+ 5_000,
+ serialized_overlapping_partials,
+ );
+ for &(overlapping, rounds) in &[(false, 10_000), (true, 6_000)] {
+ runner.case(
+
format!("tdigest::merge::serialized_partials_from_iter/{overlapping:?}"),
+ rounds,
+ |b| serialized_partials_from_iter(b, overlapping),
+ );
+ }
+ for &(overlapping, rounds) in &[(false, 9_000), (true, 7_000)] {
+ runner.case(
+
format!("tdigest::merge::serialized_partials_in_batches/{overlapping:?}"),
+ rounds,
+ |b| serialized_partials_in_batches(b, overlapping),
+ );
+ }
+}
diff --git a/benchmarks/tdigest/mod.rs b/benchmarks/tdigest/mod.rs
index ce18622..cf12e9f 100644
--- a/benchmarks/tdigest/mod.rs
+++ b/benchmarks/tdigest/mod.rs
@@ -22,3 +22,12 @@ mod query;
mod serde;
mod support;
mod update;
+
+pub fn register(runner: &mut benchmarks::Runner) {
+ aggregation::register(runner);
+ compress::register(runner);
+ merge::register(runner);
+ query::register(runner);
+ serde::register(runner);
+ update::register(runner);
+}
diff --git a/benchmarks/tdigest/query.rs b/benchmarks/tdigest/query.rs
index d3cd330..8a57646 100644
--- a/benchmarks/tdigest/query.rs
+++ b/benchmarks/tdigest/query.rs
@@ -15,9 +15,10 @@
// specific language governing permissions and limitations
// under the License.
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use rand::SeedableRng;
use rand::rngs::StdRng;
use rand::seq::SliceRandom;
@@ -26,42 +27,35 @@ use super::support::build_digest;
use super::support::prepared_digest;
use super::support::values;
-#[divan::bench]
fn rank(bencher: Bencher) {
let digest = prepared_digest();
bencher.bench_local(||
black_box(&digest).rank(black_box(0.531_25)).unwrap());
}
-#[divan::bench]
fn rank_mixed(bencher: Bencher) {
let digest = prepared_digest();
let queries = values(256);
- bencher
- .counter(ItemsCount::new(queries.len()))
- .bench_local(|| {
- for &value in black_box(&queries) {
- black_box(black_box(&digest).rank(black_box(value)).unwrap());
- }
- });
+ bencher.bench_local(|| {
+ for &value in black_box(&queries) {
+ black_box(black_box(&digest).rank(black_box(value)).unwrap());
+ }
+ });
}
-#[divan::bench(args = [0.0, 2.5, 5.0])]
fn rank_small(bencher: Bencher, value: f64) {
let digest = build_digest(&[0., 1., 2., 3., 4., 5.]);
bencher.bench_local(|| black_box(&digest).rank(black_box(value)).unwrap());
}
-#[divan::bench]
fn rank_mut(bencher: Bencher) {
let mut digest = prepared_digest().unfreeze();
bencher.bench_local(|| black_box(&mut
digest).rank(black_box(0.531_25)).unwrap());
}
-#[divan::bench(args = [0.0, 3.0, 3.5, 7.0])]
fn rank_repeated_means(bencher: Bencher, value: f64) {
let values = (0..100_000).map(|i| (i % 8) as f64).collect::<Vec<_>>();
let digest = build_digest(&values);
@@ -69,55 +63,44 @@ fn rank_repeated_means(bencher: Bencher, value: f64) {
bencher.bench_local(|| black_box(&digest).rank(black_box(value)).unwrap());
}
-#[divan::bench]
fn quantile(bencher: Bencher) {
let digest = prepared_digest();
bencher.bench_local(||
black_box(&digest).quantile(black_box(0.531_25)).unwrap());
}
-#[divan::bench]
fn cdf_100(bencher: Bencher) {
let digest = prepared_digest();
let split_points = (1..=100).map(|i| i as f64 / 101.0).collect::<Vec<_>>();
- bencher
- .counter(ItemsCount::new(split_points.len()))
- .bench_local(||
black_box(&digest).cdf(black_box(&split_points)).unwrap());
+ bencher.bench_local(||
black_box(&digest).cdf(black_box(&split_points)).unwrap());
}
-#[divan::bench(args = [1, 6, 100, 512])]
fn pmf(bencher: Bencher, num_split_points: usize) {
let digest = prepared_digest();
let split_points = (1..=num_split_points)
.map(|i| i as f64 / (num_split_points + 1) as f64)
.collect::<Vec<_>>();
- bencher
- .counter(ItemsCount::new(num_split_points))
- .bench_local(||
black_box(&digest).pmf(black_box(&split_points)).unwrap());
+ bencher.bench_local(||
black_box(&digest).pmf(black_box(&split_points)).unwrap());
}
-#[divan::bench(consts = [true, false], args = [2, 6, 100])]
fn quantiles_sequential<const SORTED: bool>(bencher: Bencher, num_ranks:
usize) {
let digest = prepared_digest();
let ranks = query_ranks(num_ranks, SORTED);
- bencher.counter(ItemsCount::new(num_ranks)).bench_local(|| {
+ bencher.bench_local(|| {
for &rank in black_box(&ranks) {
black_box(black_box(&digest).quantile(black_box(rank)).unwrap());
}
});
}
-#[divan::bench(consts = [true, false], args = [2, 6, 100])]
fn quantiles_batch<const SORTED: bool>(bencher: Bencher, num_ranks: usize) {
let digest = prepared_digest();
let ranks = query_ranks(num_ranks, SORTED);
- bencher
- .counter(ItemsCount::new(num_ranks))
- .bench_local(||
black_box(&digest).quantiles(black_box(&ranks)).unwrap());
+ bencher.bench_local(||
black_box(&digest).quantiles(black_box(&ranks)).unwrap());
}
fn query_ranks(num_ranks: usize, sorted: bool) -> Vec<f64> {
@@ -132,3 +115,71 @@ fn query_ranks(num_ranks: usize, sorted: bool) -> Vec<f64>
{
}
ranks
}
+
+pub fn register(runner: &mut Runner) {
+ let quantile_cases = &[(2, 2_000_000), (6, 700_000), (100, 40_000)];
+ runner.case("tdigest::query::rank", 10_000_000, rank);
+ runner.case("tdigest::query::rank_mixed", 40_000, rank_mixed);
+ for &(value, rounds) in &[(0.0, 80_000_000), (2.5, 80_000_000), (5.0,
50_000_000)] {
+ runner.case(
+ format!("tdigest::query::rank_small/{value:?}"),
+ rounds,
+ |b| rank_small(b, value),
+ );
+ }
+ runner.case("tdigest::query::rank_mut", 9_000_000, rank_mut);
+ for &(value, rounds) in &[
+ (0.0, 20_000_000),
+ (3.0, 10_000_000),
+ (3.5, 10_000_000),
+ (7.0, 7_000_000),
+ ] {
+ runner.case(
+ format!("tdigest::query::rank_repeated_means/{value:?}"),
+ rounds,
+ |b| rank_repeated_means(b, value),
+ );
+ }
+ runner.case("tdigest::query::quantile", 4_000_000, quantile);
+ runner.case("tdigest::query::cdf_100", 600_000, cdf_100);
+ for &(num_split_points, rounds) in &[
+ (1, 5_000_000),
+ (6, 2_000_000),
+ (100, 600_000),
+ (512, 100_000),
+ ] {
+ runner.case(
+ format!("tdigest::query::pmf/{num_split_points:?}"),
+ rounds,
+ |b| pmf(b, num_split_points),
+ );
+ }
+ for &(num_ranks, rounds) in quantile_cases {
+ runner.case(
+
format!("tdigest::query::quantiles_sequential<true>/{num_ranks:?}"),
+ rounds,
+ |b| quantiles_sequential::<true>(b, num_ranks),
+ );
+ }
+ for &(num_ranks, rounds) in quantile_cases {
+ runner.case(
+
format!("tdigest::query::quantiles_sequential<false>/{num_ranks:?}"),
+ rounds,
+ |b| quantiles_sequential::<false>(b, num_ranks),
+ );
+ }
+ for &(num_ranks, rounds) in quantile_cases {
+ runner.case(
+ format!("tdigest::query::quantiles_batch<true>/{num_ranks:?}"),
+ rounds,
+ |b| quantiles_batch::<true>(b, num_ranks),
+ );
+ }
+ for &(num_ranks, rounds) in quantile_cases {
+ runner.case(
+ format!("tdigest::query::quantiles_batch<false>/{num_ranks:?}"),
+ rounds,
+ |b| quantiles_batch::<false>(b, num_ranks),
+ );
+ }
+}
diff --git a/benchmarks/tdigest/serde.rs b/benchmarks/tdigest/serde.rs
index 7d5e0a8..f8bb501 100644
--- a/benchmarks/tdigest/serde.rs
+++ b/benchmarks/tdigest/serde.rs
@@ -15,12 +15,11 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::tdigest::TDigestMut;
-use divan::Bencher;
-use divan::black_box;
-use divan::black_box_drop;
-use divan::counter::BytesCount;
-use divan::counter::ItemsCount;
use super::support::DEFAULT_DIGEST_K;
use super::support::PARTIAL_GROUPS;
@@ -33,26 +32,20 @@ use super::support::serialized_partial_digests_with;
use super::support::serialized_state_shape;
use super::support::values;
-#[divan::bench(args = [1, 2, 8, 32, 64, 128])]
fn small_lifecycle(bencher: Bencher, rows: usize) {
let values = values(rows);
- let (serialized_bytes, _) = serialized_state_shape(DEFAULT_DIGEST_K,
&values);
-
- bencher
- .counter(ItemsCount::new(rows))
- .counter(BytesCount::new(serialized_bytes))
- .bench_local(|| {
- let mut digest = TDigestMut::default();
- for &value in black_box(&values) {
- digest.update(value);
- }
- let bytes = digest.serialize();
- black_box_drop(bytes);
- black_box_drop(digest);
- });
+
+ bencher.bench_local(|| {
+ let mut digest = TDigestMut::default();
+ for &value in black_box(&values) {
+ digest.update(value);
+ }
+ let bytes = digest.serialize();
+ drop(black_box(bytes));
+ drop(black_box(digest));
+ });
}
-#[divan::bench(args = [10_u16, 200_u16])]
fn partial_lifecycle_by_k(bencher: Bencher, k: u16) {
let values = values(ROWS_PER_PARTIAL);
let (serialized_bytes, centroids) = serialized_state_shape(k, &values);
@@ -61,78 +54,107 @@ fn partial_lifecycle_by_k(bencher: Bencher, k: u16) {
(10, 224, 12) | (200, 1_056, 64)
));
- bencher
- .counter(ItemsCount::new(ROWS_PER_PARTIAL))
- .counter(BytesCount::new(serialized_bytes))
- .bench_local(|| {
- let mut digest = TDigestMut::new(black_box(k)).unwrap();
- for &value in black_box(&values) {
- digest.update(value);
- }
- let bytes = digest.serialize();
- black_box_drop(bytes);
- black_box_drop(digest);
- });
+ bencher.bench_local(|| {
+ let mut digest = TDigestMut::new(black_box(k)).unwrap();
+ for &value in black_box(&values) {
+ digest.update(value);
+ }
+ let bytes = digest.serialize();
+ drop(black_box(bytes));
+ drop(black_box(digest));
+ });
}
-#[divan::bench]
fn serialize_small_partial_groups(bencher: Bencher) {
let digests = partial_digests(PARTIAL_GROUPS, SMALL_ROWS_PER_PARTIAL);
- bencher
- .counter(ItemsCount::new(PARTIAL_GROUPS))
- .with_inputs(|| digests.clone())
- .bench_local_values(|mut digests| {
- let bytes = digests
- .iter_mut()
- .map(TDigestMut::serialize)
- .collect::<Vec<_>>();
- black_box(bytes)
- });
+ bencher.bench_local(|| {
+ let mut digests = digests.clone();
+ let bytes = digests
+ .iter_mut()
+ .map(TDigestMut::serialize)
+ .collect::<Vec<_>>();
+ black_box(bytes)
+ });
}
-#[divan::bench]
fn deserialize_small_partial_groups(bencher: Bencher) {
let bytes = serialized_partial_digests(PARTIAL_GROUPS,
SMALL_ROWS_PER_PARTIAL);
- bencher
- .counter(ItemsCount::new(PARTIAL_GROUPS))
- .bench_local(|| {
- let digests = bytes
- .iter()
- .map(|bytes| TDigestMut::deserialize(bytes).unwrap())
- .collect::<Vec<_>>();
- black_box(digests)
- });
+ bencher.bench_local(|| {
+ let digests = bytes
+ .iter()
+ .map(|bytes| TDigestMut::deserialize(bytes).unwrap())
+ .collect::<Vec<_>>();
+ black_box(digests)
+ });
}
-#[divan::bench]
fn serialize_partial_groups(bencher: Bencher) {
let digests = partial_digests_with(DEFAULT_DIGEST_K, PARTIAL_GROUPS,
ROWS_PER_PARTIAL);
- bencher
- .counter(ItemsCount::new(PARTIAL_GROUPS))
- .with_inputs(|| digests.clone())
- .bench_local_values(|mut digests| {
- let bytes = digests
- .iter_mut()
- .map(TDigestMut::serialize)
- .collect::<Vec<_>>();
- black_box(bytes)
- });
+ bencher.bench_local(|| {
+ let mut digests = digests.clone();
+ let bytes = digests
+ .iter_mut()
+ .map(TDigestMut::serialize)
+ .collect::<Vec<_>>();
+ black_box(bytes)
+ });
}
-#[divan::bench]
fn deserialize_partial_groups(bencher: Bencher) {
let bytes = serialized_partial_digests_with(DEFAULT_DIGEST_K,
PARTIAL_GROUPS, ROWS_PER_PARTIAL);
- bencher
- .counter(ItemsCount::new(PARTIAL_GROUPS))
- .bench_local(|| {
- let digests = bytes
- .iter()
- .map(|bytes| TDigestMut::deserialize(bytes).unwrap())
- .collect::<Vec<_>>();
- black_box(digests)
- });
+ bencher.bench_local(|| {
+ let digests = bytes
+ .iter()
+ .map(|bytes| TDigestMut::deserialize(bytes).unwrap())
+ .collect::<Vec<_>>();
+ black_box(digests)
+ });
+}
+
+pub fn register(runner: &mut Runner) {
+ for &(rows, rounds) in &[
+ (1, 7_000_000),
+ (2, 6_000_000),
+ (8, 2_000_000),
+ (32, 2_000_000),
+ (64, 500_000),
+ (128, 200_000),
+ ] {
+ runner.case(
+ format!("tdigest::serde::small_lifecycle/{rows:?}"),
+ rounds,
+ |b| small_lifecycle(b, rows),
+ );
+ }
+ for &(k, rounds) in &[(10_u16, 400_000), (200_u16, 500_000)] {
+ runner.case(
+ format!("tdigest::serde::partial_lifecycle_by_k/{k:?}"),
+ rounds,
+ |b| partial_lifecycle_by_k(b, k),
+ );
+ }
+ runner.case(
+ "tdigest::serde::serialize_small_partial_groups",
+ 10_000,
+ serialize_small_partial_groups,
+ );
+ runner.case(
+ "tdigest::serde::deserialize_small_partial_groups",
+ 10_000,
+ deserialize_small_partial_groups,
+ );
+ runner.case(
+ "tdigest::serde::serialize_partial_groups",
+ 4_000,
+ serialize_partial_groups,
+ );
+ runner.case(
+ "tdigest::serde::deserialize_partial_groups",
+ 4_000,
+ deserialize_partial_groups,
+ );
}
diff --git a/benchmarks/tdigest/update.rs b/benchmarks/tdigest/update.rs
index 81dbd15..e26f336 100644
--- a/benchmarks/tdigest/update.rs
+++ b/benchmarks/tdigest/update.rs
@@ -15,10 +15,11 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::tdigest::TDigestMut;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use super::support::DEFAULT_DIGEST_K;
use super::support::PARTIAL_GROUPS;
@@ -28,88 +29,101 @@ use super::support::build_digest;
use super::support::partial_value;
use super::support::values;
-#[divan::bench(args = [1_000, 100_000])]
fn update(bencher: Bencher, len: usize) {
let values = values(len);
- bencher
- .counter(ItemsCount::new(len))
- .bench_local(|| build_digest(black_box(&values)));
+ bencher.bench_local(|| build_digest(black_box(&values)));
}
-#[divan::bench]
fn small_partial_groups_update(bencher: Bencher) {
- bencher
- .counter(ItemsCount::new(PARTIAL_GROUPS * SMALL_ROWS_PER_PARTIAL))
- .bench_local(|| {
- let mut digests = (0..PARTIAL_GROUPS)
- .map(|_| TDigestMut::default())
- .collect::<Vec<_>>();
- for (group, digest) in digests.iter_mut().enumerate() {
- for row in 0..SMALL_ROWS_PER_PARTIAL {
- digest.update(partial_value(group, row,
SMALL_ROWS_PER_PARTIAL));
- }
+ bencher.bench_local(|| {
+ let mut digests = (0..PARTIAL_GROUPS)
+ .map(|_| TDigestMut::default())
+ .collect::<Vec<_>>();
+ for (group, digest) in digests.iter_mut().enumerate() {
+ for row in 0..SMALL_ROWS_PER_PARTIAL {
+ digest.update(partial_value(group, row,
SMALL_ROWS_PER_PARTIAL));
}
- black_box(digests)
- });
+ }
+ black_box(digests)
+ });
}
-#[divan::bench]
fn small_partial_groups_update_two_states(bencher: Bencher) {
- bencher
- .counter(ItemsCount::new(PARTIAL_GROUPS * SMALL_ROWS_PER_PARTIAL))
- .bench_local(|| {
- let mut digests = (0..PARTIAL_GROUPS)
- .map(|_| (TDigestMut::default(), TDigestMut::default()))
- .collect::<Vec<_>>();
- for (group, (first, second)) in digests.iter_mut().enumerate() {
- for row in 0..SMALL_ROWS_PER_PARTIAL {
- let value = partial_value(group, row,
SMALL_ROWS_PER_PARTIAL);
- first.update(value);
- second.update(value);
- }
+ bencher.bench_local(|| {
+ let mut digests = (0..PARTIAL_GROUPS)
+ .map(|_| (TDigestMut::default(), TDigestMut::default()))
+ .collect::<Vec<_>>();
+ for (group, (first, second)) in digests.iter_mut().enumerate() {
+ for row in 0..SMALL_ROWS_PER_PARTIAL {
+ let value = partial_value(group, row, SMALL_ROWS_PER_PARTIAL);
+ first.update(value);
+ second.update(value);
}
- black_box(digests)
- });
+ }
+ black_box(digests)
+ });
}
-#[divan::bench]
fn partial_groups_update(bencher: Bencher) {
- bencher
- .counter(ItemsCount::new(PARTIAL_GROUPS * ROWS_PER_PARTIAL))
- .bench_local(|| {
- let mut digests = (0..PARTIAL_GROUPS)
- .map(|_| TDigestMut::new(DEFAULT_DIGEST_K).unwrap())
- .collect::<Vec<_>>();
- for (group, digest) in digests.iter_mut().enumerate() {
- for row in 0..ROWS_PER_PARTIAL {
- digest.update(partial_value(group, row, ROWS_PER_PARTIAL));
- }
+ bencher.bench_local(|| {
+ let mut digests = (0..PARTIAL_GROUPS)
+ .map(|_| TDigestMut::new(DEFAULT_DIGEST_K).unwrap())
+ .collect::<Vec<_>>();
+ for (group, digest) in digests.iter_mut().enumerate() {
+ for row in 0..ROWS_PER_PARTIAL {
+ digest.update(partial_value(group, row, ROWS_PER_PARTIAL));
}
- black_box(digests)
- });
+ }
+ black_box(digests)
+ });
}
-#[divan::bench]
fn partial_groups_update_two_states(bencher: Bencher) {
- bencher
- .counter(ItemsCount::new(PARTIAL_GROUPS * ROWS_PER_PARTIAL))
- .bench_local(|| {
- let mut digests = (0..PARTIAL_GROUPS)
- .map(|_| {
- (
- TDigestMut::new(DEFAULT_DIGEST_K).unwrap(),
- TDigestMut::new(DEFAULT_DIGEST_K).unwrap(),
- )
- })
- .collect::<Vec<_>>();
- for (group, (first, second)) in digests.iter_mut().enumerate() {
- for row in 0..ROWS_PER_PARTIAL {
- let value = partial_value(group, row, ROWS_PER_PARTIAL);
- first.update(value);
- second.update(value);
- }
+ bencher.bench_local(|| {
+ let mut digests = (0..PARTIAL_GROUPS)
+ .map(|_| {
+ (
+ TDigestMut::new(DEFAULT_DIGEST_K).unwrap(),
+ TDigestMut::new(DEFAULT_DIGEST_K).unwrap(),
+ )
+ })
+ .collect::<Vec<_>>();
+ for (group, (first, second)) in digests.iter_mut().enumerate() {
+ for row in 0..ROWS_PER_PARTIAL {
+ let value = partial_value(group, row, ROWS_PER_PARTIAL);
+ first.update(value);
+ second.update(value);
}
- black_box(digests)
+ }
+ black_box(digests)
+ });
+}
+
+pub fn register(runner: &mut Runner) {
+ for &(len, rounds) in &[(1_000, 20_000), (100_000, 100)] {
+ runner.case(format!("tdigest::update::update/{len:?}"), rounds, |b| {
+ update(b, len)
});
+ }
+ runner.case(
+ "tdigest::update::small_partial_groups_update",
+ 20_000,
+ small_partial_groups_update,
+ );
+ runner.case(
+ "tdigest::update::small_partial_groups_update_two_states",
+ 20_000,
+ small_partial_groups_update_two_states,
+ );
+ runner.case(
+ "tdigest::update::partial_groups_update",
+ 2_000,
+ partial_groups_update,
+ );
+ runner.case(
+ "tdigest::update::partial_groups_update_two_states",
+ 2_000,
+ partial_groups_update_two_states,
+ );
}
diff --git a/benchmarks/theta/mod.rs b/benchmarks/theta/mod.rs
index 1dea33b..4839176 100644
--- a/benchmarks/theta/mod.rs
+++ b/benchmarks/theta/mod.rs
@@ -16,3 +16,7 @@
// under the License.
mod update;
+
+pub fn register(runner: &mut benchmarks::Runner) {
+ update::register(runner);
+}
diff --git a/benchmarks/theta/update.rs b/benchmarks/theta/update.rs
index ca9b94f..db27f61 100644
--- a/benchmarks/theta/update.rs
+++ b/benchmarks/theta/update.rs
@@ -15,18 +15,18 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::hash::value::raw_bytes;
use datasketches::theta::ThetaSketchBuilder;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use crate::hash_inputs::ITEMS;
use crate::hash_inputs::bytes_32_values;
-#[divan::bench]
fn u64(bencher: Bencher) {
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut sketch =
ThetaSketchBuilder::default().lg_k(14).build().unwrap();
for value in 0..ITEMS as u64 {
sketch.update(black_box(value));
@@ -35,11 +35,10 @@ fn u64(bencher: Bencher) {
});
}
-#[divan::bench]
fn bytes_32(bencher: Bencher) {
let values = bytes_32_values();
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let mut sketch =
ThetaSketchBuilder::default().lg_k(14).build().unwrap();
for value in &values {
sketch.update(raw_bytes::from_slice(black_box(value)));
@@ -47,3 +46,8 @@ fn bytes_32(bencher: Bencher) {
black_box(sketch)
});
}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("theta::update::u64", 4_000, u64);
+ runner.case("theta::update::bytes_32", 3_000, bytes_32);
+}
diff --git a/benchmarks/tuple/mod.rs b/benchmarks/tuple/mod.rs
index 1dea33b..4839176 100644
--- a/benchmarks/tuple/mod.rs
+++ b/benchmarks/tuple/mod.rs
@@ -16,3 +16,7 @@
// under the License.
mod update;
+
+pub fn register(runner: &mut benchmarks::Runner) {
+ update::register(runner);
+}
diff --git a/benchmarks/tuple/update.rs b/benchmarks/tuple/update.rs
index 9041fed..be7b267 100644
--- a/benchmarks/tuple/update.rs
+++ b/benchmarks/tuple/update.rs
@@ -15,19 +15,19 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::hash::value::raw_bytes;
use datasketches::tuple::DefaultUpdatePolicy;
use datasketches::tuple::TupleSketchBuilder;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use crate::hash_inputs::ITEMS;
use crate::hash_inputs::bytes_32_values;
-#[divan::bench]
fn u64(bencher: Bencher) {
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let policy = DefaultUpdatePolicy::<u64>::default();
let mut sketch =
TupleSketchBuilder::new(policy).lg_k(14).build().unwrap();
for value in 0..ITEMS as u64 {
@@ -37,11 +37,10 @@ fn u64(bencher: Bencher) {
});
}
-#[divan::bench]
fn bytes_32(bencher: Bencher) {
let values = bytes_32_values();
- bencher.counter(ItemsCount::new(ITEMS)).bench_local(|| {
+ bencher.bench_local(|| {
let policy = DefaultUpdatePolicy::<u64>::default();
let mut sketch =
TupleSketchBuilder::new(policy).lg_k(14).build().unwrap();
for value in &values {
@@ -50,3 +49,8 @@ fn bytes_32(bencher: Bencher) {
black_box(sketch)
});
}
+
+pub fn register(runner: &mut Runner) {
+ runner.case("tuple::update::u64", 4_000, u64);
+ runner.case("tuple::update::bytes_32", 3_000, bytes_32);
+}
diff --git a/benchmarks/xor/build.rs b/benchmarks/xor/build.rs
index 153c063..482c1f9 100644
--- a/benchmarks/xor/build.rs
+++ b/benchmarks/xor/build.rs
@@ -15,118 +15,142 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::xor::XorFilterBuilder;
use datasketches::xor::XorFilterType;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use crate::xor::hashes;
-#[divan::bench(args = [1_000, 16_384, 49_999, 50_000, 50_001, 100_000])]
fn datasketches_distinct_8(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- black_box(
- XorFilterBuilder::new(XorFilterType::Xor8)
- .build_from_distinct_hashes(&keys)
- .unwrap(),
- )
- });
+ bencher.bench_local(|| {
+ black_box(
+ XorFilterBuilder::new(XorFilterType::Xor8)
+ .build_from_distinct_hashes(&keys)
+ .unwrap(),
+ )
+ });
}
-#[divan::bench(args = [1_000, 16_384, 49_999, 50_000, 50_001, 100_000])]
fn datasketches_deduplicate_8(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- black_box(
- XorFilterBuilder::new(XorFilterType::Xor8)
- .build_from_hashes(keys.iter().copied())
- .unwrap(),
- )
- });
+ bencher.bench_local(|| {
+ black_box(
+ XorFilterBuilder::new(XorFilterType::Xor8)
+ .build_from_hashes(keys.iter().copied())
+ .unwrap(),
+ )
+ });
}
-#[divan::bench(args = [1_000, 16_384, 49_999, 50_000, 50_001, 100_000])]
fn xorf_8(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(||
black_box(xorf::Xor8::from_iterator(keys.iter().copied())));
+ bencher.bench_local(||
black_box(xorf::Xor8::from_iterator(keys.iter().copied())));
}
-#[divan::bench(args = [1_000, 16_384, 49_999, 50_000, 50_001, 100_000])]
fn datasketches_distinct_16(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- black_box(
- XorFilterBuilder::new(XorFilterType::Xor16)
- .build_from_distinct_hashes(&keys)
- .unwrap(),
- )
- });
+ bencher.bench_local(|| {
+ black_box(
+ XorFilterBuilder::new(XorFilterType::Xor16)
+ .build_from_distinct_hashes(&keys)
+ .unwrap(),
+ )
+ });
}
-#[divan::bench(args = [1_000, 16_384, 49_999, 50_000, 50_001, 100_000])]
fn datasketches_deduplicate_16(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- black_box(
- XorFilterBuilder::new(XorFilterType::Xor16)
- .build_from_hashes(keys.iter().copied())
- .unwrap(),
- )
- });
+ bencher.bench_local(|| {
+ black_box(
+ XorFilterBuilder::new(XorFilterType::Xor16)
+ .build_from_hashes(keys.iter().copied())
+ .unwrap(),
+ )
+ });
}
-#[divan::bench(args = [1_000, 16_384, 49_999, 50_000, 50_001, 100_000])]
fn xorf_16(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(||
black_box(xorf::Xor16::from_iterator(keys.iter().copied())));
+ bencher.bench_local(||
black_box(xorf::Xor16::from_iterator(keys.iter().copied())));
}
-#[divan::bench(args = [1_000, 16_384, 49_999, 50_000, 50_001, 100_000])]
fn datasketches_distinct_32(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- black_box(
- XorFilterBuilder::new(XorFilterType::Xor32)
- .build_from_distinct_hashes(&keys)
- .unwrap(),
- )
- });
+ bencher.bench_local(|| {
+ black_box(
+ XorFilterBuilder::new(XorFilterType::Xor32)
+ .build_from_distinct_hashes(&keys)
+ .unwrap(),
+ )
+ });
}
-#[divan::bench(args = [1_000, 16_384, 49_999, 50_000, 50_001, 100_000])]
fn datasketches_deduplicate_32(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- black_box(
- XorFilterBuilder::new(XorFilterType::Xor32)
- .build_from_hashes(keys.iter().copied())
- .unwrap(),
- )
- });
+ bencher.bench_local(|| {
+ black_box(
+ XorFilterBuilder::new(XorFilterType::Xor32)
+ .build_from_hashes(keys.iter().copied())
+ .unwrap(),
+ )
+ });
}
-#[divan::bench(args = [1_000, 16_384, 49_999, 50_000, 50_001, 100_000])]
fn xorf_32(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(||
black_box(xorf::Xor32::from_iterator(keys.iter().copied())));
+ bencher.bench_local(||
black_box(xorf::Xor32::from_iterator(keys.iter().copied())));
+}
+
+pub fn register(runner: &mut Runner) {
+ let cases = &[
+ (1_000, 60_000),
+ (16_384, 2_000),
+ (49_999, 600),
+ (50_000, 600),
+ (50_001, 600),
+ (100_000, 200),
+ ];
+ for &(cardinality, rounds) in cases {
+ runner.compare(
+ &format!("xor::build::8/{cardinality}"),
+ rounds,
+ &[
+ ("distinct", &|b| datasketches_distinct_8(b, cardinality)),
+ ("deduplicate", &|b| {
+ datasketches_deduplicate_8(b, cardinality)
+ }),
+ ("xorf", &|b| xorf_8(b, cardinality)),
+ ],
+ );
+ }
+ for &(cardinality, rounds) in cases {
+ runner.compare(
+ &format!("xor::build::16/{cardinality}"),
+ rounds,
+ &[
+ ("distinct", &|b| datasketches_distinct_16(b, cardinality)),
+ ("deduplicate", &|b| {
+ datasketches_deduplicate_16(b, cardinality)
+ }),
+ ("xorf", &|b| xorf_16(b, cardinality)),
+ ],
+ );
+ }
+ for &(cardinality, rounds) in cases {
+ runner.compare(
+ &format!("xor::build::32/{cardinality}"),
+ rounds,
+ &[
+ ("distinct", &|b| datasketches_distinct_32(b, cardinality)),
+ ("deduplicate", &|b| {
+ datasketches_deduplicate_32(b, cardinality)
+ }),
+ ("xorf", &|b| xorf_32(b, cardinality)),
+ ],
+ );
+ }
}
diff --git a/benchmarks/xor/matched_build.rs b/benchmarks/xor/matched_build.rs
index fe1ad4d..34b8666 100644
--- a/benchmarks/xor/matched_build.rs
+++ b/benchmarks/xor/matched_build.rs
@@ -15,102 +15,135 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::xor::XorFilterBuilder;
use datasketches::xor::XorFilterType;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use crate::xor::hashes_with_offset;
-// xorf 0.13 starts SplitMix at 1. Both builders use the same seed sequence;
-// equal winning seeds therefore imply equal attempt counts. The offset 917504
-// case exercises a second attempt; every case verifies equal winning seeds.
-fn verify_seed(keys: &[u64], kind: XorFilterType) {
- let filter = XorFilterBuilder::new(kind)
- .seed(1)
- .build_from_distinct_hashes(keys)
- .unwrap();
- assert_eq!(filter.seed(), xorf::Xor8::from(keys).seed);
+// xorf 0.13 starts SplitMix at 1. These are its first two construction seeds.
+// Check each result directly: building a peer for validation inside the timed
+// workload would dilute the comparison, especially for the single-round cases.
+fn verify_seed(seed: u64, offset: u64) {
+ let expected = match offset {
+ 0 => 0x910a_2dec_8902_5cc1,
+ 917_504 => 0xbeeb_8da1_658e_ec67,
+ _ => unreachable!("unregistered input offset"),
+ };
+ assert_eq!(seed, expected, "construction attempt count changed");
}
-#[divan::bench(args = [(1_000, 0), (16_384, 0), (16_384, 917_504), (49_999,
0), (50_000, 0), (50_001, 0), (100_000, 0), (8_000_000, 0)])]
fn datasketches_8(bencher: Bencher, case: (usize, u64)) {
let (cardinality, offset) = case;
let keys = hashes_with_offset(cardinality, offset);
- verify_seed(&keys, XorFilterType::Xor8);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- black_box(
- XorFilterBuilder::new(XorFilterType::Xor8)
- .seed(1)
- .build_from_distinct_hashes(&keys)
- .unwrap(),
- )
- });
+ bencher.bench_local(|| {
+ let filter = XorFilterBuilder::new(XorFilterType::Xor8)
+ .seed(1)
+ .build_from_distinct_hashes(&keys)
+ .unwrap();
+ verify_seed(filter.seed(), offset);
+ black_box(filter)
+ });
}
-#[divan::bench(args = [(1_000, 0), (16_384, 0), (16_384, 917_504), (49_999,
0), (50_000, 0), (50_001, 0), (100_000, 0), (8_000_000, 0)])]
fn xorf_8(bencher: Bencher, case: (usize, u64)) {
let (cardinality, offset) = case;
let keys = hashes_with_offset(cardinality, offset);
- verify_seed(&keys, XorFilterType::Xor8);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(||
black_box(xorf::Xor8::from_iterator(keys.iter().copied())));
+ bencher.bench_local(|| {
+ let filter = xorf::Xor8::from_iterator(keys.iter().copied());
+ verify_seed(filter.seed, offset);
+ black_box(filter)
+ });
}
-#[divan::bench(args = [(1_000, 0), (16_384, 0), (16_384, 917_504), (49_999,
0), (50_000, 0), (50_001, 0), (100_000, 0), (8_000_000, 0)])]
fn datasketches_16(bencher: Bencher, case: (usize, u64)) {
let (cardinality, offset) = case;
let keys = hashes_with_offset(cardinality, offset);
- verify_seed(&keys, XorFilterType::Xor16);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- black_box(
- XorFilterBuilder::new(XorFilterType::Xor16)
- .seed(1)
- .build_from_distinct_hashes(&keys)
- .unwrap(),
- )
- });
+ bencher.bench_local(|| {
+ let filter = XorFilterBuilder::new(XorFilterType::Xor16)
+ .seed(1)
+ .build_from_distinct_hashes(&keys)
+ .unwrap();
+ verify_seed(filter.seed(), offset);
+ black_box(filter)
+ });
}
-#[divan::bench(args = [(1_000, 0), (16_384, 0), (16_384, 917_504), (49_999,
0), (50_000, 0), (50_001, 0), (100_000, 0), (8_000_000, 0)])]
fn xorf_16(bencher: Bencher, case: (usize, u64)) {
let (cardinality, offset) = case;
let keys = hashes_with_offset(cardinality, offset);
- verify_seed(&keys, XorFilterType::Xor16);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(||
black_box(xorf::Xor16::from_iterator(keys.iter().copied())));
+ bencher.bench_local(|| {
+ let filter = xorf::Xor16::from_iterator(keys.iter().copied());
+ verify_seed(filter.seed, offset);
+ black_box(filter)
+ });
}
-#[divan::bench(args = [(1_000, 0), (16_384, 0), (16_384, 917_504), (49_999,
0), (50_000, 0), (50_001, 0), (100_000, 0), (8_000_000, 0)])]
fn datasketches_32(bencher: Bencher, case: (usize, u64)) {
let (cardinality, offset) = case;
let keys = hashes_with_offset(cardinality, offset);
- verify_seed(&keys, XorFilterType::Xor32);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- black_box(
- XorFilterBuilder::new(XorFilterType::Xor32)
- .seed(1)
- .build_from_distinct_hashes(&keys)
- .unwrap(),
- )
- });
+ bencher.bench_local(|| {
+ let filter = XorFilterBuilder::new(XorFilterType::Xor32)
+ .seed(1)
+ .build_from_distinct_hashes(&keys)
+ .unwrap();
+ verify_seed(filter.seed(), offset);
+ black_box(filter)
+ });
}
-#[divan::bench(args = [(1_000, 0), (16_384, 0), (16_384, 917_504), (49_999,
0), (50_000, 0), (50_001, 0), (100_000, 0), (8_000_000, 0)])]
fn xorf_32(bencher: Bencher, case: (usize, u64)) {
let (cardinality, offset) = case;
let keys = hashes_with_offset(cardinality, offset);
- verify_seed(&keys, XorFilterType::Xor32);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(||
black_box(xorf::Xor32::from_iterator(keys.iter().copied())));
+ bencher.bench_local(|| {
+ let filter = xorf::Xor32::from_iterator(keys.iter().copied());
+ verify_seed(filter.seed, offset);
+ black_box(filter)
+ });
+}
+
+pub fn register(runner: &mut Runner) {
+ let cases = &[
+ ((1_000, 0), 60_000),
+ ((16_384, 0), 3_000),
+ ((16_384, 917_504), 2_000),
+ ((49_999, 0), 600),
+ ((50_000, 0), 600),
+ ((50_001, 0), 600),
+ ((100_000, 0), 200),
+ ((8_000_000, 0), 1),
+ ];
+ for &(case, rounds) in cases {
+ runner.compare(
+ &format!("xor::matched_build::8/{case:?}"),
+ rounds,
+ &[
+ ("DataSketches", &|b| datasketches_8(b, case)),
+ ("xorf", &|b| xorf_8(b, case)),
+ ],
+ );
+ }
+ for &(case, rounds) in cases {
+ runner.compare(
+ &format!("xor::matched_build::16/{case:?}"),
+ rounds,
+ &[
+ ("DataSketches", &|b| datasketches_16(b, case)),
+ ("xorf", &|b| xorf_16(b, case)),
+ ],
+ );
+ }
+ for &(case, rounds) in cases {
+ runner.compare(
+ &format!("xor::matched_build::32/{case:?}"),
+ rounds,
+ &[
+ ("DataSketches", &|b| datasketches_32(b, case)),
+ ("xorf", &|b| xorf_32(b, case)),
+ ],
+ );
+ }
}
diff --git a/benchmarks/xor/mod.rs b/benchmarks/xor/mod.rs
index cf9fbfc..bfdb47e 100644
--- a/benchmarks/xor/mod.rs
+++ b/benchmarks/xor/mod.rs
@@ -34,3 +34,9 @@ fn hashes_with_offset(count: usize, offset: u64) -> Vec<u64> {
})
.collect()
}
+
+pub fn register(runner: &mut benchmarks::Runner) {
+ build::register(runner);
+ matched_build::register(runner);
+ query::register(runner);
+}
diff --git a/benchmarks/xor/query.rs b/benchmarks/xor/query.rs
index af31987..3d3f0d2 100644
--- a/benchmarks/xor/query.rs
+++ b/benchmarks/xor/query.rs
@@ -15,179 +15,208 @@
// specific language governing permissions and limitations
// under the License.
+use std::hint::black_box;
+
+use benchmarks::Bencher;
+use benchmarks::Runner;
use datasketches::xor::XorFilterBuilder;
use datasketches::xor::XorFilterType;
-use divan::Bencher;
-use divan::black_box;
-use divan::counter::ItemsCount;
use xorf::Filter as _;
use crate::xor::hashes;
-#[divan::bench(args = [16_384, 50_000])]
fn datasketches_8_present(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality * 2);
let filter = XorFilterBuilder::new(XorFilterType::Xor8)
.build_from_distinct_hashes(&keys[..cardinality])
.unwrap();
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- for key in &keys[..cardinality] {
- black_box(filter.contains_hash(black_box(*key)));
- }
- });
+ bencher.bench_local(|| {
+ for key in &keys[..cardinality] {
+ black_box(filter.contains_hash(black_box(*key)));
+ }
+ });
}
-#[divan::bench(args = [16_384, 50_000])]
fn datasketches_8_absent(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality * 2);
let filter = XorFilterBuilder::new(XorFilterType::Xor8)
.build_from_distinct_hashes(&keys[..cardinality])
.unwrap();
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- for key in &keys[cardinality..] {
- black_box(filter.contains_hash(black_box(*key)));
- }
- });
+ bencher.bench_local(|| {
+ for key in &keys[cardinality..] {
+ black_box(filter.contains_hash(black_box(*key)));
+ }
+ });
}
-#[divan::bench(args = [16_384, 50_000])]
fn xorf_8_present(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality * 2);
let filter = xorf::Xor8::from(&keys[..cardinality]);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- for key in &keys[..cardinality] {
- black_box(filter.contains(black_box(key)));
- }
- });
+ bencher.bench_local(|| {
+ for key in &keys[..cardinality] {
+ black_box(filter.contains(black_box(key)));
+ }
+ });
}
-#[divan::bench(args = [16_384, 50_000])]
fn xorf_8_absent(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality * 2);
let filter = xorf::Xor8::from(&keys[..cardinality]);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- for key in &keys[cardinality..] {
- black_box(filter.contains(black_box(key)));
- }
- });
+ bencher.bench_local(|| {
+ for key in &keys[cardinality..] {
+ black_box(filter.contains(black_box(key)));
+ }
+ });
}
-#[divan::bench(args = [16_384, 50_000])]
fn datasketches_16_present(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality * 2);
let filter = XorFilterBuilder::new(XorFilterType::Xor16)
.build_from_distinct_hashes(&keys[..cardinality])
.unwrap();
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- for key in &keys[..cardinality] {
- black_box(filter.contains_hash(black_box(*key)));
- }
- });
+ bencher.bench_local(|| {
+ for key in &keys[..cardinality] {
+ black_box(filter.contains_hash(black_box(*key)));
+ }
+ });
}
-#[divan::bench(args = [16_384, 50_000])]
fn datasketches_16_absent(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality * 2);
let filter = XorFilterBuilder::new(XorFilterType::Xor16)
.build_from_distinct_hashes(&keys[..cardinality])
.unwrap();
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- for key in &keys[cardinality..] {
- black_box(filter.contains_hash(black_box(*key)));
- }
- });
+ bencher.bench_local(|| {
+ for key in &keys[cardinality..] {
+ black_box(filter.contains_hash(black_box(*key)));
+ }
+ });
}
-#[divan::bench(args = [16_384, 50_000])]
fn xorf_16_present(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality * 2);
let filter = xorf::Xor16::from(&keys[..cardinality]);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- for key in &keys[..cardinality] {
- black_box(filter.contains(black_box(key)));
- }
- });
+ bencher.bench_local(|| {
+ for key in &keys[..cardinality] {
+ black_box(filter.contains(black_box(key)));
+ }
+ });
}
-#[divan::bench(args = [16_384, 50_000])]
fn xorf_16_absent(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality * 2);
let filter = xorf::Xor16::from(&keys[..cardinality]);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- for key in &keys[cardinality..] {
- black_box(filter.contains(black_box(key)));
- }
- });
+ bencher.bench_local(|| {
+ for key in &keys[cardinality..] {
+ black_box(filter.contains(black_box(key)));
+ }
+ });
}
-#[divan::bench(args = [16_384, 50_000])]
fn datasketches_32_present(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality * 2);
let filter = XorFilterBuilder::new(XorFilterType::Xor32)
.build_from_distinct_hashes(&keys[..cardinality])
.unwrap();
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- for key in &keys[..cardinality] {
- black_box(filter.contains_hash(black_box(*key)));
- }
- });
+ bencher.bench_local(|| {
+ for key in &keys[..cardinality] {
+ black_box(filter.contains_hash(black_box(*key)));
+ }
+ });
}
-#[divan::bench(args = [16_384, 50_000])]
fn datasketches_32_absent(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality * 2);
let filter = XorFilterBuilder::new(XorFilterType::Xor32)
.build_from_distinct_hashes(&keys[..cardinality])
.unwrap();
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- for key in &keys[cardinality..] {
- black_box(filter.contains_hash(black_box(*key)));
- }
- });
+ bencher.bench_local(|| {
+ for key in &keys[cardinality..] {
+ black_box(filter.contains_hash(black_box(*key)));
+ }
+ });
}
-#[divan::bench(args = [16_384, 50_000])]
fn xorf_32_present(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality * 2);
let filter = xorf::Xor32::from(&keys[..cardinality]);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- for key in &keys[..cardinality] {
- black_box(filter.contains(black_box(key)));
- }
- });
+ bencher.bench_local(|| {
+ for key in &keys[..cardinality] {
+ black_box(filter.contains(black_box(key)));
+ }
+ });
}
-#[divan::bench(args = [16_384, 50_000])]
fn xorf_32_absent(bencher: Bencher, cardinality: usize) {
let keys = hashes(cardinality * 2);
let filter = xorf::Xor32::from(&keys[..cardinality]);
- bencher
- .counter(ItemsCount::new(cardinality))
- .bench_local(|| {
- for key in &keys[cardinality..] {
- black_box(filter.contains(black_box(key)));
- }
- });
+ bencher.bench_local(|| {
+ for key in &keys[cardinality..] {
+ black_box(filter.contains(black_box(key)));
+ }
+ });
+}
+
+pub fn register(runner: &mut Runner) {
+ let cases = &[(16_384, 10_000), (50_000, 3_000)];
+ for &(cardinality, rounds) in cases {
+ runner.compare(
+ &format!("xor::query::8_present/{cardinality}"),
+ rounds,
+ &[
+ ("DataSketches", &|b| datasketches_8_present(b, cardinality)),
+ ("xorf", &|b| xorf_8_present(b, cardinality)),
+ ],
+ );
+ }
+ for &(cardinality, rounds) in cases {
+ runner.compare(
+ &format!("xor::query::8_absent/{cardinality}"),
+ rounds,
+ &[
+ ("DataSketches", &|b| datasketches_8_absent(b, cardinality)),
+ ("xorf", &|b| xorf_8_absent(b, cardinality)),
+ ],
+ );
+ }
+ for &(cardinality, rounds) in cases {
+ runner.compare(
+ &format!("xor::query::16_present/{cardinality}"),
+ rounds,
+ &[
+ ("DataSketches", &|b| datasketches_16_present(b, cardinality)),
+ ("xorf", &|b| xorf_16_present(b, cardinality)),
+ ],
+ );
+ }
+ for &(cardinality, rounds) in cases {
+ runner.compare(
+ &format!("xor::query::16_absent/{cardinality}"),
+ rounds,
+ &[
+ ("DataSketches", &|b| datasketches_16_absent(b, cardinality)),
+ ("xorf", &|b| xorf_16_absent(b, cardinality)),
+ ],
+ );
+ }
+ for &(cardinality, rounds) in cases {
+ runner.compare(
+ &format!("xor::query::32_present/{cardinality}"),
+ rounds,
+ &[
+ ("DataSketches", &|b| datasketches_32_present(b, cardinality)),
+ ("xorf", &|b| xorf_32_present(b, cardinality)),
+ ],
+ );
+ }
+ for &(cardinality, rounds) in cases {
+ runner.compare(
+ &format!("xor::query::32_absent/{cardinality}"),
+ rounds,
+ &[
+ ("DataSketches", &|b| datasketches_32_absent(b, cardinality)),
+ ("xorf", &|b| xorf_32_absent(b, cardinality)),
+ ],
+ );
+ }
}
diff --git a/xtask/src/main.rs b/xtask/src/main.rs
index 9b3fb1f..ff88cc4 100644
--- a/xtask/src/main.rs
+++ b/xtask/src/main.rs
@@ -73,11 +73,21 @@ enum SubCommand {
}
#[derive(Parser)]
-struct CommandBench;
+struct CommandBench {
+ #[arg(long, help = "Compile benchmarks without running them.")]
+ no_run: bool,
+
+ #[arg(
+ last = true,
+ conflicts_with = "no_run",
+ help = "Arguments passed to the benchmark runner."
+ )]
+ args: Vec<String>,
+}
impl CommandBench {
fn run(self) {
- run_command(make_bench_cmd());
+ run_command(make_bench_cmd(self.no_run, &self.args));
}
}
@@ -190,9 +200,15 @@ fn run_command(mut cmd: StdCommand) {
assert!(status.success(), "command failed: {status}");
}
-fn make_bench_cmd() -> StdCommand {
+fn make_bench_cmd(no_run: bool, args: &[String]) -> StdCommand {
let mut cmd = find_command("cargo");
cmd.args(["bench", "--package", "benchmarks", "--bench", "benchmarks"]);
+ if no_run {
+ cmd.arg("--no-run");
+ }
+ if !args.is_empty() {
+ cmd.arg("--").args(args);
+ }
cmd
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]