This is an automated email from the ASF dual-hosted git repository.
tisonkun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datasketches-rust.git
The following commit(s) were added to refs/heads/main by this push:
new a62d1ba docs(tdigest): explain batch merge tradeoffs (#288)
a62d1ba is described below
commit a62d1ba0061a22066abc4b301b5aa21a7287d0cf
Author: tison <[email protected]>
AuthorDate: Sun Oct 4 13:54:57 2026 +0800
docs(tdigest): explain batch merge tradeoffs (#288)
---
CHANGELOG.md | 2 +-
datasketches/src/tdigest/sketch.rs | 5 +++++
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 24934e7..c5c6553 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -14,7 +14,7 @@ All significant changes to this project will be documented in
this file.
* `KllSketch` is now available behind the `kll` feature, with rank, quantile,
PMF, and CDF queries, merging, serialization, custom ordered item types, and a
`KllFloat` adapter for non-NaN floating-point values.
* Add `TDigestMut::quantiles` and `TDigest::quantiles` for querying several
ranks in one centroid scan while preserving input order.
-* Add `FromIterator<TDigestMut>` for batch construction from owned partial
sketches, avoiding repeated compression of intermediate results.
+* Add `FromIterator<TDigestMut>` as an alternative to incremental
`TDigestMut::merge` calls for combining owned partial sketches in a single
compression pass.
### Improvements
diff --git a/datasketches/src/tdigest/sketch.rs
b/datasketches/src/tdigest/sketch.rs
index eb31e87..5a7d1c6 100644
--- a/datasketches/src/tdigest/sketch.rs
+++ b/datasketches/src/tdigest/sketch.rs
@@ -1002,6 +1002,11 @@ impl TDigestMut {
/// Collecting consumes each digest without cloning it. Callers that need to
retain their inputs
/// can use [`TDigestMut::merge`] or explicitly clone them before collection.
///
+/// Batch collection can save repeated compression work when combining many
small partials, but
+/// is not always faster than repeated [`TDigestMut::merge`] calls. Larger
compressed inputs can
+/// favor incremental merging. Input count, retained centroid count, and value
distribution all
+/// affect performance; the number of original samples alone does not
determine the cost.
+///
/// Collection retains all non-empty input buffers before compression, even
with a lazy iterator.
/// Fully compressed inputs are merged directly into the result; inputs with
buffered updates also
/// require a combined centroid buffer and sorting workspace. Repeated `merge`
calls or smaller
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
