edubraqd opened a new pull request, #24901: URL: https://github.com/apache/datafusion/pull/24901
## Which issue does this PR close? - Closes #24900. ## Rationale for this change `TDigest::merge_sorted_f64` and `TDigest::merge_digests` allocated the compressed centroid list with `Vec::with_capacity(max_size)`. `max_size` is the user supplied `centroids` argument of `approx_percentile_cont`, so `approx_percentile_cont(x, 0.5, 9223372036854775807)` panicked with "capacity overflow" (and a merely huge value would abort on allocation failure). ## What changes are included in this PR? The compressed digest never holds more centroids than the inputs being merged, so the preallocation is now `max_size.min(<input size>)` in both places. Behaviour for ordinary `max_size` values is unchanged. ## Are these changes tested? Yes. `test_huge_max_size_does_not_preallocate` builds a digest with `max_size = usize::MAX`, merges values and digests into it, and checks the estimates; it used to panic. ## Are there any user-facing changes? `approx_percentile_cont` with a very large `centroids` argument returns a result instead of panicking. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
