tisonkun opened a new pull request, #191: URL: https://github.com/apache/datasketches-rust/pull/191
## Summary Preserve Frequencies stream and error state when a purge removes every active counter. The public `is_empty()` method retains its existing active-item semantics. A new internal virgin-state check is used for serialization and merge, so a zero-active sketch with stream history is no longer treated as an unused sketch. Such a sketch uses the existing non-empty preamble with `active_items == 0`, preserving `stream_weight` and `offset` without introducing a new wire format. Closes #188. ## Regression coverage The tests deterministically produce a purged sketch with: - zero active items, - total weight 25, - maximum error 1, and - an upper bound of 1 for an untracked item. They verify that serialization round trips and merges preserve all of those observations, that reserialization is stable, and that a genuinely virgin sketch still uses the eight-byte empty representation. ## Relationship to other implementations Current Java, C++, and Go share the previous behavior: they define empty from the active-item count, use it to select the short empty serialization, and skip the sketch during merge. - Java state and merge: https://github.com/apache/datasketches-java/blob/4067ffefabbcd03944dc3617ff2948ab760f3b75/src/main/java/org/apache/datasketches/frequencies/FrequentItemsSketch.java#L455-L477 - Java serialization: https://github.com/apache/datasketches-java/blob/4067ffefabbcd03944dc3617ff2948ab760f3b75/src/main/java/org/apache/datasketches/frequencies/FrequentItemsSketch.java#L496-L533 - C++ state and merge: https://github.com/apache/datasketches-cpp/blob/c22888581964fc490feee835766cc8c3adb722e0/fi/include/frequent_items_sketch_impl.hpp#L68-L93 - C++ serialization: https://github.com/apache/datasketches-cpp/blob/c22888581964fc490feee835766cc8c3adb722e0/fi/include/frequent_items_sketch_impl.hpp#L165-L209 - Go state and merge: https://github.com/apache/datasketches-go/blob/c558cc2d64f9a307c196a7adb0067eadd1b776f7/frequencies/items_sketch.go#L327-L408 - Go serialization: https://github.com/apache/datasketches-go/blob/c558cc2d64f9a307c196a7adb0067eadd1b776f7/frequencies/items_sketch.go#L442-L481 This is therefore a shared reference-family state-loss issue, not a Rust-only format mismatch. Existing Java, C++, and Go deserializers accept a non-empty preamble with zero active items and retain its stream weight and offset on read. However, they still consider that result empty and may collapse it again during their own reserialization or merge. This PR fixes Rust round trips and Rust merges while documenting that remaining cross-language limitation. ## Validation - `cargo x prepare-testdata` - `cargo x check` - `cargo x test` - `cargo x lint` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
