viirya commented on code in PR #7629:
URL: https://github.com/apache/datafusion-comet/pull/7629#discussion_r4589157041


##########
native/spark-expr/src/agg_funcs/hll_sketch.rs:
##########
@@ -452,8 +350,8 @@ mod tests {
     }
 
     /// A union's gadget stays small while it takes small sketches. Merging a 
sketch that holds the
-    /// register array promotes it however little that sketch has seen, so it 
is the preamble's
-    /// mode, not the estimate, that has to decide.
+    /// register array promotes it however little that sketch has seen, so the 
charge has to follow
+    /// the gadget's mode, not its estimate.

Review Comment:
   The hand-edited preamble below existed to defeat the estimate-based mode 
inference, which is gone. Since `heap_size()` now reads the container, merging 
any dense sketch and asserting `1 << 16` covers the same promotion, without 
tying the test to the HLL preamble offsets. The "not its estimate" here can go 
with it.



##########
native/spark-expr/src/agg_funcs/hll_sketch.rs:
##########
@@ -205,9 +120,9 @@ impl SparkHllUnion {
         self.inner.lg_max_k()
     }
 
-    /// Merge a sketch into the union.
+    /// Merge a sketch into the union without querying it. Querying after each 
dense merge would
+    /// rebuild the register cache, which apache/datasketches-rust#317 defers 
until it is needed.

Review Comment:
   On 0.6.0 the rebuild happens inside `update` (`merge_array_same_lgk` calls 
`rebuild_estimator_from_registers`), and `estimate()` only reads the cache, so 
this reads as if querying is what triggers it today. Maybe:
   
   ```rust
   /// Merge a sketch into the union without querying it. With 
apache/datasketches-rust#317 the
   /// register cache rebuild after a dense merge waits for the next query, and 
an `estimate()`
   /// here would force it on every merge.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to