adriangb commented on code in PR #11641:
URL: https://github.com/apache/arrow-rs/pull/11641#discussion_r4568054087


##########
parquet/src/bloom_filter/mod.rs:
##########
@@ -563,6 +579,64 @@ impl Sbbf {
         self.0[block_index].check(hash as u32)
     }
 
+    /// Same result as [`Self::check_hash`], without data-dependent branches.
+    #[cfg_attr(not(any(feature = "arrow", test)), expect(dead_code))]
+    #[inline(always)]
+    fn check_hash_branch_free(&self, hash: u64) -> bool {
+        let block_index = self.hash_to_block_index(hash);
+        self.0[block_index].check_branch_free(hash as u32)
+    }
+
+    /// Inserts precomputed 64-bit hashes.
+    ///
+    /// The filter can hold hashes from any well-mixed 64-bit hash function,
+    /// but [`Self::check`] only finds values whose hashes were computed as
+    /// the Parquet specification requires (xxHash64, seed 0, of the plain
+    /// encoding). Use [`Self::check_hashes`] with the same hash function.
+    pub fn insert_hashes(&mut self, hashes: &[u64]) {
+        for &hash in hashes {
+            self.insert_hash(hash);
+        }
+    }
+
+    /// Checks precomputed 64-bit hashes (see [`Self::insert_hashes`]).
+    ///
+    /// Bit `i` of the result is set if `hashes[i]` may be in the filter. The
+    /// check has no data-dependent branches, which is much faster than
+    /// checking one hash at a time.
+    #[cfg(feature = "arrow")]
+    pub fn check_hashes(&self, hashes: &[u64]) -> arrow_buffer::BooleanBuffer {
+        arrow_buffer::BooleanBuffer::collect_bool(hashes.len(), |i| {
+            self.check_hash_branch_free(hashes[i])
+        })
+    }
+
+    /// Adds the values of `other` to this filter, so that afterwards it
+    /// contains every value that either filter contained.
+    ///
+    /// `other` must have at least as many blocks as this filter, and a
+    /// multiple of them. When it has more, it is folded to the size of this
+    /// filter on the fly (see [`Self::fold_to_target_fpp`]), without changing
+    /// `other`. Folding and union commute: the result is the same as
+    /// inserting the values of both filters into one filter and folding it
+    /// to this size. This lets independent filters (for example, one per
+    /// thread) be built and then merged.

Review Comment:
   👍🏻  done in 0b135b53c17cdddf8d7d0f5f297d87d5f1618518



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to