mrhhsg commented on code in PR #67962:
URL: https://github.com/apache/doris/pull/67962#discussion_r4226481986
##########
be/src/core/column/column_struct.cpp:
##########
@@ -218,24 +220,92 @@ void ColumnStruct::update_hash_with_value(size_t n,
SipHash& hash) const {
}
}
+namespace {
+
+// Invoke `func(run_start, run_end)` for every maximal run of rows in [start,
end) that are not
+// NULL according to `null_data`. An all-zero mask yields the single run
[start, end), so the
+// masked and unmasked range paths produce identical hashes and the same
number of calls.
+template <typename Func>
+void for_each_non_null_run(const uint8_t* __restrict null_data, size_t start,
size_t end,
+ Func&& func) {
+ size_t i = start;
+ while (i < end) {
+ while (i < end && null_data[i] != 0) {
+ ++i;
+ }
+ size_t run_start = i;
+ while (i < end && null_data[i] == 0) {
+ ++i;
+ }
+ if (run_start < i) {
+ func(run_start, i);
+ }
+ }
+}
+
+// Batch-hash every row through the field columns, then restore the rows that
are NULL at the
+// outer level. Surviving rows keep each field's batch hash contract (which
may differ from the
+// single/range contract, e.g. the width of a NULL default), and the field
columns are still
+// called once per column instead of once per row.
+template <typename HashType, typename Func>
+void batch_hash_skipping_null_rows(HashType* __restrict hashes, size_t rows,
+ const uint8_t* __restrict null_data, Func&&
hash_all_rows) {
+ DorisVector<HashType> saved(hashes, hashes + rows);
Review Comment:
Fixed in `1ed9b8212bc9933ce85f7aced0fd4302abb33a03`.
`batch_hash_skipping_null_rows` now returns when the outer mask contains no
non-NULL row, before creating `DorisVector` scratch or dispatching to child
columns. Added a 33-row ARRAY child traversal oracle for both xxHash and CRC32C
batch hooks; it verifies zero hidden-element hashing for all-NULL masks and
continued traversal for a mixed mask. `ColumnStructTest.*` passed 10/10 and the
BE ASAN build passed. I am resolving this specific P2 thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
