Copilot commented on code in PR #51000:
URL: https://github.com/apache/arrow/pull/51000#discussion_r4047480017
##########
cpp/src/arrow/compute/kernels/scalar_validity.cc:
##########
@@ -101,6 +104,34 @@ static void SetNanBits(const ArraySpan& arr, uint8_t*
out_bitmap, int64_t out_of
}
}
+// Maps `is_null` over the dictionary values and then through the indices, so
that
+// both NaN and null dictionary entries are reported, whatever the index type.
+static Status SetNullBitsFromDictionary(KernelContext* ctx, const ArraySpan&
arr,
+ const NullOptions& options, uint8_t*
out_bitmap,
+ int64_t out_offset) {
+ if (arr.length == 0) {
+ return Status::OK();
+ }
+ const auto& dict_type = checked_cast<const DictionaryType&>(*arr.type);
+ ARROW_ASSIGN_OR_RAISE(Datum dict_is_null,
+ CallFunction("is_null",
{arr.dictionary().ToArrayData()},
+ &options, ctx->exec_context()));
+
+ std::shared_ptr<ArrayData> indices = arr.ToArrayData();
+ indices->type = dict_type.index_type();
+ indices->dictionary = nullptr;
+ ARROW_ASSIGN_OR_RAISE(Datum taken,
+ Take(dict_is_null, Datum(std::move(indices)),
+ TakeOptions::BoundsCheck(), ctx->exec_context()));
Review Comment:
`arr.ToArrayData()` builds a full DICTIONARY ArrayData (including
constructing `dictionary = arr.dictionary().ToArrayData()`), only to
immediately mutate it into an indices array and drop the dictionary. This adds
avoidable allocation/work on the hot path. Construct an indices `ArrayData`
directly from the span buffers instead of round-tripping through
`ToArrayData()`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
