adamreeve commented on code in PR #55293:
URL: https://github.com/apache/arrow/pull/55293#discussion_r4536801238
##########
cpp/src/arrow/compute/kernels/scalar_set_lookup.cc:
##########
@@ -36,9 +44,217 @@ using internal::HashTraits;
namespace compute::internal {
namespace {
-// This base class enables non-templated access to the value set type
-struct SetLookupStateBase : public KernelState {
+/// Value sets up to this length are sorted for
RangeEvaluator::EvaluateOverRange.
+/// Sorting costs more than building the lookup table of the kernel, so longer
value
Review Comment:
I don't follow the logic here. Why is the sorting time compared to a lookup
table build? Avoiding row group reads is the cost that this is trying to avoid.
It probably does make sense to fall-back to using the min/max of the set at
some point, but I'd just like to better understand how the cutoff point is
chosen.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
