cloud-fan commented on code in PR #55498:
URL: https://github.com/apache/spark/pull/55498#discussion_r3128453425
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ComplexTypeSuite.scala:
##########
@@ -321,6 +321,30 @@ class ComplexTypeSuite extends SparkFunSuite with
ExpressionEvalHelper {
}
}
+ test("GetMapValue - non-foldable map always uses linear scan") {
+ // With threshold=0, a foldable map would take the hash path. A
non-foldable map (backed by
+ // a row column) must still fall back to linear scan, because its hash
index cannot be
+ // reused across rows (building it per row is a perf regression vs.
linear).
+ withSQLConf(SQLConf.MAP_LOOKUP_HASH_THRESHOLD.key -> "0") {
Review Comment:
Great suggestion - done. Added `private[expressions] def
usesFoldableHashLookup` on `GetMapValueUtil` so suites can assert strategy
choice without reflection on path-dependent inner types. The existing
non-foldable test now asserts `!usesFoldableHashLookup`, and there's a new
`GetMapValue - strategy choice for foldable maps` test that locks in three
cases:
- foldable + above threshold + hashable key type --> `PrebuiltHashExecutor`
- foldable but below threshold --> `LinearExecutor`
- foldable + unsupported key type (used `BinaryType` - simpler than a
collated string) --> `LinearExecutor` even with `threshold=0`
If you'd prefer an explicit collated-StringType case too, happy to add it; I
went with BinaryType because it hits the same `typeWithProperEquals = false`
branch with less ceremony.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
