LukaZdravic commented on code in PR #58827:
URL: https://github.com/apache/spark/pull/58827#discussion_r4063878193
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala:
##########
@@ -2988,8 +3000,30 @@ object AsOfJoin {
private def buildArrayOrderExpression(
leftOperand: Expression,
rightOperand: Expression,
- elementType: DataType,
operator: MatchComparisonOperator): Expression = {
+ val leftElementType =
leftOperand.dataType.asInstanceOf[ArrayType].elementType
+ val rightElementType =
rightOperand.dataType.asInstanceOf[ArrayType].elementType
+ // The ZipWith lambda variables and both array inputs must share the
element type the `>=`
+ // comparison coerces to. Coercible elements (e.g. INT vs BIGINT, or INT
vs FLOAT which widens
+ // to DOUBLE under ANSI) widen to their tightest common type and both
arrays are cast to it.
+ // Structurally equal elements (BinaryComparison ignores struct field
names) need no cast and
+ // compare element-wise by ordinal.
+ val elementsStructurallyEqual =
+ DataType.equalsStructurally(leftElementType, rightElementType,
ignoreNullability = true)
+ val (leftArray, rightArray, elementType) =
+ if (elementsStructurallyEqual) {
+ (leftOperand, rightOperand, leftElementType)
+ } else {
+ MatchConditionTypes.arrayElementCommonType(leftElementType,
rightElementType) match {
+ case Some(widerElementType) =>
+ (castArrayElementType(leftOperand, widerElementType),
+ castArrayElementType(rightOperand, widerElementType),
+ widerElementType)
+ case None =>
+ // Unreachable: usesArrayOrderExpression already required a common
element type here.
+ throw SparkException.internalError("MATCH_CONDITION array elements
have no common type")
Review Comment:
Moved it up in the common helper, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
