srielau commented on code in PR #58584:
URL: https://github.com/apache/spark/pull/58584#discussion_r3969635578
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/hiveUDFEvaluators.scala:
##########
@@ -162,9 +162,12 @@ class HiveGenericUDFEvaluator(
}
@transient
- private lazy val unwrapper: Any => Any = unwrapperFor(returnInspector)
+ private lazy val catalystReturnType = inspectorToDataType(returnInspector)
Review Comment:
Fixed in 3b611b326d4. The analyzed GenericUDF return type and UDAF
partial/final types are now retained across expression copies and evaluator
reconstruction. The persisted-view regression covers first-class creation
queried under legacy settings and the inverse legacy-to-first-class direction.
##########
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala:
##########
@@ -892,6 +892,42 @@ class HiveUDFSuite extends QueryTest with
TestHiveSingleton {
hiveContext.reset()
}
+ test("SPARK-59277: Hive UDF and UDTF support first-class CHAR/VARCHAR") {
+ withSQLConf(SQLConf.CHAR_VARCHAR_STANDARD_SEMANTICS.key -> "true") {
Review Comment:
Added preserve-only coverage in 3b611b326d4. It asserts both CharType(5) and
the padded value with standard semantics disabled.
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveInspectors.scala:
##########
@@ -829,6 +864,26 @@ private[hive] trait HiveInspectors {
null
}
}
+ case (_, c: CharType) =>
Review Comment:
Updated the Scaladoc in 3b611b326d4 to document CHAR/VARCHAR read-side
length and padding checks.
##########
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationExec.scala:
##########
@@ -130,7 +132,7 @@ private[hive] case class HiveScriptTransformationExec(
if (dataList.get(i) == null) {
mutableRow.setNullAt(i)
} else {
- unwrappers(i)(dataList.get(i), mutableRow, i)
+ mutableRow.update(i, unwrappers(i)(dataList.get(i)))
Review Comment:
Fixed in 3b611b326d4 with a DataType-aware in-place overload. Primitive
fields continue to use the specialized setters; only CHAR/VARCHAR targets use
the target-aware generic conversion path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
