Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r227640284
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -635,13 +675,17 @@ object ScalaReflection extends ScalaReflection {
"cannot be used as field name\n" +
walkedTypePath.mkString("\n"))
}
+ // as a field, value class is represented by its underlying type
+ val trueFieldType =
+ if (isValueClass(fieldType)) getUnderlyingTypeOf(fieldType)
else fieldType
+
val fieldValue = Invoke(
- AssertNotNull(inputObject, walkedTypePath), fieldName,
dataTypeFor(fieldType),
- returnNullable = !fieldType.typeSymbol.asClass.isPrimitive)
- val clsName = getClassNameFromType(fieldType)
+ AssertNotNull(inputObject, walkedTypePath), fieldName,
dataTypeFor(trueFieldType),
+ returnNullable = !trueFieldType.typeSymbol.asClass.isPrimitive)
+ val clsName = getClassNameFromType(trueFieldType)
--- End diff --
Why we need such special handling? There is new serialization handling for
value class added above, can't we simple get the object of value class here and
let recursively call of `serializerFor` to handle it?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
