Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228733161
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -379,6 +388,28 @@ object ScalaReflection extends ScalaReflection {
dataType = ObjectType(udt.getClass))
Invoke(obj, "deserialize", ObjectType(udt.userClass), getPath ::
Nil)
+ case t if isValueClass(t) =>
+ val (_, underlyingType) = getUnderlyingParameterOf(t)
+ val underlyingClsName = getClassNameFromType(underlyingType)
+ val clsName = getUnerasedClassNameFromType(t)
+ val newTypePath = s"""- Scala value class:
$clsName($underlyingClsName)""" +:
+ walkedTypePath
+
+ // Nested value class is treated as its underlying type
+ // because the compiler will convert value class in the schema to
+ // its underlying type.
+ // However, for top-level value class, if it is used as another
type
+ // (e.g. as its parent trait or generic), the compiler keeps the
class
+ // so we must provide an instance of the class too. In other cases,
+ // the compiler will handle wrapping/unwrapping for us
automatically.
+ val arg = deserializerFor(underlyingType, path, newTypePath)
+ if (path.isDefined) {
--- End diff --
No, I haven't rebased since last week.
Hmm, now since `path` is not `Option` anymore, I think I have to use `if
(walkedTypePath.length > 1)` to have the same logic. But this seems a little
bit hacky. Do you have any suggestion on this?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
