It should be UserDefinedType. You can refer to https://github.com/apache/spark/blob/master/sql/core/src/test/scala/org/apache/spark/sql/UserDefinedTypeSuite.scala
From: Koert Kuipers [mailto:ko...@tresata.com] Sent: Monday, September 26, 2016 5:42 AM To: user@spark.apache.org Subject: udf forces usage of Row for complex types? after having gotten used to have case classes represent complex structures in Datasets, i am surprised to find out that when i work in DataFrames with udfs no such magic exists, and i have to fall back to manipulating Row objects, which is error prone and somewhat ugly. for example: case class Person(name: String, age: Int) val df = Seq((Person("john", 33), 5), (Person("mike", 30), 6)).toDF("person", "id") val df1 = df.withColumn("person", udf({ (p: Person) => p.copy(age = p.age + 1) }).apply(col("person"))) df1.printSchema df1.show leads to: java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to Person
