[ https://issues.apache.org/jira/browse/SPARK-6054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Armbrust reassigned SPARK-6054: --------------------------------------- Assignee: Michael Armbrust > SQL UDF returning object of case class; regression from 1.2.0 > ------------------------------------------------------------- > > Key: SPARK-6054 > URL: https://issues.apache.org/jira/browse/SPARK-6054 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.3.0 > Environment: Windows 8, Scala 2.11.2, Spark 1.3.0 RC1 > Reporter: Spiro Michaylov > Assignee: Michael Armbrust > Priority: Blocker > > The following code fails with a stack trace beginning with: > {code} > 15/02/26 23:21:32 ERROR Executor: Exception in task 2.0 in stage 7.0 (TID 422) > org.apache.spark.sql.catalyst.errors.package$TreeNodeException: makeCopy, > tree: scalaUDF(sales#2,discounts#3) > at > org.apache.spark.sql.catalyst.errors.package$.attachTree(package.scala:47) > at > org.apache.spark.sql.catalyst.trees.TreeNode.makeCopy(TreeNode.scala:309) > at > org.apache.spark.sql.catalyst.trees.TreeNode.transformChildrenDown(TreeNode.scala:237) > at > org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:192) > at > org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:207) > {code} > Here is the 1.3.0 version of the code: > {code} > case class SalesDisc(sales: Double, discounts: Double) > def makeStruct(sales: Double, disc:Double) = SalesDisc(sales, disc) > sqlContext.udf.register("makeStruct", makeStruct _) > val withStruct = > sqlContext.sql("SELECT id, sd.sales FROM (SELECT id, makeStruct(sales, > discounts) AS sd FROM customerTable) AS d") > withStruct.foreach(println) > {code} > This used to work in 1.2.0. Interestingly, the following simplified version > fails similarly, even though it seems to me to be VERY similar to the last > test in the UDFSuite: > {code} > SELECT makeStruct(sales, discounts) AS sd FROM customerTable > {code} > The data table is defined thus: > {code} > val custs = Seq( > Cust(1, "Widget Co", 120000.00, 0.00, "AZ"), > Cust(2, "Acme Widgets", 410500.00, 500.00, "CA"), > Cust(3, "Widgetry", 410500.00, 200.00, "CA"), > Cust(4, "Widgets R Us", 410500.00, 0.0, "CA"), > Cust(5, "Ye Olde Widgete", 500.00, 0.0, "MA") > ) > val customerTable = sc.parallelize(custs, 4).toDF() > customerTable.registerTempTable("customerTable") > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org