[ https://issues.apache.org/jira/browse/SPARK-16883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408994#comment-15408994 ]
Miao Wang commented on SPARK-16883: ----------------------------------- I found the root cause. I am trying to propose a clean solution to handle this case. PRIMITIVE_TYPES <- as.environment(list( ... "decimal" = "numeric", ... But when backend returns decimal(10, 0), it can't handle this type in collect() and coltypes(). collect() won't throw Error, but print the list as shown in the above example. > SQL decimal type is not properly cast to number when collecting SparkDataFrame > ------------------------------------------------------------------------------ > > Key: SPARK-16883 > URL: https://issues.apache.org/jira/browse/SPARK-16883 > Project: Spark > Issue Type: Bug > Components: SparkR > Affects Versions: 2.0.0 > Reporter: Hossein Falaki > > To reproduce run following code. As you can see "y" is a list of values. > {code} > registerTempTable(createDataFrame(iris), "iris") > str(collect(sql("select cast('1' as double) as x, cast('2' as decimal) as y > from iris limit 5"))) > 'data.frame': 5 obs. of 2 variables: > $ x: num 1 1 1 1 1 > $ y:List of 5 > ..$ : num 2 > ..$ : num 2 > ..$ : num 2 > ..$ : num 2 > ..$ : num 2 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org