[ https://issues.apache.org/jira/browse/SPARK-16883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15414476#comment-15414476 ]
Miao Wang edited comment on SPARK-16883 at 8/10/16 12:03 AM: ------------------------------------------------------------- OK.I will try both. For the frontend solution, I can create a special handling function which will be used for handling all special cases like decimal(10, 0) with return value of Boolean type. The callers will only change a little bit by calling this function when it is not the PRIMITIVE types. In the future, we can add more in this special function to handle other cases. These cases are those which are not Complex types but not covered by PRIMITIVE types mapping. For the second solution, I will make changes as you suggested. I will learn how serialize StructType(StructField(x,DoubleType,true), StructField(y,DecimalType(10,0),true)) by adding the value type. The second solution could be simpler, I guess. was (Author: wm624): OK.I will try both. For the frontend solution, I can create a special handling function which will be used for handling all special cases like decimal(10, 0) with return value of Boolean type. The callers will only change a little bit by calling this function when it is not the PRIMITIVE types. In the future, we can add more in this special function to handle other cases. These cases are those which are not Complex types but not covered by PRIMITIVE types mapping. For the second solution, I will make changes as you suggested. I will learn how serialize StructType(StructField(x,DoubleType,true), StructField(y,DecimalType(10,0),true)) by adding the value type. > SQL decimal type is not properly cast to number when collecting SparkDataFrame > ------------------------------------------------------------------------------ > > Key: SPARK-16883 > URL: https://issues.apache.org/jira/browse/SPARK-16883 > Project: Spark > Issue Type: Bug > Components: SparkR > Affects Versions: 2.0.0 > Reporter: Hossein Falaki > > To reproduce run following code. As you can see "y" is a list of values. > {code} > registerTempTable(createDataFrame(iris), "iris") > str(collect(sql("select cast('1' as double) as x, cast('2' as decimal) as y > from iris limit 5"))) > 'data.frame': 5 obs. of 2 variables: > $ x: num 1 1 1 1 1 > $ y:List of 5 > ..$ : num 2 > ..$ : num 2 > ..$ : num 2 > ..$ : num 2 > ..$ : num 2 > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org