[
https://issues.apache.org/jira/browse/SPARK-16883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15414476#comment-15414476
]
Miao Wang commented on SPARK-16883:
-----------------------------------
OK.I will try both. For the frontend solution, I can create a special handling
function which will be used for handling all special cases like decimal(10, 0)
with return value of Boolean type. The callers will only change a little bit by
calling this function when it is not the PRIMITIVE types. In the future, we can
add more in this special function to handle other cases. These cases are those
which are not Complex types but not covered by PRIMITIVE types mapping.
For the second solution, I will make changes as you suggested. I will learn how
serialize StructType(StructField(x,DoubleType,true),
StructField(y,DecimalType(10,0),true)) by adding the value type.
> SQL decimal type is not properly cast to number when collecting SparkDataFrame
> ------------------------------------------------------------------------------
>
> Key: SPARK-16883
> URL: https://issues.apache.org/jira/browse/SPARK-16883
> Project: Spark
> Issue Type: Bug
> Components: SparkR
> Affects Versions: 2.0.0
> Reporter: Hossein Falaki
>
> To reproduce run following code. As you can see "y" is a list of values.
> {code}
> registerTempTable(createDataFrame(iris), "iris")
> str(collect(sql("select cast('1' as double) as x, cast('2' as decimal) as y
> from iris limit 5")))
> 'data.frame': 5 obs. of 2 variables:
> $ x: num 1 1 1 1 1
> $ y:List of 5
> ..$ : num 2
> ..$ : num 2
> ..$ : num 2
> ..$ : num 2
> ..$ : num 2
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
