[
https://issues.apache.org/jira/browse/SPARK-9139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15530990#comment-15530990
]
Dongjoon Hyun commented on SPARK-9139:
--------------------------------------
First, of course, it is. The commit is just my commit which make me notice this
issue and come to here.
Second, of course again, you can pick this up. You had better ask [~joshrosen]
the goal of this issue before you start.
> Add backwards-compatibility tests for DataType.fromJson()
> ---------------------------------------------------------
>
> Key: SPARK-9139
> URL: https://issues.apache.org/jira/browse/SPARK-9139
> Project: Spark
> Issue Type: Test
> Components: SQL
> Reporter: Josh Rosen
> Priority: Critical
>
> SQL's DataType.fromJson is a public API and thus must be
> backwards-compatible; there are also backwards-compatibility concerns related
> to persistence of DataType JSON in metastores.
> Unfortunately, we do not have any backwards-compatibility tests which attempt
> to read old JSON values that were written by earlier versions of Spark.
> DataTypeSuite has "roundtrip" tests that test fromJson(toJson(foo)), but this
> doesn't ensure compatibility.
> I think that we should address this by capuring the JSON strings produced in
> Spark 1.3's DataFrameSuite and adding test cases that try to create DataTypes
> from those strings.
> This might be a good starter task for someone who wants to contribute to SQL
> tests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
