[GitHub] spark pull request #20219: [SPARK-23025][SQL] Support Null type in scala ref...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/20219


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20219: [SPARK-23025][SQL] Support Null type in scala ref...

[gatorsmile]

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/20219#discussion_r160990065
  
--- Diff: 
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala
 ---
@@ -356,4 +356,13 @@ class ScalaReflectionSuite extends SparkFunSuite {
 assert(deserializerFor[Int].isInstanceOf[AssertNotNull])
 assert(!deserializerFor[String].isInstanceOf[AssertNotNull])
   }
+
+  test("SPARK-23025: schemaFor shuold support Null type") {
--- End diff --

should?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20219: [SPARK-23025][SQL] Support Null type in scala ref...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/20219#discussion_r160688553
  
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala 
---
@@ -1441,6 +1441,13 @@ class DatasetSuite extends QueryTest with 
SharedSQLContext {
   assert(e.getCause.isInstanceOf[NullPointerException])
 }
   }
+
+  test("SPARK-23025: Add support for null type in scala reflection") {
+val data = Seq(("a", null))
+checkDataset(
+  data.toDS(),
+  data: _*)
--- End diff --

Nope, let's only fix this one if there are many places to update.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20219: [SPARK-23025][SQL] Support Null type in scala ref...

[mgaido91]

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/spark/pull/20219#discussion_r160687957
  
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala 
---
@@ -1441,6 +1441,13 @@ class DatasetSuite extends QueryTest with 
SharedSQLContext {
   assert(e.getCause.isInstanceOf[NullPointerException])
 }
   }
+
+  test("SPARK-23025: Add support for null type in scala reflection") {
+val data = Seq(("a", null))
+checkDataset(
+  data.toDS(),
+  data: _*)
--- End diff --

sure, since in this file in many places this syntax is used, should I 
change also other usages accordingly?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20219: [SPARK-23025][SQL] Support Null type in scala ref...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/20219#discussion_r160682423
  
--- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala 
---
@@ -1441,6 +1441,13 @@ class DatasetSuite extends QueryTest with 
SharedSQLContext {
   assert(e.getCause.isInstanceOf[NullPointerException])
 }
   }
+
+  test("SPARK-23025: Add support for null type in scala reflection") {
+val data = Seq(("a", null))
+checkDataset(
+  data.toDS(),
+  data: _*)
--- End diff --

nit: could we make this inlined?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20219: [SPARK-23025][SQL] Support Null type in scala ref...

[mgaido91]

GitHub user mgaido91 opened a pull request:

https://github.com/apache/spark/pull/20219

[SPARK-23025][SQL] Support Null type in scala reflection

## What changes were proposed in this pull request?

Add support for `Null` type in the `schemaFor` method for Scala reflection.

## How was this patch tested?

Added UT


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mgaido91/spark SPARK-23025

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20219.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20219


commit 98010236c013b24c0ecfa6357efaacba1a30ee84
Author: Marco Gaido 
Date:   2018-01-10T12:08:37Z

[SPARK-23025][SQL] Support Null type in scala reflection




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org