Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r234085471
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -373,6 +383,32 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r229156708
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -128,6 +128,15 @@ object ScalaReflection extends
Github user mt40 commented on the issue:
https://github.com/apache/spark/pull/22309
@cloud-fan while what you said is true, I think it's not that bad. Overall,
the only major special logic is inside the case class `case` and this is due to
the special nature of value class
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228879109
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -180,11 +189,13 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228863565
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -373,6 +383,32 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228741586
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -374,6 +383,29 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228737728
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -374,6 +383,29 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228734075
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -376,6 +387,23 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228733771
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -376,6 +387,23 @@ object ScalaReflection extends
Github user mt40 commented on the issue:
https://github.com/apache/spark/pull/22309
@cloud-fan @viirya I was silly @_@, I forgot to push the change for that
part and missed that error message from Jenkins as well
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228733409
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -376,6 +387,23 @@ object ScalaReflection extends
Github user mt40 commented on the issue:
https://github.com/apache/spark/pull/22309
hi @viirya, may I ask whether Jenkins is having issue? It reports
```
[error] (catalyst/compile:compileIncremental) Compilation failed
[error] Total time: 175 s, completed Oct 27, 2018 8:33
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228733161
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -379,6 +388,28 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228733107
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -376,6 +387,23 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228733087
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoderSuite.scala
---
@@ -297,11 +307,16 @@ class
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228732805
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -376,6 +387,23 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228731280
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala
---
@@ -358,4 +368,20 @@ class ScalaReflectionSuite extends
Github user mt40 commented on the issue:
https://github.com/apache/spark/pull/22309
@cloud-fan It works now. Actually, top level value class is supported from
[SPARK-17368](https://issues.apache.org/jira/browse/SPARK-17368). I try to
maintain that and add support for nested value
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228716352
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -635,13 +675,17 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228713038
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -622,6 +654,14 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r228713022
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -128,6 +128,16 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r226510002
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -632,13 +667,17 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r226509492
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -376,6 +386,23 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r226508827
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -376,6 +386,23 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r226144119
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -125,6 +125,17 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r226142977
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -376,6 +387,23 @@ object ScalaReflection extends
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r226142827
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
---
@@ -376,6 +387,23 @@ object ScalaReflection extends
Github user mt40 commented on the issue:
https://github.com/apache/spark/pull/22309
@cloud-fan may I ask what is `error code -9`? I cannot find the error from
the log in Jenkin.
---
-
To unsubscribe, e-mail
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r224951179
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala
---
@@ -108,6 +108,16 @@ object TestingUDT
Github user mt40 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22309#discussion_r224944923
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala
---
@@ -108,6 +108,16 @@ object TestingUDT
Github user mt40 commented on the issue:
https://github.com/apache/spark/pull/22309
@cloud-fan @liancheng @marmbrus could you please take a look at this and
start the tests?
---
-
To unsubscribe, e-mail: reviews
GitHub user mt40 opened a pull request:
https://github.com/apache/spark/pull/22309
[SPARK-20384][core] Support value class in schema of Dataset
## What changes were proposed in this pull request?
This PR adds support for [Scala value class][1] in schema of Datasets (as
both top
32 matches
Mail list logo
