[GitHub] spark pull request #22812: [SPARK-25817][SQL] Dataset encoder should support...

2018-10-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22812 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22812: [SPARK-25817][SQL] Dataset encoder should support combin...

2018-10-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22812 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-27 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/22309 According to the second rule, value class in Array needs to be instantiated. In current code, I think it should go to the special handling case of value class. Can you also add a test to verify it

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-27 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/22309 How about value class in Array? Do we support it? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228734242 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -635,13 +675,17 @@ object ScalaReflection extends

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread mt40
Github user mt40 commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228734075 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -376,6 +387,23 @@ object ScalaReflection extends

[GitHub] spark issue #22867: [SPARK-25778] WriteAheadLogBackedBlockRDD in YARN Cluste...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22867 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22867: [SPARK-25778] WriteAheadLogBackedBlockRDD in YARN Cluste...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22867 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22867: [SPARK-25778] WriteAheadLogBackedBlockRDD in YARN Cluste...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22867 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22867: [SPARK-25778] WriteAheadLogBackedBlockRDD in YARN...

2018-10-27 Thread gss2002
GitHub user gss2002 opened a pull request: https://github.com/apache/spark/pull/22867 [SPARK-25778] WriteAheadLogBackedBlockRDD in YARN Cluster Mode Fails … …due lack of access to tmpDir from $PWD to HDFS WriteAheadLogBackedBlockRDD usage of java.io.tmpdir will fail if

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228733929 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -376,6 +387,23 @@ object ScalaReflection extends

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread mt40
Github user mt40 commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228733771 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -376,6 +387,23 @@ object ScalaReflection extends

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228733678 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -376,6 +387,23 @@ object ScalaReflection extends

[GitHub] spark pull request #22797: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-27 Thread dilipbiswal
Github user dilipbiswal closed the pull request at: https://github.com/apache/spark/pull/22797 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-27 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22809 Thanks a lot @cloud-fan @mgaido91 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22047: [SPARK-19851] Add support for EVERY and ANY (SOME...

2018-10-27 Thread dilipbiswal
Github user dilipbiswal closed the pull request at: https://github.com/apache/spark/pull/22047 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22812: [SPARK-25817][SQL] Dataset encoder should support combin...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22812 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98147/ Test PASSed. ---

[GitHub] spark issue #22812: [SPARK-25817][SQL] Dataset encoder should support combin...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22812 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22812: [SPARK-25817][SQL] Dataset encoder should support combin...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22812 **[Test build #98147 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98147/testReport)** for PR 22812 at commit

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22309 **[Test build #98155 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98155/testReport)** for PR 22309 at commit

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228733525 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -376,6 +387,23 @@ object ScalaReflection extends

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-27 Thread mt40
Github user mt40 commented on the issue: https://github.com/apache/spark/pull/22309 @cloud-fan @viirya I was silly @_@, I forgot to push the change for that part and missed that error message from Jenkins as well ---

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228733511 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -376,6 +387,23 @@ object ScalaReflection extends

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-27 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/22309 ``` [error] /home/jenkins/workspace/SparkPullRequestBuilder/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala:401: value isDefined is not a member of

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread mt40
Github user mt40 commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228733409 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -376,6 +387,23 @@ object ScalaReflection extends

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22309 try to build sql/core? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-27 Thread mt40
Github user mt40 commented on the issue: https://github.com/apache/spark/pull/22309 hi @viirya, may I ask whether Jenkins is having issue? It reports ``` [error] (catalyst/compile:compileIncremental) Compilation failed [error] Total time: 175 s, completed Oct 27, 2018

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228733287 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -376,6 +387,23 @@ object ScalaReflection extends

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22309 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98154/ Test FAILed. ---

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22309 **[Test build #98154 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98154/testReport)** for PR 22309 at commit

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22309 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread mt40
Github user mt40 commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228733161 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -379,6 +388,28 @@ object ScalaReflection extends

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread mt40
Github user mt40 commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228733107 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -376,6 +387,23 @@ object ScalaReflection extends

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22309 **[Test build #98154 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98154/testReport)** for PR 22309 at commit

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread mt40
Github user mt40 commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228733087 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoderSuite.scala --- @@ -297,11 +307,16 @@ class

[GitHub] spark issue #21402: SPARK-24355 Spark external shuffle server improvement to...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21402 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228732860 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -376,6 +387,23 @@ object ScalaReflection extends

[GitHub] spark issue #21732: [SPARK-24762][SQL] Enable Option of Product encoders

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21732 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22858: [SPARK-24709][SQL][2.4] map basestring to str for...

2018-10-27 Thread cloud-fan
Github user cloud-fan closed the pull request at: https://github.com/apache/spark/pull/22858 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22858: [SPARK-24709][SQL][2.4] map basestring to str for python...

2018-10-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22858 title updated --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21732: [SPARK-24762][SQL] Enable Option of Product encoders

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21732 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98146/ Test PASSed. ---

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread mt40
Github user mt40 commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228732805 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -376,6 +387,23 @@ object ScalaReflection extends

[GitHub] spark issue #21732: [SPARK-24762][SQL] Enable Option of Product encoders

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21732 **[Test build #98146 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98146/testReport)** for PR 21732 at commit

[GitHub] spark issue #22858: [SPARK-24709][SQL][2.4] use str instead of basestring in...

2018-10-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22858 Oops, mind fixing PR title too? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22858: [SPARK-24709][SQL][2.4] use str instead of basestring in...

2018-10-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22858 @cloud-fan, thanks for doing this backport! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22858: [SPARK-24709][SQL][2.4] use str instead of basestring in...

2018-10-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22858 Merged to branch-2.4. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98153/ Test FAILed. ---

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22866 **[Test build #98153 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98153/testReport)** for PR 22866 at commit

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22866 **[Test build #98153 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98153/testReport)** for PR 22866 at commit

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22817: [SPARK-25816][SQL] Fix attribute resolution in ne...

2018-10-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22817#discussion_r228731805 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2578,4 +2578,12 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark issue #22858: [SPARK-24709][SQL][2.4] use str instead of basestring in...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22858 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98151/ Test PASSed. ---

[GitHub] spark issue #22858: [SPARK-24709][SQL][2.4] use str instead of basestring in...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22858 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22858: [SPARK-24709][SQL][2.4] use str instead of basestring in...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22858 **[Test build #98151 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98151/testReport)** for PR 22858 at commit

[GitHub] spark pull request #22817: [SPARK-25816][SQL] Fix attribute resolution in ne...

2018-10-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22817#discussion_r228731772 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2578,4 +2578,12 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98152/ Test FAILed. ---

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22866 **[Test build #98152 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98152/testReport)** for PR 22866 at commit

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22809 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22809 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22865: [DOC] Fix doc for spark.sql.parquet.recordLevelFilter.en...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22865 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98142/ Test PASSed. ---

[GitHub] spark pull request #22865: [DOC] Fix doc for spark.sql.parquet.recordLevelFi...

2018-10-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22865#discussion_r228731568 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -462,7 +462,7 @@ object SQLConf { val

[GitHub] spark issue #22865: [DOC] Fix doc for spark.sql.parquet.recordLevelFilter.en...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22865 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22865: [DOC] Fix doc for spark.sql.parquet.recordLevelFilter.en...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22865 **[Test build #98142 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98142/testReport)** for PR 22865 at commit

[GitHub] spark issue #22865: [DOC] Fix doc for spark.sql.parquet.recordLevelFilter.en...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22865 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98143/ Test PASSed. ---

[GitHub] spark issue #22865: [DOC] Fix doc for spark.sql.parquet.recordLevelFilter.en...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22865 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22865: [DOC] Fix doc for spark.sql.parquet.recordLevelFilter.en...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22865 **[Test build #98143 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98143/testReport)** for PR 22865 at commit

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228731507 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -376,6 +387,23 @@ object ScalaReflection extends

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22866 **[Test build #98152 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98152/testReport)** for PR 22866 at commit

[GitHub] spark issue #22853: [SPARK-25845][SQL] Fix MatchError for calendar interval ...

2018-10-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22853 LGTM. Do we target it to 2.4? The API in 2.4 is deprecated so I'm not sure if we still need to backport bug fixes. --- - To

[GitHub] spark pull request #22865: [DOC] Fix doc for spark.sql.parquet.recordLevelFi...

2018-10-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22865#discussion_r228731385 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -462,7 +462,7 @@ object SQLConf { val

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22858: [SPARK-24709][SQL][2.4] use str instead of basestring in...

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22858 **[Test build #98151 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98151/testReport)** for PR 22858 at commit

[GitHub] spark issue #22858: [SPARK-24709][SQL][2.4] use str instead of basestring in...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22858 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22866 **[Test build #98150 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98150/testReport)** for PR 22866 at commit

[GitHub] spark issue #22858: [SPARK-24709][SQL][2.4] use str instead of basestring in...

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22858 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22866 **[Test build #98150 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98150/testReport)** for PR 22866 at commit

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98150/ Test FAILed. ---

[GitHub] spark issue #22858: [SPARK-24709][SQL][2.4] use str instead of basestring in...

2018-10-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22858 Yup, I think strictly we should change. Looks there are two occurrences at `udf` and `pands_udf` `isinstance(..., str)`. Another problem at PySpark is, inconsistent type comparison like

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22866 **[Test build #98149 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98149/testReport)** for PR 22866 at commit

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread mt40
Github user mt40 commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228731280 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala --- @@ -358,4 +368,20 @@ class ScalaReflectionSuite extends

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98149/ Test FAILed. ---

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22866 **[Test build #98149 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98149/testReport)** for PR 22866 at commit

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228731247 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -379,6 +388,28 @@ object ScalaReflection extends

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22866 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98148/ Test FAILed. ---

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22866 **[Test build #98148 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98148/testReport)** for PR 22866 at commit

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-10-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r228731209 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoderSuite.scala --- @@ -297,11 +307,16 @@ class

[GitHub] spark issue #22866: [SPARK-12172][SPARKR] Remove internal-only RDD methods

2018-10-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22866 **[Test build #98148 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98148/testReport)** for PR 22866 at commit

[GitHub] spark pull request #22866: [SPARK-12172][SPARKR] Remove internal-only RDD me...

2018-10-27 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/22866 [SPARK-12172][SPARKR] Remove internal-only RDD methods ## What changes were proposed in this pull request? Remove non-public internal only methods for RDD in SparkR ## How was

[GitHub] spark pull request #22858: [SPARK-24709][SQL][2.4] use str instead of basest...

2018-10-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22858#discussion_r228731178 --- Diff: python/pyspark/sql/functions.py --- @@ -2326,7 +2326,7 @@ def schema_of_json(json): >>> df.select(schema_of_json('{"a":

  1   2   3   4   >