[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200887096 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3261,3 +3261,323 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200886228 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -450,7 +450,7 @@ public UnsafeArrayData copy() {

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing conditional expressions...

2018-07-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r200886142 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -695,6 +695,56 @@ abstract class TernaryExpression

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200885976 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -450,7 +450,7 @@ public UnsafeArrayData copy() {

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200884043 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -450,7 +450,7 @@ public UnsafeArrayData copy() {

[GitHub] spark issue #21731: Update example to work locally

2018-07-08 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21731 This seems not a necessary fix. `master` can be configured via spark-submit argument `--master`, not a best practice to set it in code. ---

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200883268 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -450,7 +450,7 @@ public UnsafeArrayData copy() {

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing conditional expressions...

2018-07-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r200879951 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -695,6 +695,56 @@ abstract class TernaryExpression

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200875757 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -450,7 +450,7 @@ public UnsafeArrayData copy() {

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r189432089 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1882,3 +1882,311 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200878344 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3261,3 +3261,323 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200875456 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -463,14 +463,27 @@ private static

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200875538 --- Diff: python/pyspark/sql/functions.py --- @@ -2013,6 +2013,25 @@ def array_distinct(col): return

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200876039 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3261,3 +3261,323 @@ case class

[GitHub] spark issue #21582: [SPARK-24576][BUILD] Upgrade Apache ORC to 1.5.2

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21582 **[Test build #92730 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92730/testReport)** for PR 21582 at commit

[GitHub] spark issue #21582: [SPARK-24576][BUILD] Upgrade Apache ORC to 1.5.2

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21582 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21582: [SPARK-24576][BUILD] Upgrade Apache ORC to 1.5.2

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21582 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92730/ Test FAILed. ---

[GitHub] spark issue #21728: [SPARK-24759] [SQL] No reordering keys for broadcast has...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21728 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21728: [SPARK-24759] [SQL] No reordering keys for broadcast has...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21728 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/762/

[GitHub] spark issue #21728: [SPARK-24759] [SQL] No reordering keys for broadcast has...

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21728 **[Test build #92732 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92732/testReport)** for PR 21728 at commit

[GitHub] spark issue #21728: [SPARK-24759] [SQL] No reordering keys for broadcast has...

2018-07-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21728 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20208: [SPARK-23007][SQL][TEST] Add schema evolution test suite...

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20208 **[Test build #92731 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92731/testReport)** for PR 20208 at commit

[GitHub] spark issue #20208: [SPARK-23007][SQL][TEST] Add schema evolution test suite...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20208 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/761/

[GitHub] spark issue #20208: [SPARK-23007][SQL][TEST] Add schema evolution test suite...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20208 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20208: [SPARK-23007][SQL][TEST] Add schema evolution test suite...

2018-07-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20208 Rebased to the master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21582: [SPARK-24576][BUILD] Upgrade Apache ORC to 1.5.2

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21582 **[Test build #92730 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92730/testReport)** for PR 21582 at commit

[GitHub] spark issue #21582: [SPARK-24576][BUILD] Upgrade Apache ORC to 1.5.2

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21582 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/760/

[GitHub] spark issue #21582: [SPARK-24576][BUILD] Upgrade Apache ORC to 1.5.2

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21582 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21732: [SPARK-24762][SQL] Aggregator should be able to use Opti...

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21732 **[Test build #92729 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92729/testReport)** for PR 21732 at commit

[GitHub] spark issue #21732: [SPARK-24762][SQL] Aggregator should be able to use Opti...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21732 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/759/

[GitHub] spark issue #21732: [SPARK-24762][SQL] Aggregator should be able to use Opti...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21732 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21732: [SPARK-24762][SQL] Aggregator should be able to u...

2018-07-08 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/21732 [SPARK-24762][SQL] Aggregator should be able to use Option of Product encoder ## What changes were proposed in this pull request? Encoders has a limitation that we can't construct encoders

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200874571 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala --- @@ -1166,4 +1166,88 @@ class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200874190 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3261,3 +3261,323 @@ case class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r200874014 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3261,3 +3261,323 @@ case class

[GitHub] spark issue #21537: [SPARK-24505][SQL] Convert strings in codegen to blocks:...

2018-07-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21537 ping @cloud-fan @kiszk --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21073 **[Test build #92728 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92728/testReport)** for PR 21073 at commit

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-07-08 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21073 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21073: [SPARK-23936][SQL] Implement map_concat

2018-07-08 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21073 I'd retrigger the build, just in case. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21659: [SPARK-24530][PYTHON] Add a control to force Pyth...

2018-07-08 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21659#discussion_r200870956 --- Diff: python/docs/Makefile --- @@ -1,19 +1,44 @@ # Makefile for Sphinx documentation # +ifndef SPHINXBUILD +ifndef

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21659 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92726/ Test PASSed. ---

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21659 **[Test build #92726 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92726/testReport)** for PR 21659 at commit

[GitHub] spark issue #21658: [SPARK-24678][Spark-Streaming] Give priority in use of '...

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21658 **[Test build #92727 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92727/testReport)** for PR 21658 at commit

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21659 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21659 **[Test build #92725 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92725/testReport)** for PR 21659 at commit

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92725/ Test PASSed. ---

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92723/ Test PASSed. ---

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21305 **[Test build #92723 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92723/testReport)** for PR 21305 at commit

[GitHub] spark pull request #21659: [SPARK-24530][PYTHON] Add a control to force Pyth...

2018-07-08 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21659#discussion_r200869393 --- Diff: python/docs/Makefile --- @@ -1,19 +1,44 @@ # Makefile for Sphinx documentation # +ifndef SPHINXBUILD +ifndef

[GitHub] spark pull request #21659: [SPARK-24530][PYTHON] Add a control to force Pyth...

2018-07-08 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21659#discussion_r200869531 --- Diff: python/docs/Makefile --- @@ -1,19 +1,44 @@ # Makefile for Sphinx documentation # +ifndef SPHINXBUILD +ifndef

[GitHub] spark pull request #21633: [SPARK-24646][CORE] Minor change to spark.yarn.di...

2018-07-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21633 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21542: [SPARK-24529][Build][test-maven] Add spotbugs int...

2018-07-08 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21542#discussion_r200867924 --- Diff: pom.xml --- @@ -2606,6 +2606,28 @@ + +com.github.spotbugs +

[GitHub] spark issue #21633: [SPARK-24646][CORE] Minor change to spark.yarn.dist.forc...

2018-07-08 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21633 Thanks @jiangxb1987 , merging to master branch. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21659 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/758/

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21659 **[Test build #92726 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92726/testReport)** for PR 21659 at commit

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-07-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21542 Seems fine to me otherwise. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21542: [SPARK-24529][Build][test-maven] Add spotbugs int...

2018-07-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21542#discussion_r200866847 --- Diff: pom.xml --- @@ -2606,6 +2606,28 @@ + +com.github.spotbugs +

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21659 **[Test build #92725 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92725/testReport)** for PR 21659 at commit

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/757/

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21659 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-07-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21542 ping @cloud-fan @viirya @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92724/ Test PASSed. ---

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21659 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21659 **[Test build #92724 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92724/testReport)** for PR 21659 at commit

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21659 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/756/

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21659 **[Test build #92724 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92724/testReport)** for PR 21659 at commit

[GitHub] spark issue #21659: [SPARK-24530][PYTHON] Add a control to force Python vers...

2018-07-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21659 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21730: [SPARK-24761][SQL] Adding of isModifiable() to RuntimeCo...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21730 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21730: [SPARK-24761][SQL] Adding of isModifiable() to RuntimeCo...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21730 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92722/ Test PASSed. ---

[GitHub] spark issue #21730: [SPARK-24761][SQL] Adding of isModifiable() to RuntimeCo...

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21730 **[Test build #92722 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92722/testReport)** for PR 21730 at commit

[GitHub] spark issue #21730: [SPARK-24761][SQL] Adding of isModifiable() to RuntimeCo...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21730 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21730: [SPARK-24761][SQL] Adding of isModifiable() to RuntimeCo...

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21730 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92721/ Test PASSed. ---

[GitHub] spark issue #21730: [SPARK-24761][SQL] Adding of isModifiable() to RuntimeCo...

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21730 **[Test build #92721 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92721/testReport)** for PR 21730 at commit

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-07-08 Thread ijuma
Github user ijuma commented on the issue: https://github.com/apache/spark/pull/21488 Any luck getting to the bottom of the issue? It would be great to include this in the next version of Spark? --- - To

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/755/

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21305 **[Test build #92723 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92723/testReport)** for PR 21305 at commit

[GitHub] spark issue #20276: [SPARK-14948][SQL] disambiguate attributes in join condi...

2018-07-08 Thread ssimeonov
Github user ssimeonov commented on the issue: https://github.com/apache/spark/pull/20276 @cloud-fan do you expect to resolve conflict + merge at some point? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-07-08 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/21583 actually, i think we're running a supported version of docker: ```sknapp@amp-jenkins-staging-worker-02:~$ docker -v Docker version 17.05.0-ce, build 89658be ``` On

[GitHub] spark issue #21583: [SPARK-23984][K8S][Test] Added Integration Tests for PyS...

2018-07-08 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/21583 @ifilonenko and @foxish, i will get to the docker upgrade this coming week. On Sat, Jul 7, 2018 at 12:48 AM, Ilan Filonenko wrote: > @foxish The

[GitHub] spark issue #21731: Update example to work locally

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21731 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21731: Update example to work locally

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21731 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21731: Update example to work locally

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21731 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21731: Update example to work locally

2018-07-08 Thread ets
GitHub user ets opened a pull request: https://github.com/apache/spark/pull/21731 Update example to work locally Without setting Master to "local" the example throws a SparkException "A master URL must be set in your configuration" ## What changes were proposed in this

[GitHub] spark issue #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21061 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92720/ Test PASSed. ---

[GitHub] spark issue #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21061 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21061: [SPARK-23914][SQL] Add array_union function

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21061 **[Test build #92720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92720/testReport)** for PR 21061 at commit

[GitHub] spark issue #12904: [SPARK-15125][SQL] Changing CSV data source mapping of e...

2018-07-08 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/12904 The issue has been already solved by https://github.com/apache/spark/commit/7a2d4895c75d4c232c377876b61c05a083eab3c8 . The PR can be closed. ---

[GitHub] spark issue #21730: [SPARK-24761][SQL] Adding of isModifiable() to RuntimeCo...

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21730 **[Test build #92722 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92722/testReport)** for PR 21730 at commit

[GitHub] spark pull request #21730: [SPARK-24761][SQL] Adding of isModifiable() to Ru...

2018-07-08 Thread ssimeonov
Github user ssimeonov commented on a diff in the pull request: https://github.com/apache/spark/pull/21730#discussion_r200852355 --- Diff: python/pyspark/sql/conf.py --- @@ -63,6 +63,12 @@ def _checkType(self, obj, identifier): raise TypeError("expected %s '%s' to

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21699 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92719/ Test PASSed. ---

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-07-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21699 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21699: [SPARK-24722][SQL] pivot() with Column type argument

2018-07-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21699 **[Test build #92719 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92719/testReport)** for PR 21699 at commit

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing conditional expressions...

2018-07-08 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r200852015 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -695,6 +695,56 @@ abstract class

[GitHub] spark pull request #21687: [SPARK-24165][SQL] Fixing conditional expressions...

2018-07-08 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/21687#discussion_r200851845 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala --- @@ -695,6 +695,56 @@ abstract class

[GitHub] spark pull request #21730: [SPARK-24761][SQL] Adding of isModifiable() to Ru...

2018-07-08 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/21730#discussion_r200851659 --- Diff: python/pyspark/sql/conf.py --- @@ -63,6 +63,12 @@ def _checkType(self, obj, identifier): raise TypeError("expected %s '%s' to be

  1   2   >