[GitHub] spark pull request #20806: [SPARK-23661][SQL] Implement treeAggregate on Dat...

2018-03-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20806#discussion_r174358970 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1658,6 +1659,43 @@ class Dataset[T] private[sql]( def groupByKey[K](func:

[GitHub] spark issue #20817: [SPARK-23599][SQL] Add a UUID generator from Pseudo-Rand...

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20817 **[Test build #88222 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88222/testReport)** for PR 20817 at commit

[GitHub] spark issue #20817: [SPARK-23599][SQL] Add a UUID generator from Pseudo-Rand...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20817 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20817: [SPARK-23599][SQL] Add a UUID generator from Pseudo-Rand...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20817 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1500/

[GitHub] spark issue #20817: [SPARK-23599][SQL] Add a UUID generator from Pseudo-Rand...

2018-03-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20817 cc @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20817: [SPARK-23599][SQL] Add a UUID generator from Pseu...

2018-03-13 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/20817 [SPARK-23599][SQL] Add a UUID generator from Pseudo-Random Numbers ## What changes were proposed in this pull request? This patch adds a UUID generator from Pseudo-Random Numbers. We can

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20659 **[Test build #88221 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88221/testReport)** for PR 20659 at commit

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88221/ Test FAILed. ---

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20659 **[Test build #88221 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88221/testReport)** for PR 20659 at commit

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1499/

[GitHub] spark issue #20727: [SPARK-23577][SQL] Supports custom line separator for te...

2018-03-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20727 @cloud-fan and @MaxGekk, I believe this is ready for another look. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20796: [SPARK-23649][SQL] Prevent crashes on schema infe...

2018-03-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20796#discussion_r174355311 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -1279,4 +1279,22 @@ class CSVSuite extends

[GitHub] spark issue #20796: [SPARK-23649][SQL] Prevent crashes on schema inferring o...

2018-03-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20796 LGTM for the fix. +1 for more tests. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20579: [SPARK-23372][SQL] Writing empty struct in parquet fails...

2018-03-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20579 I think this should be applied to all data sources not only parquet. I can't think of any cases that a data source needs to write data with empty schema, cc @rdblue for confirmation. ---

[GitHub] spark issue #20790: [SPARK-23642][DOCS] AccumulatorV2 subclass isZero scalad...

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20790 **[Test build #88220 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88220/testReport)** for PR 20790 at commit

[GitHub] spark issue #20790: [SPARK-23642][DOCS] AccumulatorV2 subclass isZero scalad...

2018-03-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20790 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-13 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r174350660 --- Diff: bin/docker-image-tool.sh --- @@ -63,9 +63,11 @@ function build { error "Cannot find docker image. This script must be run from a

[GitHub] spark pull request #20806: [SPARK-23661][SQL] Implement treeAggregate on Dat...

2018-03-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20806#discussion_r174350193 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1658,6 +1659,43 @@ class Dataset[T] private[sql]( def

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-13 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r174348342 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/ClientSuite.scala --- @@ -108,62 +111,42 @@ class

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-13 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r174345430 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala --- @@ -89,26 +56,16 @@ private[spark]

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-13 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r174345374 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala --- @@ -89,26 +56,16 @@ private[spark]

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-13 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r174345265 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala --- @@ -180,6 +179,26

[GitHub] spark pull request #20806: [SPARK-23661][SQL] Implement treeAggregate on Dat...

2018-03-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20806#discussion_r174337702 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1658,6 +1659,43 @@ class Dataset[T] private[sql]( def groupByKey[K](func:

[GitHub] spark issue #20690: [SPARK-23532][Standalone]Improve data locality when laun...

2018-03-13 Thread 10110346
Github user 10110346 commented on the issue: https://github.com/apache/spark/pull/20690 @jiangxb1987 @jerryshao Could you help review it ? thanks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20816: [SPARK-21479][SQL] Outer join filter pushdown in null su...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20816 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20816: [SPARK-21479][SQL] Outer join filter pushdown in null su...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20816 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88216/ Test FAILed. ---

[GitHub] spark issue #20816: [SPARK-21479][SQL] Outer join filter pushdown in null su...

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20816 **[Test build #88216 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88216/testReport)** for PR 20816 at commit

[GitHub] spark issue #20799: [SPARK-23635][YARN] AM env variable should not overwrite...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20799 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88219/ Test PASSed. ---

[GitHub] spark issue #20799: [SPARK-23635][YARN] AM env variable should not overwrite...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20799 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20799: [SPARK-23635][YARN] AM env variable should not overwrite...

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20799 **[Test build #88219 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88219/testReport)** for PR 20799 at commit

[GitHub] spark pull request #20806: [SPARK-23661][SQL] Implement treeAggregate on Dat...

2018-03-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20806#discussion_r174336159 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1658,6 +1659,43 @@ class Dataset[T] private[sql]( def groupByKey[K](func:

[GitHub] spark issue #20799: [SPARK-23635][YARN] AM env variable should not overwrite...

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20799 **[Test build #88219 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88219/testReport)** for PR 20799 at commit

[GitHub] spark issue #20799: [SPARK-23635][YARN] AM env variable should not overwrite...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20799 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20799: [SPARK-23635][YARN] AM env variable should not overwrite...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20799 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1498/

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r174334386 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -49,55 +50,81 @@ public static int hashInt(int input, int seed)

[GitHub] spark issue #20812: [SPARK-23669] Executors fetch jars and name the jars wit...

2018-03-13 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/20812 @vanzin @zsxwing @jerryshao How do you think about this ? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r174333828 --- Diff: common/unsafe/src/main/java/org/apache/spark/sql/catalyst/expressions/HiveHasher.java --- @@ -38,12 +39,18 @@ public static int hashLong(long

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r174333675 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/ByteArrayMemoryBlock.java --- @@ -0,0 +1,131 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88218/ Test FAILed. ---

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20659 **[Test build #88218 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88218/testReport)** for PR 20659 at commit

[GitHub] spark pull request #20802: [SPARK-23651][core]Add a check for host name

2018-03-13 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/20802#discussion_r174332968 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -975,6 +975,8 @@ private[spark] object Utils extends Logging { def

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20659 **[Test build #88218 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88218/testReport)** for PR 20659 at commit

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1497/

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20659 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88217/ Test FAILed. ---

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20659 **[Test build #88217 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88217/testReport)** for PR 20659 at commit

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20799: [SPARK-23635][YARN] AM env variable should not ov...

2018-03-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20799#discussion_r174331279 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala --- @@ -247,6 +241,18 @@ private[yarn] class

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20659 **[Test build #88217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88217/testReport)** for PR 20659 at commit

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20659: [DO-NOT-MERGE] Try to update Hive to 2.3.2

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20659 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1496/

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-13 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r174329806 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/ClientSuite.scala --- @@ -108,62 +111,42 @@ class

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-13 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r174329513 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala --- @@ -89,26 +56,16 @@ private[spark] object

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-13 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r174329568 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala --- @@ -89,26 +56,16 @@ private[spark] object

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-13 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r174328774 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala --- @@ -180,6 +179,26 @@

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-13 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r174328964 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala --- @@ -202,6 +221,10 @@

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-13 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r174328623 --- Diff: bin/docker-image-tool.sh --- @@ -63,9 +63,11 @@ function build { error "Cannot find docker image. This script must be run from a

[GitHub] spark issue #20816: SPARK-21479 Outer join filter pushdown in null supplying...

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20816 **[Test build #88216 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88216/testReport)** for PR 20816 at commit

[GitHub] spark issue #20816: SPARK-21479 Outer join filter pushdown in null supplying...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20816 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20816: SPARK-21479 Outer join filter pushdown in null supplying...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20816 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1495/

[GitHub] spark issue #20640: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2018-03-13 Thread susanxhuynh
Github user susanxhuynh commented on the issue: https://github.com/apache/spark/pull/20640 @skonto Yes, I'm ok with that. Sorry for the delayed response. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20816: SPARK-21479 Outer join filter pushdown in null supplying...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20816 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88215/ Test FAILed. ---

[GitHub] spark issue #20816: SPARK-21479 Outer join filter pushdown in null supplying...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20816 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20816: SPARK-21479 Outer join filter pushdown in null supplying...

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20816 **[Test build #88215 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88215/testReport)** for PR 20816 at commit

[GitHub] spark issue #20815: [SPARK-23658][LAUNCHER] InProcessAppHandle uses the wron...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20815 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88213/ Test PASSed. ---

[GitHub] spark issue #20815: [SPARK-23658][LAUNCHER] InProcessAppHandle uses the wron...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20815 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20815: [SPARK-23658][LAUNCHER] InProcessAppHandle uses the wron...

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20815 **[Test build #88213 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88213/testReport)** for PR 20815 at commit

[GitHub] spark issue #20796: [SPARK-23649][SQL] Prevent crashes on schema inferring o...

2018-03-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20796 the fix LGTM, we can add more tests for different ranges of the invalid chars. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20816: SPARK-21479 Outer join filter pushdown in null su...

2018-03-13 Thread maryannxue
Github user maryannxue commented on a diff in the pull request: https://github.com/apache/spark/pull/20816#discussion_r174303151 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1071,6 +1072,66 @@ object

[GitHub] spark pull request #20816: SPARK-21479 Outer join filter pushdown in null su...

2018-03-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20816#discussion_r174302717 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1071,6 +1072,66 @@ object

[GitHub] spark pull request #20816: SPARK-21479 Outer join filter pushdown in null su...

2018-03-13 Thread maryannxue
Github user maryannxue commented on a diff in the pull request: https://github.com/apache/spark/pull/20816#discussion_r174302419 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1071,6 +1072,66 @@ object

[GitHub] spark pull request #20816: SPARK-21479 Outer join filter pushdown in null su...

2018-03-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20816#discussion_r174301967 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/InferFiltersFromConstraintsSuite.scala --- @@ -204,4 +205,40 @@ class

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r174295581 --- Diff: common/unsafe/src/main/java/org/apache/spark/sql/catalyst/expressions/HiveHasher.java --- @@ -38,12 +39,18 @@ public static int hashLong(long

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r174300387 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/hash/Murmur3_x86_32.java --- @@ -49,55 +50,81 @@ public static int hashInt(int input, int

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r174300982 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/OffHeapMemoryBlock.java --- @@ -0,0 +1,108 @@ +/* + * Licensed to the

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r174301203 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/memory/ByteArrayMemoryBlock.java --- @@ -0,0 +1,131 @@ +/* + * Licensed to the

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r174295402 --- Diff: common/unsafe/src/main/java/org/apache/spark/sql/catalyst/expressions/HiveHasher.java --- @@ -38,12 +39,18 @@ public static int hashLong(long

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2018-03-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r174296632 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/LongArray.java --- @@ -80,7 +76,7 @@ public void zeroOut() { public void

[GitHub] spark pull request #20816: SPARK-21479 Outer join filter pushdown in null su...

2018-03-13 Thread maryannxue
Github user maryannxue commented on a diff in the pull request: https://github.com/apache/spark/pull/20816#discussion_r174300996 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/InferFiltersFromConstraintsSuite.scala --- @@ -204,4 +205,40 @@ class

[GitHub] spark pull request #20779: [SPARK-23598][SQL] Make methods in BufferedRowIte...

2018-03-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20779 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20816: SPARK-21479 Outer join filter pushdown in null su...

2018-03-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20816#discussion_r174299561 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/InferFiltersFromConstraintsSuite.scala --- @@ -204,4 +205,40 @@ class

[GitHub] spark pull request #20816: SPARK-21479 Outer join filter pushdown in null su...

2018-03-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20816#discussion_r174299321 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1071,6 +1072,66 @@ object

[GitHub] spark issue #20814: [SPARK-23671][core] Fix condition to enable the SHS thre...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20814 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88211/ Test PASSed. ---

[GitHub] spark issue #20814: [SPARK-23671][core] Fix condition to enable the SHS thre...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20814 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20814: [SPARK-23671][core] Fix condition to enable the SHS thre...

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20814 **[Test build #88211 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88211/testReport)** for PR 20814 at commit

[GitHub] spark pull request #20816: SPARK-21479 Outer join filter pushdown in null su...

2018-03-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20816#discussion_r174297591 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1071,6 +1072,66 @@ object

[GitHub] spark issue #20779: [SPARK-23598][SQL] Make methods in BufferedRowIterator p...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20779 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20779: [SPARK-23598][SQL] Make methods in BufferedRowIterator p...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20779 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88212/ Test PASSed. ---

[GitHub] spark issue #20779: [SPARK-23598][SQL] Make methods in BufferedRowIterator p...

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20779 **[Test build #88212 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88212/testReport)** for PR 20779 at commit

[GitHub] spark pull request #20799: [SPARK-23635][YARN] AM env variable should not ov...

2018-03-13 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/20799#discussion_r174294428 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ExecutorRunnable.scala --- @@ -247,6 +241,18 @@ private[yarn] class

[GitHub] spark issue #20816: SPARK-21479 Outer join filter pushdown in null supplying...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20816 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1494/

[GitHub] spark issue #20816: SPARK-21479 Outer join filter pushdown in null supplying...

2018-03-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20816 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20796: [SPARK-23649][SQL] Prevent crashes on schema infe...

2018-03-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20796#discussion_r174293523 --- Diff: sql/core/src/test/resources/test-data/utf8xFF.csv --- @@ -0,0 +1,3 @@ +channel,code +United,123 +ABGUN�,456 --- End diff --

[GitHub] spark issue #20816: SPARK-21479 Outer join filter pushdown in null supplying...

2018-03-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20816 **[Test build #88215 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88215/testReport)** for PR 20816 at commit

[GitHub] spark pull request #20816: SPARK-21479 Outer join filter pushdown in null su...

2018-03-13 Thread maryannxue
GitHub user maryannxue opened a pull request: https://github.com/apache/spark/pull/20816 SPARK-21479 Outer join filter pushdown in null supplying table when condition is on one of the joined columns ## What changes were proposed in this pull request? Added

[GitHub] spark pull request #20806: [SPARK-23661][SQL] Implement treeAggregate on Dat...

2018-03-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20806#discussion_r174277864 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1658,6 +1659,43 @@ class Dataset[T] private[sql]( def

  1   2   3   >