[GitHub] spark issue #19692: [SPARK-22469][SQL] Accuracy problem in comparison with s...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19692 **[Test build #83833 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83833/testReport)** for PR 19692 at commit [`cb1c8b8`](https://github.com/apache/spark/commit/cb

[GitHub] spark issue #19208: [SPARK-21087] [ML] CrossValidator, TrainValidationSplit ...

2017-11-13 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19208 I manually tested backwards compatibility and it works fine. I paste the test code for `CrossValidator` here. Run following code in spark-2.2 shell first: ``` import java.io.Fil

[GitHub] spark issue #19740: [SPARK-22514][SQL] move ColumnVector.Array and ColumnarB...

2017-11-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19740 `ColumnVectorArray` was my first idea but it sounds like array of column vector. If we don't mind to be verbose, how about `ArrayInColumnVector`? ---

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19691 **[Test build #83832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83832/testReport)** for PR 19691 at commit [`f79c6f4`](https://github.com/apache/spark/commit/f7

[GitHub] spark issue #15049: [SPARK-17310][SQL] Add an option to disable record-level...

2017-11-13 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15049 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apa

[GitHub] spark pull request #15049: [SPARK-17310][SQL] Add an option to disable recor...

2017-11-13 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/15049#discussion_r150754360 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -327,6 +327,13 @@ object SQLConf { .booleanConf .c

[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...

2017-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19738 Yup, it looks so. Could we add another small test case for it as well? --- - To unsubscribe, e-mail: reviews-unsubscr...@spar

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150754184 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1,427

[GitHub] spark issue #17972: [SPARK-20723][ML]Add intermediate storage level to tree ...

2017-11-13 Thread phatak-dev
Github user phatak-dev commented on the issue: https://github.com/apache/spark/pull/17972 I have added to all the algorithms I know that uses intermediate state. --- - To unsubscribe, e-mail: reviews-unsubscr...@spar

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19691 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19691 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83831/ Test FAILed. ---

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19691 **[Test build #83831 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83831/testReport)** for PR 19691 at commit [`f18caeb`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19691 **[Test build #83831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83831/testReport)** for PR 19691 at commit [`f18caeb`](https://github.com/apache/spark/commit/f1

[GitHub] spark pull request #19631: [SPARK-22372][core, yarn] Make cluster submission...

2017-11-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19631#discussion_r150751268 --- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala --- @@ -551,13 +553,10 @@ private[spark] class SecurityManager( private[spa

[GitHub] spark pull request #19631: [SPARK-22372][core, yarn] Make cluster submission...

2017-11-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19631#discussion_r150751761 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -92,6 +92,11 @@ object SparkSubmit extends CommandLineUtils with Logging {

[GitHub] spark pull request #19631: [SPARK-22372][core, yarn] Make cluster submission...

2017-11-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19631#discussion_r150752055 --- Diff: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala --- @@ -216,7 +216,9 @@ private[spark] object CoarseGrainedExe

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150752598 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1,427

[GitHub] spark issue #19692: [SPARK-22469][SQL] Accuracy problem in comparison with s...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19692 **[Test build #83829 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83829/testReport)** for PR 19692 at commit [`9c5bba0`](https://github.com/apache/spark/commit/9c

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19468 **[Test build #83830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83830/testReport)** for PR 19468 at commit [`df03462`](https://github.com/apache/spark/commit/df

[GitHub] spark pull request #19692: [SPARK-22469][SQL] Accuracy problem in comparison...

2017-11-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19692#discussion_r150751617 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -137,6 +137,8 @@ object TypeCoercion { ca

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19691 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19691 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83828/ Test FAILed. ---

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19691 **[Test build #83828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83828/testReport)** for PR 19691 at commit [`85fdb46`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19691 **[Test build #83828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83828/testReport)** for PR 19691 at commit [`85fdb46`](https://github.com/apache/spark/commit/85

[GitHub] spark issue #19692: [SPARK-22469][SQL] Accuracy problem in comparison with s...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19692 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19692: [SPARK-22469][SQL] Accuracy problem in comparison with s...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19692 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83820/ Test PASSed. ---

[GitHub] spark issue #19692: [SPARK-22469][SQL] Accuracy problem in comparison with s...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19692 **[Test build #83820 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83820/testReport)** for PR 19692 at commit [`cf09358`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2017-11-13 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19691 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150749210 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/ConfigurationUtils.scala --- @@ -0,0 +1,50 @@ +/* + * Lic

[GitHub] spark issue #15049: [SPARK-17310][SQL] Add an option to disable record-level...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15049 **[Test build #83827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83827/testReport)** for PR 15049 at commit [`8b3f4c7`](https://github.com/apache/spark/commit/8b

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150748702 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1

[GitHub] spark issue #19588: [SPARK-12375][ML] VectorIndexerModel support handle unse...

2017-11-13 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/19588 Also we need jira for python. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19588: [SPARK-12375][ML] VectorIndexerModel support hand...

2017-11-13 Thread hhbyyh
Github user hhbyyh commented on a diff in the pull request: https://github.com/apache/spark/pull/19588#discussion_r150748259 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/VectorIndexer.scala --- @@ -311,22 +346,39 @@ class VectorIndexerModel private[ml] ( // TOD

[GitHub] spark issue #17972: [SPARK-20723][ML]Add intermediate storage level to tree ...

2017-11-13 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/17972 Have you checked other algorithms which can also apply this parameter ? --- - To unsubscribe, e-mail: reviews-unsubscr...@sp

[GitHub] spark pull request #15049: [SPARK-17310][SQL] Add an option to disable recor...

2017-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15049#discussion_r150747804 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -327,6 +327,13 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150747604 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala --- @@ -0,0 +1,227 @@ +/*

[GitHub] spark pull request #15049: [SPARK-17310][SQL] Add an option to disable recor...

2017-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15049#discussion_r150747161 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -327,6 +327,13 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #15049: [SPARK-17310][SQL] Add an option to disable recor...

2017-11-13 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/15049#discussion_r150746926 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -327,6 +327,13 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #19633: [SPARK-22411][SQL] Disable the heuristic to calcu...

2017-11-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19633#discussion_r150746876 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -424,11 +424,19 @@ case class FileSourceScanExec(

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150745794 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150745701 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1

[GitHub] spark issue #19742: [SPARK-22511][BUILD] Update maven central repo address

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19742 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83818/ Test PASSed. ---

[GitHub] spark issue #19742: [SPARK-22511][BUILD] Update maven central repo address

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19742 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #15049: [SPARK-17310][SQL] Add an option to disable recor...

2017-11-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/15049#discussion_r150745483 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -327,6 +327,13 @@ object SQLConf { .booleanConf

[GitHub] spark issue #19742: [SPARK-22511][BUILD] Update maven central repo address

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19742 **[Test build #83818 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83818/testReport)** for PR 19742 at commit [`6164916`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150744643 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1

[GitHub] spark pull request #19408: [SPARK-22180][CORE] Allow IPv6 address in org.apa...

2017-11-13 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/19408#discussion_r150743565 --- Diff: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala --- @@ -1146,6 +1146,20 @@ class UtilsSuite extends SparkFunSuite with ResetSystemP

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150743750 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1

[GitHub] spark issue #19408: [SPARK-22180][CORE] Allow IPv6 address in org.apache.spa...

2017-11-13 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19408 Sounds good. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #19640: [SPARK-16986][WEB-UI] Converter Started, Completed and L...

2017-11-13 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/19640 IMO we'd better have timeZone after each datetime string, that would avoid a lot of confusions. --- - To unsubscribe, e-mail

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150743119 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150743127 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150743147 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150743139 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150743124 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1

[GitHub] spark pull request #15049: [SPARK-17310][SQL] Add an option to disable recor...

2017-11-13 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/15049#discussion_r150743115 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -327,6 +327,13 @@ object SQLConf { .booleanConf

[GitHub] spark issue #19726: [SPARK-22490][DOC] Add PySpark doc for SparkSession.buil...

2017-11-13 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19726 LGTM for now, but I'd like you to add some comments to explain this is a workaround for Sphinx <`1.6.6`. --- - To unsubscribe, e-

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150742085 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1

[GitHub] spark issue #18853: [SPARK-21646][SQL] Add new type coercion to compatible w...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18853 **[Test build #83826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83826/testReport)** for PR 18853 at commit [`22d0355`](https://github.com/apache/spark/commit/22

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150741867 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0 +1

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150741874 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala --- @@ -0,0 +1,227 @@ +/*

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150741888 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/config.scala --- @@ -0,0 +1,114 @@ +/* + * Licensed to th

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150741898 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala --- @@ -0,0 +1,103 @@ +/*

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150741903 --- Diff: resource-managers/kubernetes/core/pom.xml --- @@ -0,0 +1,102 @@ + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3.o

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150741895 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/config.scala --- @@ -0,0 +1,114 @@ +/* + * Licensed to th

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-11-13 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r150741879 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala --- @@ -0,0 +1,227 @@ +/*

[GitHub] spark issue #19594: [WIP] [SPARK-21984] Join estimation based on equi-height...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19594 **[Test build #83825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83825/testReport)** for PR 19594 at commit [`96776ce`](https://github.com/apache/spark/commit/96

[GitHub] spark issue #19208: [SPARK-21087] [ML] CrossValidator, TrainValidationSplit ...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19208 **[Test build #83824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83824/testReport)** for PR 19208 at commit [`7e997da`](https://github.com/apache/spark/commit/7e

[GitHub] spark issue #19208: [SPARK-21087] [ML] CrossValidator, TrainValidationSplit ...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19208 **[Test build #83823 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83823/testReport)** for PR 19208 at commit [`2bb6835`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #19208: [SPARK-21087] [ML] CrossValidator, TrainValidationSplit ...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19208 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83823/ Test FAILed. ---

[GitHub] spark issue #19208: [SPARK-21087] [ML] CrossValidator, TrainValidationSplit ...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19208 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19208: [SPARK-21087] [ML] CrossValidator, TrainValidationSplit ...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19208 **[Test build #83823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83823/testReport)** for PR 19208 at commit [`2bb6835`](https://github.com/apache/spark/commit/2b

[GitHub] spark issue #17972: [SPARK-20723][ML]Add intermediate storage level to tree ...

2017-11-13 Thread phatak-dev
Github user phatak-dev commented on the issue: https://github.com/apache/spark/pull/17972 @WeichenXu123 Done the changes to mima. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comm

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17702 **[Test build #83822 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83822/testReport)** for PR 17702 at commit [`ec9c1c1`](https://github.com/apache/spark/commit/ec

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83821 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83821/testReport)** for PR 19439 at commit [`c4a6dfe`](https://github.com/apache/spark/commit/c4

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-11-13 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/19439 @HyukjinKwon and @jkbradley I've updated the documentation based on your latest comments. I believe all comments have been resolved for this PR at this point, please let me know if I missed a

[GitHub] spark issue #19739: [SPARK-22513][BUILD] Provide build profile for hadoop 2....

2017-11-13 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19739 as RM, I'd say big -100 :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2017-11-13 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/17702 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-13 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150737147 --- Diff: python/pyspark/ml/image.py --- @@ -129,7 +129,7 @@ def toImage(self, array, origin=""): :param array array: The array to c

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-13 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150737174 --- Diff: python/pyspark/ml/image.py --- @@ -168,7 +167,7 @@ def readImages(self, path, recursive=False, numPartitions=-1, :param bool dr

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-13 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150736890 --- Diff: python/pyspark/ml/image.py --- @@ -80,7 +80,7 @@ def imageFields(self): """ Returns field names of image columns.

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-13 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150736880 --- Diff: python/pyspark/ml/image.py --- @@ -65,7 +65,7 @@ def ocvTypes(self): """ Returns the OpenCV type mapping supported

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-11-13 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r150735461 --- Diff: python/pyspark/ml/image.py --- @@ -48,7 +48,7 @@ def imageSchema(self): """ Returns the image schema. -

[GitHub] spark pull request #19692: [SPARK-22469][SQL] Accuracy problem in comparison...

2017-11-13 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19692#discussion_r150734871 --- Diff: sql/core/src/test/resources/sql-tests/results/predicate-functions.sql.out --- @@ -197,7 +197,7 @@ true -- !query 24 select 2.0 <= '2.

[GitHub] spark pull request #18906: [SPARK-21692][PYSPARK][SQL] Add nullability suppo...

2017-11-13 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/18906#discussion_r150733716 --- Diff: dev/sparktestsupport/modules.py --- @@ -389,18 +389,6 @@ def __hash__(self): "python/pyspark/sql" ], python_test_goa

[GitHub] spark issue #19740: [SPARK-22514][SQL] move ColumnVector.Array and ColumnarB...

2017-11-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19740 Moving `ColumnVector.Array` and `ColumnarBatch.Row` to individual files is also good for reading column vector sources actually. --- ---

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2017-11-13 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/18906 CC @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #19740: [SPARK-22514][SQL] move ColumnVector.Array and ColumnarB...

2017-11-13 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19740 `ColumnVectorArray` and `ColumnVectorRow`? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19692: [SPARK-22469][SQL] Accuracy problem in comparison with s...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19692 **[Test build #83820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83820/testReport)** for PR 19692 at commit [`cf09358`](https://github.com/apache/spark/commit/cf

[GitHub] spark pull request #19208: [SPARK-21087] [ML] CrossValidator, TrainValidatio...

2017-11-13 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19208#discussion_r150731403 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tuning/TrainValidationSplit.scala --- @@ -177,7 +202,9 @@ class TrainValidationSplit @Since("1.5.0")

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17702 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83819/ Test FAILed. ---

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17702 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17702 **[Test build #83819 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83819/testReport)** for PR 17702 at commit [`ec9c1c1`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #19740: [SPARK-22514][SQL] move ColumnVector.Array and ColumnarB...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19740 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19740: [SPARK-22514][SQL] move ColumnVector.Array and ColumnarB...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19740 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83817/ Test PASSed. ---

[GitHub] spark issue #19740: [SPARK-22514][SQL] move ColumnVector.Array and ColumnarB...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19740 **[Test build #83817 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83817/testReport)** for PR 19740 at commit [`ba28709`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #17702: [SPARK-20408][SQL] Get the glob path in parallel to redu...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17702 **[Test build #83819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83819/testReport)** for PR 17702 at commit [`ec9c1c1`](https://github.com/apache/spark/commit/ec

[GitHub] spark issue #19740: [SPARK-22514][SQL] move ColumnVector.Array and ColumnarB...

2017-11-13 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19740 I think that it is good to have top-level classes. I will check code later today. I feel that `VectorBased...` is not clear. IMHO, `Vector` and `Column` is orthogonal. `Vector` can be used w

[GitHub] spark issue #19666: [SPARK-22451][ML] Reduce decision tree aggregate size fo...

2017-11-13 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19666 OK. I will waiting @smurching to merge split parts of #19433 get merged first, and then I will update this PR. --- - To uns

[GitHub] spark pull request #19692: [SPARK-22469][SQL] Accuracy problem in comparison...

2017-11-13 Thread liutang123
Github user liutang123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19692#discussion_r150726583 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -137,6 +137,8 @@ object TypeCoercion { c

  1   2   3   4   5   6   >