[GitHub] spark issue #20209: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20209 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85866/ Test FAILed. ---

[GitHub] spark issue #20209: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20209 **[Test build #85866 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85866/testReport)** for PR 20209 at commit [`f6215fc`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #20209: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20209 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-09 Thread tomasatdatabricks
Github user tomasatdatabricks commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r160496086 --- Diff: python/pyspark/ml/image.py --- @@ -71,9 +88,30 @@ def ocvTypes(self): """ if self._ocvTypes is None:

[GitHub] spark issue #19290: [SPARK-22063][R] Fixes lint check failures in R by lates...

2018-01-09 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/19290 ok sounds good -- we'll keep things 'old' for now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addi

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20204 **[Test build #85856 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85856/testReport)** for PR 20204 at commit [`9f2c400`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85856/ Test FAILed. ---

[GitHub] spark issue #18991: [SPARK-21783][SQL][WIP] Turn on ORC filter push-down by ...

2018-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18991 I reopen it to re-test the master branch with this option before Apache Spark 2.3. --- - To unsubscribe, e-mail: reviews-u

[GitHub] spark pull request #18991: [SPARK-21783][SQL][WIP] Turn on ORC filter push-d...

2018-01-09 Thread dongjoon-hyun
GitHub user dongjoon-hyun reopened a pull request: https://github.com/apache/spark/pull/18991 [SPARK-21783][SQL][WIP] Turn on ORC filter push-down by default ## What changes were proposed in this pull request? ORC filter push-down is disabled by default from the beginning,

[GitHub] spark issue #18991: [SPARK-21783][SQL][WIP] Turn on ORC filter push-down by ...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18991 **[Test build #85868 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85868/testReport)** for PR 18991 at commit [`2bc2b17`](https://github.com/apache/spark/commit/2b

[GitHub] spark issue #20013: [SPARK-20657][core] Speed up rendering of the stages pag...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20013 **[Test build #85867 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85867/testReport)** for PR 20013 at commit [`86275b0`](https://github.com/apache/spark/commit/86

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20203 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20203 **[Test build #85869 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85869/testReport)** for PR 20203 at commit [`8d736c1`](https://github.com/apache/spark/commit/8d

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20203 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #19893: [SPARK-16139][TEST] Add logging functionality for...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19893#discussion_r160501177 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/SharedSQLContext.scala --- @@ -17,4 +17,22 @@ package org.apache.spark.sql.test

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20203 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85869/ Test FAILed. ---

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20203 **[Test build #85869 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85869/testReport)** for PR 20203 at commit [`8d736c1`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #20097: [SPARK-22912] v2 data source support in MicroBatchExecut...

2018-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20097 Hi, @tdas . Could you merge this to `branch-2.3` , too? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-09 Thread MrBago
Github user MrBago commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r160502167 --- Diff: python/pyspark/ml/image.py --- @@ -71,9 +88,30 @@ def ocvTypes(self): """ if self._ocvTypes is None: -

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160502356 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/executor/Dockerfile --- @@ -1,35 +0,0 @@ -# -# Licensed to the Apache Software Fou

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160502410 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile --- @@ -41,7 +41,8 @@ COPY ${spark_jars} /opt/spark/jars COPY bin /o

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160502618 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh --- @@ -0,0 +1,97 @@ +#!/bin/bash +# +# Licensed to the Ap

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160503103 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -29,17 +29,23 @@ private[spark] object Config ext

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160504887 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/DriverConfigOrchestratorSuite.scala --- @@ -75,8 +73,8 @@ clas

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160504833 --- Diff: docs/running-on-kubernetes.md --- @@ -56,14 +56,13 @@ be run in a container runtime environment that Kubernetes supports. Docker is a frequent

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20204 **[Test build #85855 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85855/testReport)** for PR 20204 at commit [`a3179d7`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #19290: [SPARK-22063][R] Fixes lint check failures in R by lates...

2018-01-09 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/19290 The minimum R version supported is something that we can revisit though. I think we do this for Python, Java versions as well in the project --- --

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160463657 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160483562 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala --- @@ -1044,6 +1056,50 @@ class LinearRegressionSuite extends MLT

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160461560 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -85,12 +87,55 @@ private[util] sealed trait BaseReadWrite { protected fi

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160506592 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala --- @@ -1044,6 +1056,50 @@ class LinearRegressionSuite extends MLT

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85855/ Test PASSed. ---

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160496808 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160462794 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -85,12 +87,55 @@ private[util] sealed trait BaseReadWrite { protected fi

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160502536 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -85,12 +87,55 @@ private[util] sealed trait BaseReadWrite { protected fi

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160501723 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160463225 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -85,12 +87,55 @@ private[util] sealed trait BaseReadWrite { protected fi

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160503322 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160471845 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -710,15 +711,57 @@ class LinearRegressionModel private[ml] (

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160503640 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -85,12 +87,55 @@ private[util] sealed trait BaseReadWrite { protected fi

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160503466 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160484001 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala --- @@ -1044,6 +1056,50 @@ class LinearRegressionSuite extends MLT

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160461644 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20192 > users with custom docker images can override the classpath by I wrote this in a comment above, but there needs to be a proper definition of how to customize these docker images. There needs

[GitHub] spark issue #20189: [SPARK-22975] MetricsReporter should not throw exception...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20189 **[Test build #85861 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85861/testReport)** for PR 20189 at commit [`b7dc922`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20189: [SPARK-22975] MetricsReporter should not throw exception...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20189 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20189: [SPARK-22975] MetricsReporter should not throw exception...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20189 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85861/ Test PASSed. ---

[GitHub] spark issue #20201: [SPARK-22389][SQL] data source v2 partitioning reporting...

2018-01-09 Thread RussellSpitzer
Github user RussellSpitzer commented on the issue: https://github.com/apache/spark/pull/20201 This looks very exciting to me --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20192 **[Test build #85870 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85870/testReport)** for PR 20192 at commit [`e771ed9`](https://github.com/apache/spark/commit/e7

[GitHub] spark issue #20097: [SPARK-22912] v2 data source support in MicroBatchExecut...

2018-01-09 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20097 Yes. My bad. I didnt realize the branch had already been cut. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20192 @vanzin, do you have some time to modify the integration tests as well? The change LGTM, but a clean run on minikube would give us a lot more confidence. Until the integration tests get checked in to

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20203 **[Test build #85872 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85872/testReport)** for PR 20203 at commit [`d8c214b`](https://github.com/apache/spark/commit/d8

[GitHub] spark issue #20209: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20209 **[Test build #85871 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85871/testReport)** for PR 20209 at commit [`2c10416`](https://github.com/apache/spark/commit/2c

[GitHub] spark pull request #20210: [SPARK-23009][PYTHON] Fix for non-str col names t...

2018-01-09 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/20210 [SPARK-23009][PYTHON] Fix for non-str col names to createDataFrame from Pandas ## What changes were proposed in this pull request? This the case when calling `SparkSession.createDataFr

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20210 Just came across this issue, ping @HyukjinKwon @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org Fo

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160511180 --- Diff: docs/running-on-kubernetes.md --- @@ -56,14 +56,13 @@ be run in a container runtime environment that Kubernetes supports. Docker is a freq

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20210 **[Test build #85873 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85873/testReport)** for PR 20210 at commit [`e2b1a41`](https://github.com/apache/spark/commit/e2

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160512231 --- Diff: docs/running-on-kubernetes.md --- @@ -56,14 +56,13 @@ be run in a container runtime environment that Kubernetes supports. Docker is a frequent

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20192 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.a

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20210 **[Test build #85874 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85874/testReport)** for PR 20210 at commit [`e2e6025`](https://github.com/apache/spark/commit/e2

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160512758 --- Diff: docs/running-on-kubernetes.md --- @@ -56,14 +56,13 @@ be run in a container runtime environment that Kubernetes supports. Docker is a freq

[GitHub] spark issue #20097: [SPARK-22912] v2 data source support in MicroBatchExecut...

2018-01-09 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20097 Done. https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=44763d93c0d923977c114d63586abfc1b68ad7fc --- - To unsubsc

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20204 **[Test build #85858 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85858/testReport)** for PR 20204 at commit [`3c3c3cb`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85858/ Test PASSed. ---

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20097: [SPARK-22912] v2 data source support in MicroBatchExecut...

2018-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20097 Thank you, @tdas ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revi

[GitHub] spark pull request #20096: [SPARK-22908] Add kafka source and sink for conti...

2018-01-09 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20096#discussion_r160516177 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaContinuousSuite.scala --- @@ -0,0 +1,133 @@ +/* + * Licensed to th

[GitHub] spark issue #20209: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20209 **[Test build #85871 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85871/testReport)** for PR 20209 at commit [`2c10416`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #20209: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20209 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85871/ Test PASSed. ---

[GitHub] spark issue #20209: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20209 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20210 **[Test build #85873 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85873/testReport)** for PR 20210 at commit [`e2b1a41`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20210 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85873/ Test FAILed. ---

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20210 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20192 > do you have some time to modify the integration tests as well I can try to look, but really you guys should be putting that code into the Spark repo. I don't see a task under SPARK-18278 fo

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20023 **[Test build #85864 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85864/testReport)** for PR 20023 at commit [`20616fd`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20023 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20023 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85864/ Test FAILed. ---

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20210 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20210 **[Test build #85874 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85874/testReport)** for PR 20210 at commit [`e2e6025`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20210 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85874/ Test FAILed. ---

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20192 Thanks @vanzin. I was waiting on spark-dev [thread on integration testing](http://apache-spark-developers-list.1001551.n3.nabble.com/Integration-testing-and-Scheduler-Backends-td23105.html) to conclu

[GitHub] spark issue #20151: [SPARK-22959][PYTHON] Configuration to select the module...

2018-01-09 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/20151 So I think this could be the basis for solving a lot of related problems and I like the minimally invasive approach to it. I think the error message for setting it to a bad module rather than a none

[GitHub] spark issue #20096: [SPARK-22908] Add kafka source and sink for continuous p...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20096 **[Test build #85875 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85875/testReport)** for PR 20096 at commit [`f825155`](https://github.com/apache/spark/commit/f8

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-09 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13599 So for what its worth I made a quick POC of supporting similar functionality without requiring any changes to Spark its self ( https://github.com/nteract/coffee_boat ) which should also work in stan

[GitHub] spark pull request #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pys...

2018-01-09 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/13599#discussion_r160521328 --- Diff: python/pyspark/context.py --- @@ -1023,6 +1032,35 @@ def getConf(self): conf.setAll(self._conf.getAll()) return conf

[GitHub] spark issue #20096: [SPARK-22908] Add kafka source and sink for continuous p...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20096 **[Test build #85876 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85876/testReport)** for PR 20096 at commit [`9101ea6`](https://github.com/apache/spark/commit/91

[GitHub] spark issue #20188: [SPARK-22993][ML] Clarify HasCheckpointInterval param do...

2018-01-09 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/20188 Good call @felixcheung! Will update shortly. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional com

[GitHub] spark pull request #20211: [SPARK-23011][PYTHON][SQL] Prepend missing groupi...

2018-01-09 Thread icexelloss
GitHub user icexelloss opened a pull request: https://github.com/apache/spark/pull/20211 [SPARK-23011][PYTHON][SQL] Prepend missing grouping key in groupby apply ## What changes were proposed in this pull request? See https://issues.apache.org/jira/browse/SPARK-23011

[GitHub] spark issue #20211: [SPARK-23011][PYTHON][SQL] Prepend missing grouping key ...

2018-01-09 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20211 cc @HyukjinKwon @ueshin @cloud-fan @viirya --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20211: [SPARK-23011][PYTHON][SQL] Prepend missing grouping colu...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20211 **[Test build #85877 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85877/testReport)** for PR 20211 at commit [`f2822b5`](https://github.com/apache/spark/commit/f2

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20168 **[Test build #85878 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85878/testReport)** for PR 20168 at commit [`eee25ce`](https://github.com/apache/spark/commit/ee

[GitHub] spark issue #20211: [SPARK-23011][PYTHON][SQL] Prepend missing grouping colu...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20211 **[Test build #85877 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85877/testReport)** for PR 20211 at commit [`f2822b5`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #20211: [SPARK-23011][PYTHON][SQL] Prepend missing grouping colu...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20211 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85877/ Test FAILed. ---

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20168 **[Test build #85878 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85878/testReport)** for PR 20168 at commit [`eee25ce`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20168 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85878/ Test FAILed. ---

[GitHub] spark issue #20211: [SPARK-23011][PYTHON][SQL] Prepend missing grouping colu...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20211 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20168 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20211: [SPARK-23011][PYTHON][SQL] Prepend missing groupi...

2018-01-09 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20211#discussion_r160524679 --- Diff: python/pyspark/sql/group.py --- @@ -233,6 +233,27 @@ def apply(self, udf): | 2| 1.1094003924504583| +---+

<    1   2   3   4   5   6   >