[GitHub] spark issue #20013: [SPARK-20657][core] Speed up rendering of the stages pag...

2018-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20013 > the memory usage will be much larger with InMemoryStore. No it won't. The in-memory store does not use extra memory for indices. It sorts the data when the index is read, and discards that

[GitHub] spark pull request #20153: [SPARK-22392][SQL] data source v2 columnar batch ...

2018-01-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20153#discussion_r160477594 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala --- @@ -17,21 +17,24 @@ package

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20204 Sorry for noice. I made some clean up and just rebased. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20205: [SPARK-16060][SQL][follow-up] add a wrapper solution for...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20205 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85857/ Test FAILed. ---

[GitHub] spark issue #20207: [SPARK-23000] [TEST-HADOOP2.6] Fix Flaky test suite Data...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20207 **[Test build #85860 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85860/testReport)** for PR 20207 at commit

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85856/ Test FAILed. ---

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20203 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19290: [SPARK-22063][R] Fixes lint check failures in R by lates...

2018-01-09 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/19290 The minimum R version supported is something that we can revisit though. I think we do this for Python, Java versions as well in the project ---

[GitHub] spark issue #20189: [SPARK-22975] MetricsReporter should not throw exception...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20189 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20189: [SPARK-22975] MetricsReporter should not throw exception...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20189 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85861/ Test PASSed. ---

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20210 **[Test build #85873 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85873/testReport)** for PR 20210 at commit

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20192 Thanks @vanzin. I was waiting on spark-dev [thread on integration testing](http://apache-spark-developers-list.1001551.n3.nabble.com/Integration-testing-and-Scheduler-Backends-td23105.html) to

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20204 **[Test build #85855 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85855/testReport)** for PR 20204 at commit

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20192 > users with custom docker images can override the classpath by I wrote this in a comment above, but there needs to be a proper definition of how to customize these docker images. There

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160512231 --- Diff: docs/running-on-kubernetes.md --- @@ -56,14 +56,13 @@ be run in a container runtime environment that Kubernetes supports. Docker is a

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20204 **[Test build #85858 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85858/testReport)** for PR 20204 at commit

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85858/ Test PASSed. ---

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20097: [SPARK-22912] v2 data source support in MicroBatchExecut...

2018-01-09 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20097 Thank you, @tdas ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20023 **[Test build #85864 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85864/testReport)** for PR 20023 at commit

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20023 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20023 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85864/ Test FAILed. ---

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20210 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20210 **[Test build #85874 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85874/testReport)** for PR 20210 at commit

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20210 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85874/ Test FAILed. ---

[GitHub] spark issue #20151: [SPARK-22959][PYTHON] Configuration to select the module...

2018-01-09 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/20151 So I think this could be the basis for solving a lot of related problems and I like the minimally invasive approach to it. I think the error message for setting it to a bad module rather than a

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-09 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/13599 So for what its worth I made a quick POC of supporting similar functionality without requiring any changes to Spark its self ( https://github.com/nteract/coffee_boat ) which should also work in

[GitHub] spark issue #20188: [SPARK-22993][ML] Clarify HasCheckpointInterval param do...

2018-01-09 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/20188 Good call @felixcheung! Will update shortly. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20168 **[Test build #85878 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85878/testReport)** for PR 20168 at commit

[GitHub] spark issue #20211: [SPARK-23011][PYTHON][SQL] Prepend missing grouping colu...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20211 **[Test build #85877 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85877/testReport)** for PR 20211 at commit

[GitHub] spark issue #20211: [SPARK-23011][PYTHON][SQL] Prepend missing grouping colu...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20211 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85877/ Test FAILed. ---

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20168 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85878/ Test FAILed. ---

[GitHub] spark issue #20211: [SPARK-23011][PYTHON][SQL] Prepend missing grouping colu...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20211 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20168 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20168: [SPARK-22730][ML] Add ImageSchema support for non...

2018-01-09 Thread MrBago
Github user MrBago commented on a diff in the pull request: https://github.com/apache/spark/pull/20168#discussion_r160502167 --- Diff: python/pyspark/ml/image.py --- @@ -71,9 +88,30 @@ def ocvTypes(self): """ if self._ocvTypes is None: -

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160503103 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala --- @@ -29,17 +29,23 @@ private[spark] object Config

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160504887 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/submit/DriverConfigOrchestratorSuite.scala --- @@ -75,8 +73,8 @@

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160504833 --- Diff: docs/running-on-kubernetes.md --- @@ -56,14 +56,13 @@ be run in a container runtime environment that Kubernetes supports. Docker is a

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20210 **[Test build #85874 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85874/testReport)** for PR 20210 at commit

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160512758 --- Diff: docs/running-on-kubernetes.md --- @@ -56,14 +56,13 @@ be run in a container runtime environment that Kubernetes supports. Docker is a

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20210 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85873/ Test FAILed. ---

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20210 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20096: [SPARK-22908] Add kafka source and sink for continuous p...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20096 **[Test build #85875 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85875/testReport)** for PR 20096 at commit

[GitHub] spark issue #20096: [SPARK-22908] Add kafka source and sink for continuous p...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20096 **[Test build #85876 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85876/testReport)** for PR 20096 at commit

[GitHub] spark pull request #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pys...

2018-01-09 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/13599#discussion_r160521328 --- Diff: python/pyspark/context.py --- @@ -1023,6 +1032,35 @@ def getConf(self): conf.setAll(self._conf.getAll()) return conf

[GitHub] spark pull request #20211: [SPARK-23011][PYTHON][SQL] Prepend missing groupi...

2018-01-09 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/20211#discussion_r160524679 --- Diff: python/pyspark/sql/group.py --- @@ -233,6 +233,27 @@ def apply(self, udf): | 2| 1.1094003924504583|

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20168 **[Test build #85880 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85880/testReport)** for PR 20168 at commit

[GitHub] spark issue #20211: [SPARK-23011][PYTHON][SQL] Prepend missing grouping colu...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20211 **[Test build #85879 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85879/testReport)** for PR 20211 at commit

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160502356 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/executor/Dockerfile --- @@ -1,35 +0,0 @@ -# -# Licensed to the Apache Software

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160502410 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile --- @@ -41,7 +41,8 @@ COPY ${spark_jars} /opt/spark/jars COPY bin

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160463657 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160483562 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala --- @@ -1044,6 +1056,50 @@ class LinearRegressionSuite extends

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160461560 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -85,12 +87,55 @@ private[util] sealed trait BaseReadWrite { protected

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160506592 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala --- @@ -1044,6 +1056,50 @@ class LinearRegressionSuite extends

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85855/ Test PASSed. ---

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160496808 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160462794 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -85,12 +87,55 @@ private[util] sealed trait BaseReadWrite { protected

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160502536 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -85,12 +87,55 @@ private[util] sealed trait BaseReadWrite { protected

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160501723 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160463225 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -85,12 +87,55 @@ private[util] sealed trait BaseReadWrite { protected

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160503322 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160471845 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala --- @@ -710,15 +711,57 @@ class LinearRegressionModel private[ml] (

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160503640 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -85,12 +87,55 @@ private[util] sealed trait BaseReadWrite { protected

[GitHub] spark issue #20204: [SPARK-7721][PYTHON][TESTS] Adds PySpark coverage genera...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20204 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160503466 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160484001 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala --- @@ -1044,6 +1056,50 @@ class LinearRegressionSuite extends

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-01-09 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r160461644 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -126,15 +180,69 @@ abstract class MLWriter extends BaseReadWrite with Logging

[GitHub] spark issue #20189: [SPARK-22975] MetricsReporter should not throw exception...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20189 **[Test build #85861 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85861/testReport)** for PR 20189 at commit

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160511180 --- Diff: docs/running-on-kubernetes.md --- @@ -56,14 +56,13 @@ be run in a container runtime environment that Kubernetes supports. Docker is a

[GitHub] spark issue #20209: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20209 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20209: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20209 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85871/ Test PASSed. ---

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/20192 > do you have some time to modify the integration tests as well I can try to look, but really you guys should be putting that code into the Spark repo. I don't see a task under SPARK-18278

[GitHub] spark issue #20211: [SPARK-23011][PYTHON][SQL] Prepend missing grouping key ...

2018-01-09 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20211 cc @HyukjinKwon @ueshin @cloud-fan @viirya --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20192: [SPARK-22994][k8s] Use a single image for all Spa...

2018-01-09 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20192#discussion_r160502618 --- Diff: resource-managers/kubernetes/docker/src/main/dockerfiles/spark/entrypoint.sh --- @@ -0,0 +1,97 @@ +#!/bin/bash +# +# Licensed to the

[GitHub] spark issue #20201: [SPARK-22389][SQL] data source v2 partitioning reporting...

2018-01-09 Thread RussellSpitzer
Github user RussellSpitzer commented on the issue: https://github.com/apache/spark/pull/20201 This looks very exciting to me --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20097: [SPARK-22912] v2 data source support in MicroBatchExecut...

2018-01-09 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20097 Yes. My bad. I didnt realize the branch had already been cut. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20192 **[Test build #85870 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85870/testReport)** for PR 20192 at commit

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20203 **[Test build #85872 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85872/testReport)** for PR 20203 at commit

[GitHub] spark issue #20209: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20209 **[Test build #85871 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85871/testReport)** for PR 20209 at commit

[GitHub] spark pull request #20210: [SPARK-23009][PYTHON] Fix for non-str col names t...

2018-01-09 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/20210 [SPARK-23009][PYTHON] Fix for non-str col names to createDataFrame from Pandas ## What changes were proposed in this pull request? This the case when calling

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20210 Just came across this issue, ping @HyukjinKwon @ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/20192 @vanzin, do you have some time to modify the integration tests as well? The change LGTM, but a clean run on minikube would give us a lot more confidence. Until the integration tests get checked in

[GitHub] spark issue #20210: [SPARK-23009][PYTHON] Fix for non-str col names to creat...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20210 **[Test build #85873 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85873/testReport)** for PR 20210 at commit

[GitHub] spark issue #20192: [SPARK-22994][k8s] Use a single image for all Spark cont...

2018-01-09 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/20192 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20097: [SPARK-22912] v2 data source support in MicroBatchExecut...

2018-01-09 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/20097 Done. https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=44763d93c0d923977c114d63586abfc1b68ad7fc --- - To

[GitHub] spark issue #20209: [SPARK-23008][ML] OnehotEncoderEstimator python API

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20209 **[Test build #85871 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85871/testReport)** for PR 20209 at commit

[GitHub] spark pull request #20096: [SPARK-22908] Add kafka source and sink for conti...

2018-01-09 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/20096#discussion_r160516177 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaContinuousSuite.scala --- @@ -0,0 +1,133 @@ +/* + * Licensed to

[GitHub] spark pull request #20211: [SPARK-23011][PYTHON][SQL] Prepend missing groupi...

2018-01-09 Thread icexelloss
GitHub user icexelloss opened a pull request: https://github.com/apache/spark/pull/20211 [SPARK-23011][PYTHON][SQL] Prepend missing grouping key in groupby apply ## What changes were proposed in this pull request? See https://issues.apache.org/jira/browse/SPARK-23011

[GitHub] spark issue #20211: [SPARK-23011][PYTHON][SQL] Prepend missing grouping colu...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20211 **[Test build #85877 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85877/testReport)** for PR 20211 at commit

[GitHub] spark issue #20168: [SPARK-22730][ML] Add ImageSchema support for non-intege...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20168 **[Test build #85878 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85878/testReport)** for PR 20168 at commit

[GitHub] spark issue #20190: [SPARK-22976][Core]: Cluster mode driver directories can...

2018-01-09 Thread RussellSpitzer
Github user RussellSpitzer commented on the issue: https://github.com/apache/spark/pull/20190 @zsxwing I think you were the last to touch this code, could you please review? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20153: [SPARK-22392][SQL] data source v2 columnar batch ...

2018-01-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20153#discussion_r160472490 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ColumnarBatchScan.scala --- @@ -137,4 +147,25 @@ private[sql] trait ColumnarBatchScan

[GitHub] spark issue #20153: [SPARK-22392][SQL] data source v2 columnar batch reader

2018-01-09 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20153 Is `ColumnarBatchScan` appropriate name for now? This is because if `supportBatch` is false, the class handles scan from a `row`, not `columnar` or `batch`. ---

[GitHub] spark pull request #20179: [SPARK-22982] Remove unsafe asynchronous close() ...

2018-01-09 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/20179#discussion_r160481843 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -196,11 +196,24 @@ private[spark] class

[GitHub] spark issue #20023: [SPARK-22036][SQL] Decimal multiplication with high prec...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20023 **[Test build #85864 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85864/testReport)** for PR 20023 at commit

[GitHub] spark issue #20200: [SPARK-23005][Core] Improve RDD.take on small number of ...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20200 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20200: [SPARK-23005][Core] Improve RDD.take on small number of ...

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20200 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85854/ Test PASSed. ---

[GitHub] spark issue #20203: [SPARK-22577] [core] executor page blacklist status shou...

2018-01-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20203 **[Test build #85869 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85869/testReport)** for PR 20203 at commit

[GitHub] spark issue #20202: [MINOR] fix a typo in BroadcastJoinSuite

2018-01-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20202 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85853/ Test PASSed. ---

  1   2   3   4   5   6   >