[GitHub] spark pull request #18477: [SPARK-21261][DOCS]SQL Regex document fix

2017-10-23 Thread visaxin
Github user visaxin commented on a diff in the pull request: https://github.com/apache/spark/pull/18477#discussion_r146457833 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -268,7 +268,7 @@ case class

[GitHub] spark issue #18477: [SPARK-21261][DOCS]SQL Regex document fix

2017-10-23 Thread visaxin
Github user visaxin commented on the issue: https://github.com/apache/spark/pull/18477 @gatorsmile Done --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18527: [SPARK-21101][SQL] Catch IllegalStateException when CREA...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18527 **[Test build #83007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83007/testReport)** for PR 18527 at commit

[GitHub] spark issue #19557: [SPARK-22281][SPARKR][WIP] Handle R method breaking sign...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19557 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19557: [SPARK-22281][SPARKR][WIP] Handle R method breaking sign...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19557 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83005/ Test PASSed. ---

[GitHub] spark issue #19557: [SPARK-22281][SPARKR][WIP] Handle R method breaking sign...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19557 **[Test build #83005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83005/testReport)** for PR 19557 at commit

[GitHub] spark issue #19557: [SPARK-22281][SPARKR][WIP] Handle R method breaking sign...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19557 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19557: [SPARK-22281][SPARKR][WIP] Handle R method breaking sign...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19557 **[Test build #83004 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83004/testReport)** for PR 19557 at commit

[GitHub] spark issue #19557: [SPARK-22281][SPARKR][WIP] Handle R method breaking sign...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19557 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83004/ Test PASSed. ---

[GitHub] spark issue #19529: [SPARK-22308] Support alternative unit testing styles in...

2017-10-23 Thread nkronenfeld
Github user nkronenfeld commented on the issue: https://github.com/apache/spark/pull/19529 Yeah, as predicted, that made PlanTest very easy to review, but didn't do as well with SQLTestUtils. I suspect I reordered functions and what-not when I was moving stuff around. If

[GitHub] spark issue #19529: [SPARK-22308] Support alternative unit testing styles in...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19529 **[Test build #83006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83006/testReport)** for PR 19529 at commit

[GitHub] spark issue #19557: [SPARK-22281][SPARKR][WIP] Handle R method breaking sign...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19557 **[Test build #83005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83005/testReport)** for PR 19557 at commit

[GitHub] spark issue #19557: [SPARK-22281][SPARKR][WIP] Handle R method breaking sign...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19557 **[Test build #83004 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83004/testReport)** for PR 19557 at commit

[GitHub] spark pull request #19560: [SPARK-22334][SQL] Check table size from filesyst...

2017-10-23 Thread jinxing64
Github user jinxing64 commented on a diff in the pull request: https://github.com/apache/spark/pull/19560#discussion_r146449741 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -120,22 +120,41 @@ class DetermineTableStats(session:

[GitHub] spark issue #19557: [SPARK-22281][SPARKR][WIP] Handle R method breaking sign...

2017-10-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19557 unfortunately `attach` part didn't work on r-devel. I think you are right, I'm going try to apply the similar approach for `attach` - i started that way before I found out method signature

[GitHub] spark pull request #19560: [SPARK-22334][SQL] Check table size from filesyst...

2017-10-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19560#discussion_r146448976 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -120,22 +120,41 @@ class DetermineTableStats(session: SparkSession)

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18747 **[Test build #83003 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83003/testReport)** for PR 18747 at commit

[GitHub] spark pull request #19560: [SPARK-22334][SQL] Check table size from filesyst...

2017-10-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19560#discussion_r146448519 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -187,6 +187,15 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #17589: [SPARK-16544][SQL] Support for conversion from nu...

2017-10-23 Thread HyukjinKwon
Github user HyukjinKwon closed the pull request at: https://github.com/apache/spark/pull/17589 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #17589: [SPARK-16544][SQL] Support for conversion from numeric c...

2017-10-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17589 Closing this. Will take another look and make a cleaner fix next time, or reopen if I see some more interests in this. ---

[GitHub] spark pull request #19535: [SPARK-22313][PYTHON] Mark/print deprecation warn...

2017-10-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19535 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19535: [SPARK-22313][PYTHON] Mark/print deprecation warnings as...

2017-10-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19535 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19535: [SPARK-22313][PYTHON] Mark/print deprecation warnings as...

2017-10-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19535 Thanks @srowen, @rxin and @felixcheung. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19556: [SPARK-22328][Core] ClosureCleaner should not miss refer...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19556 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82999/ Test PASSed. ---

[GitHub] spark issue #19556: [SPARK-22328][Core] ClosureCleaner should not miss refer...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19556 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19556: [SPARK-22328][Core] ClosureCleaner should not miss refer...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19556 **[Test build #82999 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82999/testReport)** for PR 19556 at commit

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19459 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19459 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83001/ Test PASSed. ---

[GitHub] spark issue #19560: [SPARK-22334][SQL] Check table size from filesystem in c...

2017-10-23 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/19560 @gatorsmile @dongjoon-hyun Thanks a lot for looking into this. This pr aims to avoid OOM if metastore fails to update table properties after the data is already produced. With the

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19459 **[Test build #83001 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83001/testReport)** for PR 19459 at commit

[GitHub] spark issue #10466: [SPARK-12375] [ML] add handleinvalid for vectorindexer

2017-10-23 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/10466 @hhbyyh Do you get time to continue this PR ? thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #19560: [SPARK-22334][SQL] Check table size from HDFS in case th...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19560 **[Test build #83002 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83002/testReport)** for PR 19560 at commit

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146439253 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146439099 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark issue #19559: [SPARK-22333][SQL]ColumnReference should get higher prio...

2017-10-23 Thread DonnyZone
Github user DonnyZone commented on the issue: https://github.com/apache/spark/pull/19559 @hvanhovell Yes! I made something wrong. The `timeFunctionCall` has conflicts with `columnReference`. This fix will break every use of CURRENT_DATE/CURRENT_TIMESTAMP. For

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146437952 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146437447 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r146435148 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala --- @@ -159,6 +159,12 @@ class

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r146436646 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/security/AMCredentialRenewer.scala --- @@ -59,8 +59,7 @@ private[yarn] class

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r146434999 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterManager.scala --- @@ -17,7 +17,7 @@

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r146436571 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCredentialRenewer.scala --- @@ -0,0 +1,169 @@ +/* +

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r146436883 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCredentialRenewer.scala --- @@ -0,0 +1,169 @@ +/* +

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r146437027 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCredentialRenewer.scala --- @@ -0,0 +1,169 @@ +/* +

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r146436327 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -213,6 +216,24 @@

[GitHub] spark pull request #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark ...

2017-10-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19459#discussion_r146436616 --- Diff: python/pyspark/sql/session.py --- @@ -414,6 +415,52 @@ def _createFromLocal(self, data, schema): data = [schema.toInternal(row)

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83000/ Test PASSed. ---

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19527 **[Test build #83000 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83000/testReport)** for PR 19527 at commit

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19529: [SPARK-22308] Support alternative unit testing styles in...

2017-10-23 Thread nkronenfeld
Github user nkronenfeld commented on the issue: https://github.com/apache/spark/pull/19529 @gatorsmile sounds good, giving that a try now... assuming tests pass, I'll check it in and see if it's any better. I've so far done this for PlanTest and SQLTestUtils PlanTest I

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r146434552 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -194,6 +198,27 @@

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146434438 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark issue #19562: [SPARK-21912][SQL][FOLLOW-UP] ORC/Parquet table should n...

2017-10-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19562 Thank you! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-10-23 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19459 Merged in PR from @ueshin and added case for when schema is a string single datatype. In addition using a `StructType`, now this handles specifying the schema with the following: ```

[GitHub] spark pull request #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark ...

2017-10-23 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19459#discussion_r146431078 --- Diff: python/pyspark/sql/session.py --- @@ -414,6 +415,73 @@ def _createFromLocal(self, data, schema): data = [schema.toInternal(row)

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19459 **[Test build #83001 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83001/testReport)** for PR 19459 at commit

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146430483 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark issue #18125: [SPARK-20891][SQL] Reduce duplicate code typedaggregator...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18125 **[Test build #82998 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82998/testReport)** for PR 18125 at commit

[GitHub] spark issue #18125: [SPARK-20891][SQL] Reduce duplicate code typedaggregator...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18125 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82998/ Test FAILed. ---

[GitHub] spark issue #18125: [SPARK-20891][SQL] Reduce duplicate code typedaggregator...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18125 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146429823 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19527 **[Test build #83000 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83000/testReport)** for PR 19527 at commit

[GitHub] spark issue #19556: [SPARK-22328][Core] ClosureCleaner should not miss refer...

2017-10-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19556 cc @cloud-fan for review too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146429113 --- Diff: pom.xml --- @@ -2649,6 +2649,13 @@ + kubernetes + +resource-managers/kubernetes/core ---

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146428556 --- Diff: pom.xml --- @@ -2649,6 +2649,13 @@ + kubernetes + +resource-managers/kubernetes/core

[GitHub] spark pull request #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator f...

2017-10-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19527#discussion_r146428519 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/OneHotEncoderEstimator.scala --- @@ -0,0 +1,464 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19556: [SPARK-22328][Core] ClosureCleaner should not miss refer...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19556 **[Test build #82999 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82999/testReport)** for PR 19556 at commit

[GitHub] spark pull request #19562: [SPARK-21912][SQL][FOLLOW-UP] ORC/Parquet table s...

2017-10-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19562 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146427690 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark issue #18125: [SPARK-20891][SQL] Reduce duplicate code typedaggregator...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18125 **[Test build #82998 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82998/testReport)** for PR 18125 at commit

[GitHub] spark issue #19562: [SPARK-21912][SQL][FOLLOW-UP] ORC/Parquet table should n...

2017-10-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19562 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18125: [SPARK-20891][SQL] Reduce duplicate code typedaggregator...

2017-10-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18125 @setjet Could you address the conflicts? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18125: [SPARK-20891][SQL] Reduce duplicate code typedaggregator...

2017-10-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18125 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146426881 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146426810 --- Diff: pom.xml --- @@ -2649,6 +2649,13 @@ + kubernetes + +resource-managers/kubernetes/core

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL] Add Date and Timestamp suppo...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18664 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL] Add Date and Timestamp suppo...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18664 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82997/ Test FAILed. ---

[GitHub] spark issue #18664: [SPARK-21375][PYSPARK][SQL] Add Date and Timestamp suppo...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18664 **[Test build #82997 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82997/testReport)** for PR 18664 at commit

[GitHub] spark issue #18477: [SPARK-21261][DOCS]SQL Regex document fix

2017-10-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18477 @visaxin Could you address the comment? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146426271 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146426317 --- Diff: pom.xml --- @@ -2649,6 +2649,13 @@ + kubernetes + +resource-managers/kubernetes/core

[GitHub] spark issue #18527: [SPARK-21101][SQL] Catch IllegalStateException when CREA...

2017-10-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18527 ping @wangyum This sounds a reasonable fix. Could you resolve the conflicts? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146426033 --- Diff: pom.xml --- @@ -2649,6 +2649,13 @@ + kubernetes + +resource-managers/kubernetes/core

[GitHub] spark pull request #19556: [SPARK-22328][Core] ClosureCleaner should not mis...

2017-10-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19556#discussion_r146426012 --- Diff: core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala --- @@ -91,6 +91,52 @@ private[spark] object ClosureCleaner extends Logging {

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146425629 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146425503 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146425133 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146425100 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/SparkKubernetesClientFactory.scala --- @@ -0,0 +1,103 @@ +/*

[GitHub] spark issue #19562: [SPARK-21912][SQL][FOLLOW-UP] ORC/Parquet table should n...

2017-10-23 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19562 Thank you for review and approval, @gatorsmile ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #17100: [SPARK-13947][SQL] The error message from using an inval...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17100 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82994/ Test PASSed. ---

[GitHub] spark issue #17100: [SPARK-13947][SQL] The error message from using an inval...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17100 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17100: [SPARK-13947][SQL] The error message from using an inval...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17100 **[Test build #82994 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82994/testReport)** for PR 17100 at commit

[GitHub] spark issue #18268: [SPARK-21054] [SQL] Reset Command support reset specific...

2017-10-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18268 ping @ericsahit --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18833: [SPARK-21625][SQL] sqrt(negative number) should be null.

2017-10-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18833 Can we document this difference in https://spark.apache.org/docs/latest/sql-programming-guide.html#compatibility-with-apache-hive? ---

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146423831 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread mccheah
Github user mccheah commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146423757 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146423117 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark issue #19562: [SPARK-21912][SQL][FOLLOW-UP] ORC/Parquet table should n...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19562 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19562: [SPARK-21912][SQL][FOLLOW-UP] ORC/Parquet table should n...

2017-10-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19562 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82996/ Test PASSed. ---

[GitHub] spark issue #19562: [SPARK-21912][SQL][FOLLOW-UP] ORC/Parquet table should n...

2017-10-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19562 **[Test build #82996 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82996/testReport)** for PR 19562 at commit

[GitHub] spark pull request #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark ...

2017-10-23 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/19459#discussion_r146421804 --- Diff: python/pyspark/sql/session.py --- @@ -414,6 +415,73 @@ def _createFromLocal(self, data, schema): data = [schema.toInternal(row)

  1   2   3   4   >