[GitHub] spark issue #19272: [Spark-21842][Mesos] Support Kerberos ticket renewal and...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19272 **[Test build #82941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82941/testReport)** for PR 19272 at commit

[GitHub] spark pull request #18664: [SPARK-21375][PYSPARK][SQL] Add Date and Timestam...

2017-10-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18664#discussion_r146096087 --- Diff: python/pyspark/sql/types.py --- @@ -1619,11 +1619,39 @@ def to_arrow_type(dt): arrow_type = pa.decimal(dt.precision, dt.scale)

[GitHub] spark issue #19545: [SPARK-21929][SQL] Support `ALTER TABLE table_name ADD C...

2017-10-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19545 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19545: [SPARK-21929][SQL] Support `ALTER TABLE table_nam...

2017-10-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19545#discussion_r146096042 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -2202,56 +2202,64 @@ abstract class DDLSuite extends

[GitHub] spark pull request #19545: [SPARK-21929][SQL] Support `ALTER TABLE table_nam...

2017-10-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19545#discussion_r146096038 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala --- @@ -2202,56 +2202,64 @@ abstract class DDLSuite extends

[GitHub] spark pull request #19539: [SPARK-22326] [SQL] Remove unnecessary hashCode a...

2017-10-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19539 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19539: [SPARK-22326] [SQL] Remove unnecessary hashCode and equa...

2017-10-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19539 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19539: [SPARK-22326] [SQL] Remove unnecessary hashCode and equa...

2017-10-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19539 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19528: [SPARK-20393][WEBU UI][1.6] Strengthen Spark to p...

2017-10-20 Thread ambauma
Github user ambauma commented on a diff in the pull request: https://github.com/apache/spark/pull/19528#discussion_r146095022 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/deploy/mesos/ui/DriverPage.scala --- @@ -0,0 +1,180 @@ +/* --- End diff --

[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-10-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19514 Thanks, I have been following it @shivaram and @felixcheung. Separate JIRA sounds good to me and I am okay witn merging it. ---

[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-10-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19514 LGTM too BTW. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-10-20 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r146094019 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -194,6 +198,27

[GitHub] spark issue #19539: [SPARK-22326] [SQL] Remove unnecessary hashCode and equa...

2017-10-20 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/19539 @gatorsmile JIRA created. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19468 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82940/ Test PASSed. ---

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19468 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19468 **[Test build #82940 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82940/testReport)** for PR 19468 at commit

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19468 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19468 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82938/ Test PASSed. ---

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19468 **[Test build #82938 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82938/testReport)** for PR 19468 at commit

[GitHub] spark issue #19545: [SPARK-21929][SQL] Support `ALTER TABLE table_name ADD C...

2017-10-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19545 Hi, @gatorsmile . Could you review this PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #19545: [SPARK-21929][SQL] Support `ALTER TABLE table_nam...

2017-10-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19545#discussion_r146090102 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -235,11 +235,10 @@ case class

[GitHub] spark issue #19545: [SPARK-21929][SQL] Support `ALTER TABLE table_name ADD C...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19545 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82939/ Test PASSed. ---

[GitHub] spark issue #19545: [SPARK-21929][SQL] Support `ALTER TABLE table_name ADD C...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19545 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19545: [SPARK-21929][SQL] Support `ALTER TABLE table_name ADD C...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19545 **[Test build #82939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82939/testReport)** for PR 19545 at commit

[GitHub] spark issue #19534: [SPARK-22312][CORE] Fix bug in Executor allocation manag...

2017-10-20 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/19534 I think other PR is fixing one more issue on top of runningTasks being negative, so we can proceed with the other one. What do you think @jerryshao ? ---

[GitHub] spark pull request #19528: [SPARK-20393][WEBU UI][1.6] Strengthen Spark to p...

2017-10-20 Thread ambauma
Github user ambauma commented on a diff in the pull request: https://github.com/apache/spark/pull/19528#discussion_r146084730 --- Diff: python/pyspark/mllib/classification.py --- @@ -173,7 +173,7 @@ def __init__(self, weights, intercept, numFeatures, numClasses):

[GitHub] spark pull request #19528: [SPARK-20393][WEBU UI][1.6] Strengthen Spark to p...

2017-10-20 Thread ambauma
Github user ambauma commented on a diff in the pull request: https://github.com/apache/spark/pull/19528#discussion_r146084021 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobsTab.scala --- @@ -16,9 +16,9 @@ */ package org.apache.spark.ui.jobs -

[GitHub] spark pull request #19528: [SPARK-20393][WEBU UI][1.6] Strengthen Spark to p...

2017-10-20 Thread ambauma
Github user ambauma commented on a diff in the pull request: https://github.com/apache/spark/pull/19528#discussion_r146080377 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/deploy/mesos/ui/DriverPage.scala --- @@ -0,0 +1,180 @@ +/* --- End diff --

[GitHub] spark pull request #19528: [SPARK-20393][WEBU UI][1.6] Strengthen Spark to p...

2017-10-20 Thread ambauma
Github user ambauma commented on a diff in the pull request: https://github.com/apache/spark/pull/19528#discussion_r146080311 --- Diff: python/pyspark/mllib/classification.py --- @@ -173,7 +173,7 @@ def __init__(self, weights, intercept, numFeatures, numClasses):

[GitHub] spark pull request #19528: [SPARK-20393][WEBU UI][1.6] Strengthen Spark to p...

2017-10-20 Thread ambauma
Github user ambauma commented on a diff in the pull request: https://github.com/apache/spark/pull/19528#discussion_r146080177 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/JobsTab.scala --- @@ -16,9 +16,9 @@ */ package org.apache.spark.ui.jobs -

[GitHub] spark pull request #19528: [SPARK-20393][WEBU UI][1.6] Strengthen Spark to p...

2017-10-20 Thread ambauma
Github user ambauma commented on a diff in the pull request: https://github.com/apache/spark/pull/19528#discussion_r146080089 --- Diff: python/pyspark/mllib/classification.py --- @@ -173,7 +173,7 @@ def __init__(self, weights, intercept, numFeatures, numClasses):

[GitHub] spark issue #19538: [SPARK-20393][WEBU UI][BACKPORT-2.0] Strengthen Spark to...

2017-10-20 Thread ambauma
Github user ambauma commented on the issue: https://github.com/apache/spark/pull/19538 I'm not looking for an official release. My goal is to get the fix into the official branch 1.6 to reduce the number of forks necessary and so that if CVE-2018- comes and I've moved on my

[GitHub] spark issue #19543: [SPARK-19606][MESOS] Support constraints in spark-dispat...

2017-10-20 Thread pmackles
Github user pmackles commented on the issue: https://github.com/apache/spark/pull/19543 @felixcheung - fixed scala-style issues and also updated the docs to include the new property --- - To unsubscribe, e-mail:

[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-10-20 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19514 I think we can safely merge this change as it clearly passes any tests whose functionality it would affect. We could defer further discussion about what to do about CRAN versions elsewhere, yes.

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-20 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146074603 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-10-20 Thread foxish
Github user foxish commented on the issue: https://github.com/apache/spark/pull/19468 @vanzin, you were right, the YARN constants were left overs and made no sense wrt k8s. We discussed it in our weekly meeting - it was simply dead code. I've addressed most of the style comments and

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19468 **[Test build #82940 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82940/testReport)** for PR 19468 at commit

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-20 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146072523 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-20 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146072553 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-20 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146072501 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark pull request #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - B...

2017-10-20 Thread foxish
Github user foxish commented on a diff in the pull request: https://github.com/apache/spark/pull/19468#discussion_r146072126 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -0,0

[GitHub] spark issue #19545: [SPARK-21929][SQL] Support `ALTER TABLE table_name ADD C...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19545 **[Test build #82939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82939/testReport)** for PR 19545 at commit

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19468 **[Test build #82938 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82938/testReport)** for PR 19468 at commit

[GitHub] spark pull request #19545: [SPARK-21929][SQL] Support `ALTER TABLE table_nam...

2017-10-20 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/19545 [SPARK-21929][SQL] Support `ALTER TABLE table_name ADD COLUMNS(..)` for ORC data source ## What changes were proposed in this pull request? When SPARK-19261 implements `ALTER TABLE

[GitHub] spark pull request #18664: [SPARK-21375][PYSPARK][SQL] Add Date and Timestam...

2017-10-20 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18664#discussion_r146066999 --- Diff: python/pyspark/sql/types.py --- @@ -1619,11 +1619,39 @@ def to_arrow_type(dt): arrow_type = pa.decimal(dt.precision, dt.scale)

[GitHub] spark pull request #18664: [SPARK-21375][PYSPARK][SQL] Add Date and Timestam...

2017-10-20 Thread wesm
Github user wesm commented on a diff in the pull request: https://github.com/apache/spark/pull/18664#discussion_r146062157 --- Diff: python/pyspark/sql/types.py --- @@ -1619,11 +1619,39 @@ def to_arrow_type(dt): arrow_type = pa.decimal(dt.precision, dt.scale)

[GitHub] spark pull request #18664: [SPARK-21375][PYSPARK][SQL] Add Date and Timestam...

2017-10-20 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18664#discussion_r146058020 --- Diff: python/pyspark/sql/types.py --- @@ -1619,11 +1619,39 @@ def to_arrow_type(dt): arrow_type = pa.decimal(dt.precision, dt.scale)

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-10-20 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r146052502 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -194,6 +198,27 @@

[GitHub] spark pull request #19517: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby()....

2017-10-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19517 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19517: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19517 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19517: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19517 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82936/ Test PASSed. ---

[GitHub] spark issue #19517: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19517 **[Test build #82936 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82936/testReport)** for PR 19517 at commit

[GitHub] spark pull request #19541: ABCD

2017-10-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19541 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19542: Branch 1.1.

2017-10-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19542 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #18664: [SPARK-21375][PYSPARK][SQL] Add Date and Timestam...

2017-10-20 Thread wesm
Github user wesm commented on a diff in the pull request: https://github.com/apache/spark/pull/18664#discussion_r146044463 --- Diff: python/pyspark/sql/types.py --- @@ -1619,11 +1619,39 @@ def to_arrow_type(dt): arrow_type = pa.decimal(dt.precision, dt.scale)

[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-10-20 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/19514 Good point. I'm not sure it counteracts it completely. We should run it to see the behavior I guess. I am not a big fan of mucking with Jenkins versions because it fundamentally looks

[GitHub] spark issue #19538: [SPARK-20393][WEBU UI][2.0] Strengthen Spark to prevent ...

2017-10-20 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19538 link to 1.6 PR #19528 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19535: [SPARK-22313][PYTHON] Mark/print deprecation warn...

2017-10-20 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19535#discussion_r146024871 --- Diff: python/pyspark/streaming/kafka.py --- @@ -58,6 +60,7 @@ def createStream(ssc, zkQuorum, groupId, topics, kafkaParams=None,

[GitHub] spark pull request #19535: [SPARK-22313][PYTHON] Mark/print deprecation warn...

2017-10-20 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/19535#discussion_r146024844 --- Diff: python/pyspark/streaming/flume.py --- @@ -56,6 +56,7 @@ def createStream(ssc, hostname, port, .. note:: Deprecated in 2.3.0

[GitHub] spark issue #19538: [SPARK-20393][WEBU UI][2.0] Strengthen Spark to prevent ...

2017-10-20 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19538 could you update the PR title to say `[BACKPORT-2.0]` instead of `[2.0]`. also please type to PR # for the earlier commit to link them here. you mention there is a discussion, could you

[GitHub] spark issue #19543: [SPARK-19606][MESOS] Support constraints in spark-dispat...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19543 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19543: [SPARK-19606][MESOS] Support constraints in spark-dispat...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19543 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82937/ Test FAILed. ---

[GitHub] spark issue #19543: [SPARK-19606][MESOS] Support constraints in spark-dispat...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19543 **[Test build #82937 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82937/testReport)** for PR 19543 at commit

[GitHub] spark issue #19538: [SPARK-20393][WEBU UI][2.0] Strengthen Spark to prevent ...

2017-10-20 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19538 ignore SparkR test failure for now, we are looking into it. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19543: [SPARK-19606][MESOS] Support constraints in spark-dispat...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19543 **[Test build #82937 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82937/testReport)** for PR 19543 at commit

[GitHub] spark issue #19543: [SPARK-19606][MESOS] Support constraints in spark-dispat...

2017-10-20 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19543 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19267: [WIP][SPARK-20628][CORE] Blacklist nodes when they trans...

2017-10-20 Thread juanrh
Github user juanrh commented on the issue: https://github.com/apache/spark/pull/19267 Hi @vanzin and @tgravescs, do you have any other comments on this proposal? Thanks, Juan --- - To

[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-10-20 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19514 I haven't tried it, but it might sound like it will counter-act -as-cran check sets completely? ``` R_CHECK_CRAN_INCOMING_ Check whether package is suitable for publication on CRAN.

[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-10-20 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19514 whoa. `_R_CHECK_CRAN_INCOMING_= false` sounds like the right approach. I'm a bit concerned with blindly letting through one more warning though, perhaps grep for the specific warning text

[GitHub] spark issue #19485: [SPARK-20055] [Docs] Added documentation for loading csv...

2017-10-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19485 Thanks for explanation. I guess there would be a big doc change soon? Will check those changes too. --- - To unsubscribe,

[GitHub] spark issue #19514: [SPARK-21551][Python] Increase timeout for PythonRDD.ser...

2017-10-20 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/19514 We didn't foresee this but it looks like `R CMD check --as-cran` throws this error if we try to build a package with a version number older than the one uploaded to CRAN. There are a

[GitHub] spark issue #19539: [MINOR] [SQL] Remove unnecessary hashCode and equals met...

2017-10-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19539 Could you open a JIRA? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18270: [SPARK-21055][SQL] replace grouping__id with grou...

2017-10-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18270 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18270: [SPARK-21055][SQL] replace grouping__id with grouping_id...

2017-10-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18270 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18270: [SPARK-21055][SQL] replace grouping__id with grouping_id...

2017-10-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18270 LGTM Let us resolve the issue as the follow-up PR. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19517: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19517 **[Test build #82936 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82936/testReport)** for PR 19517 at commit

[GitHub] spark issue #19517: [SPARK-20396][SQL][PySpark][FOLLOW-UP] groupby().apply()...

2017-10-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19517 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19544: [SPARK-22323] Design doc for pandas_udf

2017-10-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19544 @jiangxb1987 will reorg the existing Spark SQL doc. We can think about how to put this into the new version of Spark SQL doc. ---

[GitHub] spark issue #19485: [SPARK-20055] [Docs] Added documentation for loading csv...

2017-10-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19485 The reference manual and API docs are different. Below is a link of DB2 LUW: http://www-01.ibm.com/support/docview.wss?uid=swg27038855 ---

[GitHub] spark issue #19437: [SPARK-22131][MESOS] Mesos driver secrets

2017-10-20 Thread susanxhuynh
Github user susanxhuynh commented on the issue: https://github.com/apache/spark/pull/19437 @vanzin Ping, would you mind reviewing this PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19544: [SPARK-22323] Design doc for pandas_udf

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19544 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82935/ Test PASSed. ---

[GitHub] spark issue #19544: [SPARK-22323] Design doc for pandas_udf

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19544 **[Test build #82935 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82935/testReport)** for PR 19544 at commit

[GitHub] spark issue #19544: [SPARK-22323] Design doc for pandas_udf

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19544 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19515: [SPARK-22287][MESOS] SPARK_DAEMON_MEMORY not honored by ...

2017-10-20 Thread pmackles
Github user pmackles commented on the issue: https://github.com/apache/spark/pull/19515 @ArtRand - WDYT? I was going to switch it to ```SPARK_DISPATCHER_MEMORY``` but then I noticed that the other env vars for MesosClusterDispatcher or also prefixed with ```SPARK_DAEMON_*``` so I

[GitHub] spark issue #19544: [SPARK-22323] Design doc for pandas_udf

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19544 **[Test build #82935 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82935/testReport)** for PR 19544 at commit

[GitHub] spark issue #19544: [SPARK-22323] Design doc for pandas_udf

2017-10-20 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/19544 cc @cloud-fan @ueshin @HyukjinKwon @gatorsmile @viirya To continue the discussion on #19505 --- - To unsubscribe,

[GitHub] spark pull request #19544: [SPARK-22323] Design doc for pandas_udf

2017-10-20 Thread icexelloss
GitHub user icexelloss opened a pull request: https://github.com/apache/spark/pull/19544 [SPARK-22323] Design doc for pandas_udf I open this PR so we can have a place to discuss the design. We don't necessary need to merge a md file for the doc - this be embeded python

[GitHub] spark issue #19543: [SPARK-19606][MESOS] Support constraints in spark-dispat...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19543 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19543: [SPARK-19606][MESOS] Support constraints in spark...

2017-10-20 Thread pmackles
GitHub user pmackles opened a pull request: https://github.com/apache/spark/pull/19543 [SPARK-19606][MESOS] Support constraints in spark-dispatcher ## What changes were proposed in this pull request? A discussed in SPARK-19606, the addition of a new config property named

[GitHub] spark issue #19485: [SPARK-20055] [Docs] Added documentation for loading csv...

2017-10-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19485 @gatorsmile, sure, detailed doc is great and defintely I support it. Just one thing I am worried of is duplication. If we add or change option, we have to update those together and ..

[GitHub] spark issue #18974: [SPARK-21750][SQL] Use Arrow 0.6.0

2017-10-20 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18974 Hi, All. Two more Arrow releases seem to be out. How about the Python side? Can we catch up some? - 0.7.1 (1 October 2017) - 0.7.0 (17 September 2017) ---

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82934/ Test PASSed. ---

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19527 **[Test build #82934 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82934/testReport)** for PR 19527 at commit

[GitHub] spark issue #19479: [SPARK-17074] [SQL] Generate equi-height histogram in co...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19479 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19479: [SPARK-17074] [SQL] Generate equi-height histogram in co...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19479 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82931/ Test PASSed. ---

[GitHub] spark issue #19479: [SPARK-17074] [SQL] Generate equi-height histogram in co...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19479 **[Test build #82931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82931/testReport)** for PR 19479 at commit

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82933/ Test PASSed. ---

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19527 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19527: [SPARK-13030][ML] Create OneHotEncoderEstimator for OneH...

2017-10-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19527 **[Test build #82933 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82933/testReport)** for PR 19527 at commit

  1   2   3   >