[GitHub] spark issue #18926: [SPARK-21712] [PySpark] Clarify type error for Column.su...

2017-08-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18926 The current codes around what this PR changes look not quite clean to me too and we should clean around this. But I think this PR itself is quite well-formed with the fix that is valid,

[GitHub] spark issue #18640: [SPARK-21422][BUILD] Depend on Apache ORC 1.4.0

2017-08-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18640 Thank you so much, @rxin , @cloud-fan , @sameeragarwal , @mridulm , @viirya ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18956 **[Test build #80718 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80718/testReport)** for PR 18956 at commit

[GitHub] spark issue #18926: [SPARK-21712] [PySpark] Clarify type error for Column.su...

2017-08-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18926 To be honest, the current codes do not look good to me. Since this does not make the code worse, I will not revert it back. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-08-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r133360995 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -37,6 +37,12 @@ import org.apache.spark.sql.types._

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-15 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r133360674 --- Diff: mllib/src/test/scala/org/apache/spark/ml/evaluation/ClusteringEvaluatorSuite.scala --- @@ -0,0 +1,225 @@ +/* + * Licensed to the

[GitHub] spark issue #18640: [SPARK-21422][BUILD] Depend on Apache ORC 1.4.0

2017-08-15 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/18640 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #18957: [SPARK-21744][CORE] Add retry logic for new broadcast in...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18957 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18538: [SPARK-14516][ML] Adding ClusteringEvaluator with...

2017-08-15 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/18538#discussion_r133360284 --- Diff: mllib/src/main/scala/org/apache/spark/ml/evaluation/ClusteringEvaluator.scala --- @@ -0,0 +1,240 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #18957: [SPARK-21744][CORE] Add retry logic for new broad...

2017-08-15 Thread caneGuy
GitHub user caneGuy opened a pull request: https://github.com/apache/spark/pull/18957 [SPARK-21744][CORE] Add retry logic for new broadcast in BroadcastManager ## What changes were proposed in this pull request? When driver submit new stage and there is a bad disk before

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18956 **[Test build #80717 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80717/testReport)** for PR 18956 at commit

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-08-15 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/18956#discussion_r133360047 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -37,6 +37,12 @@ import org.apache.spark.sql.types._

[GitHub] spark issue #18956: [SPARK-21726][SQL] Check for structural integrity of the...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18956 **[Test build #80715 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80715/testReport)** for PR 18956 at commit

[GitHub] spark issue #18855: [SPARK-3151] [Block Manager] DiskStore.getBytes fails fo...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18855 **[Test build #80716 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80716/testReport)** for PR 18855 at commit

[GitHub] spark pull request #18955: [SPARK-21743][SQL] top-most limit should not caus...

2017-08-15 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18955#discussion_r133359698 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2658,4 +2658,9 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark pull request #18956: [SPARK-21726][SQL] Check for structural integrity...

2017-08-15 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/18956 [SPARK-21726][SQL] Check for structural integrity of the plan in Optimzer in test mode. ## What changes were proposed in this pull request? We have many optimization rules now in

[GitHub] spark issue #18492: [SPARK-19326] Speculated task attempts do not get launch...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18492 **[Test build #80714 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80714/testReport)** for PR 18492 at commit

[GitHub] spark issue #18955: [SPARK-21743][SQL] top-most limit should not cause memor...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18955 **[Test build #80713 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80713/testReport)** for PR 18955 at commit

[GitHub] spark issue #18955: [SPARK-21743][SQL] top-most limit should not cause memor...

2017-08-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18955 cc @gengliangwang @sameeragarwal @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #18955: [SPARK-21743][SQL] top-most limit should not caus...

2017-08-15 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/18955 [SPARK-21743][SQL] top-most limit should not cause memory leak ## What changes were proposed in this pull request? For top-most limit, we will use a special operator to execute it:

[GitHub] spark issue #18492: [SPARK-19326] Speculated task attempts do not get launch...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18492 **[Test build #80712 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80712/testReport)** for PR 18492 at commit

[GitHub] spark issue #18954: [SPARK-17654] [SQL] Enable creating hive bucketed tables

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18954 **[Test build #80711 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80711/testReport)** for PR 18954 at commit

[GitHub] spark issue #18954: [SPARK-17654] [SQL] Enable creating hive bucketed tables

2017-08-15 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/18954 Jenkins test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #18954: [SPARK-17654] [SQL] Enable creating hive bucketed...

2017-08-15 Thread tejasapatil
GitHub user tejasapatil opened a pull request: https://github.com/apache/spark/pull/18954 [SPARK-17654] [SQL] Enable creating hive bucketed tables ## What changes were proposed in this pull request? ### Semantics: - If the Hive table is bucketed, then INSERT node expect

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18953 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18953 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80707/ Test FAILed. ---

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #80707 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80707/testReport)** for PR 18953 at commit

[GitHub] spark pull request #18492: [SPARK-19326] Speculated task attempts do not get...

2017-08-15 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18492#discussion_r133355548 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -291,6 +297,16 @@ private[spark] trait SparkListenerInterface {

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #80710 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80710/testReport)** for PR 18953 at commit

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18896 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80708/ Test PASSed. ---

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18896 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18896 **[Test build #80708 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80708/testReport)** for PR 18896 at commit

[GitHub] spark issue #18951: [SPARK-21738] Thriftserver doesn't cancel jobs when sess...

2017-08-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18951 LGTM cc @cloud-fan @jiangxb1987 @wangyum @debugger87 @jerryshao --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #18640: [SPARK-21422][BUILD] Depend on Apache ORC 1.4.0

2017-08-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18640 Hi, @cloud-fan , @rxin , @sameeragarwal and @mridulm . Could you merge this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18810 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18949: [SPARK-12961][CORE][FOLLOW-UP] Remove wrapper code for S...

2017-08-15 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/18949 @viirya aha, ok. thanks. (btw, since the comment is still important, we better keep it in code comment, maybe). --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #16763: [SPARK-19422][ML][WIP] Cache input data in algorithms

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16763 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16763: [SPARK-19422][ML][WIP] Cache input data in algorithms

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16763 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80709/ Test FAILed. ---

[GitHub] spark issue #16763: [SPARK-19422][ML][WIP] Cache input data in algorithms

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16763 **[Test build #80709 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80709/testReport)** for PR 16763 at commit

[GitHub] spark issue #18949: [SPARK-12961][CORE][FOLLOW-UP] Remove wrapper code for S...

2017-08-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18949 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #18949: [SPARK-12961][CORE][FOLLOW-UP] Remove wrapper code for S...

2017-08-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18949 @maropu There is another reason we leave the workaround in place: https://github.com/apache/spark/pull/11524#issuecomment-192409933 --- If your project is set up for it, you can reply to this email

[GitHub] spark issue #18902: [SPARK-21690][ML] one-pass imputer

2017-08-15 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/18902 @hhbyyh I rewrite the impl, and now all `NaN` and `missingValue` will be transform to `null` at first, then current methods are used. For columns only containing `null`, `null` is

[GitHub] spark issue #16763: [SPARK-19422][ML][WIP] Cache input data in algorithms

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16763 **[Test build #80709 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80709/testReport)** for PR 16763 at commit

[GitHub] spark issue #16763: [SPARK-19422][ML][WIP] Cache input data in algorithms

2017-08-15 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/16763 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #18798: [SPARK-19634][ML] Multivariate summarizer - dataf...

2017-08-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18798 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18798: [SPARK-19634][ML] Multivariate summarizer - dataframes A...

2017-08-15 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/18798 Merged into master, thanks for all. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18896 **[Test build #80708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80708/testReport)** for PR 18896 at commit

[GitHub] spark pull request #18930: [SPARK-21677][SQL] json_tuple throws NullPointExc...

2017-08-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18930#discussion_r133347400 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -2034,4 +2034,25 @@ class JsonSuite extends

[GitHub] spark issue #18926: [SPARK-21712] [PySpark] Clarify type error for Column.su...

2017-08-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18926 Merged to master. Please open JIRAs / PRs related with the discussion above if anyone is willing to proceed. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #18926: [SPARK-21712] [PySpark] Clarify type error for Co...

2017-08-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18926 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18926: [SPARK-21712] [PySpark] Clarify type error for Column.su...

2017-08-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18926 I am merging this as it looks there is an explicit objection for the current change itself and it looks the issue is fixed by this. To summarize the discussion here: -

[GitHub] spark pull request #18950: [SPARK-20589][Core][Scheduler] Allow limiting tas...

2017-08-15 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/18950#discussion_r133344532 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -602,6 +604,21 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18810 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80703/ Test PASSed. ---

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80703 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80703/testReport)** for PR 18810 at commit

[GitHub] spark issue #18953: [SPARK-20682][SQL] Implement new ORC data source based o...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18953 **[Test build #80707 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80707/testReport)** for PR 18953 at commit

[GitHub] spark pull request #18953: [SPARK-20682][SQL] Implement new ORC data source ...

2017-08-15 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/18953 [SPARK-20682][SQL] Implement new ORC data source based on Apache ORC ## What changes were proposed in this pull request? Since #17924, #17943, and #17980 are a little large PRs, this

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12646 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80706/ Test FAILed. ---

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12646 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12646 **[Test build #80706 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80706/testReport)** for PR 12646 at commit

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18887 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80701/ Test PASSed. ---

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18887 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18887 **[Test build #80701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80701/testReport)** for PR 18887 at commit

[GitHub] spark issue #18896: [SPARK-21681][ML] fix bug of MLOR do not work correctly ...

2017-08-15 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/18896 LGTM except for making the test's title more descriptive. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #18488: [SPARK-21255][SQL][WIP] Fixed NPE when creating encoder ...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18488 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80700/ Test PASSed. ---

[GitHub] spark issue #18488: [SPARK-21255][SQL][WIP] Fixed NPE when creating encoder ...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18488 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18488: [SPARK-21255][SQL][WIP] Fixed NPE when creating encoder ...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18488 **[Test build #80700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80700/testReport)** for PR 18488 at commit

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-08-15 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/15770 info] Main Scala API documentation successful. [error] (spark/javaunidoc:doc) javadoc returned nonzero exit code [error] Total time: 95 s, completed Aug 15, 2017 4:59:59 PM [error]

[GitHub] spark pull request #18923: [SPARK-21710][StSt] Fix OOM on ConsoleSink with l...

2017-08-15 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/18923#discussion_r15831 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/console.scala --- @@ -49,7 +49,7 @@ class ConsoleSink(options: Map[String,

[GitHub] spark issue #18951: [SPARK-21738] Thriftserver doesn't cancel jobs when sess...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18951 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80705/ Test PASSed. ---

[GitHub] spark issue #18951: [SPARK-21738] Thriftserver doesn't cancel jobs when sess...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18951 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18951: [SPARK-21738] Thriftserver doesn't cancel jobs when sess...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18951 **[Test build #80705 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80705/testReport)** for PR 18951 at commit

[GitHub] spark issue #18926: [SPARK-21712] [PySpark] Clarify type error for Column.su...

2017-08-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18926 I don't think this suggestion / discussion blocks this PR for few days. Let's go as is and make a followup as another improvement if anyone feels so. I will review that at my best. --- If

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2017-08-15 Thread justinuang
Github user justinuang commented on the issue: https://github.com/apache/spark/pull/15009 That would be incredible. Launching a new jvm and loading all of hadoop takes about 4 seconds extra each time, versus reusing the launcher jvm, which is really significant for us since we launch

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12646 **[Test build #80706 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80706/testReport)** for PR 12646 at commit

[GitHub] spark issue #18951: [SPARK-21738] Thriftserver doesn't cancel jobs when sess...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18951 **[Test build #80705 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80705/testReport)** for PR 18951 at commit

[GitHub] spark issue #18951: [SPARK-21738] Thriftserver doesn't cancel jobs when sess...

2017-08-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18951 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15770 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15770 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80704/ Test FAILed. ---

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15770 **[Test build #80704 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80704/testReport)** for PR 15770 at commit

[GitHub] spark issue #12646: [SPARK-14878][SQL] Trim characters string function suppo...

2017-08-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/12646 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15770 **[Test build #80704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80704/testReport)** for PR 15770 at commit

[GitHub] spark pull request #18952: [MINOR] Fix a typo in the method name `UserDefine...

2017-08-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18952 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18952: [MINOR] Fix a typo in the method name `UserDefinedFuncti...

2017-08-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18952 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #18907: [SPARK-18464][SQL][followup] support old table which doe...

2017-08-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18907 I'll send new PRs for 2.2 and 2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #18947: [SPARK-21721][SQL][Backport-2.1] Clear FileSystem delete...

2017-08-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18947 Thanks @gatorsmile @dongjoon-hyun --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #18947: [SPARK-21721][SQL][Backport-2.1] Clear FileSystem...

2017-08-15 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/18947 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18810 **[Test build #80703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80703/testReport)** for PR 18810 at commit

[GitHub] spark issue #18810: [SPARK-21603][SQL]The wholestage codegen will be much sl...

2017-08-15 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18810 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #17373: [SPARK-12664][ML] Expose probability in mlp model

2017-08-15 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/17373#discussion_r133324363 --- Diff: mllib/src/main/scala/org/apache/spark/ml/ann/Layer.scala --- @@ -527,9 +550,21 @@ private[ml] class FeedForwardModel private(

[GitHub] spark pull request #17373: [SPARK-12664][ML] Expose probability in mlp model

2017-08-15 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/17373#discussion_r133322927 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/MultilayerPerceptronClassifierSuite.scala --- @@ -82,6 +83,49 @@ class

[GitHub] spark pull request #17373: [SPARK-12664][ML] Expose probability in mlp model

2017-08-15 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/17373#discussion_r133323889 --- Diff: mllib/src/main/scala/org/apache/spark/ml/ann/Layer.scala --- @@ -374,6 +380,22 @@ private[ann] trait TopologyModel extends Serializable {

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15770 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15770 **[Test build #80702 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80702/testReport)** for PR 15770 at commit

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-08-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15770 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80702/ Test FAILed. ---

[GitHub] spark pull request #18950: [SPARK-20589][Core][Scheduler] Allow limiting tas...

2017-08-15 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/18950#discussion_r133321125 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -602,6 +604,21 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #18887: [SPARK-20642][core] Store FsHistoryProvider listing data...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18887 **[Test build #80701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80701/testReport)** for PR 18887 at commit

[GitHub] spark issue #15770: [SPARK-15784][ML]:Add Power Iteration Clustering to spar...

2017-08-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15770 **[Test build #80702 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/80702/testReport)** for PR 15770 at commit

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-08-15 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r133319377 --- Diff: docs/monitoring.md --- @@ -220,6 +220,13 @@ The history server can be configured as follows: Number of threads that will be used by

[GitHub] spark pull request #18887: [SPARK-20642][core] Store FsHistoryProvider listi...

2017-08-15 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/18887#discussion_r133318822 --- Diff: scalastyle-config.xml --- @@ -86,7 +86,7 @@ This file is divided into 3 sections: - + --- End diff --

  1   2   3   4   5   >