[GitHub] spark pull request: [SPARK-11954][SQL] Encoder for JavaBeans

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9937#issuecomment-160546461 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11954][SQL] Encoder for JavaBeans

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9937#issuecomment-160546462 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11954][SQL] Encoder for JavaBeans

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9937#issuecomment-160546435 **[Test build #46871 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46871/consoleFull)** for PR 9937 at commit [`c634aa4`](https://git

[GitHub] spark pull request: [SPARK-11569] [ML] Fix StringIndexer to handle...

2015-11-29 Thread jliwork
Github user jliwork commented on the pull request: https://github.com/apache/spark/pull/9920#issuecomment-160546154 @jkbradley @holdenk Could you please take a look my latest fix and let me know of any further comments? thanks! --- If your project is set up for it, you can reply to t

[GitHub] spark pull request: [SPARK-11955][SQL] Mark one side fields in mer...

2015-11-29 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/9940#issuecomment-160544801 @yhuai Here is the background: Parquet filter push-down is disabled when schema merging is turned on because of [PARQUET-389][1]. I'm somewhat hesitant to have

[GitHub] spark pull request: [SPARK-11954][SQL] Encoder for JavaBeans

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9937#issuecomment-160544668 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11954][SQL] Encoder for JavaBeans

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9937#issuecomment-160544670 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [Spark-8426] [scheduler] enhance blacklist mec...

2015-11-29 Thread mwws
Github user mwws commented on the pull request: https://github.com/apache/spark/pull/8760#issuecomment-160544588 @kayousterhout For example, task can failed because some host can't access source/network, local storage of some host is full, broken class linkage and other run-time excep

[GitHub] spark pull request: [SPARK-11774][SPARKR] Implement struct(), enco...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9804#issuecomment-160544011 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11774][SPARKR] Implement struct(), enco...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9804#issuecomment-160543983 **[Test build #46869 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46869/consoleFull)** for PR 9804 at commit [`f5add6e`](https://git

[GitHub] spark pull request: [SPARK-11774][SPARKR] Implement struct(), enco...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9804#issuecomment-160544008 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11954][SQL] Encoder for JavaBeans

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9937#issuecomment-160543814 **[Test build #46871 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46871/consoleFull)** for PR 9937 at commit [`c634aa4`](https://gith

[GitHub] spark pull request: [SPARK-12043] wrong comment for "gt" in Column...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10036#issuecomment-160543802 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-11954][SQL] Encoder for JavaBeans

2015-11-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/9937#discussion_r46114322 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala --- @@ -191,6 +193,19 @@ abstract class TreeNode[BaseType <: Tree

[GitHub] spark pull request: [SPARK-12043] wrong comment for "gt" in Column...

2015-11-29 Thread gepuro
GitHub user gepuro opened a pull request: https://github.com/apache/spark/pull/10036 [SPARK-12043] wrong comment for "gt" in Column.scala comment of "gt" is wrong. ```scala def gt(other: Any): Column = this > other /** * Less than. * {{{

[GitHub] spark pull request: [SPARK-11954][SQL] Encoder for JavaBeans

2015-11-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/9937#discussion_r46114267 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala --- @@ -87,31 +96,293 @@ object JavaTypeInference {

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-11-29 Thread sun-rui
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/10030#issuecomment-160541696 @shivaram, I tried , that does not work. It seems right to put test cases in R/pkg/inst/tests/testthat, and after installation, the cases cases are available at 'R/lib/

[GitHub] spark pull request: [SPARK-11774][SPARKR] Implement struct(), enco...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9804#issuecomment-160541653 **[Test build #46869 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46869/consoleFull)** for PR 9804 at commit [`f5add6e`](https://gith

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-11-29 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/10030#issuecomment-160530822 Hmm the recommendation in testthat [1] seems to be to put the tests in `R/pkg/tests/testthat` instead of `R/pkg/inst/tests` -- Could we try to do that ? Or are there a

[GitHub] spark pull request: [SPARK-12019][SPARKR] Check param and fix doc ...

2015-11-29 Thread felixcheung
Github user felixcheung commented on the pull request: https://github.com/apache/spark/pull/10034#issuecomment-160525963 It is. I could add that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10030#issuecomment-160519881 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10030#issuecomment-160519877 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10030#issuecomment-160519771 **[Test build #46867 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46867/consoleFull)** for PR 10030 at commit [`d2d4edb`](https://g

[GitHub] spark pull request: [SPARK-11985][STREAMING][KINESIS][DOCS] Update...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9970#issuecomment-160517946 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11985][STREAMING][KINESIS][DOCS] Update...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9970#issuecomment-160517949 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11985][STREAMING][KINESIS][DOCS] Update...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9970#issuecomment-160517901 **[Test build #46868 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46868/consoleFull)** for PR 9970 at commit [`948e2ba`](https://git

[GitHub] spark pull request: [SPARK-12031][Core][BUG]: Integer overflow whe...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10023#issuecomment-160517154 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12031][Core][BUG]: Integer overflow whe...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10023#issuecomment-160517151 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10030#issuecomment-160517090 **[Test build #46867 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46867/consoleFull)** for PR 10030 at commit [`d2d4edb`](https://gi

[GitHub] spark pull request: [SPARK-12031][Core][BUG]: Integer overflow whe...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10023#issuecomment-160517109 **[Test build #46866 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46866/consoleFull)** for PR 10023 at commit [`3cafda5`](https://g

[GitHub] spark pull request: [SPARK-11955][SQL] Mark one side fields in mer...

2015-11-29 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/9940#issuecomment-160516788 @yhuai I think so. We completely skip pushdown filters now if schema merging is enabled. This patch is to improve that. --- If your project is set up for it, you can rep

[GitHub] spark pull request: [SPARK-11985][STREAMING][KINESIS][DOCS] Update...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9970#issuecomment-160516718 **[Test build #46868 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46868/consoleFull)** for PR 9970 at commit [`948e2ba`](https://gith

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-11-29 Thread sun-rui
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/10030#issuecomment-160516196 rebased to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-11985][STREAMING][KINESIS][DOCS] Update...

2015-11-29 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/9970#issuecomment-160516113 @tdas removed SparkR DESCRIPTION file changes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: [SPARK-12019][SPARKR] Check param and fix doc ...

2015-11-29 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/10034#issuecomment-160515406 Hmm I think its better to just support both forms ? Isn't `c("jar1", "jar2")` more natural ? --- If your project is set up for it, you can reply to this email and hav

[GitHub] spark pull request: [SPARK-11955][SQL] Mark one side fields in mer...

2015-11-29 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9940#issuecomment-160513996 oh, I thought it's a bug fix (so, I was wondering if it's a regression from 1.5 or not). But, it is actually an improvement? --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-11-29 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-160513571 By the way, it looks like the test failure which motivated this patch occurs more frequently than I originally thought. It seems to happen about once a day, give or ta

[GitHub] spark pull request: [SPARK-5682][Core] Add encrypted shuffle in sp...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8880#issuecomment-160512402 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-5682][Core] Add encrypted shuffle in sp...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8880#issuecomment-160512401 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-5682][Core] Add encrypted shuffle in sp...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8880#issuecomment-160512345 **[Test build #46864 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46864/consoleFull)** for PR 8880 at commit [`a0cade0`](https://git

[GitHub] spark pull request: [SPARK-11206] (Followup) Fix SQLListenerMemory...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9991#issuecomment-160512250 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11206] (Followup) Fix SQLListenerMemory...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9991#issuecomment-160512249 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11206] (Followup) Fix SQLListenerMemory...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9991#issuecomment-160512207 **[Test build #46862 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46862/consoleFull)** for PR 9991 at commit [`00df329`](https://git

[GitHub] spark pull request: [SPARK-11619][SQL] cannot use UDTF in DataFram...

2015-11-29 Thread dilipbiswal
Github user dilipbiswal closed the pull request at: https://github.com/apache/spark/pull/9981 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-11619][SQL] cannot use UDTF in DataFram...

2015-11-29 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/9981#issuecomment-160509630 @cloud-fan Sure.. I will close this PR for now. If you think of a better approach and need my help in anyway pl let me know. Thanks for your feedback. --- If your p

[GitHub] spark pull request: [SPARK-9563][SQL] Remove repartition operators...

2015-11-29 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/7959 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-11-29 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10030#discussion_r46106851 --- Diff: R/pkg/inst/tests/testthat/test_mllib.R --- @@ -26,7 +26,7 @@ sc <- sparkR.init() sqlContext <- sparkRSQL.init(sc) test_that("glm and

[GitHub] spark pull request: [SPARK-11698][Streaming] Add option to ignore ...

2015-11-29 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/9665#issuecomment-160507917 @tdas Thanks for comment. I would like to close this first. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-11698][Streaming] Add option to ignore ...

2015-11-29 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/9665 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-12009][Yarn]Avoid to re-allocating yarn...

2015-11-29 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/10031#issuecomment-160507811 no... actually, it is not good ideaExecutor may be killed by Yarn due to some reason...or just akka disconnected... and I hope you can close this, I will worki

[GitHub] spark pull request: [SPARK-11955][SQL] Mark one side fields in mer...

2015-11-29 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/9940#issuecomment-160507740 Hi @yhuai Can you explain more? What you meant about regression? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-12018][SQL] Refactor common subexpressi...

2015-11-29 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/10009#issuecomment-160507676 ping @rxin @cloud-fan @nongli --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11619][SQL] cannot use UDTF in DataFram...

2015-11-29 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/9981#issuecomment-160507517 Thanks for working on it! Actually I'm not sure if this is the right appraoch... We still hande `explode` and `json_tuple` specially, what's worse, we hardcode

[GitHub] spark pull request: [Spark-8426] [scheduler] enhance blacklist mec...

2015-11-29 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/8760#issuecomment-160506398 This seems potentially useful, but adds a lot of complexity. Can you elaborate on the use cases where this problem has been arising? My vague memory of previous d

[GitHub] spark pull request: [SPARK-12039] [SQL] Ignore HiveSparkSubmitSuit...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10035#issuecomment-160506192 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12039] [SQL] Ignore HiveSparkSubmitSuit...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10035#issuecomment-160506193 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12031][Core][BUG]: Integer overflow whe...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10023#issuecomment-160506162 **[Test build #46866 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46866/consoleFull)** for PR 10023 at commit [`3cafda5`](https://gi

[GitHub] spark pull request: [SPARK-12031][Core][BUG]: Integer overflow whe...

2015-11-29 Thread uncleGen
Github user uncleGen commented on a diff in the pull request: https://github.com/apache/spark/pull/10023#discussion_r46105601 --- Diff: core/src/main/scala/org/apache/spark/util/random/SamplingUtils.scala --- @@ -52,16 +52,21 @@ private[spark] object SamplingUtils { (tri

[GitHub] spark pull request: [Spark-8426] [scheduler] enhance blacklist mec...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8760#issuecomment-160505191 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [Spark-8426] [scheduler] enhance blacklist mec...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8760#issuecomment-160505192 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-12039] [SQL] Ignore HiveSparkSubmitSuit...

2015-11-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10035 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [Spark-8426] [scheduler] enhance blacklist mec...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8760#issuecomment-160505134 **[Test build #46861 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46861/consoleFull)** for PR 8760 at commit [`19e8cc0`](https://git

[GitHub] spark pull request: [SPARK-12039] [SQL] Ignore HiveSparkSubmitSuit...

2015-11-29 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10035#issuecomment-160504962 I am merging it to master and branch 1.6. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-12039] [SQL] Ignore HiveSparkSubmitSuit...

2015-11-29 Thread yhuai
GitHub user yhuai opened a pull request: https://github.com/apache/spark/pull/10035 [SPARK-12039] [SQL] Ignore HiveSparkSubmitSuite's "SPARK-9757 Persist Parquet relation with decimal column". https://issues.apache.org/jira/browse/SPARK-12039 Since it is pretty flaky in had

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-11-29 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/10030#discussion_r46105330 --- Diff: R/pkg/inst/tests/testthat/test_mllib.R --- @@ -26,7 +26,7 @@ sc <- sparkR.init() sqlContext <- sparkRSQL.init(sc) test_that("glm

[GitHub] spark pull request: [SPARK-12037][CORE] initialize heartbeatReceiv...

2015-11-29 Thread lianhuiwang
Github user lianhuiwang commented on the pull request: https://github.com/apache/spark/pull/10032#issuecomment-160500147 LGTM @andrewor14 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-5682][Core] Add encrypted shuffle in sp...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8880#issuecomment-160498136 **[Test build #46864 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46864/consoleFull)** for PR 8880 at commit [`a0cade0`](https://gith

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-11-29 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10030#discussion_r46103264 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -832,10 +846,10 @@ test_that("test HiveContext", { expect_equal(count(df2), 3)

[GitHub] spark pull request: [SPARK-11206] (Followup) Fix SQLListenerMemory...

2015-11-29 Thread carsonwang
Github user carsonwang commented on the pull request: https://github.com/apache/spark/pull/9991#issuecomment-160495888 @zsxwing , do you have any further comments regarding how the `SQLListener` is cleaned up? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-11-29 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10030#discussion_r46103245 --- Diff: R/pkg/inst/tests/testthat/test_mllib.R --- @@ -26,7 +26,7 @@ sc <- sparkR.init() sqlContext <- sparkRSQL.init(sc) test_that("glm and

[GitHub] spark pull request: [SPARK-11774][SPARKR] Implement struct(), enco...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9804#issuecomment-160495870 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11774][SPARKR] Implement struct(), enco...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9804#issuecomment-160495869 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11774][SPARKR] Implement struct(), enco...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9804#issuecomment-160495866 **[Test build #46863 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46863/consoleFull)** for PR 9804 at commit [`b4b0a61`](https://git

[GitHub] spark pull request: [SPARK-11774][SPARKR] Implement struct(), enco...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9804#issuecomment-160495223 **[Test build #46863 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46863/consoleFull)** for PR 9804 at commit [`b4b0a61`](https://gith

[GitHub] spark pull request: [SPARK-11206] (Followup) Fix SQLListenerMemory...

2015-11-29 Thread carsonwang
Github user carsonwang commented on the pull request: https://github.com/apache/spark/pull/9991#issuecomment-160494777 @vanzin , I wrapped the calls to the hooks with `Utils.tryLogNonFatalError`. I didn't clean up the `SQLListener` after a application end event because another `SQLCon

[GitHub] spark pull request: [SPARK-11102] [SQL] Uninformative exception wh...

2015-11-29 Thread zjffdu
Github user zjffdu commented on the pull request: https://github.com/apache/spark/pull/9490#issuecomment-160493625 @yhuai Only changing JSONRelation/TextRelation can be a short-term solution. But since this is a general issue for HadoopFsRelation. so it would be better to do it in Ha

[GitHub] spark pull request: [SPARK-11206] (Followup) Fix SQLListenerMemory...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9991#issuecomment-160493581 **[Test build #46862 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46862/consoleFull)** for PR 9991 at commit [`00df329`](https://gith

[GitHub] spark pull request: [SPARK-11774][SPARKR] Implement struct(), enco...

2015-11-29 Thread sun-rui
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/9804#issuecomment-160493234 rebased to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request: [Spark-8426] [scheduler] enhance blacklist mec...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8760#issuecomment-160490927 **[Test build #46861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46861/consoleFull)** for PR 8760 at commit [`19e8cc0`](https://gith

[GitHub] spark pull request: [SPARK-12019][SPARKR] Check param and fix doc ...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10034#issuecomment-160487807 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12019][SPARKR] Check param and fix doc ...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10034#issuecomment-160487808 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12019][SPARKR] Check param and fix doc ...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10034#issuecomment-160487774 **[Test build #46860 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46860/consoleFull)** for PR 10034 at commit [`11d4ddc`](https://g

[GitHub] spark pull request: [SPARK-12019][SPARKR] Check param and fix doc ...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10034#issuecomment-160486682 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12019][SPARKR] Check param and fix doc ...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10034#issuecomment-160486681 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12019][SPARKR] Check param and fix doc ...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10034#issuecomment-160486648 **[Test build #46860 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46860/consoleFull)** for PR 10034 at commit [`11d4ddc`](https://gi

[GitHub] spark pull request: [SPARK-11619][SQL] cannot use UDTF in DataFram...

2015-11-29 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/9981#issuecomment-160486254 ping @cloud-fan @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-12039] [TESTS] [test-hadoop1.0] Add mav...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10033#issuecomment-160485304 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12039] [TESTS] [test-hadoop1.0] Add mav...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10033#issuecomment-160485305 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12039] [TESTS] [test-hadoop1.0] Add mav...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10033#issuecomment-160485294 **[Test build #46858 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46858/consoleFull)** for PR 10033 at commit [`53138fe`](https://g

[GitHub] spark pull request: [SPARK-12019][SPARKR] Check param and fix doc ...

2015-11-29 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/10034 [SPARK-12019][SPARKR] Check param and fix doc for sparkR.init() and add tests. Spark submit expects comma-separated list You can merge this pull request into a Git repository by running:

[GitHub] spark pull request: [SPARK-11482][SQL] Make maven repo for Hive me...

2015-11-29 Thread xguo27
Github user xguo27 commented on the pull request: https://github.com/apache/spark/pull/9543#issuecomment-160482218 Thanks @yhuai for reviewing my code! I have updated per your suggestion. To answer your question, I personally do not have a use case for this. My take on the JIR

[GitHub] spark pull request: [SPARK-12039] [TESTS] [test-hadoop1.0] Add mav...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10033#issuecomment-160480747 **[Test build #46857 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46857/consoleFull)** for PR 10033 at commit [`53138fe`](https://g

[GitHub] spark pull request: [SPARK-12039] [TESTS] [test-hadoop1.0] Add mav...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10033#issuecomment-160480768 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12039] [TESTS] [test-hadoop1.0] Add mav...

2015-11-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10033#issuecomment-160480769 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12039] [TESTS] [test-hadoop1.0] Add mav...

2015-11-29 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10033#issuecomment-160478928 https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46858/consoleFull is in worker 6. --- If your project is set up for it, you can reply to this email a

[GitHub] spark pull request: [SPARK-12039] [TESTS] [test-hadoop1.0] Add mav...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10033#issuecomment-160478873 **[Test build #46858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46858/consoleFull)** for PR 10033 at commit [`53138fe`](https://gi

[GitHub] spark pull request: [SPARK-8641][SQL] Native Spark Window function...

2015-11-29 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/9819#issuecomment-160478413 Yes. You can use any Spark aggregate function as a window function. Most Hive UDAFs should also work except for the pivoted ones... --- If your project is set up for

[GitHub] spark pull request: [SPARK-12039] [TESTS] [test-hadoop1.0] Add mav...

2015-11-29 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10033#issuecomment-160478206 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark pull request: [SPARK-12039] [TESTS] [test-hadoop1.0] Add mav...

2015-11-29 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10033#issuecomment-160478184 **[Test build #46857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46857/consoleFull)** for PR 10033 at commit [`53138fe`](https://gi

[GitHub] spark pull request: [SPARK-12039] [TESTS] [test-hadoop1.0] Add mav...

2015-11-29 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/10033#issuecomment-160478218 https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46857/consoleFull is execurted by worker 3. Let me find another worker. --- If your project is set up

  1   2   >