[GitHub] spark pull request: [SPARK-12165] Fix bug in eviction of storage m...

2015-12-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10170#issuecomment-162441738 I think that the test failures in `BlockManagerSuite` are related to the semantics of `maxNumBytesToFree` in `StaticMemoryManager.acquireUnrollMemory`. [Prev

[GitHub] spark pull request: [SPARK-11593][SQL] Replace catalyst converter ...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9565#issuecomment-162441586 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-11593][SQL] Replace catalyst converter ...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9565#issuecomment-162441582 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-12166][TEST] Unset hadoop related envir...

2015-12-06 Thread zjffdu
GitHub user zjffdu opened a pull request: https://github.com/apache/spark/pull/10172 [SPARK-12166][TEST] Unset hadoop related environment in testing You can merge this pull request into a Git repository by running: $ git pull https://github.com/zjffdu/spark SPARK-12166 Altern

[GitHub] spark pull request: [SPARK-10259] [ML] Add @since annotation to ml...

2015-12-06 Thread taishi-oss
Github user taishi-oss commented on the pull request: https://github.com/apache/spark/pull/8534#issuecomment-162440255 I modified the code that was pointed out. @sarutak @yu-iskw Thank you for reviewing! --- If your project is set up for it, you can reply to this email and have yo

[GitHub] spark pull request: [SPARK-10259] [ML] Add @since annotation to ml...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8534#issuecomment-162440197 **[Test build #47260 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47260/consoleFull)** for PR 8534 at commit [`6831e7a`](https://gith

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10030#issuecomment-162439940 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10030#issuecomment-162439938 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-11593][SQL] Replace catalyst converter ...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9565#issuecomment-162439812 **[Test build #47259 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47259/consoleFull)** for PR 9565 at commit [`f806755`](https://gith

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10030#issuecomment-162439769 **[Test build #47257 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47257/consoleFull)** for PR 10030 at commit [`37d8080`](https://g

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10030#issuecomment-162437605 **[Test build #47257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47257/consoleFull)** for PR 10030 at commit [`37d8080`](https://gi

[GitHub] spark pull request: [SPARK-11395][SPARKR] Support over and window ...

2015-12-06 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10094#discussion_r46792542 --- Diff: R/pkg/R/window.R --- @@ -0,0 +1,88 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license ag

[GitHub] spark pull request: [SPARK-10582][Yarn][Core] Fix AM failure situa...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9963#issuecomment-162437086 **[Test build #47256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47256/consoleFull)** for PR 9963 at commit [`0e1d796`](https://gith

[GitHub] spark pull request: [SPARK-12103][Streaming][Kafka][Doc] document ...

2015-12-06 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/10132#issuecomment-162437123 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-11395][SPARKR] Support over and window ...

2015-12-06 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10094#discussion_r46792190 --- Diff: R/pkg/R/window.R --- @@ -0,0 +1,88 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license ag

[GitHub] spark pull request: [SPARK-10582][Yarn][Core] Fix AM failure situa...

2015-12-06 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/9963#issuecomment-162436010 Hi @vanzin , from my test and understanding so far, I think calling `reset()` on the first registration should be OK. But here I still don't change to that way, for me

[GitHub] spark pull request: [SPARK-12034][SPARKR] Eliminate warnings in Sp...

2015-12-06 Thread sun-rui
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/10030#issuecomment-162435647 @shivaram, rebased again. I can successfully rebase it to master on my dev machine, don't know any other merge conflicts. Could you merge it before merging any other PR

[GitHub] spark pull request: [SPARK-12167][SQL] Invoke the right sameResult...

2015-12-06 Thread watermen
Github user watermen commented on the pull request: https://github.com/apache/spark/pull/10169#issuecomment-162435295 @dilipbiswal Yes, SPARK-11246 has already fixed it, Thanks and I'll close this PR. --- If your project is set up for it, you can reply to this email and have your rep

[GitHub] spark pull request: [SPARK-12167][SQL] Invoke the right sameResult...

2015-12-06 Thread watermen
Github user watermen closed the pull request at: https://github.com/apache/spark/pull/10169 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread sun-rui
Github user sun-rui commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162431824 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162428939 @shivaram @felixcheung @sun-rui Thank you everyone! Hopefully, my code changes resolve all your concerns. I learned a lot from you! : ) --- If your project is set u

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162427732 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162427729 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162427500 **[Test build #47255 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47255/consoleFull)** for PR 10160 at commit [`a78109e`](https://g

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162424451 **[Test build #47255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47255/consoleFull)** for PR 10160 at commit [`a78109e`](https://gi

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10160#discussion_r46789803 --- Diff: R/pkg/R/DataFrame.R --- @@ -692,8 +696,8 @@ setMethod("sample", setMethod("sample_frac", signature(x = "DataFrame", withRepla

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10160#discussion_r46789542 --- Diff: R/pkg/R/DataFrame.R --- @@ -692,8 +696,8 @@ setMethod("sample", setMethod("sample_frac", signature(x = "DataFrame", withRepla

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162419713 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162419711 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162419603 **[Test build #47254 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47254/consoleFull)** for PR 10160 at commit [`4337c35`](https://g

[GitHub] spark pull request: [SPARK-7729][UI]Executor which has been killed...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10058#issuecomment-16241 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-7729][UI]Executor which has been killed...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10058#issuecomment-162418828 **[Test build #47249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47249/consoleFull)** for PR 10058 at commit [`122d3f2`](https://g

[GitHub] spark pull request: [SPARK-7729][UI]Executor which has been killed...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10058#issuecomment-162418887 Build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10160#discussion_r46789026 --- Diff: R/pkg/R/DataFrame.R --- @@ -692,8 +696,8 @@ setMethod("sample", setMethod("sample_frac", signature(x = "DataFrame", withReplacem

[GitHub] spark pull request: [SPARK-12060][CORE] Avoid memory copy in JavaS...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10167#issuecomment-162418221 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12060][CORE] Avoid memory copy in JavaS...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10167#issuecomment-162418222 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12060][CORE] Avoid memory copy in JavaS...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10167#issuecomment-162418159 **[Test build #47248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47248/consoleFull)** for PR 10167 at commit [`c100251`](https://g

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162418157 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162418156 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162418099 **[Test build #47253 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47253/consoleFull)** for PR 10160 at commit [`34d0118`](https://g

[GitHub] spark pull request: [SPARK-12165] Fix bug in eviction of storage m...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10170#issuecomment-162417990 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12165] Fix bug in eviction of storage m...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10170#issuecomment-162417956 **[Test build #47251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47251/consoleFull)** for PR 10170 at commit [`0eac7da`](https://g

[GitHub] spark pull request: [SPARK-12165] Fix bug in eviction of storage m...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10170#issuecomment-162417991 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162417739 **[Test build #47254 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47254/consoleFull)** for PR 10160 at commit [`4337c35`](https://gi

[GitHub] spark pull request: [SPARK-10852][PySpark][SQL] Override built-in ...

2015-12-06 Thread guoxu1231
Github user guoxu1231 commented on the pull request: https://github.com/apache/spark/pull/8934#issuecomment-162417597 BTW, There is a JIRA to suggest user to prefer using c['column'] instead of c.column. @davies, we encountered similar issue, could you paste the JIRA number fo

[GitHub] spark pull request: [SPARK-12167][SQL] Invoke the right sameResult...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10169#issuecomment-162417059 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12167][SQL] Invoke the right sameResult...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10169#issuecomment-162417058 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-10582][Yarn][Core] Fix AM failure situa...

2015-12-06 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/9963#discussion_r46788434 --- Diff: yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala --- @@ -51,6 +51,9 @@ private[spark] abstract class YarnSchedul

[GitHub] spark pull request: [SPARK-12167][SQL] Invoke the right sameResult...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10169#issuecomment-162417004 **[Test build #47250 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47250/consoleFull)** for PR 10169 at commit [`f1ef856`](https://g

[GitHub] spark pull request: [SPARK-11932][STREAMING] Partition previous Tr...

2015-12-06 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9988#issuecomment-162416976 LGTM except a nit --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162416996 : ) @sun-rui Done. Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/10160#discussion_r46788421 --- Diff: R/pkg/R/DataFrame.R --- @@ -677,13 +677,15 @@ setMethod("unique", #' collect(sample(df, TRUE, 0.5)) #'} setMethod("sample", -

[GitHub] spark pull request: [SPARK-12168][SPARKR] Add automated tests for ...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10171#issuecomment-162416419 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12168][SPARKR] Add automated tests for ...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10171#issuecomment-162416422 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12168][SPARKR] Add automated tests for ...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10171#issuecomment-162416192 **[Test build #47252 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47252/consoleFull)** for PR 10171 at commit [`7c7c5d3`](https://g

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10160#discussion_r46788139 --- Diff: R/pkg/R/DataFrame.R --- @@ -677,13 +677,15 @@ setMethod("unique", #' collect(sample(df, TRUE, 0.5)) #'} setMethod("sample", -

[GitHub] spark pull request: [SPARK-12156] Make SPARK_EXECUTOR_INSTANCES be...

2015-12-06 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/10157#discussion_r46788170 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -177,7 +177,7 @@ private[deploy] class SparkSubmitArguments(args:

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread sun-rui
Github user sun-rui commented on a diff in the pull request: https://github.com/apache/spark/pull/10160#discussion_r46788107 --- Diff: R/pkg/R/DataFrame.R --- @@ -692,8 +694,12 @@ setMethod("sample", setMethod("sample_frac", signature(x = "DataFrame", withReplace

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162415989 **[Test build #47253 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47253/consoleFull)** for PR 10160 at commit [`34d0118`](https://gi

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162415645 LGTM. I'll wait for @felixcheung and Jenkins before merging --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] spark pull request: [SPARK-12156] Make SPARK_EXECUTOR_INSTANCES be...

2015-12-06 Thread jerryshao
Github user jerryshao commented on the pull request: https://github.com/apache/spark/pull/10157#issuecomment-162415650 Looks like this is a hidden environment, do we need to support it explicitly? --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162415274 @felixcheung @shivaram Sure, just added that test case. Please review it. Thank you! : ) --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request: [SPARK-11932][STREAMING] Partition previous Tr...

2015-12-06 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9988#discussion_r46787805 --- Diff: streaming/src/test/resources/log4j.properties --- @@ -25,4 +25,5 @@ log4j.appender.file.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss.SSS} %t %p %c{

[GitHub] spark pull request: [SPARK-12106][STREAMING][FLAKY-TEST] BatchedWA...

2015-12-06 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/10110#issuecomment-162414966 LGTM for the patch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: [SPARK-12106][STREAMING][FLAKY-TEST] BatchedWA...

2015-12-06 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/10110#discussion_r46787547 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/BatchedWriteAheadLog.scala --- @@ -165,10 +165,12 @@ private[util] class BatchedWriteAhea

[GitHub] spark pull request: [SPARK-12168][SPARKR] Add automated tests for ...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10171#issuecomment-162414262 **[Test build #47252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47252/consoleFull)** for PR 10171 at commit [`7c7c5d3`](https://gi

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162413477 Yeah thats a good idea @felixcheung --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] spark pull request: [SPARK-12168][SPARKR] Add automated tests for ...

2015-12-06 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/10171 [SPARK-12168][SPARKR] Add automated tests for conflicted function in R Currently this is reported when loading the SparkR package in R (probably would add is.nan) ``` Loading required p

[GitHub] spark pull request: [SPARK-12158] [SparkR] [SQL] Fix 'sample' func...

2015-12-06 Thread felixcheung
Github user felixcheung commented on the pull request: https://github.com/apache/spark/pull/10160#issuecomment-162412228 @gatorsmile Sure - I guess the main thing is to ensure the seed is getting set. How about: ``` count1 <- count(sample(df, FALSE, 0.1, 0)) count2 <- cou

[GitHub] spark pull request: [SPARK-12167][SQL] Invoke the right sameResult...

2015-12-06 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/10169#issuecomment-162407562 @watermen I believe SPARK-11246 was a very similar defect and was fixed in 1.5. Is this a different scenario that what was addressed in that defect ? @yhuai had

[GitHub] spark pull request: [SPARK-11102] [SQL] Uninformative exception wh...

2015-12-06 Thread zjffdu
Github user zjffdu commented on the pull request: https://github.com/apache/spark/pull/9490#issuecomment-162406408 @yhuai Sure, np --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-12165] Fix bug in eviction of storage m...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10170#issuecomment-162405970 **[Test build #47251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47251/consoleFull)** for PR 10170 at commit [`0eac7da`](https://gi

[GitHub] spark pull request: [SPARK-12165] Fix bug in eviction of storage m...

2015-12-06 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/10170 [SPARK-12165] Fix bug in eviction of storage memory by execution This patch fixes a bug in the eviction of storage memory by execution. ## The bug: In general, execution should b

[GitHub] spark pull request: [SPARK-12167][SQL] Invoke the right sameResult...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10169#issuecomment-162404057 **[Test build #47250 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47250/consoleFull)** for PR 10169 at commit [`f1ef856`](https://gi

[GitHub] spark pull request: [SPARK-12167][SQL] Invoke the right sameResult...

2015-12-06 Thread watermen
GitHub user watermen opened a pull request: https://github.com/apache/spark/pull/10169 [SPARK-12167][SQL] Invoke the right sameResult function when plan is warpped with SubQueries ### Bug I find this bug when I use cache table, ``` spark-sql> create table src_p(key int,

[GitHub] spark pull request: [SPARK-11593][SQL] Replace catalyst converter ...

2015-12-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/9565#discussion_r46785072 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala --- @@ -66,980 +85,305 @@ case class ScalaUDF( */

[GitHub] spark pull request: [SPARK-11102] [SQL] Uninformative exception wh...

2015-12-06 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9490#issuecomment-162402641 @zjffdu How about we revisit it after we release 1.6? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] spark pull request: [SPARK-11102] [SQL] Uninformative exception wh...

2015-12-06 Thread zjffdu
Github user zjffdu commented on the pull request: https://github.com/apache/spark/pull/9490#issuecomment-162402210 @yhuai @liancheng any more comments ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: Fix kstreaming war

2015-12-06 Thread microwishing
Github user microwishing closed the pull request at: https://github.com/apache/spark/pull/10168 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: Fix kstreaming war

2015-12-06 Thread microwishing
GitHub user microwishing opened a pull request: https://github.com/apache/spark/pull/10168 Fix kstreaming war You can merge this pull request into a Git repository by running: $ git pull https://github.com/microwishing/spark FIX_KSTREAMING_WAR Alternatively you can review and

[GitHub] spark pull request: Fix kstreaming war

2015-12-06 Thread microwishing
Github user microwishing commented on the pull request: https://github.com/apache/spark/pull/10168#issuecomment-162402045 wrong PR. sorry --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-11593][SQL] Replace catalyst converter ...

2015-12-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/9565#discussion_r46784210 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala --- @@ -66,980 +85,305 @@ case class ScalaUDF( */

[GitHub] spark pull request: [SPARK-12164] [SQL] Display the binary/encoded...

2015-12-06 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/10165#issuecomment-162400140 I have the exact same question when calling the show function. From the perspectives of users, they might not care the encoded values at all when calling the functio

[GitHub] spark pull request: [SPARK-7729][UI]Executor which has been killed...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10058#issuecomment-162397994 **[Test build #47249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47249/consoleFull)** for PR 10058 at commit [`122d3f2`](https://gi

[GitHub] spark pull request: [SPARK-12161][SQL] Ignore order of predicates ...

2015-12-06 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10163#issuecomment-162397946 This is a great feature! Can we implement it in individual expressions instead of centralizing them in `LogicalPlan.samResult`? A lof of commutative operators need it

[GitHub] spark pull request: [SPARK-12060][CORE] Avoid memory copy in JavaS...

2015-12-06 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10167#issuecomment-162397948 **[Test build #47248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47248/consoleFull)** for PR 10167 at commit [`c100251`](https://gi

[GitHub] spark pull request: [SPARK-12164] [SQL] Display the binary/encoded...

2015-12-06 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10165#issuecomment-162396682 Shoud we print the decoded values(user objects) in `Dataset.show`? cc @marmbrus @rxin --- If your project is set up for it, you can reply to this email and have you

[GitHub] spark pull request: [SPARK-12060][CORE] Avoid memory copy in JavaS...

2015-12-06 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/10167 [SPARK-12060][CORE] Avoid memory copy in JavaSerializerInstance.serialize Merged #10051 again since #10083 is resolved. This reverts commit 328b757d5d4486ea3c2e246780792d7a57ee85e5. You ca

[GitHub] spark pull request: [SPARK-7729][UI]Executor which has been killed...

2015-12-06 Thread lianhuiwang
Github user lianhuiwang commented on the pull request: https://github.com/apache/spark/pull/10058#issuecomment-162396051 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have thi

[GitHub] spark pull request: Update StructType.scala

2015-12-06 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10141#discussion_r46781460 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala --- @@ -40,7 +40,8 @@ import org.apache.spark.sql.catalyst.util.DataTypeParser

[GitHub] spark pull request: [SPARK-12152][PROJECT-INFRA] Speed up Scalasty...

2015-12-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10151 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-12159] [ML] Add user guide section for ...

2015-12-06 Thread holdenk
Github user holdenk commented on the pull request: https://github.com/apache/spark/pull/10166#issuecomment-162386176 It might be useful to also document the different ways "missing" labels can be handled - what are your thoughts? --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-12152][PROJECT-INFRA] Speed up Scalasty...

2015-12-06 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10151#issuecomment-162386145 We can handle retention of the output in a followup, since it'd probably be best to also do it for Java, Python, and R style checks. Therefore, I'm going to merge thi

[GitHub] spark pull request: [SPARK-12044] [SparkR] Fix usage of isnan, isN...

2015-12-06 Thread yanboliang
Github user yanboliang commented on the pull request: https://github.com/apache/spark/pull/10037#issuecomment-162385822 Thanks for your help @sun-rui @felixcheung @shivaram . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: [SPARK-12149] [Web UI] Executor UI improvement...

2015-12-06 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/10154#discussion_r46776484 --- Diff: core/src/test/resources/HistoryServerExpectations/executor_list_json_expectation.json --- @@ -4,11 +4,13 @@ "rddBlocks" : 8, "memory

[GitHub] spark pull request: [SPARK-12102][SQL] Cast a non-nullable struct ...

2015-12-06 Thread dilipbiswal
Github user dilipbiswal commented on the pull request: https://github.com/apache/spark/pull/10156#issuecomment-162349876 @yhuai Hi Yin, Thanks for your comments. In this fix, we are only changing the nullability of the value for comparision purpose to see if the left and right types a

[GitHub] spark pull request: [SPARK-7857][MLLIB] Prevent IDFModel from retu...

2015-12-06 Thread karlhigley
Github user karlhigley closed the pull request at: https://github.com/apache/spark/pull/9843 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12160][MLLIB] Use SQLContext.getOrCreat...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10161#issuecomment-162344223 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12164] [SQL] Display the binary/encoded...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10165#issuecomment-162344227 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12164] [SQL] Display the binary/encoded...

2015-12-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10165#issuecomment-162344226 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

  1   2   >