[GitHub] spark pull request: SPARK-2533 - Add locality levels (with tasks c...

2015-11-02 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/9117#discussion_r43668261 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -122,6 +123,24 @@ private[ui] class StagePage(parent: StagesTab) extends W

[GitHub] spark pull request: SPARK-2533 - Add locality levels (with tasks c...

2015-11-02 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/9117#discussion_r43668168 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -122,6 +123,24 @@ private[ui] class StagePage(parent: StagesTab) extends W

[GitHub] spark pull request: SPARK-2533 - Add locality levels (with tasks c...

2015-11-02 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/9117#discussion_r43667980 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -122,6 +123,24 @@ private[ui] class StagePage(parent: StagesTab) extends W

[GitHub] spark pull request: SPARK-2533 - Add locality levels (with tasks c...

2015-11-02 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/9117#discussion_r43667755 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -122,6 +123,24 @@ private[ui] class StagePage(parent: StagesTab) extends W

[GitHub] spark pull request: [SPARK-10827] [CORE] AppClient should not use ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9317#issuecomment-153128012 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-2533 - Add locality levels (with tasks c...

2015-11-02 Thread kayousterhout
Github user kayousterhout commented on a diff in the pull request: https://github.com/apache/spark/pull/9117#discussion_r43667618 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -21,6 +21,7 @@ import java.net.URLEncoder import java.util.Date i

[GitHub] spark pull request: [SPARK-10827] [CORE] AppClient should not use ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9317#issuecomment-153127965 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: SPARK-2533 - Add locality levels (with tasks c...

2015-11-02 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/9117#issuecomment-153127544 Two suggestions: 1) can we change the format of this to say "Process local: 2 tasks; Rack local: 4 tasks; Any: 15 tasks"? I think that's easier to read and

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153127602 **[Test build #44823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44823/consoleFull)** for PR 9383 at commit [`2e341f5`](https://gith

[GitHub] spark pull request: [SPARK-10827] [CORE] AppClient should not use ...

2015-11-02 Thread BryanCutler
Github user BryanCutler commented on the pull request: https://github.com/apache/spark/pull/9317#issuecomment-153127474 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-8029][core] first successful shuffle ta...

2015-11-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9214#discussion_r43666862 --- Diff: core/src/main/scala/org/apache/spark/shuffle/ShuffleOutputCoordinator.scala --- @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] spark pull request: [SPARK-11333][Streaming]Add executorId to Rece...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9418#issuecomment-153126403 **[Test build #44822 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44822/consoleFull)** for PR 9418 at commit [`051afb5`](https://gith

[GitHub] spark pull request: [SPARK-11333][Streaming]Add executorId to Rece...

2015-11-02 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/9418 [SPARK-11333][Streaming]Add executorId to ReceiverInfo and display it in UI Expose executorId to `ReceiverInfo` and UI since it's helpful when there are multiple executors running in the same host.

[GitHub] spark pull request: [SPARK-10827] [CORE] AppClient should not use ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9317#issuecomment-153125623 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153125548 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153125505 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10827] [CORE] AppClient should not use ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9317#issuecomment-153125626 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-8029][core] first successful shuffle ta...

2015-11-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9214#discussion_r43666391 --- Diff: core/src/main/scala/org/apache/spark/shuffle/ShuffleOutputCoordinator.scala --- @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] spark pull request: [SPARK-8029][core] first successful shuffle ta...

2015-11-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9214#discussion_r43666157 --- Diff: core/src/main/scala/org/apache/spark/shuffle/ShuffleOutputCoordinator.scala --- @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] spark pull request: [DOC] Missing link to R DataFrame API doc

2015-11-02 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/9394#issuecomment-153125114 Yeah but the DataFrame.html doesn't seem very useful the way it is right now -- can you just update the roxygen docs in this PR as well ? --- If your project is set up

[GitHub] spark pull request: [SPARK-11333][Streaming]Add executorId to Rece...

2015-11-02 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/9418#discussion_r43665920 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala --- @@ -402,7 +402,7 @@ private[ui] class StreamingPage(parent: Streaming

[GitHub] spark pull request: [SPARK-8029][core] first successful shuffle ta...

2015-11-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9214#discussion_r43665840 --- Diff: core/src/main/scala/org/apache/spark/shuffle/ShuffleOutputCoordinator.scala --- @@ -0,0 +1,105 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] spark pull request: [SPARK-11333][Streaming]Add executorId to Rece...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9418#issuecomment-153124316 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11333][Streaming]Add executorId to Rece...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9418#issuecomment-153124282 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-8029][core] first successful shuffle ta...

2015-11-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9214#discussion_r43665022 --- Diff: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala --- @@ -72,16 +72,23 @@ private[spark] class IndexShuffleBlockResolver(c

[GitHub] spark pull request: [SPARK-11437] [PySpark] Don't .take when conve...

2015-11-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9392 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-8029][core] first successful shuffle ta...

2015-11-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9214#discussion_r43664936 --- Diff: core/src/main/scala/org/apache/spark/shuffle/FileShuffleBlockResolver.scala --- @@ -132,6 +134,15 @@ private[spark] class FileShuffleBlockResolver(co

[GitHub] spark pull request: [SPARK-11437] [PySpark] Don't .take when conve...

2015-11-02 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9392#issuecomment-153122295 Merged into master, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8029][core] first successful shuffle ta...

2015-11-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9214#discussion_r43664703 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/BypassMergeSortShuffleWriter.java --- @@ -121,13 +125,22 @@ public BypassMergeSortShuffleWriter(

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153121664 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-10827] [CORE] AppClient should not use ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9317#issuecomment-153121675 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153121656 **[Test build #44820 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44820/consoleFull)** for PR 9383 at commit [`53dbdf2`](https://git

[GitHub] spark pull request: [SPARK-10827] [CORE] AppClient should not use ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9317#issuecomment-153121649 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153121661 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153121113 **[Test build #44820 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44820/consoleFull)** for PR 9383 at commit [`53dbdf2`](https://gith

[GitHub] spark pull request: [SPARK-8029][core] first successful shuffle ta...

2015-11-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9214#discussion_r43664109 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/BypassMergeSortShuffleWriter.java --- @@ -121,13 +125,22 @@ public BypassMergeSortShuffleWriter(

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153119226 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-10622] [core] [yarn] Differentiate dead...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-153119411 **[Test build #44819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44819/consoleFull)** for PR 8887 at commit [`81061a6`](https://gith

[GitHub] spark pull request: [SPARK-10997] [core] Add "client mode" to nett...

2015-11-02 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9210#issuecomment-153119171 Just took another look. LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not hav

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153119257 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11425] Improve Hybrid aggregation

2015-11-02 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9383#issuecomment-153118810 After some benchmark, realized that using hashcode as prefix in timsort will cause regression in timsort and snappy compression (especially for aggregation after join, th

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9404#discussion_r43662693 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala --- @@ -353,4 +354,44 @@ class CachedTableSuite extends QueryTest with SharedSQLC

[GitHub] spark pull request: SPARK-11420 Updating Stddev support via Impera...

2015-11-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/9380#discussion_r43662570 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/functions.scala --- @@ -1135,7 +992,76 @@ abstract class CentralMome

[GitHub] spark pull request: [SPARK-10622] [core] [yarn] Differentiate dead...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-153117844 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-10622] [core] [yarn] Differentiate dead...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-153117825 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-11198][STREAMING][KINESIS] Support de-a...

2015-11-02 Thread brkyvz
Github user brkyvz commented on the pull request: https://github.com/apache/spark/pull/9403#issuecomment-153117592 Tested that this patch successfully de-aggregates in recovery as well. --- If your project is set up for it, you can reply to this email and have your reply appear on Git

[GitHub] spark pull request: [SPARK-10622] [core] [yarn] Differentiate dead...

2015-11-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8887#issuecomment-153117330 Hi @kayousterhout , are you ok with my explanation above? I'd like to get this is soon. --- If your project is set up for it, you can reply to this email and have your r

[GitHub] spark pull request: [SPARK-11437] [PySpark] Don't .take when conve...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9392#issuecomment-153116887 **[Test build #1969 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1969/consoleFull)** for PR 9392 at commit [`a7c395f`](https://gi

[GitHub] spark pull request: [SPARK-10997] [core] Add "client mode" to nett...

2015-11-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9210 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-10997] [core] Add "client mode" to nett...

2015-11-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9210#issuecomment-153116416 Merging this to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-9817][YARN] Improve the locality calcul...

2015-11-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/8100 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-9817][YARN] Improve the locality calcul...

2015-11-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/8100#issuecomment-153115647 Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark pull request: [SPARK-11457][Streaming][YARN] Fix incorrect A...

2015-11-02 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/9412#issuecomment-153115388 LGTM as far as I understand this code. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark pull request: [SPARK-11457][Streaming][YARN] Fix incorrect A...

2015-11-02 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/9412#discussion_r43661295 --- Diff: streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala --- @@ -66,6 +67,14 @@ class Checkpoint(ssc: StreamingContext, val checkpointTi

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9404#issuecomment-153104224 **[Test build #44817 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44817/consoleFull)** for PR 9404 at commit [`e711e05`](https://gith

[GitHub] spark pull request: [SPARK-9858][SPARK-9859][SPARK-9861][SQL] Add ...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9276#issuecomment-153114509 **[Test build #44818 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44818/consoleFull)** for PR 9276 at commit [`51ca17e`](https://gith

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9404#discussion_r43660128 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala --- @@ -353,4 +354,44 @@ class CachedTableSuite extends QueryTest with SharedSQLC

[GitHub] spark pull request: [SPARK-9858][SPARK-9859][SPARK-9861][SQL] Add ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9276#issuecomment-153106415 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-9858][SPARK-9859][SPARK-9861][SQL] Add ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9276#issuecomment-153106533 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-11437] [PySpark] Don't .take when conve...

2015-11-02 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9392#issuecomment-153104500 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark pull request: [SPARK-11437] [PySpark] Don't .take when conve...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9392#issuecomment-153104431 **[Test build #1969 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1969/consoleFull)** for PR 9392 at commit [`a7c395f`](https://git

[GitHub] spark pull request: SPARK-11371 Make "mean" an alias for "avg" ope...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9332#issuecomment-153103900 @ted-yu Can you modify a test to use this alias? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SPARK-9162][SQL] Implement code generation fo...

2015-11-02 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/9270#issuecomment-153103783 @viirya Could you add unit tests for every data types? it's easy to be wrong for some types. --- If your project is set up for it, you can reply to this email and have y

[GitHub] spark pull request: [SPARK-9162][SQL] Implement code generation fo...

2015-11-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9270#discussion_r43659293 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala --- @@ -959,6 +963,122 @@ case class ScalaUDF( }

[GitHub] spark pull request: [SPARK-9162][SQL] Implement code generation fo...

2015-11-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9270#discussion_r43659185 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala --- @@ -959,6 +963,122 @@ case class ScalaUDF( }

[GitHub] spark pull request: [SPARK-8467][MLlib][PySpark] Add LDAModel.desc...

2015-11-02 Thread yu-iskw
Github user yu-iskw commented on a diff in the pull request: https://github.com/apache/spark/pull/8643#discussion_r43658405 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala --- @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9404#issuecomment-153100484 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9404#issuecomment-153100427 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-9162][SQL] Implement code generation fo...

2015-11-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/9270#discussion_r43658099 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala --- @@ -959,6 +963,122 @@ case class ScalaUDF( }

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9404#issuecomment-153099409 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9404#issuecomment-153099394 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-5354] [SQL] Cached tables should preser...

2015-11-02 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/9404#issuecomment-153099382 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9393#discussion_r43657874 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/UDFSuite.scala --- @@ -191,4 +193,86 @@ class UDFSuite extends QueryTest with SharedSQLContext {

[GitHub] spark pull request: [SPARK-8467][MLlib][PySpark] Add LDAModel.desc...

2015-11-02 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/8643#discussion_r43657655 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/api/python/LDAModelWrapper.scala --- @@ -0,0 +1,45 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] spark pull request: [SPARK-11449][Core] PortableDataStream should ...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9417#issuecomment-153095896 **[Test build #1968 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1968/consoleFull)** for PR 9417 at commit [`91b8c6c`](https://git

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153095366 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153095363 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/9393#discussion_r43656205 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/UDFRegistration.scala --- @@ -86,9 +104,9 @@ class UDFRegistration private[sql] (sqlContext: SQLContext)

[GitHub] spark pull request: [SPARK-10978] [SQL] Allow data sources to elim...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9399#issuecomment-153095199 **[Test build #44814 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44814/consoleFull)** for PR 9399 at commit [`7c17dd1`](https://git

[GitHub] spark pull request: [SPARK-11449][Core] PortableDataStream should ...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9417#issuecomment-153095133 **[Test build #1967 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1967/consoleFull)** for PR 9417 at commit [`91b8c6c`](https://gi

[GitHub] spark pull request: [SPARK-11112] DAG visualization: display RDD c...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9398#issuecomment-153091212 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44

[GitHub] spark pull request: [SPARK-11112] DAG visualization: display RDD c...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9398#issuecomment-153091204 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-11112] DAG visualization: display RDD c...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9398#issuecomment-153090817 **[Test build #44816 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44816/consoleFull)** for PR 9398 at commit [`6cd12f1`](https://git

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9393#discussion_r43651016 --- Diff: project/MimaExcludes.scala --- @@ -112,6 +112,53 @@ object MimaExcludes { "org.apache.spark.rdd.MapPartitionsWithPreparationRDD"),

[GitHub] spark pull request: [SPARK-11438] [SQL] Allow users to define nond...

2015-11-02 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9393#discussion_r43650981 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala --- @@ -30,13 +30,16 @@ case class ScalaUDF( functio

[GitHub] spark pull request: [SPARK-11458][SQL] add word count example for ...

2015-11-02 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/9415#issuecomment-153079772 This is great! Its really helpful to see the API in use to think about what parts might be hard for users. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-11458][SQL] add word count example for ...

2015-11-02 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9415#discussion_r43649975 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/DatasetWordCount.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software F

[GitHub] spark pull request: [SPARK-11458][SQL] add word count example for ...

2015-11-02 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9415#discussion_r43649615 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/DatasetWordCount.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software F

[GitHub] spark pull request: [SPARK-11458][SQL] add word count example for ...

2015-11-02 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9415#discussion_r43649263 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/DatasetWordCount.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software F

[GitHub] spark pull request: [SPARK-11458][SQL] add word count example for ...

2015-11-02 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/9415#discussion_r43648991 --- Diff: examples/src/main/scala/org/apache/spark/examples/sql/DatasetWordCount.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software F

[GitHub] spark pull request: [SPARK-9034] [SQL] Reflect field names defined...

2015-11-02 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/8456#discussion_r43645500 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala --- @@ -53,7 +53,7 @@ trait Generator extends Expression {

[GitHub] spark pull request: [SPARK-11449][Core] PortableDataStream should ...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9417#issuecomment-153060623 **[Test build #1967 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1967/consoleFull)** for PR 9417 at commit [`91b8c6c`](https://git

[GitHub] spark pull request: [SPARK-11449][Core] PortableDataStream should ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9417#issuecomment-153059875 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SPARK-11449][Core] PortableDataStream should ...

2015-11-02 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/9417#issuecomment-153059923 @kmader what do you think of this one? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark pull request: [SPARK-11449][Core] PortableDataStream should ...

2015-11-02 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/9417 [SPARK-11449][Core] PortableDataStream should be a factory ```PortableDataStream``` maintains some internal state. This makes it tricky to reuse a stream (one needs to call ```close``` on both th

[GitHub] spark pull request: Spark-6373 Add SSL/TLS for the Netty based Blo...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9416#issuecomment-153058457 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: Spark-6373 Add SSL/TLS for the Netty based Blo...

2015-11-02 Thread turp1twin
GitHub user turp1twin opened a pull request: https://github.com/apache/spark/pull/9416 Spark-6373 Add SSL/TLS for the Netty based BlockTransferService Sorry if this pull request is premature, but I have received very little feedback, so I am going ahead and creating it. I am still o

[GitHub] spark pull request: [SPARK-11112] DAG visualization: display RDD c...

2015-11-02 Thread sarutak
Github user sarutak commented on the pull request: https://github.com/apache/spark/pull/9398#issuecomment-153057423 Thanks @andrewor14 , I'll look into this soon! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SPARK-11112] DAG visualization: display RDD c...

2015-11-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9398#issuecomment-153053162 **[Test build #44816 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44816/consoleFull)** for PR 9398 at commit [`6cd12f1`](https://gith

[GitHub] spark pull request: [SPARK-11458][SQL] add word count example for ...

2015-11-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9415#issuecomment-153050635 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

<    4   5   6   7   8   9   10   11   >