[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2016-04-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/7373 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2016-04-20 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-212303752 Merging in master. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2016-04-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-212302231 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/5

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2016-04-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-212302225 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2016-04-20 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-212301712 **[Test build #56328 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56328/consoleFull)** for PR 7373 at commit [`19f2d81`](https://git

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2016-04-19 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-212286079 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2016-04-19 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/7373#discussion_r60355233 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/Window.scala --- @@ -120,22 +120,22 @@ case class Window( val (exprs, current,

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2016-04-19 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-212277420 **[Test build #56328 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56328/consoleFull)** for PR 7373 at commit [`19f2d81`](https://gith

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-12-15 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-164879251 @cloud-fan can you rebase? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-137010014 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-137010016 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-137009749 [Test build #41924 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41924/console) for PR 7373 at commit [`62233a9`](https://github.

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-09-02 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-136964866 cc @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-09-02 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-136964557 [Test build #41924 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41924/consoleFull) for PR 7373 at commit [`62233a9`](https://gith

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-136962574 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-09-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-136962553 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-07-15 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-121815633 A new question here, now we support non-deterministic/stateful expressions in codegen, so all generated class is not thread safe, not only `MutableProjection`. If

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-07-15 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-121654210 @davies which "local" do you mean? If you mean `LocalBackend`(--master "local[*]"), we will still serialize and deserialize task sets, if you mean "running locally",

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-07-14 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-121287094 @cloud-fan How about local mode? We don't always create the projection inside `mapPartition` --- If your project is set up for it, you can reply to this email and have y

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-07-13 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-121114584 I think we will do codegen at executor side for every partition, so multiple tasks won't share one codegened `MutableProject`, is that correct? --- If your project is

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-121070225 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-121029208 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-07-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-121029045 [Test build #37136 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37136/console) for PR 7373 at commit [`4e7372c`](https://github.

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-07-13 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-121014294 @davies that is correct. The goal was to be able to amortize the cost of getting the code and then create a new copy with its own row multiple times. Though looking a

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-07-13 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-121001588 [Test build #37136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/37136/consoleFull) for PR 7373 at commit [`4e7372c`](https://gith

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-121001035 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-07-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-121001069 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-07-13 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/7373#issuecomment-120990724 I think MutableProject will create a `MutableRow` inside it, it's not thread-safe, can not be used by multiple tasks. cc @marmbrus, Is it correct? --- If your project i

[GitHub] spark pull request: [SPARK-9013][SQL] generate MutableProjection d...

2015-07-13 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/7373 [SPARK-9013][SQL] generate MutableProjection directly instead of return a function Why did we do this at the very beginning? You can merge this pull request into a Git repository by running: