[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-222646568 **[Test build #59645 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59645/consoleFull)** for PR 12850 at commit [`8956a1e`](https://github.com/apache/spark/commit/8956a1e0300c162cc8b71e836ee54c182cdd6a50). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/12850#discussion_r65127553
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -751,6 +751,16 @@ object ConstantFolding extends Rule[LogicalPlan] {
// Fold expressions that are foldable.
case e if e.foldable => Literal.create(e.eval(EmptyRow), e.dataType)
+
+ // Use associative property for integral type
+ case e if e.isInstanceOf[BinaryArithmetic] &&
e.dataType.isInstanceOf[IntegralType]
+=> e match {
+case Add(Add(a, b), c) if b.foldable && c.foldable => Add(a,
Add(b, c))
--- End diff --
Thank you for review, @cloud-fan !
I see. That sounds great.
Let me think about how to eliminate all constants then.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/12850#discussion_r65126741
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
---
@@ -751,6 +751,16 @@ object ConstantFolding extends Rule[LogicalPlan] {
// Fold expressions that are foldable.
case e if e.foldable => Literal.create(e.eval(EmptyRow), e.dataType)
+
+ // Use associative property for integral type
+ case e if e.isInstanceOf[BinaryArithmetic] &&
e.dataType.isInstanceOf[IntegralType]
+=> e match {
+case Add(Add(a, b), c) if b.foldable && c.foldable => Add(a,
Add(b, c))
--- End diff --
what about `a + 1 + b + 2`? I think we need a more general approach, like
reordering the `Add` nodes to put all literals together.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-75618 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-75620 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59531/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-75527 **[Test build #59531 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59531/consoleFull)** for PR 12850 at commit [`8c8ea7a`](https://github.com/apache/spark/commit/8c8ea7a571636ee2c8cc70aa1cadced87784060e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-65243 **[Test build #59531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59531/consoleFull)** for PR 12850 at commit [`8c8ea7a`](https://github.com/apache/spark/commit/8c8ea7a571636ee2c8cc70aa1cadced87784060e). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-221334943 Hi, @marmbrus and @rxin . Could you review this PR when you have some time? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-220862945 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/59114/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-220862943 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-220862881 **[Test build #59114 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59114/consoleFull)** for PR 12850 at commit [`eeae56d`](https://github.com/apache/spark/commit/eeae56d9b743e24a2657329f38a617ffa163ce5a). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-220859004 **[Test build #59114 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/59114/consoleFull)** for PR 12850 at commit [`eeae56d`](https://github.com/apache/spark/commit/eeae56d9b743e24a2657329f38a617ffa163ce5a). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-220447148 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-220447151 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58884/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-220446855 **[Test build #58884 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58884/consoleFull)** for PR 12850 at commit [`0ffd004`](https://github.com/apache/spark/commit/0ffd0043e54685885cc5a17947a5ac6281e8cf89). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-220417362 **[Test build #58884 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58884/consoleFull)** for PR 12850 at commit [`0ffd004`](https://github.com/apache/spark/commit/0ffd0043e54685885cc5a17947a5ac6281e8cf89). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-220416079 Rebased to trigger Jenkins test again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-220413082 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58875/ Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-220413078 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-220412842 **[Test build #58875 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58875/consoleFull)** for PR 12850 at commit [`65c7db7`](https://github.com/apache/spark/commit/65c7db78bfcac3ce025a858101ea53383f79a523). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-220390947 **[Test build #58875 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58875/consoleFull)** for PR 12850 at commit [`65c7db7`](https://github.com/apache/spark/commit/65c7db78bfcac3ce025a858101ea53383f79a523). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-219502167 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58648/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-219502166 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-219501871 **[Test build #58648 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58648/consoleFull)** for PR 12850 at commit [`0b60464`](https://github.com/apache/spark/commit/0b60464e3eed1ebf6e40e4216b9e521cbf539a52). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-219479063 **[Test build #58648 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58648/consoleFull)** for PR 12850 at commit [`0b60464`](https://github.com/apache/spark/commit/0b60464e3eed1ebf6e40e4216b9e521cbf539a52). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-218922877 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58519/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-218922875 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-218922758 **[Test build #58519 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58519/consoleFull)** for PR 12850 at commit [`18f5a8a`](https://github.com/apache/spark/commit/18f5a8a08ccfa75cb34ba2902883b9b1c0b7b8eb). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-218910104 **[Test build #58519 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58519/consoleFull)** for PR 12850 at commit [`18f5a8a`](https://github.com/apache/spark/commit/18f5a8a08ccfa75cb34ba2902883b9b1c0b7b8eb). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-218256024 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-218256026 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58246/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-218255698 **[Test build #58246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58246/consoleFull)** for PR 12850 at commit [`06e9b36`](https://github.com/apache/spark/commit/06e9b36c0381f6c987147803e894f2e29236deba). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-218231707 **[Test build #58246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58246/consoleFull)** for PR 12850 at commit [`06e9b36`](https://github.com/apache/spark/commit/06e9b36c0381f6c987147803e894f2e29236deba). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-218231334 Rebased to see the result on re-enable hive queries. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-217751647 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-217751648 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58113/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-217751614 **[Test build #58113 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58113/consoleFull)** for PR 12850 at commit [`3802255`](https://github.com/apache/spark/commit/3802255328481261949976913dfd2df6248a9bb8). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-217748244 **[Test build #58113 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58113/consoleFull)** for PR 12850 at commit [`3802255`](https://github.com/apache/spark/commit/3802255328481261949976913dfd2df6248a9bb8). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-217522739 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58006/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-217522736 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-217522469 **[Test build #58006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58006/consoleFull)** for PR 12850 at commit [`a4a3ce3`](https://github.com/apache/spark/commit/a4a3ce31f323e7cdfe4216c63ba156f8226ede52). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-217499612 **[Test build #58006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/58006/consoleFull)** for PR 12850 at commit [`a4a3ce3`](https://github.com/apache/spark/commit/a4a3ce31f323e7cdfe4216c63ba156f8226ede52). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-216980322 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57781/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-216980320 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-216980083 **[Test build #57781 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57781/consoleFull)** for PR 12850 at commit [`6898e0a`](https://github.com/apache/spark/commit/6898e0a4d1f4ba23dc7c19e73e1621c2a57cd8eb). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-216955720 **[Test build #57781 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57781/consoleFull)** for PR 12850 at commit [`6898e0a`](https://github.com/apache/spark/commit/6898e0a4d1f4ba23dc7c19e73e1621c2a57cd8eb). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-216400402 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/57567/ Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-216400399 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-216400248 **[Test build #57567 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57567/consoleFull)** for PR 12850 at commit [`71c3c73`](https://github.com/apache/spark/commit/71c3c73ad0f507837539ed58489e63e501bd4c3d). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12850#issuecomment-216386855 **[Test build #57567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57567/consoleFull)** for PR 12850 at commit [`71c3c73`](https://github.com/apache/spark/commit/71c3c73ad0f507837539ed58489e63e501bd4c3d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request: [SPARK-15076][SQL] Improve ConstantFolding opt...
GitHub user dongjoon-hyun opened a pull request:
https://github.com/apache/spark/pull/12850
[SPARK-15076][SQL] Improve ConstantFolding optimizer by using integral
associative property
## What changes were proposed in this pull request?
This issue improves `ConstantFolding` optimizer by taking advantage of
integral associative property. Currently, `ConstantFolding` works like the
following.
1) Can optimize `1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + a` into `45 + a`.
2) Cannot optimize `a + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9`.
This issue enables `ConstantFolding` to handle Case 2 for Add/Multiply
expression whose data types are `ByteType`, `ShortType`, `IntegerType`, and
`LongType`. The followings are the plan comparision between `before` and
`after` this issue.
**Before**
```
scala> sql("select a+1+2+3+4+5+6+7+8+9 from (select explode(array(1))
a)").explain
== Physical Plan ==
WholeStageCodegen
: +- Project [(a#7 + 1) + 2) + 3) + 4) + 5) + 6) + 7) + 8) + 9) AS
(a + 1) + 2) + 3) + 4) + 5) + 6) + 7) + 8) + 9)#8]
: +- INPUT
+- Generate explode([1]), false, false, [a#7]
+- Scan OneRowRelation[]
```
**After**
```
scala> sql("select a+1+2+3+4+5+6+7+8+9 from (select explode(array(1))
a)").explain
== Physical Plan ==
WholeStageCodegen
: +- Project [(a#7 + 45) AS (a + 1) + 2) + 3) + 4) + 5) + 6) + 7)
+ 8) + 9)#8]
: +- INPUT
+- Generate explode([1]), false, false, [a#7]
+- Scan OneRowRelation[]
```
## How was this patch tested?
Pass the Jenkins tests including new and updated testcases.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dongjoon-hyun/spark SPARK-15076
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12850.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12850
commit 71c3c73ad0f507837539ed58489e63e501bd4c3d
Author: Dongjoon Hyun
Date: 2016-05-02T22:38:31Z
[SPARK-15076][SQL] Improve ConstantFolding optimizer by using integral
associative property.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
