[ https://issues.apache.org/jira/browse/SPARK-15076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-15076: ---------------------------------- Component/s: SQL > Add ReorderAssociativeOperator optimizer > ---------------------------------------- > > Key: SPARK-15076 > URL: https://issues.apache.org/jira/browse/SPARK-15076 > Project: Spark > Issue Type: Improvement > Components: Optimizer, SQL > Reporter: Dongjoon Hyun > Assignee: Dongjoon Hyun > Fix For: 2.1.0 > > > This issue add a new optimizer `ReorderAssociativeOperator` by taking > advantage of integral associative property. Currently, Spark works like the > following. > 1) Can optimize `1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + a` into `45 + a`. > 2) Cannot optimize `a + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9`. > This issue can handle Case 2 for *Add/Multiply* expression whose data types > are `ByteType`, `ShortType`, `IntegerType`, and `LongType`. The followings > are the plan comparison between `before` and `after` this issue. > **Before** > {code} > scala> sql("select a+1+2+3+4+5+6+7+8+9 from (select explode(array(1)) > a)").explain > == Physical Plan == > WholeStageCodegen > : +- Project [(((((((((a#7 + 1) + 2) + 3) + 4) + 5) + 6) + 7) + 8) + 9) AS > (((((((((a + 1) + 2) + 3) + 4) + 5) + 6) + 7) + 8) + 9)#8] > : +- INPUT > +- Generate explode([1]), false, false, [a#7] > +- Scan OneRowRelation[] > {code} > **After** > {code} > scala> sql("select a+1+2+3+4+5+6+7+8+9 from (select explode(array(1)) > a)").explain > == Physical Plan == > WholeStageCodegen > : +- Project [(a#7 + 45) AS (((((((((a + 1) + 2) + 3) + 4) + 5) + 6) + 7) + > 8) + 9)#8] > : +- INPUT > +- Generate explode([1]), false, false, [a#7] > +- Scan OneRowRelation[] > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org