> On April 29, 2014, 5:11 a.m., Jacques Nadeau wrote: > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/StreamAggPrule.java, > > line 80 > > <https://reviews.apache.org/r/20741/diff/1/?file=569068#file569068line80> > > > > same as hash agg, why removed? > > Aman Sinha wrote: > That was just for testing.. I have uncommented it.
I found an issue with single grouping key distribution for StreamingAggregate...some TPCH tests timeout and seem to be stuck in VolcanoPlanner.canonize() method. Note that the HashAggregate with single grouping key distribution works ok. One possibility is that for StreamingAggr even if the distribution is on single key, the collation is on all keys, so that might be causing some problem. I will re-visit this later. The current plans I am generating (and comparing by cost) are: (1) HashAggr with distribution on all grouping keys, (2) HashAggr with distribution on single grouping key, (3) StreamingAggr with distribution on all grouping keys. - Aman ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20741/#review41681 ----------------------------------------------------------- On April 26, 2014, 1:54 a.m., Aman Sinha wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/20741/ > ----------------------------------------------------------- > > (Updated April 26, 2014, 1:54 a.m.) > > > Review request for drill, Jacques Nadeau and Jinfeng Ni. > > > Repository: drill-git > > > Description > ------- > > 1. Added DrillCostBase that includes distribution (network) cost. > 2. Added cost formulas for computeSelfCost() for various Exchange operators, > joins, aggregations etc. > 3. Added Prels for HashJoin, HashAggregate and generate plans for those. > 4. Added exchange Prels: BroadcastExchangePrel and a new type called > HashToMergeExchangePrel. > > > Diffs > ----- > > exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java > 7e3b63d > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/HashToMergeExchange.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/common/HashTableConfig.java > 98892c0 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillFilterRelBase.java > 955729b > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillProjectRelBase.java > cf3d188 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillScanRelBase.java > b370352 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/common/DrillScreenRelBase.java > 51ed442 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/cost/DrillCostBase.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/cost/DrillRelOptCost.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/cost/DrillRelOptCostFactory.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java > 1492a28 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/BroadcastExchangePrel.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/DrillDistributionTrait.java > b75fb40 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/DrillDistributionTraitDef.java > c2ebb7a > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/FilterPrel.java > 0fc3abd > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashAggPrel.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashAggPrule.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashJoinPrel.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashJoinPrule.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashToMergeExchangePrel.java > PRE-CREATION > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/HashToRandomExchangePrel.java > e5c9661 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/MergeJoinPrel.java > 978a531 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/MergeJoinPrule.java > 8298e50 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PlannerSettings.java > e6e99c0 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/ScanPrel.java > a945129 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/SingleMergeExchangePrel.java > d9431cc > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/SortPrel.java > 344be4e > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/StreamAggPrel.java > c2880da > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/StreamAggPrule.java > a561a61 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/TopNPrel.java > e981a45 > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/UnionExchangePrel.java > f89cbaa > > exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DrillSqlWorker.java > 8892a8f > exec/java-exec/src/test/java/org/apache/drill/TestTpchSingleMode.java > 1ccb65c > > exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestHashJoin.java > ba067e2 > exec/java-exec/src/test/resources/join/hj_exchanges.json PRE-CREATION > exec/java-exec/src/test/resources/queries/tpch/01.sql f33416f > > Diff: https://reviews.apache.org/r/20741/diff/ > > > Testing > ------- > > Existing unit tests. Some TPCH tests show memory leaks but that is not > directly related to the costing changes...however those leaks would have to > be resolved before enabling the new plans. However, I wanted to get this > review request out to get feedback. > > > Thanks, > > Aman Sinha > >
