----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22374/#review47486 -----------------------------------------------------------
tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java <https://reviews.apache.org/r/22374/#comment83342> According to my understanding, the parameter determines the input volume that each task processes for table partition. So, the config should belong to 'Distributed Query Execution Parameters'. Please take a look at the section 'Distributed Query Execution Parameters' in TajoConf. In addition the config is too deep. According to our convention, I'd like to recommend 'tajo.dist-query.table-partition.task-volume-mb' tajo-core/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java <https://reviews.apache.org/r/22374/#comment83363> This comment explains the problem when hash shuffle is used for table partition. I think that It is enough that we just explain what is scattered hash shuffle. tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java <https://reviews.apache.org/r/22374/#comment83362> It works well because each query has only one query. But, it is not intuitive because a loop seems to overwrite the variable multiple times. Why don't you traverse the MasterPlan via the graph visitor in order to find your interesting subquery? - Hyunsik Choi On July 4, 2014, 6:40 p.m., Jung JaeHwa wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/22374/ > ----------------------------------------------------------- > > (Updated July 4, 2014, 6:40 p.m.) > > > Review request for Tajo. > > > Bugs: TAJO-673 > https://issues.apache.org/jira/browse/TAJO-673 > > > Repository: tajo > > > Description > ------- > > When inserting into partitioned table, if the number of partitions is smaller > than cluster concurrency capacity, a query execution is too slow. > > > Diffs > ----- > > tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java 6298d27 > > tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java > f41d61d > > tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java > edd5674 > > tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/HashShuffleFileWriteExec.java > 678b745 > tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java > 6cada07 > > tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnitAttempt.java > 361f88f > > tajo-core/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java > 80274e2 > tajo-core/src/main/java/org/apache/tajo/worker/Task.java c6e2b73 > tajo-core/src/main/java/org/apache/tajo/worker/TaskAttemptContext.java > b1246ec > tajo-core/src/main/proto/TajoWorkerProtocol.proto 3bf6e13 > > tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java > 8c989b5 > > tajo-yarn-pullserver/src/main/java/org/apache/tajo/pullserver/PullServerAuxService.java > b8fda29 > > tajo-yarn-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java > cc3cb2e > > Diff: https://reviews.apache.org/r/22374/diff/ > > > Testing > ------- > > mvn clean install > > > Thanks, > > Jung JaeHwa > >
