-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22374/
-----------------------------------------------------------
(Updated July 4, 2014, 9:40 a.m.)
Review request for Tajo.
Changes
-------
I updated the patch as follows:
- Divide fetch uris into the the proper number of tasks by IntermediateData
output volume. The output volume is 256MB, but you can set it at tajo
configuration file. This property name is
tajo.scattered.hash.shuffle.split.volume.
- Adding shuffle output volume to TajoWorkerProtocol. If task complete, then
Task::getTaskCompletionReport will set this property.
For reference, I tested lots of cases on TPC-H benchmarking cluster, and I
found that it ran successfully.
Bugs: TAJO-673
https://issues.apache.org/jira/browse/TAJO-673
Repository: tajo
Description
-------
When inserting into partitioned table, if the number of partitions is smaller
than cluster concurrency capacity, a query execution is too slow.
Diffs (updated)
-----
tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java 6298d27
tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java
f41d61d
tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java
edd5674
tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/HashShuffleFileWriteExec.java
678b745
tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnit.java
6cada07
tajo-core/src/main/java/org/apache/tajo/master/querymaster/QueryUnitAttempt.java
361f88f
tajo-core/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java
80274e2
tajo-core/src/main/java/org/apache/tajo/worker/Task.java c6e2b73
tajo-core/src/main/java/org/apache/tajo/worker/TaskAttemptContext.java
b1246ec
tajo-core/src/main/proto/TajoWorkerProtocol.proto 3bf6e13
tajo-core/src/test/java/org/apache/tajo/engine/query/TestTablePartitions.java
8c989b5
tajo-yarn-pullserver/src/main/java/org/apache/tajo/pullserver/PullServerAuxService.java
b8fda29
tajo-yarn-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java
cc3cb2e
Diff: https://reviews.apache.org/r/22374/diff/
Testing
-------
mvn clean install
Thanks,
Jung JaeHwa
