> On Feb. 10, 2014, 6:44 p.m., Jung JaeHwa wrote: > > +1 for the patch. > > It looks good overall.
Sorry. Would you wait some more please? I'm reviewing the patch and have some comments even thoug it is trivial. - Jihoon ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17726/#review34063 ----------------------------------------------------------- On Feb. 8, 2014, 2:17 a.m., Hyunsik Choi wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/17726/ > ----------------------------------------------------------- > > (Updated Feb. 8, 2014, 2:17 a.m.) > > > Review request for Tajo. > > > Bugs: TAJO-584 > https://issues.apache.org/jira/browse/TAJO-584 > > > Repository: tajo > > > Description > ------- > > In Tajo, sort operator is similar to merge sort, but it works in a > distributed manner. The first sort phase sorts each fragment in local > machine, the intermediate data are shuffled in range partition, and then the > the second sort phase in each node sorts the range-partitioned data. > > However, the second sort phase reads all shuffled data via one scanner. It > causes performance degrade. This patch improves the second sort phase to > merge directly all already-sorted intermediate data. It significantly reduces > the response time of sort queries. > > I carried out some simple benchmark with the following query on TPC-H 100GB > data sets: > {code:sql} > select l_orderkey from lineitem order by l_orderkey; > {code} > > The lineitem table occupies 75GB. The query response time are dramatically > reduced from 480 to 260 secs. This patch exploits the design of TAJO-36. So, > this patch requires TAJO-36. > > > Diffs > ----- > > CHANGES.txt 8dc5ee5f0021acacb75220d1a3ba04b22ebf925b > > tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/exception/AlreadyExistsTableException.java > 98c04b57e4d72f7ef02f7ec72e4df05f14ac30eb > tajo-client/src/main/java/org/apache/tajo/jdbc/TajoResultSet.java > 3977d76e37888defb27715c249b336bb4cc8a5f5 > > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/BaseAlgebraVisitor.java > 6cc6fd053093480136184fc7d293c59265070f17 > > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/LogicalPlan.java > 6afaaad7b067258ec0cefcca634164b8ae9afa06 > > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java > dfa2e40e46e5658e664c7593fd7e224fe32cbfb4 > > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PlannerUtil.java > b59cddafadd0c254aaef97c482cacab6ca4742c1 > > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/PreLogicalPlanVerifier.java > 1843b5adc298a003a9934b180d0056c655bf44fb > > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/RangePartitionAlgorithm.java > fc56b55dd4bfee16a8650de107c07f616ffc4a4b > > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/UniformRangePartition.java > 4f18c95225935b7104da6866c7ad98f790b965e6 > > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java > 2aa93d7f251848b267f98c6a8c1e5a193696e633 > > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/ExternalSortExec.java > a3b37fc6445249c23ec2a99bbac312e4c2d4bbe9 > > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/UnaryPhysicalExec.java > a8dd877fdd3a230db86aaa13a66d24933db08735 > > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/utils/TupleUtil.java > 42508a0767e9c5848fed705042d7d14fde8747db > > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/querymaster/Repartitioner.java > 71ab8f9bbdffb140de3b2601e8982d2c0ca3f09e > > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/RangeRetrieverHandler.java > 26991a00dd62256d724bfd614c71e68f005020e2 > tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/Task.java > 1b72d1398adc097422f2bb8990561ad528ff733c > > tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/TestUniformRangePartition.java > acd663f18d539f0af24b789b082d323fc7db60b2 > > tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestBSTIndexExec.java > cf240542552b492bdee9c2df69cfdfab1a238c98 > > tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestExternalSortExec.java > d7a3eb109728a968b9bc3f2861e1e5dec6e6e265 > > tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestLeftOuterHashJoinExec.java > 4ac423b8ce8c143217523de149e840773aca0040 > > tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestLeftOuterNLJoinExec.java > ce31851333c2929e4141b8d2b3e327a41d14f90b > > tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestPhysicalPlanner.java > c97ed00782a395e84ce643e764e4f9ef88fbd17b > > tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestRightOuterHashJoinExec.java > d582e2b10b25a0b067ce640af2f8b39b6e35bbca > > tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/planner/physical/TestSortExec.java > 4aa1ba2339458d12ea8d297b7397f70340ccbb26 > > tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestCTASQuery.java > 9b940daac43cbb95164803d37e48973ea0a39ab7 > > tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestSortQuery.java > 5e5c70cd9233b4aba8f6bcd6321731c41b14bd55 > > tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/util/TestTupleUtil.java > 3c4d3c2d8ead6c70b863dd8a848c1396522c6e2d > > tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/worker/TestRangeRetrieverHandler.java > 78e846de731c7fa2739dbb6dcc3bec45d4ba1ab3 > > tajo-core/tajo-core-backend/src/test/resources/dataset/TestSortQuery/table2.tbl > PRE-CREATION > > tajo-core/tajo-core-backend/src/test/resources/org/apache/tajo/jdbc/TestTajoResultSet.java > PRE-CREATION > > tajo-core/tajo-core-backend/src/test/resources/queries/TestCTASQuery/testError.sql > PRE-CREATION > > tajo-core/tajo-core-backend/src/test/resources/queries/TestJoinQuery/testOuterJoinAndCaseWhen1.sql > ccb58a58f0303e01a228fc5774e2180d90a20f27 > > tajo-core/tajo-core-backend/src/test/resources/queries/TestSortQuery/create_table_with_asc_desc_keys.sql > PRE-CREATION > > tajo-core/tajo-core-backend/src/test/resources/queries/TestSortQuery/testSortWithAscDescKeys.sql > PRE-CREATION > > tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/pullserver/PullServerAuxService.java > 1531fadaf9bbc2cb601989618936a400815d592c > > tajo-core/tajo-core-pullserver/src/main/java/org/apache/tajo/pullserver/TajoPullServerService.java > 1b0cadcc67c134ffe7e09c7bf7af08e970871542 > tajo-storage/src/main/java/org/apache/tajo/storage/MergeScanner.java > e4439f318d080f064848543586e538b1e73b3122 > tajo-storage/src/main/java/org/apache/tajo/storage/RawFile.java > 73b36926144b0797615b88ad728b889a7849c8b2 > tajo-storage/src/main/java/org/apache/tajo/storage/TupleComparator.java > 69c1c046775d7c7193864912d54a4fa3d7024cce > tajo-storage/src/main/java/org/apache/tajo/storage/TupleRange.java > 7d0f674e9af104b9f13f6f77cac18115b8224d1a > > Diff: https://reviews.apache.org/r/17726/diff/ > > > Testing > ------- > > mvn clean install > > > Thanks, > > Hyunsik Choi > >
