-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18764/
-----------------------------------------------------------
(Updated March 6, 2014, 2:36 a.m.)
Review request for Tajo.
Changes
-------
Hyunsik.
Thank you for your review and detailed comments.
I added unit test cases for other join types. (eg: right outer join, full outer
join, cross join)
Bugs: TAJO-427
https://issues.apache.org/jira/browse/TAJO-427
Repository: tajo
Description
-------
If empty table use at LEFT OUTER JOIN clause, it makes
IndexOutOfBoundsException as follows:
{code:xml}
tajo> select * from table1;
Progress: 100%, response time: 0.146 sec
final state: QUERY_SUCCEEDED, response time: 0.146 sec
result:
hdfs://localhost:9010/tmp/tajo-blrunner/staging/q_1387348492386_0013/RESULT, 5
rows (60 B)
id, name, score, type
-------------------------------
1, ooo, 1.1, a
2, ppp, 2.3, b
3, qqq, 3.4, c
4, rrr, 4.5, d
5, xxx, 5.6, e
tajo> select * from table3;
Progress: 100%, response time: 0.053 sec
final state: QUERY_SUCCEEDED, response time: 0.053 sec
result:
hdfs://localhost:9010/tmp/tajo-blrunner/staging/q_1387348492386_0014/RESULT, 0
rows (0 B)
id, name, score, type
-------------------------------
tajo> select a.id, a.name, a.score, case when b.name is null then 'zzz' else
b.name end as name2 from table1 a left outer join table3 b on a.id = b.id;
Internal error!
{code}
Tajo master makes log as follows:
{code:xml}
2013-12-18 15:39:25,678 INFO service.AbstractService
(AbstractService.java:start(94)) -
Service:org.apache.tajo.worker.AbstractResourceAllocator is started.
2013-12-18 15:39:25,678 INFO service.AbstractService
(AbstractService.java:start(94)) -
Service:org.apache.tajo.master.TajoAsyncDispatcher is started.
2013-12-18 15:39:25,679 INFO master.TajoAsyncDispatcher
(TajoAsyncDispatcher.java:start(101)) - AsyncDispatcher
started:q_1387348492386_0015
2013-12-18 15:39:25,679 INFO service.AbstractService
(AbstractService.java:start(94)) -
Service:org.apache.tajo.master.querymaster.QueryMasterTask is started.
2013-12-18 15:39:25,679 INFO querymaster.Query (Query.java:handle(452)) -
Processing q_1387348492386_0015 of type START
2013-12-18 15:39:25,682 INFO storage.AbstractStorageManager
(AbstractStorageManager.java:listStatus(384)) - Total input paths to process : 0
2013-12-18 15:39:25,682 INFO storage.AbstractStorageManager
(AbstractStorageManager.java:getSplits(612)) - Total # of splits: 0
2013-12-18 15:39:25,682 ERROR querymaster.SubQuery
(SubQuery.java:transition(529)) - SubQuery (eb_1387348492386_0015_000003)
ERROR:
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at
org.apache.tajo.master.querymaster.Repartitioner.createJoinTasks(Repartitioner.java:96)
at
org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.createTasks(SubQuery.java:663)
at
org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.transition(SubQuery.java:517)
at
org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.transition(SubQuery.java:499)
at
org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:382)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
at org.apache.tajo.master.querymaster.SubQuery.handle(SubQuery.java:476)
at
org.apache.tajo.master.querymaster.Query$StartTransition.transition(Query.java:288)
at
org.apache.tajo.master.querymaster.Query$StartTransition.transition(Query.java:277)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:359)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:299)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:43)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:445)
at org.apache.tajo.master.querymaster.Query.handle(Query.java:457)
at org.apache.tajo.master.querymaster.Query.handle(Query.java:54)
at
org.apache.tajo.master.TajoAsyncDispatcher.dispatch(TajoAsyncDispatcher.java:137)
at
org.apache.tajo.master.TajoAsyncDispatcher$1.run(TajoAsyncDispatcher.java:79)
at java.lang.Thread.run(Thread.java:680)
{code}
Diffs (updated)
-----
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/ShuffleFileWriteNode.java
5399357
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/HashPartitioner.java
b620b22
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/physical/Partitioner.java
b67f45c
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/utils/TupleUtil.java
6d801dd
tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/engine/query/TestJoinQuery.java
0e925f1
tajo-core/tajo-core-backend/src/test/resources/queries/TestJoinQuery/testCrossJoinWithEmptyTable1.sql
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/queries/TestJoinQuery/testFullOuterJoinWithEmptyTable1.sql
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/queries/TestJoinQuery/testLeftOuterJoin1.sql
f946e1d
tajo-core/tajo-core-backend/src/test/resources/queries/TestJoinQuery/testLeftOuterJoinWithEmptyTable1.sql
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/queries/TestJoinQuery/testLeftOuterJoinWithEmptyTable2.sql
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/queries/TestJoinQuery/testLeftOuterJoinWithEmptyTable3.sql
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/queries/TestJoinQuery/testLeftOuterJoinWithEmptyTable4.sql
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/queries/TestJoinQuery/testRightOuterJoinWithEmptyTable1.sql
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/results/TestJoinQuery/testCrossJoinWithEmptyTable1.result
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/results/TestJoinQuery/testFullOuterJoinWithEmptyTable1.result
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/results/TestJoinQuery/testLeftOuterJoin1.result
81dc055
tajo-core/tajo-core-backend/src/test/resources/results/TestJoinQuery/testLeftOuterJoinWithEmptyTable1.result
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/results/TestJoinQuery/testLeftOuterJoinWithEmptyTable2.result
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/results/TestJoinQuery/testLeftOuterJoinWithEmptyTable3.result
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/results/TestJoinQuery/testLeftOuterJoinWithEmptyTable4.result
PRE-CREATION
tajo-core/tajo-core-backend/src/test/resources/results/TestJoinQuery/testRightOuterJoinWithEmptyTable1.result
PRE-CREATION
Diff: https://reviews.apache.org/r/18764/diff/
Testing
-------
mvn clean install
Thanks,
Jung JaeHwa
