-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58203/
-----------------------------------------------------------
(Updated April 6, 2017, 9:25 a.m.)
Review request for hive, Aihua Xu, Zoltan Haindrich, Yongzhi Chen, and Barna
Zsombor Klara.
Changes
-------
Instead of the needsContinuation method refactor moved the getCommands to
BeeLine, so we can reuse the whole file parsing algorithm in the tests.
Modified the test classes to use the BeeLine.getCommands method
Added a warning message, when use database command is used in the test script
Bugs: HIVE-16345
https://issues.apache.org/jira/browse/HIVE-16345
Repository: hive-git
Description
-------
The goal of the change is to run qtest files which contain queries on tables
created by the init scripts.
It adds the possibility to rewrite the src table references to default.src
This patch contains the following changes:
- Added new parameter to the driver, to control weather the rewrite the table
names or not (test.rewrite.source.tables) - default is true
- Made QTestUtil.getSrcTables() available for QFile class
- Run the QFile not with "!run testfile.q", but reading the file, and
assembling the commands - enable us to parse the queries, and provide better
feedback about the failing queries
- QFile rewrites the source tables, if it is required
- Used 9 qtest files from the CliDriver, and added them to BeeLine tests
- Added new filters, and removed redundant ones - I was able to remove every
QFile specific filter, and corresponding setter methods as well
- Moved QFile classes to org.apache.hive.beeline package, so it can use package
private methods from BeeLine, and Commands
- Refactored needsContinuation method in BeeLine, so it can be called from a
static context as well
And one important change is:
- In Utilities.setMapRedWork, change the INPUT_NAME value in the conf to a
mapreduce task specific value. This one is used by the IOContextMap to cache
the IOContext objects. Using the same value for every mapred task prevented
them to run in the same JVM. The test were running sequencially, but failed
randomly in parallel
Diffs (updated)
-----
beeline/src/java/org/apache/hive/beeline/BeeLine.java 11526a7
beeline/src/java/org/apache/hive/beeline/Commands.java 2578728
itests/src/test/resources/testconfiguration.properties 7a70c9c
itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CoreBeeLineDriver.java
0d63f5d
itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java 2abf252
itests/util/src/main/java/org/apache/hive/beeline/qfile/QFile.java ae5a349
itests/util/src/main/java/org/apache/hive/beeline/qfile/QFileBeeLineClient.java
760fde6
itests/util/src/main/java/org/apache/hive/beeline/qfile/package-info.java
fcd50ec
ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 79955e9
ql/src/test/results/clientpositive/beeline/drop_with_concurrency.q.out
385f9b7
ql/src/test/results/clientpositive/beeline/escape_comments.q.out abc0fee
ql/src/test/results/clientpositive/beeline/smb_mapjoin_1.q.out PRE-CREATION
ql/src/test/results/clientpositive/beeline/smb_mapjoin_10.q.out PRE-CREATION
ql/src/test/results/clientpositive/beeline/smb_mapjoin_11.q.out PRE-CREATION
ql/src/test/results/clientpositive/beeline/smb_mapjoin_12.q.out PRE-CREATION
ql/src/test/results/clientpositive/beeline/smb_mapjoin_13.q.out PRE-CREATION
ql/src/test/results/clientpositive/beeline/smb_mapjoin_16.q.out PRE-CREATION
ql/src/test/results/clientpositive/beeline/smb_mapjoin_2.q.out PRE-CREATION
ql/src/test/results/clientpositive/beeline/smb_mapjoin_3.q.out PRE-CREATION
ql/src/test/results/clientpositive/beeline/smb_mapjoin_7.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/58203/diff/2/
Changes: https://reviews.apache.org/r/58203/diff/1-2/
Testing
-------
Run the test multiple times with the various combinations of the following
parameters:
- test.rewrite.source.tables - runs with true, or without it, fails when set to
false
- junit.parallel.threads - runs with 1, or without this parameter
Thanks,
Peter Vary
