[
https://issues.apache.org/jira/browse/HIVE-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168541#comment-14168541
]
Thomas Friedrich commented on HIVE-7953:
----------------------------------------
The tests
dbtxnmgr_compact1
dbtxnmgr_compact2
dbtxnmgr_compact3
dbtxnmgr_ddl1
dbtxnmgr_query2
dbtxnmgr_query5
all fail with the same error:
FAILED: LockException [Error 10280]: Error communicating with the metastore
I did some investigation and found that the metastore tables are not
initialized properly. The underlying exception is:
org.apache.thrift.TException: MetaException(message:Unable to select from
transaction database, java.sql.SQLSyntaxErrorException: Table/View 'TXNS' does
not exist.
at
org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
Source)
at
org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedStatement.executeQuery(Unknown
Source)
at
com.jolbox.bonecp.StatementHandle.executeQuery(StatementHandle.java:464)
at
org.apache.hadoop.hive.metastore.txn.TxnHandler.timeOutTxns(TxnHandler.java:1625)
at
org.apache.hadoop.hive.metastore.txn.TxnHandler.getOpenTxns(TxnHandler.java:197)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_open_txns(HiveMetaStore.java:5273)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
When I debugged, I found that in TxnHandler.java, the check
HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVE_IN_TEST) in method
checkQFileTestHack() returns false:
private void checkQFileTestHack() {
boolean hackOn = HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVE_IN_TEST)
||
HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVE_IN_TEZ_TEST);
If that is false, it won't call prepDb() which creates the required tables.
According to HiveConf.java, HIVE_IN_TEST should be true in test mode, not sure
why it's not the case here.
> Investigate query failures (2)
> ------------------------------
>
> Key: HIVE-7953
> URL: https://issues.apache.org/jira/browse/HIVE-7953
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Brock Noland
> Assignee: Thomas Friedrich
>
> I ran all q-file tests and the following failed with an exception:
> http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/HIVE-SPARK-ALL-TESTS-Build/lastCompletedBuild/testReport/
> we don't necessary want to run all these tests as part of the spark tests,
> but we should understand why they failed with an exception. This JIRA is to
> look into these failures and document them with one of:
> * New JIRA
> * Covered under existing JIRA
> * More investigation required
> Tests:
> {noformat}
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_temp_table_external
> 0.33 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucket_num_reducers
> 4.3 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_2
> 11 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_load_hdfs_file_with_space_in_the_name
> 0.65 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketsortoptimize_insert_4
> 4.7 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketsortoptimize_insert_7
> 2.8 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketsortoptimize_insert_2
> 5.5 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby_position
> 1.5 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_18_part_external
> 2.4 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_6
> 11 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_smb_mapjoin_11
> 5.1 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_bucketsortoptimize_insert_8
> 10 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_parquet_join
> 5.4 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_stats_empty_dyn_part
> 0.81 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_compact1
> 0.31 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_ddl1
> 0.26 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_query2
> 0.73 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_auto_sortmerge_join_3
> 8.5 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_query5
> 0.34 sec 2
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_rcfile_bigdata
> 0.93 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_groupby_multi_single_reducer
> 6.3 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_compact3
> 2.4 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_dbtxnmgr_compact2
> 0.56 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_stats_partscan_1_23
> 3.1 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_list_bucket_dml_2
> 4.3 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_15_external_part
> 3.2 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_16_part_external
> 2.8 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_17_part_managed
> 3.4 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_20_part_managed_location
> 3.3 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_exim_19_00_part_external_location
> 6.9 sec 2
>
> org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_external_table_with_space_in_location_path
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
