[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16433310#comment-16433310
]
Vineet Garg commented on HIVE-19164:
[~alangates] Would you mind taking a quick look and see if you could figure out
what's causing this? I suspect this failure is due to recent metastore changes.
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434085#comment-16434085
]
Alan Gates commented on HIVE-19164:
---
Sure.
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.0
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16436075#comment-16436075
]
Alan Gates commented on HIVE-19164:
---
I do not believe this is an issue with metastore changes. The config file
being handed to the HiveMetaStore.getMSForConf has schema verification set to
true. However, the config file being handed to Driver in
TestMetastoreVersion.testMetastoreVersion has it set to false. This means the
driver is somewhere not passing the config file it is handed down to
HiveMetaStore. I notice as well that this test passes if run in isolation, so
I'm guessing that somewhere the Driver or Hive (the class) isn't cleaning up an
old config and is reusing it.
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.0
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437719#comment-16437719
]
Vineet Garg commented on HIVE-19164:
I think there were some discussions regarding this sometime ago and driver not
passing its own config to Hive was a deliberate decision. I might be
misremembering though. cc [~pvary] [~vihangk1] [~thejas] to see if they know
anything about this.
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19164.patch
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437890#comment-16437890
]
Alan Gates commented on HIVE-19164:
---
If we want to leave it so Driver does not pass it's config to Hive I can change
the tests to explicitly set the config in Hive. But I don't know what has
changed to make it so that these tests fail now when they once passed.
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19164.patch
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16437917#comment-16437917
]
Ashutosh Chauhan commented on HIVE-19164:
-
Lets make a test only change for now so that we get closer to green build while
investigation continues for when/how this changed.
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19164.patch
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438016#comment-16438016
]
Vihang Karajgaonkar commented on HIVE-19164:
I don't know what changed which started to break the test but it seems to me
that the {{setVersion}} method in this test directly initializes the
ObjectStore and then calls setConf. However, that would not update the version
table since that code path gets executed via {{checkSchema}} method which gets
called when its instantiated via
{{HiveMetaStore.HMSHandler.getMSForConf(hiveConf)}}. Once I make that change
the test starts working for me.
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19164.patch
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438018#comment-16438018
]
Vihang Karajgaonkar commented on HIVE-19164:
These are my diffs in case anyone is interested.
{noformat}
diff --git
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetastoreVersion.java
b/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetastoreVersion.java
index 6015405..4d26f3e 100644
---
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetastoreVersion.java
+++
b/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestMetastoreVersion.java
@@ -142,6 +142,7 @@ public void testVersionMatching () throws Exception {
ObjectStore.setSchemaVerified(false);
hiveConf.setBoolVar(HiveConf.ConfVars.METASTORE_SCHEMA_VERIFICATION, true);
+hiveConf = new HiveConf(this.getClass());
setVersion(hiveConf, metastoreSchemaInfo.getHiveSchemaVersion());
driver = DriverFactory.newDriver(hiveConf);
CommandProcessorResponse proc = driver.run("show tables");
@@ -191,37 +192,34 @@ public void testVersionCompatibility () throws Exception {
}
// write the given version to metastore
- private String getVersion(HiveConf conf) throws HiveMetaException {
+ private String getVersion(HiveConf conf) throws Exception {
return getMetaStoreVersion();
}
// write the given version to metastore
- private void setVersion(HiveConf conf, String version) throws
HiveMetaException {
+ private void setVersion(HiveConf conf, String version) throws Exception {
setMetaStoreVersion(version, "setVersion test");
}
// Load the version stored in the metastore db
- public String getMetaStoreVersion() throws HiveMetaException {
-ObjectStore objStore = new ObjectStore();
-objStore.setConf(hiveConf);
+ public String getMetaStoreVersion() throws HiveMetaException, MetaException {
+RawStore ms = HiveMetaStore.HMSHandler.getMSForConf(hiveConf);
try {
- return objStore.getMetaStoreSchemaVersion();
+ return ms.getMetaStoreSchemaVersion();
} catch (MetaException e) {
throw new HiveMetaException("Failed to get version", e);
}
}
// Store the given version and comment in the metastore
- public void setMetaStoreVersion(String newVersion, String comment) throws
HiveMetaException {
-ObjectStore objStore = new ObjectStore();
-objStore.setConf(hiveConf);
+ public void setMetaStoreVersion(String newVersion, String comment)
+ throws HiveMetaException, MetaException {
+RawStore ms = HiveMetaStore.HMSHandler.getMSForConf(hiveConf);
try {
- objStore.setMetaStoreSchemaVersion(newVersion, comment);
+ ms.setMetaStoreSchemaVersion(newVersion, comment);
} catch (MetaException e) {
throw new HiveMetaException("Failed to set version", e);
}
}
-
-
}
{noformat}
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19164.patch
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438784#comment-16438784
]
Hive QA commented on HIVE-19164:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
17s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10226/dev-support/hive-personality.sh
|
| git revision | master / 1851732 |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10226/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10226/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19164.patch
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438798#comment-16438798
]
Hive QA commented on HIVE-19164:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918987/HIVE-19164.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 13402 tests
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,nopart_load.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_uri_add_partition.q,authorization_view_disable_cbo_3.q,bad_exec_hooks.q,authorization_view_disable_cbo_2.q,fetchtask_ioexception.q,char_pad_convert_fail2.q,authorization_set_role_neg1.q,serde_regex3.q,authorization_delete_nodeletepriv.q,materialized_view_delete.q,create_or_replace_view6.q,bucket_mapjoin_wrong_table_metadata_2.q,udf_sort_array_by_wrong2.q,loca
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439510#comment-16439510
]
Ashutosh Chauhan commented on HIVE-19164:
-
Since this is failing only in test not outside, I don't see a reason to change
Hive.java to propagate HiveConf. [~vihangk1] Would you like to upload your
patch so that we get QA run on it?
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19164.patch
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439569#comment-16439569
]
Vihang Karajgaonkar commented on HIVE-19164:
JIRA doesn't allow me to attach the patch since I am not the assignee of this
issue. [~alangates] Can I take over this issue?
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19164.patch
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439773#comment-16439773
]
Ashutosh Chauhan commented on HIVE-19164:
-
[~alangates] is OOO. [~vihangk1] I have changed the assignee so that you may
upload a patch.
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19164.patch
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439827#comment-16439827
]
Vihang Karajgaonkar commented on HIVE-19164:
Thanks [~ashutoshc]. Attaching the patch.
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19164.02.patch, HIVE-19164.patch
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441005#comment-16441005
]
Hive QA commented on HIVE-19164:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10272/dev-support/hive-personality.sh
|
| git revision | master / 5404635 |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10272/yetus/patch-asflicense-problems.txt
|
| modules | C: itests/hive-unit U: itests/hive-unit |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10272/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19164.02.patch, HIVE-19164.patch
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441120#comment-16441120
]
Hive QA commented on HIVE-19164:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12919261/HIVE-19164.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 13445 tests
executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=95)
[nopart_insert.q,insert_into_with_schema.q,input41.q,having1.q,create_table_failure3.q,default_constraint_invalid_default_value.q,database_drop_not_empty_restrict.q,windowing_after_orderby.q,orderbysortby.q,subquery_select_distinct2.q,authorization_uri_alterpart_loc.q,udf_last_day_error_1.q,constraint_duplicate_name.q,create_table_failure4.q,alter_tableprops_external_with_notnull_constraint.q,semijoin5.q,udf_format_number_wrong4.q,deletejar.q,exim_11_nonpart_noncompat_sorting.q,show_tables_bad_db2.q,drop_func_nonexistent.q,nopart_load.q,alter_table_non_partitioned_table_cascade.q,check_constraint_subquery.q,load_wrong_fileformat.q,check_constraint_udtf.q,lockneg_try_db_lock_conflict.q,udf_field_wrong_args_len.q,create_table_failure2.q,create_with_fk_constraints_enforced.q,groupby2_map_skew_multi_distinct.q,authorization_update_noupdatepriv.q,show_columns2.q,authorization_insert_noselectpriv.q,orc_replace_columns3_acid.q,compare_double_bigint.q,authorization_set_nonexistent_conf.q,alter_rename_partition_failure3.q,split_sample_wrong_format2.q,create_with_fk_pk_same_tab.q,compare_double_bigint_2.q,authorization_show_roles_no_admin.q,materialized_view_authorization_rebuild_no_grant.q,unionLimit.q,authorization_revoke_table_fail2.q,duplicate_insert3.q,authorization_desc_table_nosel.q,stats_noscan_non_native.q,orc_change_serde_acid.q,create_or_replace_view7.q,exim_07_nonpart_noncompat_ifof.q,create_with_unique_constraints_enforced.q,udf_concat_ws_wrong2.q,fileformat_bad_class.q,merge_negative_2.q,exim_15_part_nonpart.q,authorization_not_owner_drop_view.q,external1.q,authorization_uri_insert.q,create_with_fk_wrong_ref.q,columnstats_tbllvl_incorrect_column.q,authorization_show_parts_nosel.q,authorization_not_owner_drop_tab.q,external2.q,authorization_deletejar.q,temp_table_create_like_partitions.q,udf_greatest_error_1.q,ptf_negative_AggrFuncsWithNoGBYNoPartDef.q,alter_view_as_select_not_exist.q,touch1.q,groupby3_map_skew_multi_distinct.q,insert_into_notnull_constraint.q,exchange_partition_neg_partition_missing.q,groupby_cube_multi_gby.q,columnstats_tbllvl.q,drop_invalid_constraint2.q,alter_table_add_partition.q,update_not_acid.q,archive5.q,alter_table_constraint_invalid_pk_col.q,ivyDownload.q,udf_instr_wrong_type.q,bad_sample_clause.q,authorization_not_owner_drop_tab2.q,authorization_alter_db_owner.q,show_columns1.q,orc_type_promotion3.q,create_view_failure8.q,strict_join.q,udf_add_months_error_1.q,groupby_cube2.q,groupby_cube1.q,groupby_rollup1.q,genericFileFormat.q,invalid_cast_from_binary_4.q,drop_invalid_constraint1.q,serde_regex.q,show_partitions1.q,check_constraint_nonboolean_expr.q,invalid_cast_from_binary_6.q,create_with_multi_pk_constraint.q,udf_field_wrong_type.q,groupby_grouping_sets4.q,groupby_grouping_sets3.q,insertsel_fail.q,udf_locate_wrong_type.q,orc_type_promotion1_acid.q,set_table_property.q,create_or_replace_view2.q,groupby_grouping_sets2.q,alter_view_failure.q,distinct_windowing_failure1.q,invalid_t_alter2.q,alter_table_constraint_invalid_fk_col1.q,invalid_varchar_length_2.q,authorization_show_grant_otheruser_alltabs.q,subquery_windowing_corr.q,compact_non_acid_table.q,authorization_view_4.q,authorization_disallow_transform.q,materialized_view_authorization_rebuild_other.q,authorization_fail_4.q,dbtxnmgr_nodblock.q,set_hiveconf_internal_variable1.q,input_part0_neg.q,udf_printf_wrong3.q,load_orc_negative2.q,druid_buckets.q,archive2.q,authorization_addjar.q,invalid_sum_syntax.q,insert_into_with_schema1.q,udf_add_months_error_2.q,dyn_part_max_per_node.q,authorization_revoke_table_fail1.q,udf_printf_wrong2.q,archive_multi3.q,udf_printf_wrong1.q,subquery_subquery_chain.q,authorization_view_disable_cbo_4.q,no_matching_udf.q,create_view_failure7.q,drop_native_udf.q,truncate_column_list_bucketing.q,authorization_uri_add_partition.q,authorization_view_disable_cbo_3.q,bad_exec_hooks.q,authorization_view_disable_cbo_2.q,fetchtask_ioexception.q,char_pad_convert_fail2.q,authorization_set_role_neg1.q,serde_regex3.q,authorization_delete_nodeletepriv.q,materialized_view_delete.q,create_or_replace_view6.q,bucket_mapjoin_wrong_table_metadata_2.q,udf_sort_array_by_wrong2.
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441137#comment-16441137
]
Ashutosh Chauhan commented on HIVE-19164:
-
+1
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-19164.02.patch, HIVE-19164.patch
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19164) TestMetastoreVersion failures
[
https://issues.apache.org/jira/browse/HIVE-19164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445352#comment-16445352
]
Vineet Garg commented on HIVE-19164:
Pushed to branch-3 as well since this fixes couple of metastore tests
> TestMetastoreVersion failures
> -
>
> Key: HIVE-19164
> URL: https://issues.apache.org/jira/browse/HIVE-19164
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Vineet Garg
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Fix For: 3.0.0, 3.1.0
>
> Attachments: HIVE-19164.02.patch, HIVE-19164.patch
>
>
> Following tests are failing consistently and are reproducible on master:
> * testVersionMatching
> * testMetastoreVersion
> I tried debugging it and found that ObjectStore.getMSSchemaVersion() throws
> an exception {{No matching version found}}.
> To fetch schema version this method executes {code:sql} SELECT FROM
> org.apache.hadoop.hive.metastore.model.MVersionTable {code} but for whatever
> reason execution returns empty result set resulting in the exception. Both
> test failures are due to this exception. I tried debugging the query
> execution but didn't really go nowhere with it. I suspect this could be due
> to recent metastore changes. I tried reproducing this outside test but with
> no success.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
