Matt McCline created HIVE-13515: ----------------------------------- Summary: Semantics of TypeInfoUtils.areColTypesCompatible changed which also changed the semantics of hive.metastore.disallow.incompatible.col.type.changes of Key: HIVE-13515 URL: https://issues.apache.org/jira/browse/HIVE-13515 Project: Hive Issue Type: Bug Components: Hive Affects Versions: 2.1.0 Reporter: Matt McCline Assignee: Ashutosh Chauhan Priority: Critical
I had this test case in my new Schema Evolution patch (not checked in yet) that verifies STRING_GROUP to DECIMAL conversion: {code} -- SUBSECTION: ALTER TABLE CHANGE COLUMNS for STRING_GROUP -> DECIMAL: (STRING, CHAR, VARCHAR) -- CREATE TABLE table_change_string_group_decimal(insert_num int, c1 STRING, c2 CHAR(50), c3 VARCHAR(50), b STRING); insert into table table_change_string_group_decimal values (1, '753.7028', '753.7028', '753.7028', 'original'), (2, '-3651.672121', '-3651.672121', '-3651.672121', 'original'), (3, '-29.0764', '-29.0764', '-29.0764', 'original'), (4, '-10.3', '-10.3', '-10.3', 'original'); -- Table-Non-Cascade CHANGE COLUMNS ... alter table table_change_string_group_decimal replace columns (insert_num int, c1 decimal(38,18), c2 decimal(38,18), c3 decimal(38,18), b STRING); insert into table table_change_string_group_decimal values (5, 30.774, 30.774, 30.774, 'new'), (6, 20.31, 20.31, 20.31, 'new'), (7, 46114.284799488, 46114.284799488, 46114.284799488, 'new'), (8, -66475.561431, -66475.561431, -66475.561431, 'new'); insert into table table_change_string_group_decimal values (9, 17808.963785, 17808.963785, 17808.963785, 'new'), (10, 9250340.75 , 9250340.75 , 9250340.75 , 'new'); explain select insert_num,c1,c2,c3,b from table_change_string_group_decimal order by insert_num; select insert_num,c1,c2,c3,b from table_change_string_group_decimal order by insert_num; drop table table_change_string_group_decimal; {code} that now fails on master (I suspect because of HIVE-13380 Decimal should have lower precedence than double in type hierachy) when: {code} set hive.metastore.disallow.incompatible.col.type.changes=true; {code} with this call stack: {code} Caused by: InvalidOperationException(message:The following columns have types incompatible with the existing columns in their respective positions : c1,c2,c3) at org.apache.hadoop.hive.metastore.MetaStoreUtils.throwExceptionIfIncompatibleColTypeChange(MetaStoreUtils.java:599) at org.apache.hadoop.hive.metastore.HiveAlterHandler.alterTable(HiveAlterHandler.java:132) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table_core(HiveMetaStore.java:3502) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_table_with_environment_context(HiveMetaStore.java:3473) {code} Key point is MetaStoreUtils.throwExceptionIfIncompatibleColTypeChange is calling TypeInfoUtils.areColTypesCompatible which changed - // Allow implicit String to Decimal conversion - if (fromPg == PrimitiveGrouping.STRING_GROUP && to == PrimitiveCategory.DECIMAL) { - return true; - } Will not current customers that have scripts with hive.metastore.disallow.incompatible.col.type.changes=true that do Schema Evolution get bitten? -- This message was sent by Atlassian JIRA (v6.3.4#6332)