[ https://issues.apache.org/jira/browse/HIVE-15995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adam Szita updated HIVE-15995: ------------------------------ Status: Patch Available (was: In Progress) > Syncing metastore table with serde schema > ----------------------------------------- > > Key: HIVE-15995 > URL: https://issues.apache.org/jira/browse/HIVE-15995 > Project: Hive > Issue Type: Bug > Components: Metastore > Affects Versions: 2.1.0, 1.2.1, 3.0.0 > Reporter: Michal Ferlinski > Assignee: Adam Szita > Priority: Major > Attachments: HIVE-15995.1.patch, HIVE-15995.2.patch, > HIVE-15995.3.patch, HIVE-15995.4.patch, HIVE-15995.patch, cx1.avsc, cx2.avsc > > > Hive enables table schema evolution via properties. For avro e.g. we can > alter the 'avro.schema.url' property to update table schema to the next > version. Updating properties however doesn't affect column list stored in > metastore DB so the table is not in the newest version when returned from > metastore API. This is problem for tools working with metastore (e.g. Presto). > To solve this issue I suggest to introduce new DDL statement syncing > metastore columns with those from serde: > {code} > ALTER TABLE user_test1 UPDATE COLUMNS > {code} > Note that this is format independent solution. > To reproduce, follow the instructions below: > - Create table based on avro schema version 1 (cxv1.avsc) > {code} > CREATE EXTERNAL TABLE user_test1 > PARTITIONED BY (dt string) > ROW FORMAT SERDE > 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' > STORED AS INPUTFORMAT > 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' > OUTPUTFORMAT > 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat' > LOCATION > '/tmp/schema-evolution/user_test1' > TBLPROPERTIES ('avro.schema.url'='/tmp/schema-evolution/cx1.avsc'); > {code} > - Update schema to version 2 (cx2.avsc) > {code} > ALTER TABLE user_test1 SET TBLPROPERTIES ('avro.schema.url' = > '/tmp/schema-evolution/cx2.avsc'); > {code} > - Print serde columns (top info) and metastore columns (Detailed Table > Information): > {code} > DESCRIBE EXTENDED user_test1 > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)