Michal Ferlinski created HIVE-15995:
---------------------------------------
Summary: Syncing metastore table with serde schema
Key: HIVE-15995
URL: https://issues.apache.org/jira/browse/HIVE-15995
Project: Hive
Issue Type: Bug
Reporter: Michal Ferlinski
Hive enables table schema evolution via properties. For avro e.g. we could
alter the 'avro.schema.url' property to update table schema to the next
version. Updating properties however doesn't affect column list stored in
metastore DB so the table is not in the newest version when returned from
metastore API. This is problem for tools working with metastore (e.g. Presto).
To solve this issue I suggest to introduce new DDL statement syncing metastore
columns with those from serde:
{code}
ALTER TABLE user_test1 UPDATE COLUMNS
{code}
This will be serde format independent solution.
Let me know what you think. Patch is in progress.
To reproduce, follow the instructions below:
- Create table based on avro schema version 1 (cxv1.avsc)
{code}
CREATE EXTERNAL TABLE user_test1
PARTITIONED BY (dt string)
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION
'/tmp/schema-evolution/user_test1'
TBLPROPERTIES ('avro.schema.url'='/tmp/schema-evolution/cx1.avsc');
{code}
- Update schema to version 2 (cx2.avsc)
{code}
ALTER TABLE user_test1 SET TBLPROPERTIES ('avro.schema.url' =
'/tmp/schema-evolution/cx2.avsc');
{code}
- Print serde columns (top info) and metastore columns (Detailed Table
Information):
{code}
DESCRIBE EXTENDED user_test1
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
