[ https://issues.apache.org/jira/browse/GOBBLIN-877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zihan Li resolved GOBBLIN-877. ------------------------------ Resolution: Fixed > Add column metadata for partition for inline hive registration > -------------------------------------------------------------- > > Key: GOBBLIN-877 > URL: https://issues.apache.org/jira/browse/GOBBLIN-877 > Project: Apache Gobblin > Issue Type: Task > Reporter: Zihan Li > Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > > Previously, we remove the schema.literal for partition. Because Avro schemas > should _only_ be defined at the table level. Hive overrides table properties > if the same property is defined on the partition. Defining them at the > partition level may lead to partitions with inconsistent schemas. And because > column metadata is calculated from schema.literal, so we remove the column > metadata as well. > Then we encounter a problem that presto cannot read data from orc file. > Because ORC (and other Hive serdes) need metadata in the partitions so that > coercion can be done between a partition schema and the table schema. > So we need to treat Avro and other formate separately to make sure hive > registration works well so that user can read right data from Presto. -- This message was sent by Atlassian Jira (v8.3.4#803005)