[
https://issues.apache.org/jira/browse/HIVE-8950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238177#comment-14238177
]
Ashish Kumar Singh commented on HIVE-8950:
------------------------------------------
[~brocknoland], [~rdblue], need suggestion on dealing with test failures.
The patch now addresses all the review comments and adds required
functionality. However, I am fighting with qtest failures to avoid changing
existing .q.out files. Below are my findings. Kindly suggest what you guys
think should be done here.
Possible ways to pass down table location to SerDe.
1. Add table path to table.sd.location.
This led to 38 failures. HbaseStorageHandler checks if table.sd.location is
null or not. If not, throws an MetaException.
2. Add table path to table.sd.serdeinfo.location
158 failures, as serdeinfo has location set. Bunch of qfiles have serdeinfo
printed.
3. Add table path to table.properties
56 failures, as table properties has location set. Bunch of qfiles have table
properties printed.
Possible solutions I can think of.
a. Modify .q.out's for failing tests, Will only add "#### A masked pattern was
here ####" at a few places in existing .q.out.
b. Guard setting of path in (1) by checking for SerDe class name. Will require
importing ParquetHiveSerDe to a SerDe agnostic class, Hive.
c. Add method, needTablePathInInitialize(), to SerDe. Hive class can then get
this method using reflection and use it to determine if table path needs to be
added to table or not. This will solve the problem in (b) of importing specific
SerDe's in Hive class. However, reflections are costly.
> Add support in ParquetHiveSerde to create table schema from a parquet file
> --------------------------------------------------------------------------
>
> Key: HIVE-8950
> URL: https://issues.apache.org/jira/browse/HIVE-8950
> Project: Hive
> Issue Type: Improvement
> Reporter: Ashish Kumar Singh
> Assignee: Ashish Kumar Singh
> Attachments: HIVE-8950.1.patch, HIVE-8950.2.patch, HIVE-8950.3.patch,
> HIVE-8950.patch
>
>
> PARQUET-76 and PARQUET-47 ask for creating parquet backed tables without
> having to specify the column names and types. As, parquet files store schema
> in their footer, it is possible to generate hive schema from parquet file's
> metadata. This will improve usability of parquet backed tables.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
