Craig Condit created HIVE-7502:
----------------------------------
Summary: Writes to parquet tables via HCatalog fail with
"java.lang.RuntimeException: Should never be used".
Key: HIVE-7502
URL: https://issues.apache.org/jira/browse/HIVE-7502
Project: Hive
Issue Type: Bug
Components: HCatalog
Reporter: Craig Condit
Attempting to store data into a Parquet table via HCatalog (using Pig) fails.
Based on the code, it appears that
org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat.getRecordWriter()
is unimplemented.
To duplicate:
Hive:
{noformat}
create table parquet_input_test stored as parquet as with x as (select test as
value) select * from x;
create table parquet_output_test (value int) stored as parquet;
{noformat}
Pig:
{noformat}
A = LOAD 'parquet_input_test' USING org.apache.hive.hcatalog.pig.HCatLoader();
STORE A INTO 'parquet_output_test' USING
org.apache.hive.hcatalog.pig.HCatStorer();
{noformat}
Result:
{noformat}
java.lang.RuntimeException: Should never be used
at
org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat.getRecordWriter(MapredParquetOutputFormat.java:76)
at
org.apache.hive.hcatalog.mapreduce.FileOutputFormatContainer.getRecordWriter(FileOutputFormatContainer.java:103)
at
org.apache.hive.hcatalog.mapreduce.HCatOutputFormat.getRecordWriter(HCatOutputFormat.java:260)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:84)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.<init>(MapTask.java:623)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:743)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
