Re: ArrayWritableGroupConverter
Anyone know anything about this? We have our errors stored in this data
structure, so it's hard to debug when we can't analyze our errors :\
Shameless bump.
On Thu, Aug 14, 2014 at 10:43 AM, Raymond Lau r...@ooyala.com wrote:
Original Thrift schema:
struct teststruct {
1: optional string field1;
2: optional string field2;
3: optional string field3;
}
struct mainstruct {
1: optional listteststruct teststructs;
}
This parquet file schema was generated:
message ParquetSchema {
optional group teststructs {
repeated group teststruct_tuple {
optional binary field1;
optional binary field2;
optional binary field3;
}
}
}
When i try to run queries involving this 'teststructs' column, I get this
error:
Failed with exception java.io.IOException:java.lang.RuntimeException:
Invalid parquet hive schema: repeated group teststruct_tuple {
optional binary field1;
optional binary field2;
optional binary field3;
}
On Thu, Aug 14, 2014 at 8:35 AM, Brock Noland br...@cloudera.com wrote:
Hi,
Can you share your parquet schema?
Brock
On Tue, Aug 12, 2014 at 5:06 PM, Raymond Lau raymond.lau...@gmail.com
wrote:
Hello. (First off, sorry if I accidentally posted to the wrong mailing
list before - dev - and you are getting this again)
Regarding the ArrayWritableGroupConverter class: I was just wondering
how come the field count has to be either 1 or 2? I'm trying to read a
column where the amount is fields is 3 and I'm getting an invalid parquet
hive schema (in hive 0.12) error when I try to do so. It looks like it
links back to here.
*https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ArrayWritableGroupConverter.java
https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ArrayWritableGroupConverter.java*
Thanks,
-Raymond
--
*Raymond Lau*
Software Engineer - Intern |
r...@ooyala.com | (925) 395-3806
--
*Raymond Lau*
Software Engineer - Intern |
r...@ooyala.com | (925) 395-3806
Re: ArrayWritableGroupConverter
Hi, Can you share your parquet schema? Brock On Tue, Aug 12, 2014 at 5:06 PM, Raymond Lau raymond.lau...@gmail.com wrote: Hello. (First off, sorry if I accidentally posted to the wrong mailing list before - dev - and you are getting this again) Regarding the ArrayWritableGroupConverter class: I was just wondering how come the field count has to be either 1 or 2? I'm trying to read a column where the amount is fields is 3 and I'm getting an invalid parquet hive schema (in hive 0.12) error when I try to do so. It looks like it links back to here. *https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ArrayWritableGroupConverter.java https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ArrayWritableGroupConverter.java* Thanks, -Raymond
Re: ArrayWritableGroupConverter
Original Thrift schema:
struct teststruct {
1: optional string field1;
2: optional string field2;
3: optional string field3;
}
struct mainstruct {
1: optional listteststruct teststructs;
}
This parquet file schema was generated:
message ParquetSchema {
optional group teststructs {
repeated group teststruct_tuple {
optional binary field1;
optional binary field2;
optional binary field3;
}
}
}
When i try to run queries involving this 'teststructs' column, I get this
error:
Failed with exception java.io.IOException:java.lang.RuntimeException:
Invalid parquet hive schema: repeated group teststruct_tuple {
optional binary field1;
optional binary field2;
optional binary field3;
}
On Thu, Aug 14, 2014 at 8:35 AM, Brock Noland br...@cloudera.com wrote:
Hi,
Can you share your parquet schema?
Brock
On Tue, Aug 12, 2014 at 5:06 PM, Raymond Lau raymond.lau...@gmail.com
wrote:
Hello. (First off, sorry if I accidentally posted to the wrong mailing
list before - dev - and you are getting this again)
Regarding the ArrayWritableGroupConverter class: I was just wondering how
come the field count has to be either 1 or 2? I'm trying to read a column
where the amount is fields is 3 and I'm getting an invalid parquet hive
schema (in hive 0.12) error when I try to do so. It looks like it links
back to here.
*https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ArrayWritableGroupConverter.java
https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ArrayWritableGroupConverter.java*
Thanks,
-Raymond
--
*Raymond Lau*
Software Engineer - Intern |
r...@ooyala.com | (925) 395-3806
