[
https://issues.apache.org/jira/browse/PARQUET-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Agarwal updated PARQUET-53:
------------------------------------
Description:
ArrayWritableGroupConverter in Parquet-Hive binding expects that any array
field contains either 1 or 2 fields. However, the schema which
PigSchemaConverter generates, doesn't honour this condition. It can be
reproduced through following code
{noformat}
MessageType messageType = new
PigSchemaConverter().convert(ThriftToPig.toSchema(Requestlist.class));
DataWritableGroupConverter converter = new
DataWritableGroupConverter(messageType, messageType);
{noformat}
Requestlist is a thrift class
{noformat}
struct Request {
1: required string id,
2: i64 time,
3: required string source
}
struct Requestlist {
1: optional list<Request> requests
}
{noformat}
was:
ArrayWritableGroupConverter in Parquet-Hive binding expects that any array
field contains either 1 or 2 fields. However, the schema which
PigSchemaConverter generates, doesn't honour this condition. It can be
reproduced through following code
{noformat}
MessageType messageType = new
PigSchemaConverter().convert(ThriftToPig.toSchema(Attributes.class));
DataWritableGroupConverter converter = new
DataWritableGroupConverter(messageType, messageType);
{noformat}
Attributes is a thrift class
{noformat}
struct Request {
1: required string id,
2: i64 time,
3: required string source
}
struct Requestlist {
1: optional list<Request> requests
}
{noformat}
> Error in reading a list in hive which was generated through pig
> ----------------------------------------------------------------
>
> Key: PARQUET-53
> URL: https://issues.apache.org/jira/browse/PARQUET-53
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Environment: Parquet-Pig - 1.3.2
> Hive - 0.13.1
> Elephant-Bird - 4.4
> Reporter: Abhishek Agarwal
>
> ArrayWritableGroupConverter in Parquet-Hive binding expects that any array
> field contains either 1 or 2 fields. However, the schema which
> PigSchemaConverter generates, doesn't honour this condition. It can be
> reproduced through following code
> {noformat}
> MessageType messageType = new
> PigSchemaConverter().convert(ThriftToPig.toSchema(Requestlist.class));
> DataWritableGroupConverter converter = new
> DataWritableGroupConverter(messageType, messageType);
> {noformat}
> Requestlist is a thrift class
> {noformat}
> struct Request {
> 1: required string id,
> 2: i64 time,
> 3: required string source
> }
> struct Requestlist {
> 1: optional list<Request> requests
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
