Re: Hive Parquet Reader and "repeated" field

2014-11-11 Thread Jean-Pascal Billaud
Hey Ryan, I take therefore that parquet-thrift structure using list/set/maps are not supported with hive as of today. Regarding the patch that I posted, since I need to make it work for my deployment regardless, does the approach make sense so far? I still need to hack into the ObjInspector so th

Re: Hive Parquet Reader and "repeated" field

2014-11-11 Thread Ryan Blue
On 11/11/2014 01:07 PM, Jean-Pascal Billaud wrote: While running "select * from parquet_requests", the whole thing crashes with the following exception: > public ArrayWritableGroupConverter(final GroupType groupType, final HiveGroupConverter parent, >final int index) { > this.pare

Hive Parquet Reader and "repeated" field

2014-11-11 Thread Jean-Pascal Billaud
Hi, I am trying to integrate parquet as the underlying storage format in our data pipeline but I am facing some issues which I hope some of you can help me with. The batch layer is fairly standard, some cascading write thrift log objecs from an input tap to a parquet output sink. As a snippet of