RE: Using Map and Basic Operators yield java.lang.ClassCastException (Parquet + Hive + Spark SQL 1.5.0 + Thrift)

2015-09-28 Thread Dominic Ricard
[mailto:lian.cs....@gmail.com] > Sent: Thursday, September 24, 2015 5:47 PM > To: Dominic Ricard; user@spark.apache.org > Subject: Re: Using Map and Basic Operators yield java.lang.ClassCastException > (Parquet + Hive + Spark SQL 1.5.0 + Thrift) > > > > On 9/24/15 11:34 AM, Domi

Re: Using Map and Basic Operators yield java.lang.ClassCastException (Parquet + Hive + Spark SQL 1.5.0 + Thrift)

2015-09-25 Thread Cheng Lian
ndling INT is all good but float and double are causing the exception. Thanks. Dominic Ricard Triton Digital -Original Message- From: Cheng Lian [mailto:lian.cs@gmail.com] Sent: Thursday, September 24, 2015 5:47 PM To: Dominic Ricard; user@spark.apache.org Subject: Re: Using Map a

Using Map and Basic Operators yield java.lang.ClassCastException (Parquet + Hive + Spark SQL 1.5.0 + Thrift)

2015-09-24 Thread Dominic Ricard
Hi, I stumbled on the following today. We have Parquet files that expose a column in a Map format. This is very convenient as we have data parts that can vary in time. Not knowing what the data will be, we simply split it in tuples and insert it as a map inside 1 column. Retrieving the data is

RE: Using Map and Basic Operators yield java.lang.ClassCastException (Parquet + Hive + Spark SQL 1.5.0 + Thrift)

2015-09-24 Thread Dominic Ricard
Thursday, September 24, 2015 5:47 PM To: Dominic Ricard; user@spark.apache.org Subject: Re: Using Map and Basic Operators yield java.lang.ClassCastException (Parquet + Hive + Spark SQL 1.5.0 + Thrift) On 9/24/15 11:34 AM, Dominic Ricard wrote: > Hi, > I stumbled on the following today.