Re: Hive On Spark - Using custom SerDe

2015-11-16 Thread Jimmy Xiang
It is better to use safety valve for hive configuration. On Mon, Nov 16, 2015 at 8:03 AM, Daniel Haviv < daniel.ha...@veracity-group.com> wrote: > Hi, > How should I set it ? just a normal set in hive or add it via the safety > valve to the hive or sparks configuartion? > > Thank you. > Daniel >

Re: Hive On Spark - Using custom SerDe

2015-11-16 Thread Daniel Haviv
Hi, How should I set it ? just a normal set in hive or add it via the safety valve to the hive or sparks configuartion? Thank you. Daniel On Mon, Nov 16, 2015 at 5:46 PM, Jimmy Xiang wrote: > Have you add your class to "spark.kryo.classesToRegister"? You also need > to make sure your jar is in

Re: Hive On Spark - Using custom SerDe

2015-11-16 Thread Jimmy Xiang
Have you add your class to "spark.kryo.classesToRegister"? You also need to make sure your jar is in ""hive.aux.jars.path". Thanks, Jimmy On Mon, Nov 16, 2015 at 1:44 AM, Daniel Haviv < daniel.ha...@veracity-group.com> wrote: > Hi, > We have a custom SerDe we would like to use with Hive on Spark

Hive On Spark - Using custom SerDe

2015-11-16 Thread Daniel Haviv
Hi, We have a custom SerDe we would like to use with Hive on Spark but I'm not sure how to. The error messages are pretty clear about the fact that it can't find my SerDE's class: Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find class: com.mycompany.hive.WhaleAvro