Upgrade to CDH 5.5 for spark. It should work On Sat, Jan 9, 2016 at 12:17 AM, Ophir Etzion <op...@foursquare.com> wrote:
> It didn't work. assuming I did the right thing. > in the properties you could see > > {"key":"hive.aux.jars.path","value":"file:///data/loko/foursquare.web-hiverc/current/hadoop-hive-serde.jar,file:///data/loko/foursquare.web-hiverc/current/hadoop-hive-udf.jar","isFinal":false,"resource":"programatically"} > which includes the jar that has the class I need but I still get > > org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find > class: com.foursquare.hadoop.hive.io.HiveThriftSequenceFileInputFormat > > > > On Fri, Jan 8, 2016 at 12:24 PM, Edward Capriolo <edlinuxg...@gmail.com> > wrote: > >> You can not 'add jar' input formats and serde's. They need to be part of >> your auxlib. >> >> On Fri, Jan 8, 2016 at 12:19 PM, Ophir Etzion <op...@foursquare.com> >> wrote: >> >>> I tried now. still getting >>> >>> 16/01/08 16:37:34 ERROR exec.Utilities: Failed to load plan: >>> hdfs://hadoop-alidoro-nn-vip/tmp/hive/hive/c2af9882-38a9-42b0-8d17-3f56708383e8/hive_2016-01-08_16-36-41_370_3307331506800215903-3/-mr-10004/3c90a796-47fc-4541-bbec-b196c40aefab/map.xml: >>> org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find >>> class: com.foursquare.hadoop.hive.io.HiveThriftSequenceFileInputFormat >>> Serialization trace: >>> inputFileFormatClass (org.apache.hadoop.hive.ql.plan.PartitionDesc) >>> aliasToPartnInfo (org.apache.hadoop.hive.ql.plan.MapWork) >>> org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find >>> class: com.foursquare.hadoop.hive.io.HiveThriftSequenceFileInputFormat >>> >>> >>> HiveThriftSequenceFileInputFormat is in one of the jars I'm trying to add. >>> >>> >>> On Thu, Jan 7, 2016 at 9:58 PM, Prem Sure <premsure...@gmail.com> wrote: >>> >>>> did you try -- jars property in spark submit? if your jar is of huge >>>> size, you can pre-load the jar on all executors in a common available >>>> directory to avoid network IO. >>>> >>>> On Thu, Jan 7, 2016 at 4:03 PM, Ophir Etzion <op...@foursquare.com> >>>> wrote: >>>> >>>>> I' trying to add jars before running a query using hive on spark on >>>>> cdh 5.4.3. >>>>> I've tried applying the patch in >>>>> https://issues.apache.org/jira/browse/HIVE-12045 (manually as the >>>>> patch is done on a different hive version) but still hasn't succeeded. >>>>> >>>>> did anyone manage to do ADD JAR successfully with CDH? >>>>> >>>>> Thanks, >>>>> Ophir >>>>> >>>> >>>> >>> >> >