Thanks! In certain use cases you could but forgot about the aux thing, thats probably it.
On Fri, Jan 8, 2016 at 12:24 PM, Edward Capriolo <edlinuxg...@gmail.com> wrote: > You can not 'add jar' input formats and serde's. They need to be part of > your auxlib. > > On Fri, Jan 8, 2016 at 12:19 PM, Ophir Etzion <op...@foursquare.com> > wrote: > >> I tried now. still getting >> >> 16/01/08 16:37:34 ERROR exec.Utilities: Failed to load plan: >> hdfs://hadoop-alidoro-nn-vip/tmp/hive/hive/c2af9882-38a9-42b0-8d17-3f56708383e8/hive_2016-01-08_16-36-41_370_3307331506800215903-3/-mr-10004/3c90a796-47fc-4541-bbec-b196c40aefab/map.xml: >> org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find >> class: com.foursquare.hadoop.hive.io.HiveThriftSequenceFileInputFormat >> Serialization trace: >> inputFileFormatClass (org.apache.hadoop.hive.ql.plan.PartitionDesc) >> aliasToPartnInfo (org.apache.hadoop.hive.ql.plan.MapWork) >> org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find >> class: com.foursquare.hadoop.hive.io.HiveThriftSequenceFileInputFormat >> >> >> HiveThriftSequenceFileInputFormat is in one of the jars I'm trying to add. >> >> >> On Thu, Jan 7, 2016 at 9:58 PM, Prem Sure <premsure...@gmail.com> wrote: >> >>> did you try -- jars property in spark submit? if your jar is of huge >>> size, you can pre-load the jar on all executors in a common available >>> directory to avoid network IO. >>> >>> On Thu, Jan 7, 2016 at 4:03 PM, Ophir Etzion <op...@foursquare.com> >>> wrote: >>> >>>> I' trying to add jars before running a query using hive on spark on cdh >>>> 5.4.3. >>>> I've tried applying the patch in >>>> https://issues.apache.org/jira/browse/HIVE-12045 (manually as the >>>> patch is done on a different hive version) but still hasn't succeeded. >>>> >>>> did anyone manage to do ADD JAR successfully with CDH? >>>> >>>> Thanks, >>>> Ophir >>>> >>> >>> >> >
