Re: adding jars - hive on spark cdh 5.4.3

2016-01-12 Thread Ophir Etzion
that? (cdh 5.5 is the same hive version as cdh5.4. is it spark related and not hive?) On Sun, Jan 10, 2016 at 9:26 AM, sandeep vura <sandeepv...@gmail.com> wrote: > Upgrade to CDH 5.5 for spark. It should work > > On Sat, Jan 9, 2016 at 12:17 AM, Ophir Etzion <op...@fours

Re: adding jars - hive on spark cdh 5.4.3

2016-01-08 Thread Ophir Etzion
Jan 8, 2016 at 12:24 PM, Edward Capriolo <edlinuxg...@gmail.com> wrote: > You can not 'add jar' input formats and serde's. They need to be part of > your auxlib. > > On Fri, Jan 8, 2016 at 12:19 PM, Ophir Etzion <op...@foursquare.com> > wrote: > >> I tried now. sti

Re: adding jars - hive on spark cdh 5.4.3

2016-01-08 Thread Ophir Etzion
property in spark submit? if your jar is of huge size, > you can pre-load the jar on all executors in a common available directory > to avoid network IO. > > On Thu, Jan 7, 2016 at 4:03 PM, Ophir Etzion <op...@foursquare.com> wrote: > >> I' trying to add jars before runnin

adding jars - hive on spark cdh 5.4.3

2016-01-07 Thread Ophir Etzion
I' trying to add jars before running a query using hive on spark on cdh 5.4.3. I've tried applying the patch in https://issues.apache.org/jira/browse/HIVE-12045 (manually as the patch is done on a different hive version) but still hasn't succeeded. did anyone manage to do ADD JAR successfully

hive on spark

2015-12-18 Thread Ophir Etzion
During spark-submit when running hive on spark I get: Exception in thread "main" java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.hdfs.HftpFileSystem could not be instantiated Caused by: java.lang.IllegalAccessError: tried to access method

Re: Hive on Spark - Error: Child process exited before connecting back

2015-12-15 Thread Ophir Etzion
mation in this > message shall not be understood as given or endorsed by Peridale Technology > Ltd, its subsidiaries or their employees, unless expressly so stated. It is > the responsibility of the recipient to ensure that this email is virus > free, therefore neither Peridale Ltd, its su

Hive on Spark - Error: Child process exited before connecting back

2015-12-15 Thread Ophir Etzion
Hi, when trying to do Hive on Spark on CDH5.4.3 I get the following error when trying to run a simple query using spark. I've tried setting everything written here ( https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started) as well as what the cdh recommends. any one