Did you tried this with spark-shell?
Please try this.
$spark-shell --jars /home/cloudera/Downloads/genudnvl2.jar
On the spark shell:
val hc = new org.apache.spark.sql.hive.HiveContext(sc) ;
hc.sql("create temporary function nexr_nvl2 as '
com.nexr.platform.hive.udf.GenericUDFNVL2'");
hc.sql("select nexr_nvl2(name,let,ret) from testtab5").show;
This should work.
So basically in your spark program , you can specify the jars while
submitting the job using spark-submit.
There is no need to have add jars statement in the spark program itself.
Thanks
Deepak
On Wed, Jan 18, 2017 at 8:58 AM, Sirisha Cheruvu <siri8...@gmail.com> wrote:
> This error
> org.apache.spark.sql.AnalysisException: No handler for Hive udf class
> com.nexr.platform.hive.udf.GenericUDFNVL2 because:
> com.nexr.platform.hive.udf.GenericUDFNVL2.; line 1 pos 26
> at org.apache.spark.sql.hive.HiveFunctionRegistry$$anonfun$look
> upFunction$2.apply(hiveUDFs.scala:105)
> at org.apache.spark.sql.hive.HiveFunctionRegistry$$anonfun$look
> upFunction$2.apply(hiveUDFs.scala:64)
> at scala.util.Try.getOrElse(Try.scala:77)
>
>
>
>
>
> My script is this
>
>
> import org.apache.spark.sql.hive.HiveContext
> val hc = new org.apache.spark.sql.hive.HiveContext(sc) ;
> hc.sql("add jar /home/cloudera/Downloads/genudnvl2.jar");
> hc.sql("create temporary function nexr_nvl2 as '
> com.nexr.platform.hive.udf.GenericUDFNVL2'");
> hc.sql("select nexr_nvl2(name,let,ret) from testtab5").show;
> System.exit(0);
>
>
> On Jan 17, 2017 2:01 PM, "Sirisha Cheruvu" <siri8...@gmail.com> wrote:
>
>> Hi
>>
>> Anybody has a test and tried generic udf with object inspector
>> implementaion which sucessfully ran on both hive and spark-sql
>>
>> please share me the git hub link or source code file
>>
>> Thanks in advance
>> Sirisha
>>
>
--
Thanks
Deepak
www.bigdatabig.com
www.keosha.net
