Re: Using Hive UDF in spark

2015-07-08 Thread ayan guha
You are most likely confused because you are using the UDF using HiveContext. In your case, you are using Spark UDF, not Hive UDF. For a naive scenario, I can use spark UDFs without any hive installation in my cluster. sqlContext.udf.register is for UDF in spark. Hive UDFs are stored in Hive and y

Using Hive UDF in spark

2015-07-08 Thread vinod kumar
Hi everyone Shall we use UDF defined in hive using spark sql? I've created a UDF in spark and registered it using sqlContext.udf.register,but when I restarted a service the UDF was not available. I've heared that Hive UDF's are permanently stored in hive.(Please Correct me if I am wrong). Thanks