Is there a way to access Hive UDFs in a HiveContext?
I'm trying to access a permanent Hive UDF, scala> val data = hc.sql("select func.md5(some_string) from some_table") data: org.apache.spark.sql.SchemaRDD = SchemaRDD[19] at RDD at SchemaRDD.scala:108 == Query Plan == == Physical Plan == java.lang.RuntimeException: Couldn't find function func.md5 Version: Spark 1.2 on CDH 5.3. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Is-there-a-way-to-access-Hive-UDFs-in-a-HiveContext-tp21510.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: Is there a way to access Hive UDFs in a HiveContext?
Hi, My guess is that Spark is not picking up the jar where the function is stored. You might have to add it to sparkcontext or the classpath manually. You can also register the function hc.registerFunction("myfunct", myfunct) then use it in the query. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Is-there-a-way-to-access-Hive-UDFs-in-a-HiveContext-tp21510p21527.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org