[ https://issues.apache.org/jira/browse/SPARK-18832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lokesh Yadav updated SPARK-18832: --------------------------------- Description: Spark Thriftserver is unable to run a HiveUDTF. It throws the error that it is unable to find the functions although the function registration succeeds and the funtions does show up in the list output by {{show functions}}. I am using a Hive UDTF, registering it using a jar placed on my local machine. Calling it using the following commands: //Registering the functions, this command succeeds. {{CREATE FUNCTION SampleUDTF AS 'com.fuzzylogix.experiments.udf.hiveUDF.SampleUDTF' USING JAR '/root/spark_files/experiments-1.2.jar';}} //Thriftserver is able to look up the functuion, on this command: {{DESCRIBE FUNCTION SampleUDTF;}} {quote} {noformat} Output: +-----------------------------------------------------------+--+ | function_desc | +-----------------------------------------------------------+--+ | Function: default.SampleUDTF | | Class: com.fuzzylogix.experiments.udf.hiveUDF.SampleUDTF | | Usage: N/A. | +-----------------------------------------------------------+--+ {noformat} {quote} // Calling the function: {{SELECT SampleUDTF('Paris');}} bq. Output of the above command: Error: org.apache.spark.sql.AnalysisException: Undefined function: 'SampleUDTF'. This function is neither a registered temporary function nor a permanent function registered in the database 'default'.; line 1 pos 7 (state=,code=0) I have also tried with using a non-local (on hdfs) jar, but I get the same error. My environment: HDP 2.4 with spark 2.00 was: Spark Thriftserver is unable to run a HiveUDTF. It throws the error that it is unable to find the functions although the function registration succeeds and the funtions does show up in the list output by {{show functions}}. I am using a Hive UDTF, registering it using a jar placed on my local machine. Calling it using the following commands: //Registering the functions, this command succeeds. {{CREATE FUNCTION SampleUDTF AS 'com.fuzzylogix.experiments.udf.hiveUDF.SampleUDTF' USING JAR '/root/spark_files/experiments-1.2.jar';}} //Thriftserver is able to look up the functuion, on this command: {{DESCRIBE FUNCTION SampleUDTF;}} {quote} Output: +-----------------------------------------------------------+--+ | function_desc | +-----------------------------------------------------------+--+ | Function: default.SampleUDTF | | Class: com.fuzzylogix.experiments.udf.hiveUDF.SampleUDTF | | Usage: N/A. | +-----------------------------------------------------------+--+ {quote} // Calling the function: {{SELECT SampleUDTF('Paris');}} bq. Output of the above command: Error: org.apache.spark.sql.AnalysisException: Undefined function: 'SampleUDTF'. This function is neither a registered temporary function nor a permanent function registered in the database 'default'.; line 1 pos 7 (state=,code=0) I have also tried with using a non-local (on hdfs) jar, but I get the same error. My environment: HDP 2.4 with spark 2.00 > Spark SQL: Thriftserver unable to run a registered Hive UDTF > ------------------------------------------------------------ > > Key: SPARK-18832 > URL: https://issues.apache.org/jira/browse/SPARK-18832 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.0.0, 2.0.1, 2.0.2 > Environment: HDP: 2.5 > Spark: 2.0.0 > Reporter: Lokesh Yadav > Attachments: SampleUDTF.java > > > Spark Thriftserver is unable to run a HiveUDTF. > It throws the error that it is unable to find the functions although the > function registration succeeds and the funtions does show up in the list > output by {{show functions}}. > I am using a Hive UDTF, registering it using a jar placed on my local > machine. Calling it using the following commands: > //Registering the functions, this command succeeds. > {{CREATE FUNCTION SampleUDTF AS > 'com.fuzzylogix.experiments.udf.hiveUDF.SampleUDTF' USING JAR > '/root/spark_files/experiments-1.2.jar';}} > //Thriftserver is able to look up the functuion, on this command: > {{DESCRIBE FUNCTION SampleUDTF;}} > {quote} > {noformat} > Output: > +-----------------------------------------------------------+--+ > | function_desc | > +-----------------------------------------------------------+--+ > | Function: default.SampleUDTF | > | Class: com.fuzzylogix.experiments.udf.hiveUDF.SampleUDTF | > | Usage: N/A. | > +-----------------------------------------------------------+--+ > {noformat} > {quote} > // Calling the function: > {{SELECT SampleUDTF('Paris');}} > bq. Output of the above command: Error: > org.apache.spark.sql.AnalysisException: Undefined function: 'SampleUDTF'. > This function is neither a registered temporary function nor a permanent > function registered in the database 'default'.; line 1 pos 7 (state=,code=0) > I have also tried with using a non-local (on hdfs) jar, but I get the same > error. > My environment: HDP 2.4 with spark 2.00 -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org