[jira] [Commented] (SPARK-28670) [UDF] create permanent UDF does not throw Exception if jar does not exist in HDFS path or Local
[ https://issues.apache.org/jira/browse/SPARK-28670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16903565#comment-16903565 ] Sandeep Katta commented on SPARK-28670: --- [~dongjoon] [~hyukjin.kwon], as per this Jira there is a difference in *temporary* and *permanent* functions creation. IMHO I feel this behaviour should be consistent. Whenever the user creates the function if the resource does not exists then it should fail. > [UDF] create permanent UDF does not throw Exception if jar does not exist in > HDFS path or Local > --- > > Key: SPARK-28670 > URL: https://issues.apache.org/jira/browse/SPARK-28670 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: ABHISHEK KUMAR GUPTA >Priority: Minor > > jdbc:hive2://10.18.18.214:23040/default> create function addm AS > 'com.huawei.bigdata.hive.example.udf.AddDoublesUDF' using jar > 'hdfs://hacluster/user/AddDoublesUDF1.jar'; > +-+--+ > | Result | > +-+--+ > +-+--+ > No rows selected (0.241 seconds) > 0: jdbc:hive2://10.18.18.214:23040/default> create temporary function addm > AS 'com.huawei.bigdata.hive.example.udf.AddDoublesUDF' using jar > 'hdfs://hacluster/user/AddDoublesUDF1.jar'; > INFO : converting to local hdfs://hacluster/user/AddDoublesUDF1.jar > ERROR : Failed to read external resource > hdfs://hacluster/user/AddDoublesUDF1.jar > java.lang.RuntimeException: Failed to read external resource > hdfs://hacluster/user/AddDoublesUDF1.jar > at > org.apache.hadoop.hive.ql.session.SessionState.downloadResource(SessionState.java:1288) -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-28670) [UDF] create permanent UDF does not throw Exception if jar does not exist in HDFS path or Local
[ https://issues.apache.org/jira/browse/SPARK-28670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16903559#comment-16903559 ] Sandeep Katta commented on SPARK-28670: --- [~abhishek.akg] thanks for raising this issue, I will post the PR ASAP > [UDF] create permanent UDF does not throw Exception if jar does not exist in > HDFS path or Local > --- > > Key: SPARK-28670 > URL: https://issues.apache.org/jira/browse/SPARK-28670 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.4.0 >Reporter: ABHISHEK KUMAR GUPTA >Priority: Minor > > jdbc:hive2://10.18.18.214:23040/default> create function addm AS > 'com.huawei.bigdata.hive.example.udf.AddDoublesUDF' using jar > 'hdfs://hacluster/user/AddDoublesUDF1.jar'; > +-+--+ > | Result | > +-+--+ > +-+--+ > No rows selected (0.241 seconds) > 0: jdbc:hive2://10.18.18.214:23040/default> create temporary function addm > AS 'com.huawei.bigdata.hive.example.udf.AddDoublesUDF' using jar > 'hdfs://hacluster/user/AddDoublesUDF1.jar'; > INFO : converting to local hdfs://hacluster/user/AddDoublesUDF1.jar > ERROR : Failed to read external resource > hdfs://hacluster/user/AddDoublesUDF1.jar > java.lang.RuntimeException: Failed to read external resource > hdfs://hacluster/user/AddDoublesUDF1.jar > at > org.apache.hadoop.hive.ql.session.SessionState.downloadResource(SessionState.java:1288) -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
