[
https://issues.apache.org/jira/browse/SPARK-26602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16784611#comment-16784611
]
Ajith S edited comment on SPARK-26602 at 3/5/19 4:15 PM:
---------------------------------------------------------
# I have a question about this issue in thrift-server case. If admin does a add
jar with a non-existing jar (may be a human error), it will cause all the
ongoing beeline sessions to fail ( even a query where jar is not needed at
all). and only way to recover is restart of thrift-server
# As you said, "If a user adds something to the classpath, it matters to the
whole classpath. If it's missing, I think it's surprising to ignore that fact"
- but unless the user refers to the jar, is it ok to fail all of his
operations.? (just like JVM behaviour, we get classnotfoundexception when the
missing class is actually referred, until then JVM is happily running)
Please correct me if i am wrong cc [~srowen]
was (Author: ajithshetty):
# I have a question about this issue in thrift-server case. If admin does a add
jar with a non-existing jar (may be a human error), it will cause all the
ongoing beeline sessions to fail ( even a query where jar is not needed at
all). and only way to recover is restart of thrift-server
# As you said, "If a user adds something to the classpath, it matters to the
whole classpath. If it's missing, I think it's surprising to ignore that fact"
- but unless the user refers to the jar, is it ok to fail all of his
operations.? (just like JVM behaviour)
Please correct me if i am wrong cc [~srowen]
> Insert into table fails after querying the UDF which is loaded with wrong
> hdfs path
> -----------------------------------------------------------------------------------
>
> Key: SPARK-26602
> URL: https://issues.apache.org/jira/browse/SPARK-26602
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Haripriya
> Priority: Major
> Attachments: beforeFixUdf.txt
>
>
> In sql,
> 1.Query the existing udf(say myFunc1)
> 2. create and select the udf registered with incorrect path (say myFunc2)
> 3.Now again query the existing udf in the same session - Wil throw exception
> stating that couldn't read resource of myFunc2's path
> 4.Even the basic operations like insert and select will fail giving the same
> error
> Result:
> java.lang.RuntimeException: Failed to read external resource
> hdfs:///tmp/hari_notexists1/two_udfs.jar
> at
> org.apache.hadoop.hive.ql.session.SessionState.downloadResource(SessionState.java:1288)
> at
> org.apache.hadoop.hive.ql.session.SessionState.resolveAndDownload(SessionState.java:1242)
> at
> org.apache.hadoop.hive.ql.session.SessionState.add_resources(SessionState.java:1163)
> at
> org.apache.hadoop.hive.ql.session.SessionState.add_resources(SessionState.java:1149)
> at
> org.apache.hadoop.hive.ql.processors.AddResourceProcessor.run(AddResourceProcessor.java:67)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$runHive$1.apply(HiveClientImpl.scala:737)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$runHive$1.apply(HiveClientImpl.scala:706)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$withHiveState$1.apply(HiveClientImpl.scala:275)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.liftedTree1$1(HiveClientImpl.scala:213)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.retryLocked(HiveClientImpl.scala:212)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.withHiveState(HiveClientImpl.scala:258)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.runHive(HiveClientImpl.scala:706)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.runSqlHive(HiveClientImpl.scala:696)
> at
> org.apache.spark.sql.hive.client.HiveClientImpl.addJar(HiveClientImpl.scala:841)
> at
> org.apache.spark.sql.hive.HiveSessionResourceLoader.addJar(HiveSessionStateBuilder.scala:112)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
