[ https://issues.apache.org/jira/browse/LIVY-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeffrey(Xilang) Yan closed LIVY-571. ------------------------------------ Resolution: Fixed Fix Version/s: 0.7.0 > When an exception happens running a query, we should report it to end user > -------------------------------------------------------------------------- > > Key: LIVY-571 > URL: https://issues.apache.org/jira/browse/LIVY-571 > Project: Livy > Issue Type: Bug > Components: Thriftserver > Reporter: Marco Gaido > Assignee: Jeffrey(Xilang) Yan > Priority: Major > Fix For: 0.7.0 > > Time Spent: 20m > Remaining Estimate: 0h > > When a query fails with an exception on livy thriftserver, instead of > reporting this exception to the end user, a meaningless one is reported. Eg, > with Hive support not enabled on spark, the following query causes: > {code} > 0: jdbc:hive2://localhost:10090/> create table test as select a.* from > (select 1, "2") a; > Error: java.lang.RuntimeException: java.util.NoSuchElementException: > Statement 820bb5c2-018b-46ea-9b7f-b0e3b9c31c46 not found in session > acf3712b-1f08-4111-950f-559fc3f3f10c. > org.apache.livy.thriftserver.session.ThriftSessionState.statementNotFound(ThriftSessionState.java:118) > org.apache.livy.thriftserver.session.ThriftSessionState.cleanupStatement(ThriftSessionState.java:107) > org.apache.livy.thriftserver.session.CleanupStatementJob.call(CleanupStatementJob.java:43) > org.apache.livy.thriftserver.session.CleanupStatementJob.call(CleanupStatementJob.java:26) > org.apache.livy.rsc.driver.JobWrapper.call(JobWrapper.java:64) > org.apache.livy.rsc.driver.JobWrapper.call(JobWrapper.java:31) > java.util.concurrent.FutureTask.run(FutureTask.java:266) > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > java.lang.Thread.run(Thread.java:748) > {code} > Looking at the logs, of course the real problem is: > {code} > 19/03/23 10:40:32 ERROR LivyExecuteStatementOperation: Error running hive > query: > org.apache.hive.service.cli.HiveSQLException: > java.util.concurrent.ExecutionException: java.lang.RuntimeException: > org.apache.spark.sql.AnalysisException: Hive support is required to CREATE > Hive TABLE (AS SELECT);; > 'CreateTable `test`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, > ErrorIfExists > +- Project [1#1, 2#2] > +- SubqueryAlias `a` > +- Project [1 AS 1#1, 2 AS 2#2] > +- OneRowRelation > org.apache.spark.sql.execution.datasources.HiveOnlyCheck$$anonfun$apply$12.apply(rules.scala:392) > org.apache.spark.sql.execution.datasources.HiveOnlyCheck$$anonfun$apply$12.apply(rules.scala:390) > org.apache.spark.sql.catalyst.trees.TreeNode.foreach(TreeNode.scala:117) > org.apache.spark.sql.execution.datasources.HiveOnlyCheck$.apply(rules.scala:390) > org.apache.spark.sql.execution.datasources.HiveOnlyCheck$.apply(rules.scala:388) > org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$2.apply(CheckAnalysis.scala:386) > org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$2.apply(CheckAnalysis.scala:386) > scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) > scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) > org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:386) > org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:95) > org.apache.spark.sql.catalyst.analysis.Analyzer$$anonfun$executeAndCheck$1.apply(Analyzer.scala:108) > org.apache.spark.sql.catalyst.analysis.Analyzer$$anonfun$executeAndCheck$1.apply(Analyzer.scala:105) > org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper$.markInAnalyzer(AnalysisHelper.scala:201) > org.apache.spark.sql.catalyst.analysis.Analyzer.executeAndCheck(Analyzer.scala:105) > org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:57) > org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:55) > org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:47) > org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:79) > org.apache.spark.sql.SparkSession.sql(SparkSession.scala:642) > org.apache.livy.thriftserver.session.SqlJob.executeSql(SqlJob.java:74) > org.apache.livy.thriftserver.session.SqlJob.call(SqlJob.java:64) > org.apache.livy.thriftserver.session.SqlJob.call(SqlJob.java:35) > org.apache.livy.rsc.driver.JobWrapper.call(JobWrapper.java:64) > org.apache.livy.rsc.driver.JobWrapper.call(JobWrapper.java:31) > java.util.concurrent.FutureTask.run(FutureTask.java:266) > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > java.lang.Thread.run(Thread.java:748) > at > org.apache.livy.thriftserver.LivyExecuteStatementOperation.execute(LivyExecuteStatementOperation.scala:147) > at > org.apache.livy.thriftserver.LivyExecuteStatementOperation$$anon$1$$anon$2.run(LivyExecuteStatementOperation.scala:97) > at > org.apache.livy.thriftserver.LivyExecuteStatementOperation$$anon$1$$anon$2.run(LivyExecuteStatementOperation.scala:94) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) > at > org.apache.livy.thriftserver.LivyExecuteStatementOperation$$anon$1.run(LivyExecuteStatementOperation.scala:107) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > .... > {code} > And this should be reported to the end user. -- This message was sent by Atlassian JIRA (v7.6.14#76016)