[jira] [Comment Edited] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[ https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680855#comment-17680855 ] AjaykumarDev edited comment on HIVE-17502 at 1/27/23 8:41 AM: -- [~vgarg] [~thejas] [~sershe] [~thai.bui] h4. [Stamatis Zampetakis|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=zabetak] h4. We are also facing similar issue in out Hive LLap Application as below , # user1 executes the query and user2 also executes the query both users are using the same userid. As user1 executed the query first, he gets the session from thread pool however, in case of other user he gets the below error. Error : 2023-01-25T15:40:54,852 INFO [ATS Logger 0] hooks.ATSHook: Received pre-hook notification for :hive_20230125154054_3e24f02f-c0e5-4ad2-87a0-758d025cb02a 2023-01-25T15:40:54,863 ERROR [HiveServer2-Background-Pool: Thread-228] exec.Task: Failed to execute tez graph. org.apache.hadoop.hive.ql.metadata.HiveException: The pool session sessionId=59675216-4e35-4872-8e22-835c50b8c23d, queueName=llap, user=hive, doAs=false, isOpen=true, isDefault=true, expires in 593431620ms should have been returned to the pool at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:546) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:556) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:150) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] This Jira was created to address the same issue and the patch was also available. So, when can we expect this patch to be available in any future hive release or any alternative is available ? Please let me know because the solution for this issue is very critical for success of my company project, the company has made big investment for hive llap project and we want to resolve this issue as soon as possible. Please let us know if you need more information about the same. was (Author: JIRAUSER298812): [~vgarg] [~thejas] [~sershe] [~thai.bui] Stamatis Zampetakis h4. We are also facing similar issue in out Hive LLap Application as below , # user1 executes the query and user2 also executes the query both users are using the same userid. As user1 executed the query first, he gets the session from thread pool however, in case of other user he gets the below error. Error : 2023-01-25T15:40:54,852 INFO [ATS Logger 0] hooks.ATSHook: Received pre-hook notification for :hive_20230125154054_3e24f02f-c0e5-4ad2-87a0-758d025cb02a 2023-01-25T15:40:54,863 ERROR [HiveServer2-Background-Pool: Thread-228] exec.Task: Failed to execute tez graph. org.apache.hadoop.hive.ql.metadata.HiveException: The pool session sessionId=59675216-4e35-4872-8e22-835c50b8c23d, queueName=llap, user=hive, doAs=false, isOpen=true, isDefault=true, expires in 593431620ms should have been returned to the pool at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:546) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:556) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:150) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] This Jira was created to address the same issue and the patch was also available. So, when can we expect this patch to be available in any future hive release or any alternative is available ? Please let me know because the solution for this issue is very critical for success of my company project, the company has made big investment for hive llap project and we want to resolve this issue as soon as possible. Please
[jira] [Comment Edited] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[ https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680855#comment-17680855 ] AjaykumarDev edited comment on HIVE-17502 at 1/27/23 8:40 AM: -- [~vgarg] [~thejas] [~sershe] [~thai.bui] Stamatis Zampetakis h4. We are also facing similar issue in out Hive LLap Application as below , # user1 executes the query and user2 also executes the query both users are using the same userid. As user1 executed the query first, he gets the session from thread pool however, in case of other user he gets the below error. Error : 2023-01-25T15:40:54,852 INFO [ATS Logger 0] hooks.ATSHook: Received pre-hook notification for :hive_20230125154054_3e24f02f-c0e5-4ad2-87a0-758d025cb02a 2023-01-25T15:40:54,863 ERROR [HiveServer2-Background-Pool: Thread-228] exec.Task: Failed to execute tez graph. org.apache.hadoop.hive.ql.metadata.HiveException: The pool session sessionId=59675216-4e35-4872-8e22-835c50b8c23d, queueName=llap, user=hive, doAs=false, isOpen=true, isDefault=true, expires in 593431620ms should have been returned to the pool at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:546) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:556) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:150) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] This Jira was created to address the same issue and the patch was also available. So, when can we expect this patch to be available in any future hive release or any alternative is available ? Please let me know because the solution for this issue is very critical for success of my company project, the company has made big investment for hive llap project and we want to resolve this issue as soon as possible. Please let us know if you need more information about the same. was (Author: JIRAUSER298812): [~vgarg] [~thejas] [~sershe] [~thai.bui] h4. [Vineet Garg|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=vgarg] We are also facing similar issue in out Hive LLap Application as below , # user1 executes the query and user2 also executes the query both users are using the same userid. As user1 executed the query first, he gets the session from thread pool however, in case of other user he gets the below error. Error : 2023-01-25T15:40:54,852 INFO [ATS Logger 0] hooks.ATSHook: Received pre-hook notification for :hive_20230125154054_3e24f02f-c0e5-4ad2-87a0-758d025cb02a 2023-01-25T15:40:54,863 ERROR [HiveServer2-Background-Pool: Thread-228] exec.Task: Failed to execute tez graph. org.apache.hadoop.hive.ql.metadata.HiveException: The pool session sessionId=59675216-4e35-4872-8e22-835c50b8c23d, queueName=llap, user=hive, doAs=false, isOpen=true, isDefault=true, expires in 593431620ms should have been returned to the pool at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:546) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:556) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:150) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] This Jira was created to address the same issue and the patch was also available. So, when can we expect this patch to be available in any future hive release or any alternative is available ? Please let me know because the solution for this issue is very critical for success of my company project, the company has made big investment for hive llap project and we want to resolve this issue as soon as possible. Please let us know if
[jira] [Comment Edited] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[ https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680855#comment-17680855 ] AjaykumarDev edited comment on HIVE-17502 at 1/27/23 6:36 AM: -- [~vgarg] [~thejas] [~sershe] [~thai.bui] h4. [Vineet Garg|https://issues.apache.org/jira/secure/ViewProfile.jspa?name=vgarg] We are also facing similar issue in out Hive LLap Application as below , # user1 executes the query and user2 also executes the query both users are using the same userid. As user1 executed the query first, he gets the session from thread pool however, in case of other user he gets the below error. Error : 2023-01-25T15:40:54,852 INFO [ATS Logger 0] hooks.ATSHook: Received pre-hook notification for :hive_20230125154054_3e24f02f-c0e5-4ad2-87a0-758d025cb02a 2023-01-25T15:40:54,863 ERROR [HiveServer2-Background-Pool: Thread-228] exec.Task: Failed to execute tez graph. org.apache.hadoop.hive.ql.metadata.HiveException: The pool session sessionId=59675216-4e35-4872-8e22-835c50b8c23d, queueName=llap, user=hive, doAs=false, isOpen=true, isDefault=true, expires in 593431620ms should have been returned to the pool at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:546) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:556) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:150) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] This Jira was created to address the same issue and the patch was also available. So, when can we expect this patch to be available in any future hive release or any alternative is available ? Please let me know because the solution for this issue is very critical for success of my company project, the company has made big investment for hive llap project and we want to resolve this issue as soon as possible. Please let us know if you need more information about the same. was (Author: JIRAUSER298812): [~vgarg] [~thejas] [~sershe] [~thai.bui] We are also facing similar issue in out Hive LLap Application as below , # user1 executes the query and user2 also executes the query both users are using the same userid. As user1 executed the query first, he gets the session from thread pool however, in case of other user he gets the below error. Error : 2023-01-25T15:40:54,852 INFO [ATS Logger 0] hooks.ATSHook: Received pre-hook notification for :hive_20230125154054_3e24f02f-c0e5-4ad2-87a0-758d025cb02a 2023-01-25T15:40:54,863 ERROR [HiveServer2-Background-Pool: Thread-228] exec.Task: Failed to execute tez graph. org.apache.hadoop.hive.ql.metadata.HiveException: The pool session sessionId=59675216-4e35-4872-8e22-835c50b8c23d, queueName=llap, user=hive, doAs=false, isOpen=true, isDefault=true, expires in 593431620ms should have been returned to the pool at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:546) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:556) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:150) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] This Jira was created to address the same issue and the patch was also available. So, when can we expect this patch to be available in any future hive release or any alternative is available ? Please let me know because the solution for this issue is very critical for success of my company project, the company has made big investment for hive llap project and we want to resolve this issue as soon as possible. Please let us know if you need more information
[jira] [Comment Edited] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[ https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680855#comment-17680855 ] AjaykumarDev edited comment on HIVE-17502 at 1/26/23 3:46 AM: -- [~vgarg] [~thejas] [~sershe] [~thai.bui] We are also facing similar issue in out Hive LLap Application as below , # user1 executes the query and user2 also executes the query both users are using the same userid. As user1 executed the query first, he gets the session from thread pool however, in case of other user he gets the below error. Error : 2023-01-25T15:40:54,852 INFO [ATS Logger 0] hooks.ATSHook: Received pre-hook notification for :hive_20230125154054_3e24f02f-c0e5-4ad2-87a0-758d025cb02a 2023-01-25T15:40:54,863 ERROR [HiveServer2-Background-Pool: Thread-228] exec.Task: Failed to execute tez graph. org.apache.hadoop.hive.ql.metadata.HiveException: The pool session sessionId=59675216-4e35-4872-8e22-835c50b8c23d, queueName=llap, user=hive, doAs=false, isOpen=true, isDefault=true, expires in 593431620ms should have been returned to the pool at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:546) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:556) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:150) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] This Jira was created to address the same issue and the patch was also available. So, when can we expect this patch to be available in any future hive release or any alternative is available ? Please let me know because the solution for this issue is very critical for success of my company project, the company has made big investment for hive llap project and we want to resolve this issue as soon as possible. Please let us know if you need more information about the same. was (Author: JIRAUSER298812): [~vgarg] [~thejas] [~sershe] We are also facing similar issue in out Hive LLap Application as below , # user1 executes the query and user2 also executes the query both users are using the same userid. As user1 executed the query first, he gets the session from thread pool however, in case of other user he gets the below error. Error : 2023-01-25T15:40:54,852 INFO [ATS Logger 0] hooks.ATSHook: Received pre-hook notification for :hive_20230125154054_3e24f02f-c0e5-4ad2-87a0-758d025cb02a 2023-01-25T15:40:54,863 ERROR [HiveServer2-Background-Pool: Thread-228] exec.Task: Failed to execute tez graph. org.apache.hadoop.hive.ql.metadata.HiveException: The pool session sessionId=59675216-4e35-4872-8e22-835c50b8c23d, queueName=llap, user=hive, doAs=false, isOpen=true, isDefault=true, expires in 593431620ms should have been returned to the pool at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:546) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:556) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:150) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] This Jira was created to address the same issue and the patch was also available. So, when can we expect this patch to be available in any future hive release or any alternative is available ? Please let me know because the solution for this issue is very critical for success of my company project, the company has made big investment for hive llap project and we want to resolve this issue as soon as possible. Please let us know if you need more information about the same. > Reuse of default session should not throw an exception in LLAP w/ Tez >
[jira] [Comment Edited] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[ https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680855#comment-17680855 ] AjaykumarDev edited comment on HIVE-17502 at 1/26/23 3:45 AM: -- [~vgarg] [~thejas] [~sershe] We are also facing similar issue in out Hive LLap Application as below , # user1 executes the query and user2 also executes the query both users are using the same userid. As user1 executed the query first, he gets the session from thread pool however, in case of other user he gets the below error. Error : 2023-01-25T15:40:54,852 INFO [ATS Logger 0] hooks.ATSHook: Received pre-hook notification for :hive_20230125154054_3e24f02f-c0e5-4ad2-87a0-758d025cb02a 2023-01-25T15:40:54,863 ERROR [HiveServer2-Background-Pool: Thread-228] exec.Task: Failed to execute tez graph. org.apache.hadoop.hive.ql.metadata.HiveException: The pool session sessionId=59675216-4e35-4872-8e22-835c50b8c23d, queueName=llap, user=hive, doAs=false, isOpen=true, isDefault=true, expires in 593431620ms should have been returned to the pool at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:546) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:556) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:150) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] This Jira was created to address the same issue and the patch was also available. So, when can we expect this patch to be available in any future hive release or any alternative is available ? Please let me know because the solution for this issue is very critical for success of my company project, the company has made big investment for hive llap project and we want to resolve this issue as soon as possible. Please let us know if you need more information about the same. was (Author: JIRAUSER298812): [~vgarg] [~thejas] We are also facing similar issue in out Hive LLap Application as below , # user1 executes the query and user2 also executes the query both users are using the same userid. As user1 executed the query first, he gets the session from thread pool however, in case of other user he gets the below error. Error : 2023-01-25T15:40:54,852 INFO [ATS Logger 0] hooks.ATSHook: Received pre-hook notification for :hive_20230125154054_3e24f02f-c0e5-4ad2-87a0-758d025cb02a 2023-01-25T15:40:54,863 ERROR [HiveServer2-Background-Pool: Thread-228] exec.Task: Failed to execute tez graph. org.apache.hadoop.hive.ql.metadata.HiveException: The pool session sessionId=59675216-4e35-4872-8e22-835c50b8c23d, queueName=llap, user=hive, doAs=false, isOpen=true, isDefault=true, expires in 593431620ms should have been returned to the pool at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:546) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:556) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:150) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] This Jira was created to address the same issue and the patch was also available. So, when can we expect this patch to be available in any future hive release or any alternative is available ? Please let me know because the solution for this issue is very critical for success of my company project, the company has made big investment for hive llap project and we want to resolve this issue as soon as possible. Please let us know if you need more information about the same. > Reuse of default session should not throw an exception in LLAP w/ Tez >
[jira] [Comment Edited] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[ https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680855#comment-17680855 ] AjaykumarDev edited comment on HIVE-17502 at 1/26/23 3:44 AM: -- [~vgarg] [~thejas] We are also facing similar issue in out Hive LLap Application as below , # user1 executes the query and user2 also executes the query both users are using the same userid. As user1 executed the query first, he gets the session from thread pool however, in case of other user he gets the below error. Error : 2023-01-25T15:40:54,852 INFO [ATS Logger 0] hooks.ATSHook: Received pre-hook notification for :hive_20230125154054_3e24f02f-c0e5-4ad2-87a0-758d025cb02a 2023-01-25T15:40:54,863 ERROR [HiveServer2-Background-Pool: Thread-228] exec.Task: Failed to execute tez graph. org.apache.hadoop.hive.ql.metadata.HiveException: The pool session sessionId=59675216-4e35-4872-8e22-835c50b8c23d, queueName=llap, user=hive, doAs=false, isOpen=true, isDefault=true, expires in 593431620ms should have been returned to the pool at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:546) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:556) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:150) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] This Jira was created to address the same issue and the patch was also available. So, when can we expect this patch to be available in any future hive release or any alternative is available ? Please let me know because the solution for this issue is very critical for success of my company project, the company has made big investment for hive llap project and we want to resolve this issue as soon as possible. Please let us know if you need more information about the same. was (Author: JIRAUSER298812): [~vgarg] [~vgarg] We are also facing similar issue in out Hive LLap Application as below , # user1 executes the query and user2 also executes the query both users are using the same userid. As user1 executed the query first, he gets the session from thread pool however, in case of other user he gets the below error. Error : 2023-01-25T15:40:54,852 INFO [ATS Logger 0] hooks.ATSHook: Received pre-hook notification for :hive_20230125154054_3e24f02f-c0e5-4ad2-87a0-758d025cb02a 2023-01-25T15:40:54,863 ERROR [HiveServer2-Background-Pool: Thread-228] exec.Task: Failed to execute tez graph. org.apache.hadoop.hive.ql.metadata.HiveException: The pool session sessionId=59675216-4e35-4872-8e22-835c50b8c23d, queueName=llap, user=hive, doAs=false, isOpen=true, isDefault=true, expires in 593431620ms should have been returned to the pool at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:546) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:556) ~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:150) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667) [hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] This Jira was created to address the same issue and the patch was also available. So, when can we expect this patch to be available in any future hive release or any alternative is available ? Please let me know because the solution for this issue is very critical for success of my company project, the company has made big investment for hive llap project and we want to resolve this issue as soon as possible. Please let us know if you need more information about the same. > Reuse of default session should not throw an exception in LLAP w/ Tez >
[jira] [Comment Edited] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197868#comment-16197868
]
Sergey Shelukhin edited comment on HIVE-17502 at 10/9/17 11:31 PM:
---
Because currently each SessionState (Hive session) can only have one Tez
session. With this patch it's possible to reuse the Hive session, utilizing
several Tez sessions. So the code that gets/sets/closes/etc. the session by
getting it from SessionState will become invalid I assume.
Just having a config is not good enough to allow known buggy code...
Btw, we were wondering in some discussion, why cannot Hue just not reuse the
session this way?
was (Author: sershe):
Because currently each SessionState (Hive session) can only have one Tez
session. With this patch it's possible to reuse the Hive session, utilizing
several Tez sessions. So the code that gets/sets/closes/etc. the session by
getting it from SessionState will become invalid I assume
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
> Fix For: 3.0.0
>
> Attachments: HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow
[jira] [Comment Edited] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168535#comment-16168535
]
Thejas M Nair edited comment on HIVE-17502 at 9/15/17 9:15 PM:
---
The SessionState has state for the session/jdbc connection. So its meant to be
used for everything in that session.
However, it is not a class that we have worked on to ensure thread safety.
I had created a patch to disabled parallel use of single session because of
safety issues, but forgot to commit it - HIVE-14247 .
[~thai.bui]
What is the purpose of sharing the same connection for different queries at
same time ?
Its not something that api's like JDBC easily allow and not something usually
recommended in the RDBMS world. Sharing the session won't give you any
significant advantages in terms of resource utilization in LLAP mode AFAIK.
was (Author: thejas):
The SessionState has state for the session/jdbc connection. So its meant to be
used for everything in that session.
However, it is not a class that we have worked on to ensure thread safety.
I had created a patch to disabled parallel use of single session because of
safety issues, but forgot to commit it - HIVE-11402 .
[~thai.bui]
What is the purpose of sharing the same connection for different queries at
same time ?
Its not something that api's like JDBC easily allow and not something usually
recommended in the RDBMS world. Sharing the session won't give you any
significant advantages in terms of resource utilization in LLAP mode AFAIK.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session
[jira] [Comment Edited] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162299#comment-16162299
]
Sergey Shelukhin edited comment on HIVE-17502 at 9/12/17 12:45 AM:
---
There may be a bigger problem here.
I think the logic behind this exception is related to the giant try-finally in
TezTask after SessionState.get() and then getTezSession is called.
In the finally, returnSession is called, which will ignore a non-pool session,
but for a pool session it would put it back into the pool and reset
SessionState's Tez session to null. This is not necessarily the best logic,
cause maybe pool session reuse could also be allowed, but for obvious reasons
(limits of the pool) holding a session while there may be other users waiting
is undesirable.
However it's beside the point as, unless I'm missing something, the only way
for getTezSession before the try block to get an existing _pool_ session from
the SessionState, is the case where the latter used for two Hive queries in
parallel (i.e. SessionState.get.getTezSession is called from one thread while
another is inside of TezTask try-finally block, running some query). Let me
know if it's a different scenario.
This pattern would probably break all this session setting and unsetting, which
would need to be fixed with additional checks; however I think it also has
other similar state-related issues and is not supported.
[~thejas] do you know if we allow parallel queries with the same SessionState?
was (Author: sershe):
There may be a bigger problem here.
I think the logic behind this exception is related to a giant try-finally in
TezTask after SessionState.get().getTezSession is called.
In the end of the finally, returnSession is called, which will ignore the
non-pool session, but for a pool session it would put it back in the pool and
reset SessionState's Tez session to null. This is not necessarily the best
logic, cause maybe pool session reuse could also be allowed, but for obvious
reasons (limits of the pool) holding a session from others is undesirable.
However it's beside the point as, unless I'm missing something, the only way
for getTezSession before the block to get an existing pool tez session is if
SessionState is used for two Hive queries in parallel (i.e.
SessionState.get.getTezSession is called from one thread while another is
inside of TezTask try-finally block, running some query). Let me know if it's a
different scenario.
This pattern would probably break all this session setting and unsetting, which
would need to be fixed with additional checks; however I think it also has
other similar state-related issues.
[~thejas] do you know if we allow parallel queries with the same SessionState?
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
[jira] [Comment Edited] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162299#comment-16162299
]
Sergey Shelukhin edited comment on HIVE-17502 at 9/12/17 12:43 AM:
---
There may be a bigger problem here.
I think the logic behind this exception is related to a giant try-finally in
TezTask after SessionState.get().getTezSession is called.
In the end of the finally, returnSession is called, which will ignore the
non-pool session, but for a pool session it would put it back in the pool and
reset SessionState's Tez session to null. This is not necessarily the best
logic, cause maybe pool session reuse could also be allowed, but for obvious
reasons (limits of the pool) holding a session from others is undesirable.
However it's beside the point as, unless I'm missing something, the only way
for getTezSession before the block to get an existing pool tez session is if
SessionState is used for two Hive queries in parallel (i.e.
SessionState.get.getTezSession is called from one thread while another is
inside of TezTask try-finally block, running some query). Let me know if it's a
different scenario.
This pattern would probably break all this session setting and unsetting, which
would need to be fixed with additional checks; however I think it also has
other similar state-related issues.
[~thejas] do you know if we allow parallel queries with the same SessionState?
was (Author: sershe):
There may be a bigger problem here.
I think the logic behind this exception is related to a giant try-finally in
TezTask after SessionState.get().getTezSession is called.
In the end of the finally, returnSession is called, which will ignore the
non-pool session, but for a pool session it would put it back in the pool and
reset SessionState's Tez session to null. This is not necessarily the best
logic, cause maybe pool session reuse could also be allowed, but for obvious
reasons (limits of the pool) holding a session from others is undesirable.
However it's beside the point as, unless I'm missing something, the only way
for getTezSession before the block to get an existing pool tez session is if
SessionState is used for two Hive queries in parallel (i.e.
SessionState.get.getTezSession is called from one thread while another is
inside of TezTask try-finally block, running some query). Let me know if it's a
different scenario.
This pattern would probably break all this session setting and unsetting, which
would need to be fixed with additional checks; however I think it also has
other similar state-related issues.
[~thejas] do you know if we allow parallel queries with the same SessionState.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
>
