[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695603#comment-17695603
]
edward henry commented on HIVE-17502:
-
Nice post thanks for sharing. check this [save with
blogs|https://www.savewithblogs.com/]
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Priority: Blocker
> Attachments: HIVE-17502.2.patch, HIVE-17502.3.patch, HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to master &/ the other
> branches is acceptable. I'm not an contributor &/ committer, this will be my
> first time contributing to Hive and the Apache foundation. Any early review
> is greatly appreciated, thanks!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681951#comment-17681951
]
Stamatis Zampetakis commented on HIVE-17502:
Hey [~ajaykumardev36], I don't have much context around this issue and not
enough time to push this forward.
Note that the patch submission is currently done via pull requests on GitHub
(https://github.com/apache/hive/pulls). If you want to push this forward start
by raising a pull request. When tests pass try to find a reviewer by sending an
email to the dev list; note that we are quite short on reviewers the past few
months so it might not be trivial.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Priority: Blocker
> Attachments: HIVE-17502.2.patch, HIVE-17502.3.patch, HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to master &/ the other
> branches is acceptable. I'm not an contributor &/ committer, this will be my
> first time contributing to
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681262#comment-17681262
]
AjaykumarDev commented on HIVE-17502:
-
Hi [~zabetak] ,
I am so happy that you replied. Please find my comments below ,
# Code is not merged into branch-4.0.0-alpha-2 branch hence, It will still
throw the error.
https://github.com/apache/hive/blob/branch-4.0.0-alpha-2/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java
return (queueName == null) ? confQueueName == null :
queueName.equals(confQueueName);
} else {
// this session should never be a default session unless something has
messed up.
throw new HiveException("The pool session " + session + " should have
been returned to the pool");
}
Could you please check if the patch is already merged ?
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Priority: Blocker
> Attachments: HIVE-17502.2.patch, HIVE-17502.3.patch, HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681253#comment-17681253
]
Stamatis Zampetakis commented on HIVE-17502:
Hi [~ajaykumardev36], based on the stack trace that you provided it seems that
you are using a non-Apache release of Hive (hive-exec-2.1.0.2.6.5.0-292) and a
pretty old one as well. This JIRA instance is only for official ASF releases
not for 3rd-party Hive releases. Please get in touch with your Hive provider if
you need additional support about the products you are using.
Other than that , the ticket is pretty old so there is a high chance that the
problem is already solved in Hive 4.0.0 releases. Feel free to comment again if
the problem is still present in latest Hive 4.0.0-alpha-2 release.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Priority: Blocker
> Attachments: HIVE-17502.2.patch, HIVE-17502.3.patch, HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17681170#comment-17681170
]
AjaykumarDev commented on HIVE-17502:
-
Any update on above.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>Priority: Major
> Attachments: HIVE-17502.2.patch, HIVE-17502.3.patch, HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to master &/ the other
> branches is acceptable. I'm not an contributor &/ committer, this will be my
> first time contributing to Hive and the Apache foundation. Any early review
> is greatly appreciated, thanks!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680855#comment-17680855
]
AjaykumarDev commented on HIVE-17502:
-
[~vgarg] [~vgarg]
We are also facing similar issue in out Hive LLap Application as below ,
# user1 executes the query and user2 also executes the query both users are
using the same userid. As user1 executed the query first, he gets the session
from thread pool however, in case of other user he gets the below error.
Error : 2023-01-25T15:40:54,852 INFO [ATS Logger 0] hooks.ATSHook: Received
pre-hook notification for
:hive_20230125154054_3e24f02f-c0e5-4ad2-87a0-758d025cb02a
2023-01-25T15:40:54,863 ERROR [HiveServer2-Background-Pool: Thread-228]
exec.Task: Failed to execute tez graph.
org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
sessionId=59675216-4e35-4872-8e22-835c50b8c23d, queueName=llap, user=hive,
doAs=false, isOpen=true, isDefault=true, expires in 593431620ms should have
been returned to the pool at
org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:546)
~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at
org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:556)
~[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at
org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:150)
[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at
org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at
org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1987)
[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292] at
org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1667)
[hive-exec-2.1.0.2.6.5.0-292.jar:2.1.0.2.6.5.0-292]
This Jira was created to address the same issue and the patch was also
available. So, when can we expect this patch to be available in any future hive
release or any alternative is available ? Please let me know because the
solution for this issue is very critical for success of my company project, the
company has made big investment for hive llap project and we want to resolve
this issue as soon as possible. Please let us know if you need more information
about the same.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>Priority: Major
> Attachments: HIVE-17502.2.patch, HIVE-17502.3.patch, HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
>
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528111#comment-16528111
]
Hive QA commented on HIVE-17502:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918499/HIVE-17502.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14633 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/12250/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12250/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12250/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12918499 - PreCommit-HIVE-Build
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-17502.2.patch, HIVE-17502.3.patch, HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4,
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16528068#comment-16528068
]
Hive QA commented on HIVE-17502:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 64 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 2287 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12250/dev-support/hive-personality.sh
|
| git revision | master / 35cec21 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-12250/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-17502.2.patch, HIVE-17502.3.patch, HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]:
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434762#comment-16434762
]
Hive QA commented on HIVE-17502:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12918499/HIVE-17502.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 87 failed/errored test(s), 13625 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=231)
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed
out) (batchId=247)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=96)
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16434698#comment-16434698
]
Hive QA commented on HIVE-17502:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-10144/dev-support/hive-personality.sh
|
| git revision | master / b3fe652 |
| Default Java | 1.8.0_111 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-10144/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-17502.2.patch, HIVE-17502.3.patch, HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
>
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16433350#comment-16433350
]
Vineet Garg commented on HIVE-17502:
Hi [~thai.bui]. For now we have only cut branch for hive 3.0 release and we
have yet to finalize the release date. So you have at least two or more weeks
to get this patch in 3.0. Once the patch is ready and pushed to master you can
push it to branch-3 to make it in 3.0.
As for testing you can use branch-3 for testing in your production and report
any issues you find.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-17502.2.patch, HIVE-17502.3.patch, HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to master &/ the other
> branches is acceptable. I'm not an contributor &/ committer, this will be my
> first time contributing to Hive and the Apache foundation. Any early
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16432246#comment-16432246
]
Thai Bui commented on HIVE-17502:
-
Hi [~vgarg], I would definitely want to get this patch onto 3.0.0 if possible.
At Bazaarvoice, we are currently maintain a separate branch to apply this patch
and pushing this upstream would remove that effort, we are also currently
testing using 3.0 in production since early this year (with Hadoop 3.0 but soon
to upgrade to 3.1). If you guys need help testing the branch, we are that the
perfect stage for that.
Please let me know what I need to do to get this approved & merged to 3.0.0.
Thanks!
cc: [~thejas]
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to master &/ the other
> branches is acceptable. I'm not an contributor &/
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431843#comment-16431843
]
Hive QA commented on HIVE-17502:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12890993/HIVE-17502.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/10109/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10109/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10109/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-04-10 07:28:05.554
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-10109/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-04-10 07:28:05.557
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at dcd9b59 HIVE-19146 : Delete dangling q.out
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at dcd9b59 HIVE-19146 : Delete dangling q.out
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-04-10 07:28:06.110
+ rm -rf ../yetus_PreCommit-HIVE-Build-10109
+ mkdir ../yetus_PreCommit-HIVE-Build-10109
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-10109
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-10109/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:2425
Falling back to three-way merge...
Applied patch to 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java'
with conflicts.
Going to apply patch with: git apply -p0
error: patch failed:
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:2425
Falling back to three-way merge...
Applied patch to 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java'
with conflicts.
U common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12890993 - PreCommit-HIVE-Build
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
>
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197922#comment-16197922
]
Thai Bui commented on HIVE-17502:
-
bq. Because currently each SessionState (Hive session) can only have one Tez
session. With this patch it's possible to reuse the Hive session, utilizing
several Tez sessions. So the code that gets/sets/closes/etc. the session by
getting it from SessionState will become invalid I assume.
Yes, it's possible to reuse the Hive session, *if and only if that session has
been returned to the pool*. So in effect, 1 session is used by 1 client at a
time and so SessionState only handles 1 Tez session.
What this patch does is to remove the exception being thrown when a client is
making a second request using the same session (while it's still being used).
In that case, the first session in the request will be *skipped*, and a new
session from the pool will be returned. Thus, allowing the same client to use
multiple sessions (handled by distinct threads). Since all session states are
thread-local, each session request is handled by a different thread and things
are fine, there's no bug AFAIK.
[~thejas] I totally understand the concern about long term support. My company
has a big investment in this and we do want to work with the community to find
the best solution using Hive 2 w/ LLAP + Tez. This is one of the biggest
requirements for my team since it's frustrating to use Hue 4 in
one-query-at-time mode. To change Hue 4 to use a different session per query is
very invasive whereas it makes more sense for HiveServer2 to be more permissive
and not throw an exception.
Also, I think we need to clarity the intention of this patch. It is not to use
the same session for multiple queries in parallel. It will use a session for a
query at a time. However, when the client tries to reuse the session (while
it's still being used), a new session from the pool will be returned.
The only downside to this patch is that there will be quite a bit of orphaned
sessions (since users keep requesting new sessions and leave existing
sessions). However, that is taken care of easily by setting HS2 session
expiration times more aggressively.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
> Fix For: 3.0.0
>
> Attachments: HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197870#comment-16197870
]
Thejas M Nair commented on HIVE-17502:
--
[~thai.bui]
Do you know why Hue uses same HiveServer2 session for multiple queries ?
I fear this is mostly kicking can down the road as we don't seem to have all
cases covered here or any ongoing investment to cover those cases, that I am
aware of.
In general, with JDBC, recommendations people have seem to be to not run
simultaneous queries within same jdbc connection. (Also, jdbc statement calls
are blocking, so its not easy to do without multiple threads).
Is it possible to change Hue to use different sessions for each query ?
We can still get this patch in, but I just want to get the conversation going
on the long term solution for this.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
> Fix For: 3.0.0
>
> Attachments: HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197868#comment-16197868
]
Sergey Shelukhin commented on HIVE-17502:
-
Because currently each SessionState (Hive session) can only have one Tez
session. With this patch it's possible to reuse the Hive session, utilizing
several Tez sessions. So the code that gets/sets/closes/etc. the session by
getting it from SessionState will become invalid I assume
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
> Fix For: 3.0.0
>
> Attachments: HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to master &/ the other
> branches is acceptable. I'm not an contributor &/ committer, this will be my
> first time contributing to Hive and the Apache foundation. Any early review
> is greatly appreciated, thanks!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197864#comment-16197864
]
Thai Bui commented on HIVE-17502:
-
Sure, I'm happy to add additional checks, although nothing should be affected
unless the new flag is turned on. I'm not sure what you mean
SessionState::getTezSession and all related methods will become invalid, could
you clarify?
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
> Fix For: 3.0.0
>
> Attachments: HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to master &/ the other
> branches is acceptable. I'm not an contributor &/ committer, this will be my
> first time contributing to Hive and the Apache foundation. Any early review
> is greatly appreciated, thanks!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197830#comment-16197830
]
Sergey Shelukhin commented on HIVE-17502:
-
Hmm.. SessionState::getTezSession and all related methods will become invalid
now, right? I wonder if they need additional correctness checks for this case.
[~thejas] any other input?
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
> Fix For: 3.0.0
>
> Attachments: HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to master &/ the other
> branches is acceptable. I'm not an contributor &/ committer, this will be my
> first time contributing to Hive and the Apache foundation. Any early review
> is greatly appreciated, thanks!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197356#comment-16197356
]
Thai Bui commented on HIVE-17502:
-
Those test failures are unrelated to my changes and it looks like all the
current builds are failing with the same error.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
> Fix For: 3.0.0
>
> Attachments: HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to master &/ the other
> branches is acceptable. I'm not an contributor &/ committer, this will be my
> first time contributing to Hive and the Apache foundation. Any early review
> is greatly appreciated, thanks!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196434#comment-16196434
]
Hive QA commented on HIVE-17502:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12890993/HIVE-17502.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 11191 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
(batchId=231)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
(batchId=231)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_nullscan]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14]
(batchId=239)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7186/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7186/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7186/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12890993 - PreCommit-HIVE-Build
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
> Fix For: 3.0.0
>
> Attachments: HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]:
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196398#comment-16196398
]
Thai Bui commented on HIVE-17502:
-
[~sershe] Based on our previous discussions, I've made a patch to add an option
to allow default sessions reuse. Please review when you have time and let me
know if there's anything I need to change. Thanks!
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
> Fix For: 3.0.0
>
> Attachments: HIVE-17502.patch
>
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to master &/ the other
> branches is acceptable. I'm not an contributor &/ committer, this will be my
> first time contributing to Hive and the Apache foundation. Any early review
> is greatly appreciated, thanks!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180195#comment-16180195
]
Sergey Shelukhin commented on HIVE-17502:
-
The config option sounds good to me, esp. if we can limit it to HS2 pool
sessions that are not ever directly reused anyway. [~thejas] wdyt?
Also, do we have a list or a notion of why sessionstate/hivesessionimpl object
couldn't used in parallel?
ThreadLocal is not an obstacle in itself but rather an artifact on not having a
good dependency injection-type logic for most of Hive compile, similar to other
globals.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to master &/ the other
> branches is acceptable. I'm not an contributor &/ committer, this will be my
> first time contributing to Hive and the Apache foundation. Any early review
> is greatly appreciated, thanks!
--
This message was sent by Atlassian JIRA
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16173846#comment-16173846
]
Thai Bui commented on HIVE-17502:
-
I see what you mean. So without HiveSession(Impl) and SessionState to support
reusing of sessions from the same user, the patch is not complete.
Interestingly, we have been using this patch for a couple of weeks with 3-5
concurrent users each making and reusing sessions according to this logics and
things were fine. I did observe a couple of weird problems but I don't thing
they are related.
Anyhow, I'm happy to contribute and/or solidify the HiveSession(Impl) and/or
SessionState to make them immutable and/or stateless more suitable to be reused
by multiple sessions. For example, the current SessionState is bind to a
thread-local static object, if it's bind to a unique SessionID locked in a
database (hive metadata store?), or Zookeeper, or HDFS files, things could have
been different.
It is also possible to change Hue to only issue 1 query at a time per user but
the point was to go beyond that to allow a much better user experience using
Hive 2 w/ LLAP + Hue 4.
Let me know what you guys think, we (our big data & analytics group at
Bazaarvoice) are happy to contribute. Currently we have to build a custom
hive-exec jar with this logic and deploy this jar specifically in Ambari for
this to work. Both worlds are fine but I would prefer to push the patch
upstream to make it official and potentially solidify HiveSession
implementation incremental. For example, I think making a new HS2 config option
`hive.sessions.default-session.reuse=false` could work for this patch. If the
option is false (by default), the logics stay the same and an exception is
thrown, if true, then the new patch logics apply, allowing avid users to have
multiple sessions per user. Understandably, having too many options is
confusing. If that's the case we'll just close this ticket but thanks for the
discussion either way!
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
>
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16172568#comment-16172568
]
Sergey Shelukhin commented on HIVE-17502:
-
[~thai.bui] there's a little bit of confusion because there are 2 separate
sessions, and 3 objects, involved.
There's Hive (HS2) session, related to the JDBC connection and represented by
HiveSession(Impl) and SessionState (separate for historical reasons that are
probably not relevant anymore), and then there's TezClient, that represents the
Tez session and AM.
The patch addresses the reuse of the latter correctly; it skips the one in use
and returns a new one instead. However, it still uses the same Hive session
(incl. SessionState) in parallel, which is also not supported.
Is it possible to change Hue to only issue one query at a time per JDBC/Hive
session? It might be possible to add behavior to Hive to catch such behavior
and to use/open a separate Hive session (starting with HiveSessionImpl...,
etc.), but I'm not sure how practical that is and it's doesn't seen to me like
a good solution.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170272#comment-16170272
]
Thai Bui commented on HIVE-17502:
-
[~thejas] I'm not sure if we are on the same page, but my intention was to not
share the same session concurrently (meaning a session being used by 2 distinct
query at the same time). My intention was to allow skipping of a default
session and return an unused default session when a client has made a request
that contains a currently used session.
For my use case, it is Hue4 that is making the concurrent requests from the
same session. In Hue4, each user can work on a notebook-like environment. Each
notebook could have multiple snippets (text, or Hive query), with this model, a
user could execute multiple Hive snippets in the same notebook. Thus, there's a
need for this patch. Without the patch, HS2 will complain because the currently
used session should have been returned to the pool before the second request
was made.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
>
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168537#comment-16168537
]
Sergey Shelukhin commented on HIVE-17502:
-
HIVE-14247 is the correct link for the patch
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to master &/ the other
> branches is acceptable. I'm not an contributor &/ committer, this will be my
> first time contributing to Hive and the Apache foundation. Any early review
> is greatly appreciated, thanks!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168535#comment-16168535
]
Thejas M Nair commented on HIVE-17502:
--
The SessionState has state for the session/jdbc connection. So its meant to be
used for everything in that session.
However, it is not a class that we have worked on to ensure thread safety.
I had created a patch to disabled parallel use of single session because of
safety issues, but forgot to commit it - HIVE-11402 .
[~thai.bui]
What is the purpose of sharing the same connection for different queries at
same time ?
Its not something that api's like JDBC easily allow and not something usually
recommended in the RDBMS world. Sharing the session won't give you any
significant advantages in terms of resource utilization in LLAP mode AFAIK.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to master &/ the other
> branches
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16168391#comment-16168391
]
Sergey Shelukhin commented on HIVE-17502:
-
The problem are methods in HiveSessionImpl that call
SessionState.setCurrentSessionState(sessionState), setting the same
sessionState object to different threads when this pattern is used (client
requesting the same Hive session from different threads).
So, the SessionState object is getting reused even though it's accessed thru a
threadlocal.
My understanding was that this was not supported, so the client should open
separate sessions for parallel requests. However, I'm not sure about that.
[~thejas] [~vgumashta] can you clarify on Hive session usage w/HS2?
If the same SessionState can be reused otherwise, it needs to be changed to
support multiple Tez sessions.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is provided in my branch to demonstrate how it works. If possible
> I would like this patch to be applied to version 2.1, 2.2 and master. Since
> we are using 2.1 LLAP in production with Hue 4, this patch is critical to our
> success.
> Alternatively, if this patch is too broad in scope, I propose adding an
> option to allow "skipping of currently used default sessions". With this new
> option default to "false", existing behavior won't change unless the option
> is turned on.
> I will prepare an official path if this change to
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16165060#comment-16165060
]
Thai Bui commented on HIVE-17502:
-
[~sershe] Thanks for the prompt reply. I see that the SessionState.get keeps a
thread's static SessionState object which gets set/unset every time a session
is returned to the pool. I'm not that familiar with Hive/TezPool/TezTask
session code base, but given that it is thread local, it shouldn't impact other
thread's and/or client, correct?
Also, to help us understand my use case a little bit better, I will try to
elaborate what I'm trying to do a little bit better.
Let's say a client C1 requests for a new session, it is handled by thread T1 in
HS2, which yields session S1 from the pool. The same client C1 wants to make a
parallel request and reuses S1's in the request. Now, since S1 is still being
used, the session pool should skip S1 and returns S2 from the pool, given that
S2 is unused.
Since, S2 should be created by another thread T2 since T1 was busy executing
the TezTask in the first request in HS2. Given that SessionState.get is
thread's local to only T1, T2 should gets a different SessionState.get, which
yield a different SessionState. But I think that is not the case,
SessionState.get will always yield the same currently used SessionState of the
client C1, which will be unset and that is problem, correct?
If that's the case, what's the best course of actions that I can take to make
parallel query execution of the same client in a session pool happen?
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session
[jira] [Commented] (HIVE-17502) Reuse of default session should not throw an exception in LLAP w/ Tez
[
https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16162299#comment-16162299
]
Sergey Shelukhin commented on HIVE-17502:
-
There may be a bigger problem here.
I think the logic behind this exception is related to a giant try-finally in
TezTask after SessionState.get().getTezSession is called.
In the end of the finally, returnSession is called, which will ignore the
non-pool session, but for a pool session it would put it back in the pool and
reset SessionState's Tez session to null. This is not necessarily the best
logic, cause maybe pool session reuse could also be allowed, but for obvious
reasons (limits of the pool) holding a session from others is undesirable.
However it's beside the point as, unless I'm missing something, the only way
for getTezSession before the block to get an existing pool tez session is if
SessionState is used for two Hive queries in parallel (i.e.
SessionState.get.getTezSession is called from one thread while another is
inside of TezTask try-finally block, running some query). Let me know if it's a
different scenario.
This pattern would probably break all this session setting and unsetting, which
would need to be fixed with additional checks; however I think it also has
other similar state-related issues.
[~thejas] do you know if we allow parallel queries with the same SessionState.
> Reuse of default session should not throw an exception in LLAP w/ Tez
> -
>
> Key: HIVE-17502
> URL: https://issues.apache.org/jira/browse/HIVE-17502
> Project: Hive
> Issue Type: Bug
> Components: llap, Tez
>Affects Versions: 2.1.1, 2.2.0
> Environment: HDP 2.6.1.0-129, Hue 4
>Reporter: Thai Bui
>Assignee: Thai Bui
>
> Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be
> skipped mostly because of this line
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365.
> However, some clients such as Hue 4, allow multiple sessions to be used per
> user. Under this configuration, a Thrift client will send a request to either
> reuse or open a new session. The reuse request could include the session id
> of a currently used snippet being executed in Hue, this causes HS2 to throw
> an exception:
> {noformat}
> 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user:
> hive, session user: hive
> 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task
> (TezTask.java:execute(232)) - Failed to execute tez graph.
> org.apache.hadoop.hive.ql.metadata.HiveException: The pool session
> sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive,
> doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have
> been returned to the pool
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544)
> ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79)
> [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129]
> {noformat}
> Note that every query is issued as a single 'hive' user to share the LLAP
> daemon pool, a set of pre-determined number of AMs is initialized at setup
> time. Thus, HS2 should allow new sessions from a Thrift client to be used out
> of the pool, or an existing session to be skipped and an unused session from
> the pool to be returned. The logic to throw an exception in the
> `canWorkWithSameSession` doesn't make sense to me.
> I have a solution to fix this issue in my local branch at
> https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70.
> When applied, the log will become like so
> {noformat}
> 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager
> (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default
> session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap,
> user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms
> since it is being used.
> {noformat}
> A test case is
