[ https://issues.apache.org/jira/browse/HIVE-17502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431843#comment-16431843 ]
Hive QA commented on HIVE-17502: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12890993/HIVE-17502.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/10109/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10109/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10109/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2018-04-10 07:28:05.554 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-10109/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2018-04-10 07:28:05.557 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at dcd9b59 HIVE-19146 : Delete dangling q.out + git clean -f -d + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at dcd9b59 HIVE-19146 : Delete dangling q.out + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2018-04-10 07:28:06.110 + rm -rf ../yetus_PreCommit-HIVE-Build-10109 + mkdir ../yetus_PreCommit-HIVE-Build-10109 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-10109 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-10109/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: patch failed: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:2425 Falling back to three-way merge... Applied patch to 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java' with conflicts. Going to apply patch with: git apply -p0 error: patch failed: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:2425 Falling back to three-way merge... Applied patch to 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java' with conflicts. U common/src/java/org/apache/hadoop/hive/conf/HiveConf.java + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12890993 - PreCommit-HIVE-Build > Reuse of default session should not throw an exception in LLAP w/ Tez > --------------------------------------------------------------------- > > Key: HIVE-17502 > URL: https://issues.apache.org/jira/browse/HIVE-17502 > Project: Hive > Issue Type: Bug > Components: llap, Tez > Affects Versions: 2.1.1, 2.2.0 > Environment: HDP 2.6.1.0-129, Hue 4 > Reporter: Thai Bui > Assignee: Thai Bui > Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-17502.patch > > > Hive2 w/ LLAP on Tez doesn't allow a currently used, default session to be > skipped mostly because of this line > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L365. > However, some clients such as Hue 4, allow multiple sessions to be used per > user. Under this configuration, a Thrift client will send a request to either > reuse or open a new session. The reuse request could include the session id > of a currently used snippet being executed in Hue, this causes HS2 to throw > an exception: > {noformat} > 2017-09-10T17:51:36,548 INFO [Thread-89]: tez.TezSessionPoolManager > (TezSessionPoolManager.java:canWorkWithSameSession(512)) - The current user: > hive, session user: hive > 2017-09-10T17:51:36,549 ERROR [Thread-89]: exec.Task > (TezTask.java:execute(232)) - Failed to execute tez graph. > org.apache.hadoop.hive.ql.metadata.HiveException: The pool session > sessionId=5b61a578-6336-41c5-860d-9838166f97fe, queueName=llap, user=hive, > doAs=false, isOpen=true, isDefault=true, expires in 591015330ms should have > been returned to the pool > at > org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.canWorkWithSameSession(TezSessionPoolManager.java:534) > ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129] > at > org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager.getSession(TezSessionPoolManager.java:544) > ~[hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129] > at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:147) > [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129] > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197) > [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129] > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129] > at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:79) > [hive-exec-2.1.0.2.6.1.0-129.jar:2.1.0.2.6.1.0-129] > {noformat} > Note that every query is issued as a single 'hive' user to share the LLAP > daemon pool, a set of pre-determined number of AMs is initialized at setup > time. Thus, HS2 should allow new sessions from a Thrift client to be used out > of the pool, or an existing session to be skipped and an unused session from > the pool to be returned. The logic to throw an exception in the > `canWorkWithSameSession` doesn't make sense to me. > I have a solution to fix this issue in my local branch at > https://github.com/thaibui/hive/commit/078a521b9d0906fe6c0323b63e567f6eee2f3a70. > When applied, the log will become like so > {noformat} > 2017-09-10T09:15:33,578 INFO [Thread-239]: tez.TezSessionPoolManager > (TezSessionPoolManager.java:canWorkWithSameSession(533)) - Skipping default > session sessionId=6638b1da-0f8a-405e-85f0-9586f484e6de, queueName=llap, > user=hive, doAs=false, isOpen=true, isDefault=true, expires in 591868732ms > since it is being used. > {noformat} > A test case is provided in my branch to demonstrate how it works. If possible > I would like this patch to be applied to version 2.1, 2.2 and master. Since > we are using 2.1 LLAP in production with Hue 4, this patch is critical to our > success. > Alternatively, if this patch is too broad in scope, I propose adding an > option to allow "skipping of currently used default sessions". With this new > option default to "false", existing behavior won't change unless the option > is turned on. > I will prepare an official path if this change to master &/ the other > branches is acceptable. I'm not an contributor &/ committer, this will be my > first time contributing to Hive and the Apache foundation. Any early review > is greatly appreciated, thanks! -- This message was sent by Atlassian JIRA (v7.6.3#76005)