[jira] [Commented] (HIVE-11923) allow qtests to run via a single client session for tez and llap
[ https://issues.apache.org/jira/browse/HIVE-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903790#comment-14903790 ] Siddharth Seth commented on HIVE-11923: --- [~sershe] - not much point submitting this to jenkins yet. There's a lot of failures. Some potential issues like 11924, others are caused by the fact that the TransactionManager can be set once per session. Will upload another patch which resets the SessionState, but uses the same AM. > allow qtests to run via a single client session for tez and llap > > > Key: HIVE-11923 > URL: https://issues.apache.org/jira/browse/HIVE-11923 > Project: Hive > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-11923.1.txt, HIVE-11923.branch-1.txt > > > Launching a new session - AM and containers for each test adds unnecessary > overheads. Running via a single session should reduce the run time > significantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11923) allow qtests to run via a single client session for tez and llap
[ https://issues.apache.org/jira/browse/HIVE-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14934343#comment-14934343 ] Prasanth Jayachandran commented on HIVE-11923: -- LGTM, +1. Pending tests and the resolving of the blocking issue. > allow qtests to run via a single client session for tez and llap > > > Key: HIVE-11923 > URL: https://issues.apache.org/jira/browse/HIVE-11923 > Project: Hive > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-11923.1.txt, HIVE-11923.2.branchllap.txt, > HIVE-11923.2.txt, HIVE-11923.branch-1.txt > > > Launching a new session - AM and containers for each test adds unnecessary > overheads. Running via a single session should reduce the run time > significantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11923) allow qtests to run via a single client session for tez and llap
[ https://issues.apache.org/jira/browse/HIVE-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14934411#comment-14934411 ] Prasanth Jayachandran commented on HIVE-11923: -- Committed llap patch to branch. > allow qtests to run via a single client session for tez and llap > > > Key: HIVE-11923 > URL: https://issues.apache.org/jira/browse/HIVE-11923 > Project: Hive > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-11923.1.txt, HIVE-11923.2.branchllap.txt, > HIVE-11923.2.txt, HIVE-11923.branch-1.txt > > > Launching a new session - AM and containers for each test adds unnecessary > overheads. Running via a single session should reduce the run time > significantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11923) allow qtests to run via a single client session for tez and llap
[ https://issues.apache.org/jira/browse/HIVE-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14934445#comment-14934445 ] Siddharth Seth commented on HIVE-11923: --- Removing HIVE-11924 as a blocker. SessionState is being reset in the latest patch - so the issue does not show up while running tests. However, it still appears to be a valid issue. > allow qtests to run via a single client session for tez and llap > > > Key: HIVE-11923 > URL: https://issues.apache.org/jira/browse/HIVE-11923 > Project: Hive > Issue Type: Improvement >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-11923.1.txt, HIVE-11923.2.branchllap.txt, > HIVE-11923.2.txt, HIVE-11923.branch-1.txt > > > Launching a new session - AM and containers for each test adds unnecessary > overheads. Running via a single session should reduce the run time > significantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11923) allow qtests to run via a single client session for tez and llap
[ https://issues.apache.org/jira/browse/HIVE-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942451#comment-14942451 ] Hive QA commented on HIVE-11923: {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12764701/HIVE-11923.2.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 9640 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_scriptfile1 org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5516/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5516/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5516/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12764701 - PreCommit-HIVE-TRUNK-Build > allow qtests to run via a single client session for tez and llap > > > Key: HIVE-11923 > URL: https://issues.apache.org/jira/browse/HIVE-11923 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Affects Versions: 1.3.0, 2.0.0 >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-11923.1.txt, HIVE-11923.2.branchllap.txt, > HIVE-11923.2.patch, HIVE-11923.2.txt, HIVE-11923.2.txt, > HIVE-11923.branch-1.txt > > > Launching a new session - AM and containers for each test adds unnecessary > overheads. Running via a single session should reduce the run time > significantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11923) allow qtests to run via a single client session for tez and llap
[ https://issues.apache.org/jira/browse/HIVE-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14942461#comment-14942461 ] Prasanth Jayachandran commented on HIVE-11923: -- [~sseth] the test failure looks related. Can you take a look? > allow qtests to run via a single client session for tez and llap > > > Key: HIVE-11923 > URL: https://issues.apache.org/jira/browse/HIVE-11923 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Affects Versions: 1.3.0, 2.0.0 >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-11923.1.txt, HIVE-11923.2.branchllap.txt, > HIVE-11923.2.patch, HIVE-11923.2.txt, HIVE-11923.2.txt, > HIVE-11923.branch-1.txt > > > Launching a new session - AM and containers for each test adds unnecessary > overheads. Running via a single session should reduce the run time > significantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11923) allow qtests to run via a single client session for tez and llap
[ https://issues.apache.org/jira/browse/HIVE-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949364#comment-14949364 ] Sergey Shelukhin commented on HIVE-11923: - The failure is HIVE-12072. Before a real fix is available (which as far as I understood might also need Tez changes), can you exclude this test from reuse somehow? Perhaps via a new magic test variable akin to SORT_QUERY_RESULTS? > allow qtests to run via a single client session for tez and llap > > > Key: HIVE-11923 > URL: https://issues.apache.org/jira/browse/HIVE-11923 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Affects Versions: 1.3.0, 2.0.0 >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-11923.1.txt, HIVE-11923.2.branchllap.txt, > HIVE-11923.2.patch, HIVE-11923.2.txt, HIVE-11923.2.txt, > HIVE-11923.branch-1.txt > > > Launching a new session - AM and containers for each test adds unnecessary > overheads. Running via a single session should reduce the run time > significantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11923) allow qtests to run via a single client session for tez and llap
[ https://issues.apache.org/jira/browse/HIVE-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949377#comment-14949377 ] Sergey Shelukhin commented on HIVE-11923: - Will update patch soonish > allow qtests to run via a single client session for tez and llap > > > Key: HIVE-11923 > URL: https://issues.apache.org/jira/browse/HIVE-11923 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Affects Versions: 1.3.0, 2.0.0 >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-11923.1.txt, HIVE-11923.2.branchllap.txt, > HIVE-11923.2.patch, HIVE-11923.2.txt, HIVE-11923.2.txt, > HIVE-11923.branch-1.txt > > > Launching a new session - AM and containers for each test adds unnecessary > overheads. Running via a single session should reduce the run time > significantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11923) allow qtests to run via a single client session for tez and llap
[ https://issues.apache.org/jira/browse/HIVE-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14951875#comment-14951875 ] Hive QA commented on HIVE-11923: {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12765694/HIVE-11923.03.patch {color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 9659 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_scriptfile1 org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_stats_counter org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_stats_counter_partitioned org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_temp_table_external org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_truncate_column_buckets org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_uber_reduce org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_scriptfile1 org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_scriptfile1 org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5596/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5596/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5596/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12765694 - PreCommit-HIVE-TRUNK-Build > allow qtests to run via a single client session for tez and llap > > > Key: HIVE-11923 > URL: https://issues.apache.org/jira/browse/HIVE-11923 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Affects Versions: 1.3.0, 2.0.0 >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-11923.03.patch, HIVE-11923.1.txt, > HIVE-11923.2.branchllap.txt, HIVE-11923.2.patch, HIVE-11923.2.txt, > HIVE-11923.2.txt, HIVE-11923.branch-1.txt > > > Launching a new session - AM and containers for each test adds unnecessary > overheads. Running via a single session should reduce the run time > significantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11923) allow qtests to run via a single client session for tez and llap
[ https://issues.apache.org/jira/browse/HIVE-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14955041#comment-14955041 ] Hive QA commented on HIVE-11923: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12766239/HIVE-11923.04.patch {color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 9660 tests executed *Failed tests:* {noformat} TestCliDriver-ppd_outer_join2.q-groupby3_map_skew.q-merge_dynamic_partition2.q-and-12-more - did not produce a TEST-*.xml file org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation org.apache.hive.jdbc.TestSSL.testSSLVersion {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5628/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5628/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5628/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12766239 - PreCommit-HIVE-TRUNK-Build > allow qtests to run via a single client session for tez and llap > > > Key: HIVE-11923 > URL: https://issues.apache.org/jira/browse/HIVE-11923 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Affects Versions: 1.3.0, 2.0.0 >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-11923.03.patch, HIVE-11923.04.patch, > HIVE-11923.1.txt, HIVE-11923.2.branchllap.txt, HIVE-11923.2.patch, > HIVE-11923.2.txt, HIVE-11923.2.txt, HIVE-11923.branch-1.txt > > > Launching a new session - AM and containers for each test adds unnecessary > overheads. Running via a single session should reduce the run time > significantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11923) allow qtests to run via a single client session for tez and llap
[ https://issues.apache.org/jira/browse/HIVE-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14955455#comment-14955455 ] Sergey Shelukhin commented on HIVE-11923: - [~prasanth_j] [~hagleitn] can you take a look at the updated patch? The bad instance failed due to some spark(?) download nonsense. {noformat} [exec] + /bin/pwd [exec] + BASE_DIR=./target [exec] + HIVE_ROOT=./target/../../../ [exec] + DOWNLOAD_DIR=./../thirdparty [exec] + mkdir -p ./../thirdparty [exec] /home/hiveptest/54.147.252.66-hiveptest-1/apache-github-source-source/itests/hive-unit [exec] + download http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-1.4.0-bin-hadoop2-without-hive.tgz spark [exec] + url=http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-1.4.0-bin-hadoop2-without-hive.tgz [exec] + finalName=spark [exec] ++ basename http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-1.4.0-bin-hadoop2-without-hive.tgz [exec] + tarName=spark-1.4.0-bin-hadoop2-without-hive.tgz [exec] + rm -rf ./target/spark [exec] + [[ ! -f ./../thirdparty/spark-1.4.0-bin-hadoop2-without-hive.tgz ]] [exec] + tar -zxf ./../thirdparty/spark-1.4.0-bin-hadoop2-without-hive.tgz -C ./target [exec] + mv ./target/spark-1.4.0-bin-hadoop2-without-hive ./target/spark [exec] + cp -f ./target/../../..//data/conf/spark/log4j2.xml ./target/spark/conf/ [exec] + sed '/package /d' /home/hiveptest/54.147.252.66-hiveptest-1/apache-github-source-source/itests/../contrib/src/java/org/apache/hadoop/hive/contrib/udf/example/UDFExampleAdd.java [exec] + javac -cp /home/hiveptest/54.147.252.66-hiveptest-1/maven/org/apache/hive/hive-exec/2.0.0-SNAPSHOT/hive-exec-2.0.0-SNAPSHOT.jar /tmp/UDFExampleAdd.java -d /tmp [exec] + jar -cf /tmp/udfexampleadd-1.0.jar -C /tmp UDFExampleAdd.class [exec] java.util.zip.ZipException: attempt to write past end of STORED entry [exec] at java.util.zip.ZipOutputStream.write(ZipOutputStream.java:319) [exec] at sun.tools.jar.Main.copy(Main.java:779) [exec] at sun.tools.jar.Main.copy(Main.java:793) [exec] at sun.tools.jar.Main.addFile(Main.java:740) [exec] at sun.tools.jar.Main.create(Main.java:491) [exec] at sun.tools.jar.Main.run(Main.java:201) [exec] at sun.tools.jar.Main.main(Main.java:1177) {noformat} > allow qtests to run via a single client session for tez and llap > > > Key: HIVE-11923 > URL: https://issues.apache.org/jira/browse/HIVE-11923 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Affects Versions: 1.3.0, 2.0.0 >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-11923.03.patch, HIVE-11923.04.patch, > HIVE-11923.1.txt, HIVE-11923.2.branchllap.txt, HIVE-11923.2.patch, > HIVE-11923.2.txt, HIVE-11923.2.txt, HIVE-11923.branch-1.txt > > > Launching a new session - AM and containers for each test adds unnecessary > overheads. Running via a single session should reduce the run time > significantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-11923) allow qtests to run via a single client session for tez and llap
[ https://issues.apache.org/jira/browse/HIVE-11923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14955494#comment-14955494 ] Prasanth Jayachandran commented on HIVE-11923: -- +1 for the workaround patch. > allow qtests to run via a single client session for tez and llap > > > Key: HIVE-11923 > URL: https://issues.apache.org/jira/browse/HIVE-11923 > Project: Hive > Issue Type: Improvement > Components: Testing Infrastructure >Affects Versions: 1.3.0, 2.0.0 >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-11923.03.patch, HIVE-11923.04.patch, > HIVE-11923.1.txt, HIVE-11923.2.branchllap.txt, HIVE-11923.2.patch, > HIVE-11923.2.txt, HIVE-11923.2.txt, HIVE-11923.branch-1.txt > > > Launching a new session - AM and containers for each test adds unnecessary > overheads. Running via a single session should reduce the run time > significantly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)