[jira] [Commented] (HIVE-13511) Run clidriver tests from within the qtest dir for the precommit tests
[ https://issues.apache.org/jira/browse/HIVE-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15306126#comment-15306126 ] Sergio Peña commented on HIVE-13511: I just restarted the server to reduce the number of drones per box to 2. Thanks for taking a look at this. At some point, I was going to reduce the number to 1 to start experimenting and see if the HMS issues and timeout issues were related to memory. Now that we're on this, let's see how it goes with 2 drones. > Run clidriver tests from within the qtest dir for the precommit tests > - > > Key: HIVE-13511 > URL: https://issues.apache.org/jira/browse/HIVE-13511 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Fix For: 2.1.0 > > Attachments: HIVE-13511.01.patch, HIVE-13511.02.patch, > HIVE-13511.03.addendum.patch, HIVE-13511.03.patch, example_maven-test.txt, > example_testExecution.txt, failedScriptPostPatch.txt, > miniTezKilledLogs_run432.tar.gz, sparkTestMemAllocateErrorLogs_run432.tar.gz > > > The tests are currently run from the itests directory - which means there's > additional overhead of having to at least check whether files have changed. > Will attach a sample output - this adds up to 40+ seconds per batch. Getting > rid of this should be a reasonable saving overall. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13511) Run clidriver tests from within the qtest dir for the precommit tests
[ https://issues.apache.org/jira/browse/HIVE-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1530#comment-1530 ] Siddharth Seth commented on HIVE-13511: --- Thanks [~spena]. I think the problem introduced by this patch is now resolved. However, the new runs are seeing a lot more failures. There's some interesting stuff in the logs 1. It looks like test runs are getting killed randomly (I believe this problem existed earlier as well ?) 2. Some tests fail to even start with not enough memory available for the VM to continue {code} # # There is insufficient memory for the Java Runtime Environment to continue. # Native memory allocation (mmap) failed to map 105381888 bytes for committing reserved memory. # An error report file with more information is saved as: # /home/hiveptest/54.177.86.160-hiveptest-1/apache-github-source-source/itests/qtest-spark/hs_err_pid9151.log {code} I wonder if the random kills are related as well. i.e. at some point the box gets a little overloaded and the OS decides to kill processes ? syslog output on the systems would be interesting to look at. >From looking at various process sizes maven seems to be 2GB heap per test (potentially 2 processes per testt?) TestMiniTezCliDriver - runs AMs and Containers with a heap of 819M Spark tests run AMs and executors with a heap of 1G each I can look at reducing the size occupied by Tez AMs and containers. Given the small amount of data processed in tests - this change seems reasonable. Can the same be done for Spark ? Alternately, should we try reducing the number of drones per box to 2 instead of 3, and see how that affects the system ? > Run clidriver tests from within the qtest dir for the precommit tests > - > > Key: HIVE-13511 > URL: https://issues.apache.org/jira/browse/HIVE-13511 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Fix For: 2.1.0 > > Attachments: HIVE-13511.01.patch, HIVE-13511.02.patch, > HIVE-13511.03.addendum.patch, HIVE-13511.03.patch, example_maven-test.txt, > example_testExecution.txt, failedScriptPostPatch.txt > > > The tests are currently run from the itests directory - which means there's > additional overhead of having to at least check whether files have changed. > Will attach a sample output - this adds up to 40+ seconds per batch. Getting > rid of this should be a reasonable saving overall. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13511) Run clidriver tests from within the qtest dir for the precommit tests
[ https://issues.apache.org/jira/browse/HIVE-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15305348#comment-15305348 ] Sergio Peña commented on HIVE-13511: Thanks [~sseth] for letting me know. I applied the patch (revert the old one, and re-apply both patches). I'll restart the server. > Run clidriver tests from within the qtest dir for the precommit tests > - > > Key: HIVE-13511 > URL: https://issues.apache.org/jira/browse/HIVE-13511 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Fix For: 2.1.0 > > Attachments: HIVE-13511.01.patch, HIVE-13511.02.patch, > HIVE-13511.03.addendum.patch, HIVE-13511.03.patch, example_maven-test.txt, > example_testExecution.txt, failedScriptPostPatch.txt > > > The tests are currently run from the itests directory - which means there's > additional overhead of having to at least check whether files have changed. > Will attach a sample output - this adds up to 40+ seconds per batch. Getting > rid of this should be a reasonable saving overall. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13511) Run clidriver tests from within the qtest dir for the precommit tests
[ https://issues.apache.org/jira/browse/HIVE-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15299858#comment-15299858 ] Jesus Camacho Rodriguez commented on HIVE-13511: [~sseth], ready to be pushed to 2.1.0? Thanks > Run clidriver tests from within the qtest dir for the precommit tests > - > > Key: HIVE-13511 > URL: https://issues.apache.org/jira/browse/HIVE-13511 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-13511.01.patch, HIVE-13511.02.patch, > HIVE-13511.03.patch, example_maven-test.txt, example_testExecution.txt > > > The tests are currently run from the itests directory - which means there's > additional overhead of having to at least check whether files have changed. > Will attach a sample output - this adds up to 40+ seconds per batch. Getting > rid of this should be a reasonable saving overall. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13511) Run clidriver tests from within the qtest dir for the precommit tests
[ https://issues.apache.org/jira/browse/HIVE-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15265989#comment-15265989 ] Ashutosh Chauhan commented on HIVE-13511: - +1 > Run clidriver tests from within the qtest dir for the precommit tests > - > > Key: HIVE-13511 > URL: https://issues.apache.org/jira/browse/HIVE-13511 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-13511.01.patch, HIVE-13511.02.patch, > HIVE-13511.03.patch, example_maven-test.txt, example_testExecution.txt > > > The tests are currently run from the itests directory - which means there's > additional overhead of having to at least check whether files have changed. > Will attach a sample output - this adds up to 40+ seconds per batch. Getting > rid of this should be a reasonable saving overall. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13511) Run clidriver tests from within the qtest dir for the precommit tests
[ https://issues.apache.org/jira/browse/HIVE-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15250087#comment-15250087 ] Ashutosh Chauhan commented on HIVE-13511: - This patch fails to compile when I ran {{ mvn clean install -DskipTests}} from within ptest2 dir, with following trace: {code} [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile (default-testCompile) on project hive-ptest: Compilation failure: Compilation failure: [ERROR] /Users/ashutosh/workspace/apache-master/testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/conf/TestQFileTestBatch.java:[45,28] constructor QFileTestBatch in class org.apache.hive.ptest.execution.conf.QFileTestBatch cannot be applied to given types; [ERROR] required: java.lang.String,java.lang.String,java.lang.String,java.util.Set,boolean,java.lang.String [ERROR] found: java.lang.String,java.lang.String,java.lang.String,java.util.Set,boolean [ERROR] reason: actual and formal argument lists differ in length [ERROR] /Users/ashutosh/workspace/apache-master/testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/conf/TestQFileTestBatch.java:[55,28] constructor QFileTestBatch in class org.apache.hive.ptest.execution.conf.QFileTestBatch cannot be applied to given types; [ERROR] required: java.lang.String,java.lang.String,java.lang.String,java.util.Set,boolean,java.lang.String [ERROR] found: java.lang.String,java.lang.String,java.lang.String,java.util.Set,boolean [ERROR] reason: actual and formal argument lists differ in length [ERROR] /Users/ashutosh/workspace/apache-master/testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/conf/TestQFileTestBatch.java:[61,28] constructor QFileTestBatch in class org.apache.hive.ptest.execution.conf.QFileTestBatch cannot be applied to given types; [ERROR] required: java.lang.String,java.lang.String,java.lang.String,java.util.Set,boolean,java.lang.String [ERROR] found: java.lang.String,java.lang.String,java.lang.String,java.util.Set,boolean [ERROR] reason: actual and formal argument lists differ in length [ERROR] /Users/ashutosh/workspace/apache-master/testutils/ptest2/src/test/java/org/apache/hive/ptest/execution/TestExecutionPhase.java:[72,17] constructor QFileTestBatch in class org.apache.hive.ptest.execution.conf.QFileTestBatch cannot be applied to given types; [ERROR] required: java.lang.String,java.lang.String,java.lang.String,java.util.Set,boolean,java.lang.String [ERROR] found: java.lang.String,java.lang.String,java.lang.String,java.util.HashSet,boolean [ERROR] reason: actual and formal argument lists differ in length {code} > Run clidriver tests from within the qtest dir for the precommit tests > - > > Key: HIVE-13511 > URL: https://issues.apache.org/jira/browse/HIVE-13511 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-13511.01.patch, HIVE-13511.02.patch, > example_maven-test.txt, example_testExecution.txt > > > The tests are currently run from the itests directory - which means there's > additional overhead of having to at least check whether files have changed. > Will attach a sample output - this adds up to 40+ seconds per batch. Getting > rid of this should be a reasonable saving overall. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13511) Run clidriver tests from within the qtest dir for the precommit tests
[ https://issues.apache.org/jira/browse/HIVE-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15244139#comment-15244139 ] Hive QA commented on HIVE-13511: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12798983/HIVE-13511.02.patch {color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 9982 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver_index_bitmap3 {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7615/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/7615/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-7615/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12798983 - PreCommit-HIVE-TRUNK-Build > Run clidriver tests from within the qtest dir for the precommit tests > - > > Key: HIVE-13511 > URL: https://issues.apache.org/jira/browse/HIVE-13511 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-13511.01.patch, HIVE-13511.02.patch, > example_maven-test.txt, example_testExecution.txt > > > The tests are currently run from the itests directory - which means there's > additional overhead of having to at least check whether files have changed. > Will attach a sample output - this adds up to 40+ seconds per batch. Getting > rid of this should be a reasonable saving overall. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13511) Run clidriver tests from within the qtest dir for the precommit tests
[ https://issues.apache.org/jira/browse/HIVE-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15242561#comment-15242561 ] Siddharth Seth commented on HIVE-13511: --- Same comment as HIVE-13505, lets commit and monitor a run. This one would need a dedicated run to itself. If it's high enough in the build queue - this can wait till it's turn. > Run clidriver tests from within the qtest dir for the precommit tests > - > > Key: HIVE-13511 > URL: https://issues.apache.org/jira/browse/HIVE-13511 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-13511.01.patch, example_maven-test.txt, > example_testExecution.txt > > > The tests are currently run from the itests directory - which means there's > additional overhead of having to at least check whether files have changed. > Will attach a sample output - this adds up to 40+ seconds per batch. Getting > rid of this should be a reasonable saving overall. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13511) Run clidriver tests from within the qtest dir for the precommit tests
[ https://issues.apache.org/jira/browse/HIVE-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241966#comment-15241966 ] Szehon Ho commented on HIVE-13511: -- Agree in principle with the change, though we will have to try to see if it works. > Run clidriver tests from within the qtest dir for the precommit tests > - > > Key: HIVE-13511 > URL: https://issues.apache.org/jira/browse/HIVE-13511 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-13511.01.patch, example_maven-test.txt, > example_testExecution.txt > > > The tests are currently run from the itests directory - which means there's > additional overhead of having to at least check whether files have changed. > Will attach a sample output - this adds up to 40+ seconds per batch. Getting > rid of this should be a reasonable saving overall. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13511) Run clidriver tests from within the qtest dir for the precommit tests
[ https://issues.apache.org/jira/browse/HIVE-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15241627#comment-15241627 ] Ashutosh Chauhan commented on HIVE-13511: - I think batches are created only for *CliDriver not for junit tests, so this should be ok. +1 > Run clidriver tests from within the qtest dir for the precommit tests > - > > Key: HIVE-13511 > URL: https://issues.apache.org/jira/browse/HIVE-13511 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: HIVE-13511.01.patch, example_maven-test.txt, > example_testExecution.txt > > > The tests are currently run from the itests directory - which means there's > additional overhead of having to at least check whether files have changed. > Will attach a sample output - this adds up to 40+ seconds per batch. Getting > rid of this should be a reasonable saving overall. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13511) Run clidriver tests from within the qtest dir for the precommit tests
[ https://issues.apache.org/jira/browse/HIVE-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15240335#comment-15240335 ] Siddharth Seth commented on HIVE-13511: --- {code} main: [delete] Deleting directory /home/hiveptest/54.205.21.173-hiveptest-2/apache-github-source-source/itests/qtest-spark/target/tmp [delete] Deleting directory /home/hiveptest/54.205.21.173-hiveptest-2/apache-github-source-source/itests/qtest-spark/target/warehouse [mkdir] Created dir: /home/hiveptest/54.205.21.173-hiveptest-2/apache-github-source-source/itests/qtest-spark/target/tmp [mkdir] Created dir: /home/hiveptest/54.205.21.173-hiveptest-2/apache-github-source-source/itests/qtest-spark/target/warehouse [mkdir] Created dir: /home/hiveptest/54.205.21.173-hiveptest-2/apache-github-source-source/itests/qtest-spark/target/tmp/conf [copy] Copying 15 files to /home/hiveptest/54.205.21.173-hiveptest-2/apache-github-source-source/itests/qtest-spark/target/tmp/conf [INFO] Executed tasks [INFO] [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ hive-it-qfile-spark --- [INFO] Compiling 3 source files to /home/hiveptest/54.205.21.173-hiveptest-2/apache-github-source-source/itests/qtest-spark/target/test-classes [INFO] [INFO] --- maven-surefire-plugin:2.16:test (default-test) @ hive-it-qfile-spark --- [INFO] [INFO] Reactor Summary: [INFO] [INFO] Hive Integration - Parent .. SUCCESS [ 1.781 s] [INFO] Hive Integration - Custom Serde SUCCESS [ 4.906 s] [INFO] Hive Integration - Custom udfs . SUCCESS [ 1.529 s] [INFO] Hive Integration - Custom UDFs - udf-classloader-util SUCCESS [ 1.007 s] [INFO] Hive Integration - Custom UDFs - udf-classloader-udf1 SUCCESS [ 0.903 s] [INFO] Hive Integration - Custom UDFs - udf-classloader-udf2 SUCCESS [ 1.172 s] [INFO] Hive Integration - HCatalog Unit Tests . SUCCESS [ 5.339 s] [INFO] Hive Integration - Testing Utilities ... SUCCESS [ 4.839 s] [INFO] Hive Integration - Unit Tests .. SUCCESS [ 10.251 s] [INFO] Hive Integration - Test Serde .. SUCCESS [ 0.769 s] [INFO] Hive Integration - QFile Tests . SUCCESS [05:25 min] [INFO] Hive Integration - QFile Accumulo Tests SUCCESS [ 4.824 s] [INFO] JMH benchmark: Hive SUCCESS [ 0.804 s] [INFO] Hive Integration - Unit Tests - Hadoop 2 ... SUCCESS [ 3.006 s] [INFO] Hive Integration - Unit Tests with miniKdc . SUCCESS [ 2.205 s] [INFO] Hive Integration - QFile Spark Tests ... SUCCESS [ 6.765 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 06:17 min [INFO] Finished at: 2016-04-13T16:52:49-04:00 [INFO] Final Memory: 166M/903M [INFO] {code} This is while running some clidriver tests. The only thing that should be built here is the qtests dir. > Run clidriver tests from within the qtest dir for the precommit tests > - > > Key: HIVE-13511 > URL: https://issues.apache.org/jira/browse/HIVE-13511 > Project: Hive > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Siddharth Seth > Attachments: example_maven-test.txt, example_testExecution.txt > > > The tests are currently run from the itests directory - which means there's > additional overhead of having to at least check whether files have changed. > Will attach a sample output - this adds up to 40+ seconds per batch. Getting > rid of this should be a reasonable saving overall. -- This message was sent by Atlassian JIRA (v6.3.4#6332)