Seems to always be from line 220:
Caused by: java.lang.RuntimeException: VMs did not start up within 120
seconds
at
org.apache.geode.test.dunit.standalone.DUnitLauncher.launch(DUnitLauncher.java:220)
at
org.apache.geode.test.dunit.standalone.DUnitLauncher.launchIfNeeded(DUnitLauncher.java:142)
... 34 more
Here's the code:
// Create a VM for the locator
processManager.launchVM(LOCATOR_VM_NUM);
// wait for the VM to start up
if (!processManager.waitForVMs(STARTUP_TIMEOUT)) {
220: throw new RuntimeException(STARTUP_TIMEOUT_MESSAGE);
}
And then this shows up in stderr of the ChildVM that's starting:
*[locator] Error: Could not find or load main class
org.apache.geode.test.dunit.standalone.ChildVM*
This is the full execution command for that ChildVM:
Executing [/usr/local/asfpackages/java/jdk1.8.0_102/jre/bin/java,
-classpath,
/home/jenkins/jenkins-slave/workspace/Geode-release/caches/2.14.1/workerMain/gradle-worker.jar,
-Dgemfire.DUnitLauncher.RMI_PORT=26491, -Dgemfire.DUnitLauncher.VM_NUM=-2,
-Dgemfire.DUnitLauncher.VM_VERSION=000,
-DWORKSPACE_DIR=/home/jenkins/jenkins-slave/workspace/Geode-release/geode-lucene/build/distributedTest/.,
-Dgemfire.enable-network-partition-detection=false, -Dlog-level=info,
-Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib,
-Xrunjdwp:transport=dt_socket,server=y,suspend=n,
-XX:+HeapDumpOnOutOfMemoryError, -Xmx512m,
-Dgemfire.DEFAULT_MAX_OPLOG_SIZE=10, -Dgemfire.disallowMcastDefaults=true,
-Dgemfire.use-ephemeral-ports=true, -ea, -XX:MetaspaceSize=512m,
-XX:+PrintGC, -XX:+PrintGCDetails, -XX:+PrintGCTimeStamps, -DdummyArg=true,
org.apache.geode.test.dunit.standalone.ChildVM]
Looks like the -classpath is empty except for gradle-worker.jar and that
seems wrong to me. Any chance we have a bug in how we're generating that
classpath for dunit VMs? Or maybe it's specific to the build.gradle in
geode-lucene (I'm not seeing this failure in other modules)?
-Kirk
On Mon, Jan 30, 2017 at 11:52 AM, Dan Smith <dsm...@pivotal.io> wrote:
> I created GEODE-2386 for these "Unable to launch dunit VMs" failures. It
> looks like this may actually be a gradle bug. I wonder if the some of the
> changes for the backwards compatibility test framework somehow made it more
> likely for us to hit this bug, because we're seeing it frequently now.
>
> -Dan
>
> On Sat, Jan 28, 2017 at 1:17 PM, Anthony Baker <aba...@pivotal.io> wrote:
>
> > Looks like some lucene tests are failing on the release branch.
> >
> > Anthony
> >
> >
> > > Begin forwarded message:
> > >
> > > From: Apache Jenkins Server <jenk...@builds.apache.org>
> > > Subject: Build failed in Jenkins: Geode-release #39
> > > Date: January 27, 2017 at 11:01:59 PM PST
> > > To: dev@geode.apache.org
> > > Reply-To: dev@geode.apache.org
> > >
> > > See <https://builds.apache.org/job/Geode-release/39/>
> > >
> > > ------------------------------------------
> > > [...truncated 1038 lines...]
> > > :geode-cq:spotlessCheck
> > > :geode-cq:test
> > > :geode-cq:check
> > > :geode-cq:build
> > > :geode-cq:distributedTest
> > > :geode-cq:flakyTest
> > > :geode-cq:integrationTest
> > > :geode-json:assemble
> > > :geode-json:compileTestJava UP-TO-DATE
> > > :geode-json:processTestResources UP-TO-DATE
> > > :geode-json:testClasses UP-TO-DATE
> > > :geode-json:checkMissedTests UP-TO-DATE
> > > :geode-json:spotlessJavaCheck
> > > :geode-json:spotlessCheck
> > > :geode-json:test UP-TO-DATE
> > > :geode-json:check
> > > :geode-json:build
> > > :geode-json:distributedTest UP-TO-DATE
> > > :geode-json:flakyTest UP-TO-DATE
> > > :geode-json:integrationTest UP-TO-DATE
> > > :geode-junit:javadoc
> > > :geode-junit:javadocJar
> > > :geode-junit:sourcesJar
> > > :geode-junit:signArchives SKIPPED
> > > :geode-junit:assemble
> > > :geode-junit:compileTestJava
> > > :geode-junit:processTestResources UP-TO-DATE
> > > :geode-junit:testClasses
> > > :geode-junit:checkMissedTests
> > > :geode-junit:spotlessJavaCheck
> > > :geode-junit:spotlessCheck
> > > :geode-junit:test
> > > :geode-junit:check
> > > :geode-junit:build
> > > :geode-junit:distributedTest
> > > :geode-junit:flakyTest
> > > :geode-junit:integrationTest
> > > :geode-lucene:assemble
> > > :geode-lucene:compileTestJava
> > > Download https://repo1.maven.org/maven2/org/apache/lucene/
> > lucene-test-framework/6.0.0/lucene-test-framework-6.0.0.pom
> > > Download https://repo1.maven.org/maven2/org/apache/lucene/
> > lucene-codecs/6.0.0/lucene-codecs-6.0.0.pom
> > > Download https://repo1.maven.org/maven2/com/carrotsearch/
> > randomizedtesting/randomizedtesting-runner/2.3.
> > 2/randomizedtesting-runner-2.3.2.pom
> > > Download https://repo1.maven.org/maven2/com/carrotsearch/
> > randomizedtesting/randomizedtesting-parent/2.3.
> > 2/randomizedtesting-parent-2.3.2.pom
> > > Download https://repo1.maven.org/maven2/org/apache/lucene/
> > lucene-test-framework/6.0.0/lucene-test-framework-6.0.0.jar
> > > Download https://repo1.maven.org/maven2/org/apache/lucene/
> > lucene-codecs/6.0.0/lucene-codecs-6.0.0.jar
> > > Download https://repo1.maven.org/maven2/com/carrotsearch/
> > randomizedtesting/randomizedtesting-runner/2.3.
> > 2/randomizedtesting-runner-2.3.2.jar
> > > Note: Some input files use or override a deprecated API.
> > > Note: Recompile with -Xlint:deprecation for details.
> > > Note: Some input files use unchecked or unsafe operations.
> > > Note: Recompile with -Xlint:unchecked for details.
> > > :geode-lucene:processTestResources
> > > :geode-lucene:testClasses
> > > :geode-lucene:checkMissedTests
> > > :geode-lucene:spotlessJavaCheck
> > > :geode-lucene:spotlessCheck
> > > :geode-lucene:test
> > > :geode-lucene:check
> > > :geode-lucene:build
> > > :geode-lucene:distributedTest
> > >
> > > org.apache.geode.cache.lucene.LuceneQueriesPeerFixedPRDUnitTest >
> > returnCorrectResultsWhenBucketIsMovedAndMovedBackOnIndexUpdate FAILED
> > > java.lang.AssertionError
> > >
> > > org.apache.geode.cache.lucene.internal.management.
> LuceneManagementDUnitTest
> > > classMethod FAILED
> > > java.lang.RuntimeException: Unable to launch dunit VMs
> > >
> > > Caused by:
> > > java.lang.RuntimeException: VMs did not start up within 120
> > seconds
> > >
> > > org.apache.geode.cache.lucene.LuceneIndexCreationOnFixedPRDUnitTest >
> > classMethod FAILED
> > > java.lang.RuntimeException: Unable to launch dunit VMs
> > >
> > > Caused by:
> > > java.lang.RuntimeException: VMs did not start up within 120
> > seconds
> > >
> > > org.apache.geode.cache.lucene.LuceneQueriesPeerPRRedundancyDUnitTest >
> > classMethod FAILED
> > > java.lang.RuntimeException: Unable to launch dunit VMs
> > >
> > > Caused by:
> > > java.lang.RuntimeException: VMs did not start up within 120
> > seconds
> > >
> > > org.apache.geode.cache.lucene.LuceneQueriesPeerPRDUnitTest >
> > classMethod FAILED
> > > java.lang.RuntimeException: Unable to launch dunit VMs
> > >
> > > Caused by:
> > > java.lang.RuntimeException: VMs did not start up within 120
> > seconds
> > >
> > > 86 tests completed, 5 failed
> > > :geode-lucene:distributedTest FAILED
> > > :geode-lucene:flakyTest
> > > :geode-lucene:integrationTest
> > > :geode-old-client-support:assemble
> > > :geode-old-client-support:compileTestJava
> > > :geode-old-client-support:processTestResources UP-TO-DATE
> > > :geode-old-client-support:testClasses
> > > :geode-old-client-support:checkMissedTests
> > > :geode-old-client-support:spotlessJavaCheck
> > > :geode-old-client-support:spotlessCheck
> > > :geode-old-client-support:test
> > > :geode-old-client-support:check
> > > :geode-old-client-support:build
> > > :geode-old-client-support:distributedTest
> > > :geode-old-client-support:flakyTest
> > > :geode-old-client-support:integrationTest
> > > :geode-old-versions:javadoc UP-TO-DATE
> > > :geode-old-versions:javadocJar
> > > :geode-old-versions:sourcesJar
> > > :geode-old-versions:signArchives SKIPPED
> > > :geode-old-versions:assemble
> > > :geode-old-versions:compileTestJava UP-TO-DATE
> > > :geode-old-versions:processTestResources UP-TO-DATE
> > > :geode-old-versions:testClasses UP-TO-DATE
> > > :geode-old-versions:checkMissedTests UP-TO-DATE
> > > :geode-old-versions:spotlessJavaCheck
> > > :geode-old-versions:spotlessCheck
> > > :geode-old-versions:test UP-TO-DATE
> > > :geode-old-versions:check
> > > :geode-old-versions:build
> > > :geode-old-versions:distributedTest UP-TO-DATE
> > > :geode-old-versions:flakyTest UP-TO-DATE
> > > :geode-old-versions:integrationTest UP-TO-DATE
> > > :geode-pulse:assemble
> > > :geode-pulse:compileTestJava
> > > Download https://repo1.maven.org/maven2/org/seleniumhq/
> > selenium/selenium-api/2.53.1/selenium-api-2.53.1.pom
> > > Download https://repo1.maven.org/maven2/org/seleniumhq/
> > selenium/selenium-parent/2.53.1/selenium-parent-2.53.1.pom
> > > Download https://repo1.maven.org/maven2/org/seleniumhq/
> > selenium/selenium-remote-driver/2.53.1/selenium-remote-driver-2.53.1.pom
> > > Download https://repo1.maven.org/maven2/org/seleniumhq/
> > selenium/selenium-support/2.53.1/selenium-support-2.53.1.pom
> > > Download https://repo1.maven.org/maven2/org/springframework/
> > spring-test/4.3.2.RELEASE/spring-test-4.3.2.RELEASE.pom
> > > Download https://repo1.maven.org/maven2/com/tdunning/json/1.7/
> > json-1.7.pom
> > > Download https://repo1.maven.org/maven2/org/seleniumhq/
> > selenium/selenium-api/2.53.1/selenium-api-2.53.1.jar
> > > Download https://repo1.maven.org/maven2/org/seleniumhq/
> > selenium/selenium-remote-driver/2.53.1/selenium-remote-driver-2.53.1.jar
> > > Download https://repo1.maven.org/maven2/org/seleniumhq/
> > selenium/selenium-support/2.53.1/selenium-support-2.53.1.jar
> > > Download https://repo1.maven.org/maven2/org/springframework/
> > spring-test/4.3.2.RELEASE/spring-test-4.3.2.RELEASE.jar
> > > Download https://repo1.maven.org/maven2/com/tdunning/json/1.7/
> > json-1.7.jar
> > > Note: <https://builds.apache.org/job/Geode-release/ws/geode-
> > pulse/src/test/java/org/apache/geode/tools/pulse/
> tests/ui/PulseBase.java>
> > uses or overrides a deprecated API.
> > > Note: Recompile with -Xlint:deprecation for details.
> > > Note: <https://builds.apache.org/job/Geode-release/ws/geode-
> > pulse/src/test/java/org/apache/geode/tools/pulse/controllers/
> > PulseControllerJUnitTest.java> uses unchecked or unsafe operations.
> > > Note: Recompile with -Xlint:unchecked for details.
> > > :geode-pulse:processTestResources
> > > :geode-pulse:testClasses
> > > :geode-pulse:checkMissedTests
> > > :geode-pulse:spotlessJavaCheck
> > > :geode-pulse:spotlessCheck
> > > :geode-pulse:test
> > > :geode-pulse:check
> > > :geode-pulse:build
> > > :geode-pulse:distributedTest
> > > :geode-pulse:flakyTest
> > > :geode-pulse:integrationTest
> > > :geode-rebalancer:assemble
> > > :geode-rebalancer:compileTestJava
> > > :geode-rebalancer:processTestResources UP-TO-DATE
> > > :geode-rebalancer:testClasses
> > > :geode-rebalancer:checkMissedTests
> > > :geode-rebalancer:spotlessJavaCheck
> > > :geode-rebalancer:spotlessCheck
> > > :geode-rebalancer:test
> > > :geode-rebalancer:check
> > > :geode-rebalancer:build
> > > :geode-rebalancer:distributedTest
> > > :geode-rebalancer:flakyTest
> > > :geode-rebalancer:integrationTest
> > > :geode-wan:assemble
> > > :geode-wan:compileTestJavaNote: Some input files use or override a
> > deprecated API.
> > > Note: Recompile with -Xlint:deprecation for details.
> > > Note: Some input files use unchecked or unsafe operations.
> > > Note: Recompile with -Xlint:unchecked for details.
> > >
> > > :geode-wan:processTestResources
> > > :geode-wan:testClasses
> > > :geode-wan:checkMissedTests
> > > :geode-wan:spotlessJavaCheck
> > > :geode-wan:spotlessCheck
> > > :geode-wan:test
> > > :geode-wan:check
> > > :geode-wan:build
> > > :geode-wan:distributedTest
> > > :geode-wan:flakyTest
> > > :geode-wan:integrationTest
> > > :geode-web:assemble
> > > :geode-web:compileTestJavaNote: Some input files use or override a
> > deprecated API.
> > > Note: Recompile with -Xlint:deprecation for details.
> > > Note: Some input files use unchecked or unsafe operations.
> > > Note: Recompile with -Xlint:unchecked for details.
> > >
> > > :geode-web:processTestResources UP-TO-DATE
> > > :geode-web:testClasses
> > > :geode-web:checkMissedTests
> > > :geode-web:spotlessJavaCheck
> > > :geode-web:spotlessCheck
> > > :geode-web:test
> > > :geode-web:check
> > > :geode-web:build
> > > :geode-web:distributedTest
> > > :geode-web:flakyTest
> > > :geode-web:integrationTest
> > > :geode-web-api:assemble
> > > :geode-web-api:compileTestJava UP-TO-DATE
> > > :geode-web-api:processTestResources UP-TO-DATE
> > > :geode-web-api:testClasses UP-TO-DATE
> > > :geode-web-api:checkMissedTests UP-TO-DATE
> > > :geode-web-api:spotlessJavaCheck
> > > :geode-web-api:spotlessCheck
> > > :geode-web-api:test UP-TO-DATE
> > > :geode-web-api:check
> > > :geode-web-api:build
> > > :geode-web-api:distributedTest UP-TO-DATE
> > > :geode-web-api:flakyTest UP-TO-DATE
> > > :geode-web-api:integrationTest UP-TO-DATE
> > > :combineReports
> > > All test reports at <https://builds.apache.org/
> > job/Geode-release/ws/build/reports/combined>
> > > :extensions/geode-modules:precheckin
> > > :extensions/geode-modules-assembly:precheckin
> > > :extensions/geode-modules-session:precheckin
> > > :extensions/geode-modules-session-internal:precheckin
> > > :extensions/geode-modules-tomcat7:precheckin
> > > :extensions/geode-modules-tomcat8:precheckin
> > > :geode-assembly:precheckin
> > > :geode-benchmarks:precheckin
> > > :geode-common:precheckin
> > > :geode-core:precheckin
> > > :geode-cq:precheckin
> > > :geode-json:precheckin
> > > :geode-junit:precheckin
> > > :geode-old-client-support:precheckin
> > > :geode-old-versions:precheckin
> > > :geode-pulse:precheckin
> > > :geode-rebalancer:precheckin
> > > :geode-wan:precheckin
> > > :geode-web:precheckin
> > > :geode-web-api:precheckin
> > >
> > > FAILURE: Build failed with an exception.
> > >
> > > * What went wrong:
> > > Execution failed for task ':geode-lucene:distributedTest'.
> > >> There were failing tests. See the report at: file://<
> > https://builds.apache.org/job/Geode-release/ws/geode-lucene/
> build/reports/
> > distributedTest/index.html>
> > >
> > > * Try:
> > > Run with --stacktrace option to get the stack trace. Run with --info or
> > --debug option to get more log output.
> > >
> > > BUILD FAILED
> > >
> > > Total time: 11 hrs 51 mins 28.93 secs
> > > Build step 'Invoke Gradle script' changed build result to FAILURE
> > > Build step 'Invoke Gradle script' marked build as failure
> > > Archiving artifacts
> > > Compressed 245.67 MB of artifacts by 15.3% relative to #35
> > > Recording test results
> >
> >
>
