I think we've seen something like this before when tests are logging too much. Jenkins is trying to parse the test result xml files, which contain all of the stdout from the test.
-Dan On Thu, Sep 1, 2016 at 1:41 PM, Kirk Lund <kl...@pivotal.io> wrote: > We're getting OOME failures in Hudson fairly frequently in the nightly > build. Any ideas what the issue is? Do we need to change the job > configuration or exclude more build machines? > > -Kirk > > ---------- Forwarded message ---------- > From: Apache Jenkins Server <jenk...@builds.apache.org> > Date: Thu, Sep 1, 2016 at 1:23 PM > Subject: Build failed in Jenkins: Geode-nightly #579 > To: dev@geode.incubator.apache.org, hkhame...@pivotal.io, gz...@pivotal.io > , > upthewatersp...@apache.org, sbawas...@pivotal.io, bogle...@pivotal.io, > aba...@pivotal.io > > > See <https://builds.apache.org/job/Geode-nightly/579/changes> > > Changes: > > [abaker] GEODE-1809: Improve error handling > > [gzhou] GEODE-11: create index repository using raw Lucene directory. > > [gzhou] GEODE-11: need to fix the junit test case when adding FSDirectory > > ------------------------------------------ > [...truncated 486 lines...] > [fatal 2016/09/01 10:44:41.170 UTC <JMX client heartbeat 2> tid=0x1e1] > (tid=481 msgId=19) No longer connected to localhost[27402]. > > 7606 tests completed, 1 failed, 588 skipped > :geode-core:distributedTest FAILED > :geode-core:flakyTest > :geode-core:integrationTest > :geode-cq:assemble > :geode-cq:compileTestJavaNote: Some input files use or override a > deprecated API. > Note: Recompile with -Xlint:deprecation for details. > Note: Some input files use unchecked or unsafe operations. > Note: Recompile with -Xlint:unchecked for details. > > :geode-cq:processTestResources > :geode-cq:testClasses > :geode-cq:checkMissedTests > :geode-cq:test > :geode-cq:check > :geode-cq:build > :geode-cq:distributedTest > :geode-cq:flakyTest > :geode-cq:integrationTest > :geode-json:assemble > :geode-json:compileTestJava UP-TO-DATE > :geode-json:processTestResources UP-TO-DATE > :geode-json:testClasses UP-TO-DATE > :geode-json:checkMissedTests UP-TO-DATE > :geode-json:test UP-TO-DATE > :geode-json:check > :geode-json:build > :geode-json:distributedTest UP-TO-DATE > :geode-json:flakyTest UP-TO-DATE > :geode-json:integrationTest UP-TO-DATE > :geode-junit:javadoc > :geode-junit:javadocJar > :geode-junit:sourcesJar > :geode-junit:signArchives SKIPPED > :geode-junit:assemble > :geode-junit:compileTestJava > :geode-junit:processTestResources UP-TO-DATE > :geode-junit:testClasses > :geode-junit:checkMissedTests > :geode-junit:test > :geode-junit:check > :geode-junit:build > :geode-junit:distributedTest > :geode-junit:flakyTest > :geode-junit:integrationTest > :geode-lucene:assemble > :geode-lucene:compileTestJavaNote: Some input files use or override a > deprecated API. > Note: Recompile with -Xlint:deprecation for details. > Note: Some input files use unchecked or unsafe operations. > Note: Recompile with -Xlint:unchecked for details. > > :geode-lucene:processTestResources > :geode-lucene:testClasses > :geode-lucene:checkMissedTests > :geode-lucene:test > :geode-lucene:check > :geode-lucene:build > :geode-lucene:distributedTest > :geode-lucene:flakyTest > :geode-lucene:integrationTest > :geode-pulse:assemble > :geode-pulse:compileTestJavaNote: <https://builds.apache.org/ > job/Geode-nightly/ws/geode-pulse/src/test/java/com/ > vmware/gemfire/tools/pulse/tests/PulseAbstractTest.java> uses or overrides > a deprecated API. > Note: Recompile with -Xlint:deprecation for details. > Note: Some input files use unchecked or unsafe operations. > Note: Recompile with -Xlint:unchecked for details. > > :geode-pulse:processTestResources > :geode-pulse:testClasses > :geode-pulse:checkMissedTests > :geode-pulse:test > :geode-pulse:check > :geode-pulse:build > :geode-pulse:distributedTest > :geode-pulse:flakyTest > :geode-pulse:integrationTest > :geode-rebalancer:jar > :geode-rebalancer:javadoc > :geode-rebalancer:javadocJar > :geode-rebalancer:sourcesJar > :geode-rebalancer:signArchives SKIPPED > :geode-rebalancer:assemble > :geode-rebalancer:compileTestJava > :geode-rebalancer:processTestResources UP-TO-DATE > :geode-rebalancer:testClasses > :geode-rebalancer:checkMissedTests > :geode-rebalancer:test > :geode-rebalancer:check > :geode-rebalancer:build > :geode-rebalancer:distributedTest > :geode-rebalancer:flakyTest > :geode-rebalancer:integrationTest > :geode-wan:assemble > :geode-wan:compileTestJavaNote: Some input files use or override a > deprecated API. > Note: Recompile with -Xlint:deprecation for details. > Note: Some input files use unchecked or unsafe operations. > Note: Recompile with -Xlint:unchecked for details. > > :geode-wan:processTestResources > :geode-wan:testClasses > :geode-wan:checkMissedTests > :geode-wan:test > :geode-wan:check > :geode-wan:build > :geode-wan:distributedTest > :geode-wan:flakyTest > :geode-wan:integrationTest > :geode-web:assemble > :geode-web:compileTestJavaNote: <https://builds.apache.org/ > job/Geode-nightly/ws/geode-web/src/test/java/com/ > gemstone/gemfire/management/internal/cli/commands/ > DataCommandsOverHttpDistributedTest.java> uses or overrides a deprecated > API. > Note: Recompile with -Xlint:deprecation for details. > Note: Some input files use unchecked or unsafe operations. > Note: Recompile with -Xlint:unchecked for details. > > :geode-web:processTestResources UP-TO-DATE > :geode-web:testClasses > :geode-web:checkMissedTests > :geode-web:test > :geode-web:check > :geode-web:build > :geode-web:distributedTest > :geode-web:flakyTest > :geode-web:integrationTest > :geode-web-api:assemble > :geode-web-api:compileTestJava UP-TO-DATE > :geode-web-api:processTestResources UP-TO-DATE > :geode-web-api:testClasses UP-TO-DATE > :geode-web-api:checkMissedTests UP-TO-DATE > :geode-web-api:test UP-TO-DATE > :geode-web-api:check > :geode-web-api:build > :geode-web-api:distributedTest UP-TO-DATE > :geode-web-api:flakyTest UP-TO-DATE > :geode-web-api:integrationTest UP-TO-DATE > :combineReports > All test reports at <https://builds.apache.org/job/Geode-nightly/ws/build/ > reports/combined> > :extensions/geode-modules:precheckin > :extensions/geode-modules-assembly:precheckin > :extensions/geode-modules-hibernate:precheckin > :extensions/geode-modules-session:precheckin > :extensions/geode-modules-session-internal:precheckin > :extensions/geode-modules-tomcat7:precheckin > :extensions/geode-modules-tomcat8:precheckin > :geode-assembly:precheckin > :geode-common:precheckin > :geode-cq:precheckin > :geode-json:precheckin > :geode-junit:precheckin > :geode-lucene:precheckin > :geode-pulse:precheckin > :geode-rebalancer:precheckin > :geode-wan:precheckin > :geode-web:precheckin > :geode-web-api:precheckin > :extensions/geode-modules:uploadArchives > :extensions/geode-modules-assembly:uploadArchives SKIPPED > :extensions/geode-modules-hibernate:uploadArchives > :extensions/geode-modules-session:uploadArchives > :extensions/geode-modules-session-internal:uploadArchives > :extensions/geode-modules-tomcat7:uploadArchives > :extensions/geode-modules-tomcat8:uploadArchives > :geode-assembly:uploadArchives SKIPPED > :geode-common:uploadArchives > :geode-core:uploadArchives > :geode-cq:uploadArchives > :geode-json:uploadArchives > :geode-junit:uploadArchives > :geode-lucene:uploadArchives > :geode-pulse:uploadArchives > :geode-rebalancer:uploadArchives > :geode-wan:uploadArchives > :geode-web:uploadArchives > :geode-web-api:uploadArchives > > FAILURE: Build failed with an exception. > > * What went wrong: > Execution failed for task ':geode-core:distributedTest'. > > There were failing tests. See the report at: file://< > https://builds.apache.org/job/Geode-nightly/ws/geode-core/build/reports/ > distributedTest/index.html> > > * Try: > Run with --stacktrace option to get the stack trace. Run with --info or > --debug option to get more log output. > > BUILD FAILED > > Total time: 11 hrs 10 mins 1.029 secs > Build step 'Invoke Gradle script' changed build result to FAILURE > Build step 'Invoke Gradle script' marked build as failure > Archiving artifacts > Compressed 378.83 MB of artifacts by 16.1% relative to #573 > Recording test results > ERROR: Step ?Publish JUnit test result report? aborted due to exception: > java.io.IOException: remote file operation failed: < > https://builds.apache.org/job/Geode-nightly/ws/> at > hudson.remoting.Channel@43724f7c:H4: > java.io.IOException: Remote call on H4 failed > at hudson.FilePath.act(FilePath.java:1014) > at hudson.FilePath.act(FilePath.java:996) > at hudson.tasks.junit.JUnitParser.parseResult( > JUnitParser.java:103) > at hudson.tasks.junit.JUnitResultArchiver.parse( > JUnitResultArchiver.java:128) > at hudson.tasks.junit.JUnitResultArchiver.perform( > JUnitResultArchiver.java:149) > at hudson.tasks.BuildStepCompatibilityLayer.perform( > BuildStepCompatibilityLayer.java:78) > at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor. > java:20) > at hudson.model.AbstractBuild$AbstractBuildExecution. > perform(AbstractBuild.java:779) > at hudson.model.AbstractBuild$AbstractBuildExecution. > performAllBuildSteps(AbstractBuild.java:720) > at hudson.model.Build$BuildExecution.post2(Build.java:185) > at hudson.model.AbstractBuild$AbstractBuildExecution.post( > AbstractBuild.java:665) > at hudson.model.Run.execute(Run.java:1766) > at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) > at hudson.model.ResourceController.execute( > ResourceController.java:98) > at hudson.model.Executor.run(Executor.java:410) > Caused by: java.io.IOException: Remote call on H4 failed > at hudson.remoting.Channel.call(Channel.java:789) > at hudson.FilePath.act(FilePath.java:1007) > ... 14 more > Caused by: java.lang.OutOfMemoryError: Java heap space > at com.sun.org.apache.xerces.internal.util.XMLStringBuffer. > append(XMLStringBuffer.java:208) > at com.sun.org.apache.xerces.internal.impl. > XMLEntityScanner.scanData(XMLEntityScanner.java:1507) > at com.sun.org.apache.xerces.internal.impl. > XMLDocumentFragmentScannerImpl.scanCDATASection( > XMLDocumentFragmentScannerImpl.java:1648) > at com.sun.org.apache.xerces.internal.impl. > XMLDocumentFragmentScannerImpl$FragmentContentDriver.next( > XMLDocumentFragmentScannerImpl.java:3011) > at com.sun.org.apache.xerces.internal.impl. > XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606) > at com.sun.org.apache.xerces.internal.impl. > XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:118) > at com.sun.org.apache.xerces.internal.impl. > XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl > .java:504) > at com.sun.org.apache.xerces.internal.parsers. > XML11Configuration.parse(XML11Configuration.java:848) > at com.sun.org.apache.xerces.internal.parsers. > XML11Configuration.parse(XML11Configuration.java:777) > at com.sun.org.apache.xerces.internal.parsers.XMLParser. > parse(XMLParser.java:141) > at com.sun.org.apache.xerces.internal.parsers. > AbstractSAXParser.parse(AbstractSAXParser.java:1213) > at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$ > JAXPSAXParser.parse(SAXParserImpl.java:642) > at org.dom4j.io.SAXReader.read(SAXReader.java:465) > at org.dom4j.io.SAXReader.read(SAXReader.java:264) > at hudson.tasks.junit.SuiteResult.parse(SuiteResult.java:126) > at hudson.tasks.junit.TestResult.parse(TestResult.java:301) > at hudson.tasks.junit.TestResult.parsePossiblyEmpty(TestResult. > java:244) > at hudson.tasks.junit.TestResult.parse(TestResult.java:175) > at hudson.tasks.junit.TestResult.parse(TestResult.java:154) > at hudson.tasks.junit.TestResult.<init>(TestResult.java:126) > at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke( > JUnitParser.java:132) > at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke( > JUnitParser.java:107) > at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2772) > at hudson.remoting.UserRequest.perform(UserRequest.java:153) > at hudson.remoting.UserRequest.perform(UserRequest.java:50) > at hudson.remoting.Request$2.run(Request.java:332) > at hudson.remoting.InterceptingExecutorService$1.call( > InterceptingExecutorService.java:68) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > at ......remote call to H4(Native Method) > at hudson.remoting.Channel.attachCallSiteStackTrace( > Channel.java:1416) > at hudson.remoting.UserResponse.retrieve(UserRequest.java:253) > at hudson.remoting.Channel.call(Channel.java:781) > ... 15 more >
