[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title Nikolas Falco updated JENKINS-52842 Jenkins / JENKINS-52842 xUnit plugin blocks PingThread responses Change By: Nikolas Falco Status: Fixed but Unreleased Resolved Released As: 2.3.8 Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.192770.1533150138000.379.1581857220312%40Atlassian.JIRA.
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title Nikolas Falco updated JENKINS-52842 Jenkins / JENKINS-52842 xUnit plugin blocks PingThread responses Change By: Nikolas Falco Status: Reopened Fixed but Unreleased Resolution: Fixed Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.192770.1533150138000.375.1581856920354%40Atlassian.JIRA.
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title Nikolas Falco commented on JENKINS-52842 Re: xUnit plugin blocks PingThread responses I had complete the work. The sleep time is not configurable so a futher changes in the code should not be needed. To be back compatible is 0ms for old configurations (is not the case of pipelines). By default for new definition is *10*ms. John Lengeling output log warn a message that you are using a deprecated class and you instantiate it by reflection by means pipeline (steps.step([$class : 'XUnitBuilder', testTimeMargin: '3000', ) that is subject any class changes. I heavy suggest to replace with xunit step (that is not XUnitBuilder) that manage the configuration of sleep parameter (XUnitBuilder does not). If I do not remember bad you had test 20ms each 20 processed report. The default is now 10ms every 10 processed reports. xunit thresholdMode: 2, thresholds: [failed(failureThreshold: '100', unstableThreshold: '100'), skipped(failureThreshold: '100', unstableThreshold: '100')], tools: [GoogleTest(deleteOutputFiles: true, failIfNotNew: false, pattern: $filePattern, skipNoTestFiles: true, stopProcessingIfError: true)], sleepTime: 15 Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title John Lengeling commented on JENKINS-52842 Re: xUnit plugin blocks PingThread responses Other than the 1 hung job soon after I loaded xunit-2.3.8-rc831.cbb77af6dfed, the issue looks to be improved. The test job has been running hourly for the past 2 days with 0 hangs versus several hangs per day with 2.3.7. So you might be on the right track. I did also increase the ping thread interval and timeout values to 1500/1200, but I have now restored the ping thread interval/timeout values back to the defaults (300/240). I will let that bake for a few days. Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.192770.1533150138000.8459.1581549060178%40Atlassian.JIRA.
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title Nikolas Falco commented on JENKINS-52842 Re: xUnit plugin blocks PingThread responses I can try to increase the sleep time after 20 processed input files and maybe also try to setup the maximun number of thread used by saxon. By default saxon use the maximun number of thread to make CPU full load. Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.192770.1533150138000.8190.1581523440406%40Atlassian.JIRA.
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title John Lengeling commented on JENKINS-52842 Re: xUnit plugin blocks PingThread responses Nickolas, I had 7 successful builds before the pingThread killed the node during xunit processing when running xunit version 2.3.8-rc831.cbb77af6dfed. Console Output:Console Output: [2020-02-10T13:35:50.172Z] WARNING: XUnitBuilder step is deprecated since 2.x, it has been replaced by XUnitPublisher. This builer will be remove in version 3.x [2020-02-10T13:35:50.175Z] INFO: Starting to record. [2020-02-10T13:35:50.175Z] INFO: Processing GoogleTest-1.8[2020-02-10T13:35:50.250Z] INFO: [GoogleTest-1.8] - 959 test report file(s) were found with the pattern 'j/wr/build/_TestArtifacts*/*.xml' relative to '/home/jenkins/workspace/kb/os' for the testing framework 'GoogleTest-1.8'. Theads related to this aws node i-0b11d0f08d8d59c51: dev-large (i-0b11d0f08d8d59c51) "org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution [#93] / waiting for EC2 (foo-aws) - dev-large (i-0b11d0f08d8d59c51) id=166000" daemon prio=5 TIMED_WAITING java.lang.Object.wait(Native Method) hudson.remoting.Request.call(Request.java:177) hudson.remoting.Channel.call(Channel.java:954) hudson.FilePath.act(FilePath.java:1069) hudson.FilePath.act(FilePath.java:1058) org.jenkinsci.plugins.xunit.XUnitProcessor.processTestsReport(XUnitProcessor.java:195) org.jenkinsci.plugins.xunit.XUnitProcessor.process(XUnitProcessor.java:159) org.jenkinsci.plugins.xunit.XUnitBuilder.perform(XUnitBuilder.java:126) org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:80) org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:67) org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$$Lambda$339/847132060.run(Unknown Source) java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) java.util.concurrent.FutureTask.run(FutureTask.java:266) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:748) "Channel reader thread: EC2 (foo-aws) - dev-large (i-0b11d0f08d8d59c51)" daemon prio=5 WAITING java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) com.trilead.ssh2.channel.FifoBuffer.read(FifoBuffer.java:212) com.trilead.ssh2.channel.Channel$Output.read(Channel.java:127) com.trilead.ssh2.channel.ChannelManager.getChannelData(ChannelManager.java:933) com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:58) com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:79) hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:91) hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:72) hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:103) hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39)
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title John Lengeling commented on JENKINS-52842 Re: xUnit plugin blocks PingThread responses Will test the plugin version that you provided. Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.192770.1533150138000.5151.1581310260171%40Atlassian.JIRA.
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title
John Lengeling commented on JENKINS-52842
Re: xUnit plugin blocks PingThread responses
Sorry I sent you the stacktrace from the job console which is after the job is aborted. fooTool.groovy is our Jenkins pipeline library which calls has a publishUnitTests method which calls XUnitBuilder. {{ try { steps.step([$class : 'XUnitBuilder', testTimeMargin: '3000', thresholdMode: 2, thresholds: [ [ $class : 'FailedThreshold', failureNewThreshold : '100', failureThreshold : '100', unstableNewThreshold: '100', unstableThreshold : '100' ], [ $class : 'SkippedThreshold', failureNewThreshold : '100', failureThreshold : '100', unstableNewThreshold: '100', unstableThreshold : '100' ] ], tools : [ [ $class : 'GoogleTestType', deleteOutputFiles : true, failIfNotNew : false, pattern : filePattern, skipNoTestFiles : true, stopProcessingIfError: true ] ] ]) }} Here is the thread dump of the thread that is hanging in xunit: {{ "org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution 2624 / waiting for EC2 (foo-aws) - team-bar-large (i-086328a69a4a0b130) id=8001805" daemon prio=5 TIMED_WAITING java.lang.Object.wait(Native Method) hudson.remoting.Request.call(Request.java:177) hudson.remoting.Channel.call(Channel.java:954) hudson.FilePath.act(FilePath.java:1069) hudson.FilePath.act(FilePath.java:1058) org.jenkinsci.plugins.xunit.XUnitProcessor.processTestsReport(XUnitProcessor.java:195) org.jenkinsci.plugins.xunit.XUnitProcessor.process(XUnitProcessor.java:159) org.jenkinsci.plugins.xunit.XUnitBuilder.perform(XUnitBuilder.java:126) org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:80) org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:67) }}
Add Comment
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title Nikolas Falco commented on JENKINS-52842 Re: xUnit plugin blocks PingThread responses Please try with this: https://ci.jenkins.io/job/Plugins/job/xunit-plugin/job/feature%252FJENKINS-52842/2/artifact/org/jenkins-ci/plugins/xunit/2.3.8-rc831.cbb77af6dfed/xunit-2.3.8-rc831.cbb77af6dfed.hpi Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.192770.1533150138000.5023.1581265260179%40Atlassian.JIRA.
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title Nikolas Falco edited a comment on JENKINS-52842 Re: xUnit plugin blocks PingThread responses Which step is at fooTool.groovy:597?I have no idea what to do, and the reason is that not only do I not know the exact point where it is blocked but also the reason. The only thing I can assume is that the CPU is busy by the JVM by the XSLT. The only test I can do is add a sort of thread sleep after 50 transformations. Obviously this change may have no effect if the ping thread only monitors before and after the execution of a callable (means the whole step) Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.192770.1533150138000.4587.1581110880175%40Atlassian.JIRA.
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title Nikolas Falco commented on JENKINS-52842 Re: xUnit plugin blocks PingThread responses Which step is at fooTool.groovy:597? I have no idea what to do, and the reason is that not only do I not know the exact point where it is blocked but also the reason. The only thing I can assume is that the CPU is busy by the JVM by the XSLT. The only test I can do is add a sort of thread sleep after 50 transformations. Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.192770.1533150138000.4583.1581110700212%40Atlassian.JIRA.
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title John Lengeling reopened an issue We are seeing the pingthread killing xunit processsing as described. Provided stacktrace and error mesages. Jenkins / JENKINS-52842 xUnit plugin blocks PingThread responses Change By: John Lengeling Resolution: Fixed Status: Closed Reopened Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title
John Lengeling commented on JENKINS-52842
Re: xUnit plugin blocks PingThread responses
Nikolas, Looks like I have run into this issue. We generate a lot of XML files..sometimes 800+. Looks like the ping thread killed it. We are running Jenkins 2.204.1 and xunit 2.37. Is there a workaround or a snapshot to test? Thanks! Error message: Processing xunit results failed, archiving test result files foo/wr8-64/build/_TestArtifacts*/*.xml for troubleshooting [Pipeline] archiveArtifacts 10:07:27 EC2 (foo-es-aws) - team-foo.fooTool-large (i-06488c01860a95d82) was marked offline: Connection was broken: java.util.concurrent.TimeoutException: Ping started at 1580227324607 hasn't completed by 1580227564613 10:07:27 at hudson.remoting.PingThread.ping(PingThread.java:133) 10:07:27 at hudson.remoting.PingThread.run(PingThread.java:89) {{10:07:27 }} {{[Pipeline] }}} [Pipeline] // script Error when executing always post condition: java.lang.IllegalArgumentException: Failed to prepare archiveArtifacts step {{ at org.jenkinsci.plugins.workflow.cps.DSL.invokeDescribable(DSL.java:419)}} {{ at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:182)}} {{ at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)}} {{ at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:48)}} {{ at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)}} {{ at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)}} {{ at com.cloudbees.groovy.cps.sandbox.DefaultInvoker.methodCall(DefaultInvoker.java:20)}} {{ at com.fooCorp.pipeline.fooTool.publishUnitTests(fooTool.groovy:597)}} {{ at com.fooCorp.pipeline.fooTool.publishfooToolReports(fooTool.groovy:666)}} {{ at com.fooCorp.pipeline.fooTool.publish(fooTool.groovy:610)}} {{ at WorkflowScript.run(WorkflowScript:61)}} {{ at __cps.transform__(Native Method)}} {{ at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)}} {{ at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)}} {{ at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)}} {{ at sun.reflect.GeneratedMethodAccessor960.invoke(Unknown Source)}} {{ at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}} {{ at java.lang.reflect.Method.invoke(Method.java:498)}} {{ at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)}} {{ at com.cloudbees.groovy.cps.impl.CollectionLiteralBlock$ContinuationImpl.dispatch(CollectionLiteralBlock.java:55)}} {{ at com.cloudbees.groovy.cps.impl.CollectionLiteralBlock$ContinuationImpl.item(CollectionLiteralBlock.java:45)}} {{ at sun.reflect.GeneratedMethodAccessor988.invoke(Unknown Source)}} {{ at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}} {{ at java.lang.reflect.Method.invoke(Method.java:498)}} {{ at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)}} {{ at com.cloudbees.groovy.cps.impl.LocalVariableBlock$LocalVariable.get(LocalVariableBlock.java:39)}} {{ at com.cloudbees.groovy.cps.LValueBlock$GetAdapter.receive(LValueBlock.java:30)}} {{ at com.cloudbees.groovy.cps.impl.LocalVariableBlock.evalLValue(LocalVariableBlock.java:28)}} {{ at com.cloudbees.groovy.cps.LValueBlock$BlockImpl.eval(LValueBlock.java:55)}} {{ at com.cloudbees.groovy.cps.LValueBlock.eval(LValueBlock.java:16)}} {{ at com.cloudbees.groovy.cps.Next.step(Next.java:83)}} {{ at
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title Nikolas Falco closed an issue as Fixed Reopen if happens again Jenkins / JENKINS-52842 xUnit plugin blocks PingThread responses Change By: Nikolas Falco Status: Open Closed Resolution: Fixed Add Comment This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title Nikolas Falco commented on JENKINS-52842 Re: xUnit plugin blocks PingThread responses Please keep in mind that transformation is a CPU intensive operation. If you have complex XSLT, large XML report and heavy volumn, 100% CPU may be expected this maybe cause JVM threads agent freeze and not serve the PingThread. Let me make some try threadpool, sleep time after 10 transformation update saxon libraries... Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title
Jo Shields created an issue
Jenkins / JENKINS-52842
xUnit plugin blocks PingThread responses
Issue Type:
Bug
Assignee:
Nikolas Falco
Components:
xunit-plugin
Created:
2018-08-01 19:02
Priority:
Minor
Reporter:
Jo Shields
If your xUnit parsing takes a long time (don't ask), the entire build agent can be kicked offline, by passing the timeout threshhold on pings (default 4 minutes). Agent: 12:39:02 [xUnit] [INFO] - [NUnit-2 (default)] - 110 test report file(s) were found with the pattern '*/TestResult.xml' relative to '/home/builder/jenkins/workspace/foo' for the testing framework 'NUnit-2 (default)'. 12:44:28 [xUnit] [ERROR] - The plugin hasn't been performed correctly: Remote call on JNLP4-connect connection from 1.1.1.1/1.1.1.1:1880 failed Server: INFO: Ping failed. Terminating the channel JNLP4-connect connection from 1.1.1.1/1.1.1.1:7468. java.util.concurrent.TimeoutException: Ping started at 1532979041327 hasn't completed by 1532979281327 {{ at hudson.remoting.PingThread.ping(PingThread.java:134)}} {{ at hudson.remoting.PingThread.run(PingThread.java:90)}}
Add Comment
[JIRA] (JENKINS-52842) xUnit plugin blocks PingThread responses
Title: Message Title
Jo Shields updated an issue
Jenkins / JENKINS-52842
xUnit plugin blocks PingThread responses
Change By:
Jo Shields
If your xUnit parsing takes a long time (don't ask), the entire build agent can be kicked offline, by passing the timeout threshhold on pings (default 4 minutes). Agent:{{12:39:02 [xUnit] [INFO] - [NUnit-2 (default)] - 110 test report file(s) were found with the pattern '**/TestResult*.xml' relative to '/home/builder/jenkins/workspace/foo' for the testing framework 'NUnit-2 (default)'.}}{{12:44:28 [xUnit] [ERROR] - The plugin hasn't been performed correctly: Remote call on JNLP4-connect connection from 1.1.1.1/1.1.1.1:1880 failed}}Server:{{INFO: Ping failed. Terminating the channel JNLP4-connect connection from 1.1.1.1/1.1.1.1:7468.}}{{java.util.concurrent.TimeoutException: Ping started at 1532979041327 hasn't completed by 1532979281327}}{{ at hudson.remoting.PingThread.ping(PingThread.java:134)}}{{ at hudson.remoting.PingThread.run(PingThread.java:90)}}
Add Comment
This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
