[JIRA] (JENKINS-37208) Slowness by too many calls in /wfapi/runs on complex pipelines

[fbelz...@gmail.com (JIRA)]

Title: Message Title

Félix Belzunce Arcos updated an issue   Jenkins / JENKINS-37208 Slowness by too many calls in /wfapi/runs on complex pipelines Change By: Félix Belzunce Arcos The way this plugin works is that through _javascript_ and each 5 seconds the stage-view-ui plugin will perform a HTTP GET to wfapi/runs even if there is not any build currently running, which looks very inefficient. See: * https://github.com/jenkinsci/pipeline-stage-view-plugin/blob/master/ui/src/main/js/model/job-history-listener.js#L36 * https://github.com/jenkinsci/pipeline-stage-view-plugin/blob/master/ui/src/main/js/model/job-history-listener.js#L72 Again, in my opinion, once there is not any build running the timeout should not be 5 seconds but an higher value, or even better it should be the backend to tell the frontend to continue performing HTTP GET. Also, each time that an user is on the stage-view-ui graph the rest api HTTP GET wfapi/runs will return ALL the builds of the job. I am not sure about how costly is this, but it looks like we should do a cache or something and only get the new builds. * https://github.com/jenkinsci/pipeline-stage-view-plugin/blob/master/rest-api/src/main/java/com/cloudbees/workflow/rest/endpoints/JobAPI.java#L72 This problem can be partially reproduced creating a pipeline script with iota close to 700. Something like: {code:java} node('master') {     for (i = 0; i <350; i++) {        stage 'stage'        sh 'Hello world!'        sh 'Hello world!'        sh 'Hello world!'     } } {code} Then, start performing builds and open 40-50 tabs on your browser so each 5 seconds each one is doing HTTP GET wfapi/runs The following appears a lot of times on the thread dumps taken. {code:java} 13085msec elapsed in Handling GET /jenkins/<ROOT_TO_THE_JOB>/wfapi/runs from <IP> : RequestHandlerThread[#269]     java.io.FileInputStream.readBytes(Native Method)     …     com.thoughtworks.xstream.XStream.fromXML(XStream.java:1053)     hudson.XmlFile.read(XmlFile.java:142)     org.jenkinsci.plugins.workflow.support.storage.SimpleXStreamFlowNodeStorage.load(SimpleXStreamFlowNodeStorage.java:114)     …     org.jenkinsci.plugins.workflow.support.storage.SimpleXStreamFlowNodeStorage.getNode(SimpleXStreamFlowNodeStorage.java:80)     org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.getNode(CpsFlowExecution.java:775)     org.jenkinsci.plugins.workflow.graph.FlowNode.loadParents(FlowNode.java:128)     org.jenkinsci.plugins.workflow.graph.FlowNode.getParents(FlowNode.java:119)     org.jenkinsci.plugins.workflow.graph.FlowGraphWalker$FlowGraphWalkerIterator.getNext(FlowGraphWalker.java:103)     org.jenkinsci.plugins.workflow.graph.FlowGraphWalker$FlowGraphWalkerIterator.next(FlowGraphWalker.java:86)     org.jenkinsci.plugins.workflow.graph.FlowGraphWalker$FlowGraphWalkerIterator.next(FlowGraphWalker.java:68)     com.cloudbees.workflow.flownode.FlowNodeUtil.getIdSortedExecutionNodeList(FlowNodeUtil.java:486)     com.cloudbees.workflow.rest.external.RunExt.createOld(RunExt.java:336)     com.cloudbees.workflow.rest.external.RunExt.create(RunExt.java:303)     com.cloudbees.workflow.rest.external.JobExt.create(JobExt.java:126)     com.cloudbees.workflow.rest.endpoints.JobAPI.doRuns(JobAPI.java:68) {code} The current flow graph storage is just not efficient for builds with large numbers of steps. No particular recommendation other than either get faster disk, or turn off the stage view, or try to move complex script logic into external programs rather than doing too much in Groovy. Add Comment   This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.