[JIRA] (JENKINS-57161) ec2 plugin locks queue until excessWorkload is 0
Title: Message Title FABRIZIO MANFREDI closed an issue as Fixed Jenkins / JENKINS-57161 ec2 plugin locks queue until excessWorkload is 0 Change By: FABRIZIO MANFREDI Status: In Progress Closed Resolution: Fixed Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.198911.1556047726000.9564.1561563062431%40Atlassian.JIRA. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-57161) ec2 plugin locks queue until excessWorkload is 0
Title: Message Title FABRIZIO MANFREDI commented on JENKINS-57161 Re: ec2 plugin locks queue until excessWorkload is 0 I will merge soon the patch, I am waiting to close some pending of 1.43. After that It will be part of the 1.44 Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.198911.1556047726000.700.1558084680191%40Atlassian.JIRA. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-57161) ec2 plugin locks queue until excessWorkload is 0
Title: Message Title FABRIZIO MANFREDI started work on JENKINS-57161 Change By: FABRIZIO MANFREDI Status: Open In Progress Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.198911.1556047726000.692.1558084501596%40Atlassian.JIRA. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-57161) ec2 plugin locks queue until excessWorkload is 0
Title: Message Title Adrien Tateno commented on JENKINS-57161 Re: ec2 plugin locks queue until excessWorkload is 0 Proposed fix: https://github.com/jenkinsci/ec2-plugin/pull/346 Add Comment This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-issues/JIRA.198911.1556047726000.20257.1557245160174%40Atlassian.JIRA. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-57161) ec2 plugin locks queue until excessWorkload is 0
Title: Message Title Kevin Boschert created an issue Jenkins / JENKINS-57161 ec2 plugin locks queue until excessWorkload is 0 Issue Type: Bug Assignee: FABRIZIO MANFREDI Attachments: add_many_removes_thread_dump.txt, ec2-test.png, ecs-test.png, Jenkinsfile.stress_test Components: ec2-plugin Created: 2019-04-23 19:28 Environment: Jenkins 2.172 (from the jenkins/jenkins:2.172 docker image) ec2 1.42 durable-task 1.29 Labels: ec2 Queue Priority: Major Reporter: Kevin Boschert We have periods where we enqueue a burst of jobs (> 100) and need to spin up approximately that many workers. When this happens, the ec2 plugin appears to be taking out a substantial number of locks on the Queue, which prevents builds from being processed out of the Queue and assigned to an executor. We see the number of available executors go up until the excessWorkload is under the NodeProvisioner's thresholdMargin, at which point the Queue locks are released, and jobs are assigned to the executors. In cases where we need to spin up 100s of workers, we can see the Queue locked for a fairly long time (> 15 minutes). During the time that the Queue has been locked, if any of our EC2 workers reach the specified idle timeout, Jenkins.updateComputerList (which is called during Jenkins.addNode) will decide it needs to remove those workers, despite having a large Queue that we're in the middle
