[JIRA] (JENKINS-37449) EC2-plugin preventing Jenkins from starting if a slave is dead
Title: Message Title Jeff Gaston commented on JENKINS-37449 Re: EC2-plugin preventing Jenkins from starting if a slave is dead Brent Courtois Not sure what the best option is for you, but here's what I ended up doing: The short version is I deployed the patched ec2-plugin and deployed a credentials plugin, and had to restore the ec2 credentials because the ec2 plugin added a dependency on what I think was called the Credentials Plugin. 1. Obtain an hpi file that's exactly the same as what's currently deployed in the target Jenkins, except also containing the fix: https://github.com/jenkinsci/ec2-plugin/pull/213 . See also https://wiki.jenkins-ci.org/display/JENKINS/Plugin+tutorial for info about building the plugin 2. Copy the modified .hpi file into the Jenkins master 3. Start Jenkins. At this point, the problem was fixed but the version of the ec2 plugin should was slightly old and also custom. 4. Make a backup of all of the ecs-related credentials in Jenkins because we're probably going to accidentally delete them next. I did this by just expanding all the options on $jenkinsHostname/configure and doing rightclick -> "Save As". I wouldn't be surprised if there's a better way to do this. 5. Install the Credentials Plugin, I think 6. Obtain a .hpi file that is the desired version to have be deployed in the target Jenkins (presumably the latest master (there might be a published version somewhere by now)). 7. Copy the .hpi file into the Jenkins master 8. Restart Jenkins 9. Check that all the ecs plugin credentials are still in the UI. If they're not, then manually copy them one field at a time from the backup made in #4 Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
[JIRA] (JENKINS-37449) EC2-plugin preventing Jenkins from starting if a slave is dead
Title: Message Title Jeff Gaston updated an issue Jenkins / JENKINS-37449 EC2-plugin preventing Jenkins from starting if a slave is dead Change By: Jeff Gaston Summary: EC2-plugin preventing Jenkins from starting if a slave is dead instance exists Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-37449) EC2-plugin preventing Jenkins from starting if a dead instance exists
Title: Message Title Jeff Gaston commented on JENKINS-37449 Re: EC2-plugin preventing Jenkins from starting if a dead instance exists Pull request is https://github.com/jenkinsci/ec2-plugin/pull/213 Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-37449) EC2-plugin preventing Jenkins from starting if a dead instance exists
Title: Message Title Jeff Gaston created an issue Jenkins / JENKINS-37449 EC2-plugin preventing Jenkins from starting if a dead instance exists Issue Type: Bug Assignee: Francis Upton Components: ec2-plugin Created: 2016/Aug/16 6:46 PM Priority: Major Reporter: Jeff Gaston Our Jenkins was failing to start when there existed a Jenkins Node corresponding to a dead EC2 instance. The stack trace said: SEVERE: Failed to initialize Jenkins hudson.util.HudsonFailedToLoad: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at hudson.WebAppMain$3.run(WebAppMain.java:237) Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.ArrayList.rangeCheck(ArrayList.java:604) at java.util.ArrayList.get(ArrayList.java:382) at hudson.plugins.ec2.EC2Computer._describeInstanceOnce(EC2Computer.java:166) at hudson.plugins.ec2.EC2Computer._describeInstance(EC2Computer.java:149) at hudson.plugins.ec2.EC2Computer.describeInstance(EC2Computer.java:107) at hudson.plugins.ec2.EC2Computer.getUptime(EC2Computer.java:133) at hudson.plugins.ec2.EC2RetentionStrategy.internalCheck(EC2RetentionStrategy.java:104) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:85) at hudson.plugins.ec2.EC2RetentionStrategy.check(EC2RetentionStrategy.java:43) at hudson.slaves.SlaveComputer$4.run(SlaveComputer.java:717) at hudson.model.Queue._withLock(Queue.java:1346) at hudson.model.Queue.withLock(Queue.java:1229) at hudson.slaves.SlaveComputer.setNode(SlaveComputer.java:714) at hudson.model.AbstractCIBase.updateComputer(AbstractCIBase.java:118) at hudson.model.AbstractCIBase.access$000(AbstractCIBase.java:44) at hudson.model.AbstractCIBase$2.run(AbstractCIBase.java:186) at hudson.model.Queue._withLock(Queue.java:1346) at hudson.model.Queue.withLock(Queue.java:1229) at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:169) at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1247) at jenkins.model.Jenkins.(Jenkins.java:844) at
[JIRA] [ec2-plugin] (JENKINS-35679) EC2-plugin not provisioning new slaves if slaves of same template but different label exist
Title: Message Title Jeff Gaston created an issue Jenkins / JENKINS-35679 EC2-plugin not provisioning new slaves if slaves of same template but different label exist Issue Type: Bug Assignee: Francis Upton Components: ec2-plugin Created: 2016/Jun/13 4:12 PM Priority: Minor Reporter: Jeff Gaston Our Jenkins has several pools (SlaveTemplates) of servers that are scaled up and down by the EC2 plugin. Different jobs generate different state (including downloading Maven dependencies or deploying to Tomcat) that could be recreated on other servers but that take more time to recreate from scratch. So, we have a separate SlaveTemplate for each job. Each SlaveTemplate generally generates slaves having one label and each job only runs on slaves of a specific label. The AMI required by each job is the same, however. As a result, if one job has finished all of its work and another job is backlogged, then there may be an idle slave with label A having AMI ami-1, and still be a need for another slave with label B having AMI ami-1. If the system reaches this state, then the EC2 plugin doesn't provision more slaves, thinking that the need for more slaves of label B is really just a request for another slave of ami-1, which then can be satisfied by the existing idle slave of label A (this behavior is accompanied by messages in the logs saying "true - Node has capacity - can use it"). Then the job that only runs on slaves of label B doesn't have a new server on which to run, and remains backlogged. I've made a pull request at https://github.com/jenkinsci/ec2-plugin/pull/201 , which addresses
