[JIRA] (JENKINS-36468) Tasks fail to run on ECS after upgrade to 1.4
Title: Message Title Luis Silva commented on JENKINS-36468 Re: Tasks fail to run on ECS after upgrade to 1.4 I managed to trace the problem to an empty Jenkins URL on the plugin config. On version 1.4 you introduced a new field to set the Jenkins URL. When I upgraded the plugin that field stayed empty and the plugin wasn't falling back to the Global jenkins url. As soon as I set the jenkinsURL config field to a correct value, the plugin started working. Now, even if I delete the contents of the url field on plugin config, it will get auto populated with the value from the Global jenkins url settings so the plugin continues to work just fine. So, I draw two conclusions from this: During the upgrade process from 1.3 something is not being correctly handled in order to populate the plugin's jenkins url field. When clearing the plugin's url field and saving I would expect it to stay empty the next time I loaded the jenkins settings page and the plugin should just internally get the global jenkins setting and use it. Instead, it is actually copying the global setting value and populating the config field with it (setting a value on the jenkins config.xml). I would never expect a plugin to set a value in the config.xml when I explicitly empty the field. Please consider these two inputs for an eventual improvement on the way the plugin works. You can either close this issue or use it to track an update to the plugin. Thanks for your help. Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
[JIRA] (JENKINS-36468) Tasks fail to run on ECS after upgrade to 1.4
Title: Message Title Luis Silva updated an issue Jenkins / JENKINS-36468 Tasks fail to run on ECS after upgrade to 1.4 Change By: Luis Silva Environment: Jenkins 2. 11 17 ECS-plugin 1.4 Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-36468) Tasks fail to run on ECS after upgrade to 1.4
Title: Message Title
Luis Silva edited a comment on JENKINS-36468
Re: Tasks fail to run on ECS after upgrade to 1.4
Hi Jan, Sorry for the late reply but I 've been off.I can't give you access to that Env I'm sorry, company policy. I ran this again with 1.4 with the Scenario below:- nothing running in running in Jenkins- nothing running/pending on ECS- Only 1 server in the ASG that serves ECS and correctly registered with ECS * Analysis * :It looks like Jenkins triggers the task but times out after 9/10 seconds. In the meantime, ECS is happily pulling the image from ECR but when it is ready to notify Jenkins that the container is up and running, Jenkins is long gone. During that period Jenkins keeps trying to launch the task on ECS multiple time, but there aren't any more resources available because the first task is still spooling up in ECS and it takes up almost all of the resources on that server. Check the logs below: * Jenkins log * :{quote}Aug 10, 2016 4:31:26 PM hudson.slaves.NodeProvisioner$StandardStrategyImpl applyINFO: Started provisioning ECS Slave MyTests from JenkinsSlavesECS with 1 executors. Remaining excess workload: 0Aug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Found 1 instancesAug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Resources found in instance arn:aws:ecs:eu-west-1:ACCOUNT_ID:container-instance/2506c5b8-946c-41c6-b01d-x: [{Name: CPU,Type: INTEGER,DoubleValue: 0.0,LongValue: 0,IntegerValue: 1024,StringSetValue: []}, {Name: MEMORY,Type: INTEGER,DoubleValue: 0.0,LongValue: 0,IntegerValue: 3767,StringSetValue: []}, {Name: PORTS,Type: STRINGSET,DoubleValue: 0.0,LongValue: 0,IntegerValue: 0,StringSetValue: [22, 2376, 2375, 51678]}, {Name: PORTS_UDP,Type: STRINGSET,DoubleValue: 0.0,LongValue: 0,IntegerValue: 0,StringSetValue: []}]Aug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Instance arn:aws:ecs:eu-west-1:ACCOUNT_ID:container-instance/2506c5b8-946c-41c6-b01d-x has 3,767mb of free memory. 3,200mb are requiredAug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Instance arn:aws:ecs:eu-west-1:ACCOUNT_ID:container-instance/2506c5b8-946c-41c6-b01d-x has 1,024 units of free cpu. 1,024 units are requiredAug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback callINFO: Created Slave: JenkinsSlavesECS-yyAug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback callINFO: Slave JenkinsSlavesECS-yy - Slave Task Started : arn:aws:ecs:eu-west-1:ACCOUNT_ID:task/ed743e1e-7be1-455b-a01d-zzAug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback callWARNING: ECS Slave JenkinsSlavesECS-yy (ecs task arn:aws:ecs:eu-west-1:ACCOUNT_ID:task/ed743e1e-7be1-455b-a01d-zz) not connected since 8/10/16 4:31 PM secondsAug 10, 2016 4:31:36 PM hudson.slaves.NodeProvisioner$2 runWARNING: Provisioned agent ECS Slave MyTests failed to launchjava.lang.IllegalStateException: ECS Slave JenkinsSlavesECS-yy (ecs task arn:aws:ecs:eu-west-1:ACCOUNT_ID:task/ed743e1e-7be1-455b-a01d-zz) not connected since 8/10/16 4:31 PM secondsat com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:278)at com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:219)at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
[JIRA] (JENKINS-36468) Tasks fail to run on ECS after upgrade to 1.4
Title: Message Title
Luis Silva edited a comment on JENKINS-36468
Re: Tasks fail to run on ECS after upgrade to 1.4
Hi Jan,I can't give you access to that Env I'm sorry, company policy. I ran this again with 1.4 with the Scenario below:- nothing running in running in Jenkins- nothing running/pending on ECS- Only 1 server in the ASG that serves ECS and correctly registered with ECSAnalysis:It looks like Jenkins triggers the task but times out after 9/10 seconds. In the meantime, ECS is happily pulling the image from ECR but when it is ready to notify Jenkins that the container is up and running, Jenkins is long gone. During that period Jenkins keeps trying to launch the task on ECS multiple time, but there aren't any more resources available because the first task is still spooling up in ECS and it takes up almost all of the resources on that server. Check the logs below: Jenkins log:{quote}Aug 10, 2016 4:31:26 PM hudson.slaves.NodeProvisioner$StandardStrategyImpl applyINFO: Started provisioning ECS Slave MyTests from JenkinsSlavesECS with 1 executors. Remaining excess workload: 0Aug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Found 1 instancesAug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Resources found in instance arn:aws:ecs:eu-west-1:ACCOUNT_ID:container-instance/2506c5b8-946c-41c6-b01d-x: [{Name: CPU,Type: INTEGER,DoubleValue: 0.0,LongValue: 0,IntegerValue: 1024,StringSetValue: []}, {Name: MEMORY,Type: INTEGER,DoubleValue: 0.0,LongValue: 0,IntegerValue: 3767,StringSetValue: []}, {Name: PORTS,Type: STRINGSET,DoubleValue: 0.0,LongValue: 0,IntegerValue: 0,StringSetValue: [22, 2376, 2375, 51678]}, {Name: PORTS_UDP,Type: STRINGSET,DoubleValue: 0.0,LongValue: 0,IntegerValue: 0,StringSetValue: []}]Aug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Instance arn:aws:ecs:eu-west-1:ACCOUNT_ID:container-instance/2506c5b8-946c-41c6-b01d-x has 3,767mb of free memory. 3,200mb are requiredAug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Instance arn:aws:ecs:eu-west-1:ACCOUNT_ID:container-instance/2506c5b8-946c-41c6-b01d-x has 1,024 units of free cpu. 1,024 units are requiredAug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback callINFO: Created Slave: JenkinsSlavesECS-yyAug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback callINFO: Slave JenkinsSlavesECS-yy - Slave Task Started : arn:aws:ecs:eu-west-1:ACCOUNT_ID:task/ed743e1e-7be1-455b-a01d-zzAug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback callWARNING: ECS Slave JenkinsSlavesECS-yy (ecs task arn:aws:ecs:eu-west-1:ACCOUNT_ID:task/ed743e1e-7be1-455b-a01d-zz) not connected since 8/10/16 4:31 PM secondsAug 10, 2016 4:31:36 PM hudson.slaves.NodeProvisioner$2 runWARNING: Provisioned agent ECS Slave MyTests failed to launchjava.lang.IllegalStateException: ECS Slave JenkinsSlavesECS-yy (ecs task arn:aws:ecs:eu-west-1:ACCOUNT_ID:task/ed743e1e-7be1-455b-a01d-zz) not connected since 8/10/16 4:31 PM secondsat com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:278)at com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:219)at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)at
[JIRA] (JENKINS-36468) Tasks fail to run on ECS after upgrade to 1.4
Title: Message Title
Luis Silva commented on JENKINS-36468
Re: Tasks fail to run on ECS after upgrade to 1.4
Hi Jan, I can't give you access to that Env I'm sorry, company policy. I ran this again with 1.4 with the Scenario below:
nothing running in running in Jenkins
nothing running/pending on ECS
Only 1 server in the ASG that serves ECS and correctly registered with ECS
Analysis: It looks like Jenkins triggers the task but times out after 9/10 seconds. In the meantime, ECS is happily pulling the image from ECR but when it is ready to notify Jenkins that the container is up and running, Jenkins is long gone. Check the logs below: Jenkins log:
Aug 10, 2016 4:31:26 PM hudson.slaves.NodeProvisioner$StandardStrategyImpl apply INFO: Started provisioning ECS Slave MyTests from JenkinsSlavesECS with 1 executors. Remaining excess workload: 0 Aug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResources INFO: Found 1 instances Aug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResources INFO: Resources found in instance arn:aws:ecs:eu-west-1:ACCOUNT_ID:container-instance/2506c5b8-946c-41c6-b01d-x: [
Unknown macro: {Name}
,
Unknown macro: {Name}
,
Unknown macro: {Name}
,
Unknown macro: {Name}
] Aug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResources INFO: Instance arn:aws:ecs:eu-west-1:ACCOUNT_ID:container-instance/2506c5b8-946c-41c6-b01d-x has 3,767mb of free memory. 3,200mb are required Aug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResources INFO: Instance arn:aws:ecs:eu-west-1:ACCOUNT_ID:container-instance/2506c5b8-946c-41c6-b01d-x has 1,024 units of free cpu. 1,024 units are required Aug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback call INFO: Created Slave: JenkinsSlavesECS-yy Aug 10, 2016 4:31:27 PM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback call INFO: Slave JenkinsSlavesECS-yy - Slave Task Started : arn:aws:ecs:eu-west-1:ACCOUNT_ID:task/ed743e1e-7be1-455b-a01d-zz Aug 10, 2016 4:31:27 PM
[JIRA] (JENKINS-36468) Tasks fail to run on ECS after upgrade to 1.4
Title: Message Title Luis Silva edited a comment on JENKINS-36468 Re: Tasks fail to run on ECS after upgrade to 1.4 Hi Jan,I'm using +only+ m3.medium instances with 3.7GB of memory available on a ASG. I tested this with one instance only min=max=desired=1 and it was failing.Before digging into the logs I added another instance thinking it was related to resource starvation. It still didn't work, then I grabbed the above logs.There were no other jenkins jobs running when I was testing this (master, regular slave or ECS slaves), and there were no other tasks or services (running or pending) in the ECS console when I tested this.I downgraded to 1.3 it hasn't failed once. Whenever I try to upgrade to 1.4, they immediately stop working. The attached image shows my ECS console in Standby waiting for the tests to run (that either pass or fail depending on the plugin version) Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-36468) Tasks fail to run on ECS after upgrade to 1.4
Title: Message Title Luis Silva updated an issue Jenkins / JENKINS-36468 Tasks fail to run on ECS after upgrade to 1.4 Change By: Luis Silva Attachment: ECS.jpg Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-36468) Tasks fail to run on ECS after upgrade to 1.4
Title: Message Title Luis Silva commented on JENKINS-36468 Re: Tasks fail to run on ECS after upgrade to 1.4 Hi Jan, I'm using only m3.medium instances with 3.7GB of memory available on a ASG. I tested this with one instance only min=max=desired=1 and it was failing. Before digging into the logs I added another instance thinking it was related to resource starvation. It still didn't work, then I grabbed the above logs. There were no other jenkins jobs running when I was testing this (master, regular slave or ECS slaves), and there were no other tasks or services (running or pending) in the ECS console when I tested this. I downgraded to 1.3 it hasn't failed once. Whenever I try to upgrade to 1.4, they immediately stop working. Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-36468) Tasks fail to run on ECS after upgrade to 1.4
Title: Message Title
Luis Silva updated an issue
Jenkins / JENKINS-36468
Tasks fail to run on ECS after upgrade to 1.4
Change By:
Luis Silva
With the latest plugin version (1.4) my tasks fail to run on the ecs slaves. They sit waiting for executors forever. I made sure I had two servers with plenty of resources connected to ECS. It looks like the task is assigned to a node but then immediately disconnects.After downgrading to version 1.3 it all started working perfectly again.I tested it with the same number of servers, same tasks, same jenkins versions, etc.See the logs below for both versions.Version 1.4 failures:{quote}Jul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Instance arn:aws:ecs:eu-west-1:AWS_ACC:container-instance/ceb4eece-2abe-4f19-aad3-8d3cbd18a7ee has 3,767mb of free memory. 3,000mb are requiredJul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Instance arn:aws:ecs:eu-west-1:AWS_ACC:container-instance/ceb4eece-2abe-4f19-aad3-8d3cbd18a7ee has 1,024 units of free cpu. 512 units are requiredJul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback callINFO: Created Slave: JenkinsSlavesECS-16e94238750e85Jul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService runEcsTaskWARNING: Slave JenkinsSlavesECS-16e94238750e85 - Failure to run task with definition arn:aws:ecs:eu-west-1:AWS_ACC:task-definition/jenkins-slave:78 on ECS cluster arn:aws:ecs:eu-west-1:AWS_ACC:cluster/JenkinsSlavesJul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService runEcsTaskWARNING: Slave JenkinsSlavesECS-16e94238750e85 - Failure reason=RESOURCE:MEMORY, arn=arn:aws:ecs:eu-west-1:AWS_ACC:container-instance/e949c164-1e0c-4e9c-85c3-36b035b5b601Jul 06, 2016 8:37:43 AM hudson.slaves.NodeProvisioner$2 runWARNING: Provisioned agent ECS Slave BackendTests failed to launchjava.lang.IllegalStateException: ECS Slave JenkinsSlavesECS-16e9422efc9bc3 (ecs task arn:aws:ecs:eu-west-1:AWS_ACC:task/b6621ca4-b397-46ad-9659-9c6fe1633b32) not connected since 7/6/16 8:37 AM secondsat com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:278)at com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:219)at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)at java.util.concurrent.FutureTask.run(FutureTask.java:262)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)at java.lang.Thread.run(Thread.java:745)Jul 06, 2016 8:37:43 AM hudson.slaves.NodeProvisioner$2 runWARNING: Provisioned agent ECS Slave BackendTests failed to launchhudson.AbortException: Failed to run slave container JenkinsSlavesECS-16e94238750e85at
[JIRA] (JENKINS-36468) Tasks fail to run on ECS after upgrade to 1.4
Title: Message Title
Luis Silva updated an issue
Jenkins / JENKINS-36468
Tasks fail to run on ECS after upgrade to 1.4
Change By:
Luis Silva
With the latest plugin version (1.4) my tasks fail to run on the ecs slaves. They sit waiting for executors forever. I made sure I had two servers with plenty of resources connected to ECS. It looks like the task is assigned to a node but they then immediately disconnects.After downgrading to version 1.3 is all started working perfectly again.I tested it with the same number of servers, same tasks, same jenkins versions, etc.See the logs below for both versions.Version 1.4 failures:{quote}Jul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Instance arn:aws:ecs:eu-west-1:AWS_ACC:container-instance/ceb4eece-2abe-4f19-aad3-8d3cbd18a7ee has 3,767mb of free memory. 3,000mb are requiredJul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Instance arn:aws:ecs:eu-west-1:AWS_ACC:container-instance/ceb4eece-2abe-4f19-aad3-8d3cbd18a7ee has 1,024 units of free cpu. 512 units are requiredJul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback callINFO: Created Slave: JenkinsSlavesECS-16e94238750e85Jul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService runEcsTaskWARNING: Slave JenkinsSlavesECS-16e94238750e85 - Failure to run task with definition arn:aws:ecs:eu-west-1:AWS_ACC:task-definition/jenkins-slave:78 on ECS cluster arn:aws:ecs:eu-west-1:AWS_ACC:cluster/JenkinsSlavesJul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService runEcsTaskWARNING: Slave JenkinsSlavesECS-16e94238750e85 - Failure reason=RESOURCE:MEMORY, arn=arn:aws:ecs:eu-west-1:AWS_ACC:container-instance/e949c164-1e0c-4e9c-85c3-36b035b5b601Jul 06, 2016 8:37:43 AM hudson.slaves.NodeProvisioner$2 runWARNING: Provisioned agent ECS Slave BackendTests failed to launchjava.lang.IllegalStateException: ECS Slave JenkinsSlavesECS-16e9422efc9bc3 (ecs task arn:aws:ecs:eu-west-1:AWS_ACC:task/b6621ca4-b397-46ad-9659-9c6fe1633b32) not connected since 7/6/16 8:37 AM secondsat com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:278)at com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:219)at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)at java.util.concurrent.FutureTask.run(FutureTask.java:262)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)at java.lang.Thread.run(Thread.java:745)Jul 06, 2016 8:37:43 AM hudson.slaves.NodeProvisioner$2 runWARNING: Provisioned agent ECS Slave BackendTests failed to launchhudson.AbortException: Failed to run slave container JenkinsSlavesECS-16e94238750e85at
[JIRA] (JENKINS-36468) Tasks fail to run on ECS after upgrade to 1.4
Title: Message Title
Luis Silva updated an issue
Jenkins / JENKINS-36468
Tasks fail to run on ECS after upgrade to 1.4
Change By:
Luis Silva
With the latest plugin version (1.4) my tasks fail to run on the ecs slaves. They sit waiting for executors forever. I made sure I had two servers with plenty of resources connected to ECS. It looks like the task is assigned to a node but then immediately disconnects.After downgrading to version 1.3 is it all started working perfectly again.I tested it with the same number of servers, same tasks, same jenkins versions, etc.See the logs below for both versions.Version 1.4 failures:{quote}Jul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Instance arn:aws:ecs:eu-west-1:AWS_ACC:container-instance/ceb4eece-2abe-4f19-aad3-8d3cbd18a7ee has 3,767mb of free memory. 3,000mb are requiredJul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResourcesINFO: Instance arn:aws:ecs:eu-west-1:AWS_ACC:container-instance/ceb4eece-2abe-4f19-aad3-8d3cbd18a7ee has 1,024 units of free cpu. 512 units are requiredJul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback callINFO: Created Slave: JenkinsSlavesECS-16e94238750e85Jul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService runEcsTaskWARNING: Slave JenkinsSlavesECS-16e94238750e85 - Failure to run task with definition arn:aws:ecs:eu-west-1:AWS_ACC:task-definition/jenkins-slave:78 on ECS cluster arn:aws:ecs:eu-west-1:AWS_ACC:cluster/JenkinsSlavesJul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService runEcsTaskWARNING: Slave JenkinsSlavesECS-16e94238750e85 - Failure reason=RESOURCE:MEMORY, arn=arn:aws:ecs:eu-west-1:AWS_ACC:container-instance/e949c164-1e0c-4e9c-85c3-36b035b5b601Jul 06, 2016 8:37:43 AM hudson.slaves.NodeProvisioner$2 runWARNING: Provisioned agent ECS Slave BackendTests failed to launchjava.lang.IllegalStateException: ECS Slave JenkinsSlavesECS-16e9422efc9bc3 (ecs task arn:aws:ecs:eu-west-1:AWS_ACC:task/b6621ca4-b397-46ad-9659-9c6fe1633b32) not connected since 7/6/16 8:37 AM secondsat com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:278)at com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback.call(ECSCloud.java:219)at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)at java.util.concurrent.FutureTask.run(FutureTask.java:262)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)at java.lang.Thread.run(Thread.java:745)Jul 06, 2016 8:37:43 AM hudson.slaves.NodeProvisioner$2 runWARNING: Provisioned agent ECS Slave BackendTests failed to launchhudson.AbortException: Failed to run slave container JenkinsSlavesECS-16e94238750e85at
[JIRA] (JENKINS-36468) Tasks fail to run on ECS after upgrade to 1.4
Title: Message Title Luis Silva created an issue Jenkins / JENKINS-36468 Tasks fail to run on ECS after upgrade to 1.4 Issue Type: Bug Assignee: Jan Roehrich Components: amazon-ecs-plugin Created: 2016/Jul/06 8:55 AM Environment: Jenkins 2.11 ECS-plugin 1.4 Priority: Blocker Reporter: Luis Silva With the latest plugin version (1.4) my tasks fail to run on the ecs slaves. They sit waiting for executors forever. I made sure I had two servers with plenty of resources connected to ECS. It looks like the task is assigned to a node but they immediately disconnects. After downgrading to version 1.3 is all started working perfectly again. I tested it with the same number of servers, same tasks, same jenkins versions, etc. See the logs below for both versions. Version 1.4 failures: Jul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResources INFO: Instance arn:aws:ecs:eu-west-1:AWS_ACC:container-instance/ceb4eece-2abe-4f19-aad3-8d3cbd18a7ee has 3,767mb of free memory. 3,000mb are required Jul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService waitForSufficientClusterResources INFO: Instance arn:aws:ecs:eu-west-1:AWS_ACC:container-instance/ceb4eece-2abe-4f19-aad3-8d3cbd18a7ee has 1,024 units of free cpu. 512 units are required Jul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSCloud$ProvisioningCallback call INFO: Created Slave: JenkinsSlavesECS-16e94238750e85 Jul 06, 2016 8:37:33 AM com.cloudbees.jenkins.plugins.amazonecs.ECSService runEcsTask WARNING: Slave JenkinsSlavesECS-16e94238750e85 - Failure to run task with definition
[JIRA] [amazon-ecr-plugin] (JENKINS-34958) Getting "Your Authorization Token has expired" when using ECR credentials
Title: Message Title Luis Silva edited a comment on JENKINS-34958 Re: Getting "Your Authorization Token has expired" when using ECR credentials Faced this same issue today with this plugin today .Configuring ECR in us-east-1 region ( ***.dkr.ecr.us-east-1.amazonaws.com/*** ) the push succeeds.But using ECR in eu-west-1 ( ***.dkr.ecr.eu-west-1.amazonaws.com/*** ) it fails: with an error:error parsing HTTP 403 response body: unexpected end of JSON input: ""Build step 'Docker Build and Publish' marked build as failureIt's the exact same job and settings, only changing the registry endpoint in the job.Using the command line to issue a push with the token obtained by the jenkins job (set in ~/.docker/config.json) also fails, but using the aws cli to get the token, issuing a docker login and then push works fine regardless of the region.That leads to me to believe the problem is somewhere in the token / credentials issue code as you mentioned.Any ETA on the fix for this problem?Docker version 1.11.1, build 5604cbe/1.11.1Amazon ECR plugin: 1.3CloudBees Docker Build and Publish plugin: 1.2.2Docker Commons Plugin: 1.3.1 Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [amazon-ecr-plugin] (JENKINS-34958) Getting "Your Authorization Token has expired" when using ECR credentials
Title: Message Title Luis Silva commented on JENKINS-34958 Re: Getting "Your Authorization Token has expired" when using ECR credentials Faced this same issue today with this plugin today. Configuring ECR in us-east-1 region ( **.dkr.ecr.us-east-1.amazonaws.com/** ) the push succeeds. But using ECR in eu-west-1 ( **.dkr.ecr.eu-west-1.amazonaws.com/** ) it fails: with an error: error parsing HTTP 403 response body: unexpected end of JSON input: "" Build step 'Docker Build and Publish' marked build as failure It's the exact same job and settings, only changing the registry endpoint in the job. Using the command line to issue a push with the token obtained by the jenkins job (set in ~/.docker/config.json) also fails, but using the aws cli to get the token, issuing a docker login and then push works fine regardless of the region. That leads to me to believe the problem is somewhere in the token / credentials issue code as you mentioned. Any ETA on the fix for this problem? Docker version 1.11.1, build 5604cbe/1.11.1 Amazon ECR plugin: 1.3 CloudBees Docker Build and Publish plugin: 1.2.2 Docker Commons Plugin: 1.3.1 Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [jira-trigger-plugin] (JENKINS-34135) Plugin requires anonymous to have Job Read permission
Title: Message Title
Luis Silva commented on JENKINS-34135
Re: Plugin requires anonymous to have Job Read permission
The new 0.2.2 version is not showing up on jenkins update center. URL: http://jenkins.mirror.isppower.de/updates/current/update-center.json
Response:
.
"jira-trigger": {
"buildDate": "Apr 10, 2016",
"compatibleSinceVersion": "0.2.0",
"dependencies": [],
"developers": [
{
"developerId": "ceilfors",
"email": "wi...@ceilfors.com",
"name": "Wisen Tanasa"
}
],
"excerpt": "Triggers a build when a certain condition is matched in JIRA.",
"gav": "org.jenkins-ci.plugins:jira-trigger:0.2.1",
"labels": [
"trigger",
"external"
],
"name": "jira-trigger",
"previousTimestamp": "2016-02-16T02:44:40.00Z",
"previousVersion": "0.2.0",
"releaseTimestamp": "2016-04-10T12:54:00.00Z",
"requiredCore": "1.565.3",
"scm": "github.com",
"sha1": "2HNtsq1nff0jwgvJjs7ix2SAsqc=",
"title": "JIRA Trigger Plugin",
"url": "http://updates.jenkins-ci.org/download/plugins/jira-trigger/0.2.1/jira-trigger.hpi",
"version": "0.2.1",
"wiki": "https://wiki.jenkins-ci.org/display/JENKINS/JIRA+Trigger+Plugin"
},
..
Add Comment
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [jira-trigger-plugin] (JENKINS-34301) Plugin swallows every exceptions thrown in http requests
Title: Message Title Luis Silva commented on JENKINS-34301 Re: Plugin swallows every exceptions thrown in http requests Thanks, I'll keep an eye on this and update my plugin once you release the fix. Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [jira-trigger-plugin] (JENKINS-34135) Plugin requires anonymous to have Job Read permission
Title: Message Title
Luis Silva edited a comment on JENKINS-34135
Re: Plugin requires anonymous to have Job Read permission
Hi,This last update 0.2.1 changed the behaviour of Jenkins root page.With the plugin enabled, accessing JENKINS_URL/ returns a blank page with no content:{code}< HTTP/1.1 200 OK< Content-Length: 0{code}whilst with the plugin disabled I get:{code}< HTTP/1.1 403 Forbidden< X-Content-Type-Options: nosniff< Content-Type: text/html;charset=UTF-8< X-You-Are-Authenticated-As: anonymous< X-You-Are-In-Group:< X-Required-Permission: hudson.model.Hudson.Read< X-Permission-Implied-By: hudson.security.Permission.GenericRead< X-Permission-Implied-By: hudson.model.Hudson.Administer< Content-Length: 793{code}This is what I used to get before and what allowed people to login without having to access the /login URI. The page contains just a login prompt and a jenkins page header.Both of those scenarios are without any permissions set for the anonymous user, which is how I use Jenkins. If I add overall read access then the blank page turns into a page with a bit more data then I'd like to display before allowing for a login non authenticated user .
Add Comment
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [jira-trigger-plugin] (JENKINS-34135) Plugin requires anonymous to have Job Read permission
Title: Message Title
Luis Silva edited a comment on JENKINS-34135
Re: Plugin requires anonymous to have Job Read permission
Hi,This last update 0.2.1 changed the behaviour of Jenkins root page.With the plugin enabled, accessing JENKINS_URL/ returns this a blank page with no content:{code}< HTTP/1.1 200 OK< Content-Length: 0{code}whilst with the plugin disabled I get:{code}< HTTP/1.1 403 Forbidden< X-Content-Type-Options: nosniff< Content-Type: text/html;charset=UTF-8< X-You-Are-Authenticated-As: anonymous< X-You-Are-In-Group:< X-Required-Permission: hudson.model.Hudson.Read< X-Permission-Implied-By: hudson.security.Permission.GenericRead< X-Permission-Implied-By: hudson.model.Hudson.Administer< Content-Length: 793{code}This is what I used to get before and what allowed people to login without having to access the /login URI. The page contains just a login prompt and a jenkins page header.Both of those scenarios are without any permissions set for the anonymous user, which is how I use Jenkins. If I add overall read access then the blank page turns into a page with a bit more data then I'd like to display before allowing a login.
Add Comment
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [jira-trigger-plugin] (JENKINS-34135) Plugin requires anonymous to have Job Read permission
Title: Message Title Luis Silva reopened an issue Hi, This last update 0.2.1 changed the behaviour of Jenkins root page. With the plugin enabled, accessing JENKINS_URL/ returns this a blank page with no content: < HTTP/1.1 200 OK < Content-Length: 0 whilst with the plugin disabled I get: < HTTP/1.1 403 Forbidden < X-Content-Type-Options: nosniff < Content-Type: text/html;charset=UTF-8 < X-You-Are-Authenticated-As: anonymous < X-You-Are-In-Group: < X-Required-Permission: hudson.model.Hudson.Read < X-Permission-Implied-By: hudson.security.Permission.GenericRead < X-Permission-Implied-By: hudson.model.Hudson.Administer < Content-Length: 793 This is what I used to get before and what allowed people to login without having to access the /login URI. The page contains just a login prompt and a jenkins page header. Both of those scenarios are without any permissions set for the anonymous user, which is how I use Jenkins. If I add overall read access then the blank page turns into a page with a bit more data then I'd like to display before allowing a login. Jenkins / JENKINS-34135 Plugin requires anonymous to have Job Read permission Change By: Luis Silva Resolution: Fixed Status: Resolved Reopened Add Comment
[JIRA] [jira-trigger-plugin] (JENKINS-34135) Plugin unable to find jobs configured with JiraChangelogTrigger
Title: Message Title Luis Silva edited a comment on JENKINS-34135 Re: Plugin unable to find jobs configured with JiraChangelogTrigger That works. The only required permission is Anonymous -> Job Read.It can either be set globally on 'Manage Jenkins' - 'Configure Global Security' or directly on the job(s) which we're configuring to use the jira-trigger-plugin by ticking 'Enable project-based security' and setting the Anonymous -> Job Read permission. Worth adding that to the plugin docs / github README. I don't know much about writing Jenkins plugins, but is there any chance you can require auth in the form of https://user:password:jenkins_url/jira-trigger-webhook-receiver/ (since the jira payload is not modifiable) to even trigger the plugin execution? And maybe even use those creds in the plugin itself to run as a non anonymous user?And why would you need perms to read a job settings but not to trigger the actual build? Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [jira-trigger-plugin] (JENKINS-34135) Plugin unable to find jobs configured with JiraChangelogTrigger
Title: Message Title Luis Silva commented on JENKINS-34135 Re: Plugin unable to find jobs configured with JiraChangelogTrigger That works. The only required permission is Anonymous -> Job Read. It can either be set globally on 'Manage Jenkins' - 'Configure Global Security' or directly on the job(s) which we're configuring to use the jira-trigger-plugin by ticking 'Enable project-based security' and setting the Anonymous -> Job Read permission. I don't know much about writing Jenkins plugins, but is there any chance you can require auth in the form of https://user:password:jenkins_url/jira-trigger-webhook-receiver/ (since the jira payload is not modifiable) to even trigger the plugin execution? And maybe even use those creds in the plugin itself to run as a non anonymous user? And why would you need perms to read a job settings but not to trigger the actual build? Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [jira-trigger-plugin] (JENKINS-34135) Plugin unable to find jobs configured with JiraChangelogTrigger
Title: Message Title
Luis Silva updated an issue
Jenkins / JENKINS-34135
Plugin unable to find jobs configured with JiraChangelogTrigger
Change By:
Luis Silva
Jira webhooks are working fine but the plugin doesn't seem to find any configured jobs. Check the log output below:{code}Apr 08, 2016 5:04:58 PM FINEST com.ceilfors.jenkins.plugins.jiratrigger.webhook.JiraWebhookWebhook event body:Apr 08, 2016 5:04:58 PM FINEST com.ceilfors.jenkins.plugins.jiratrigger.webhook.JiraWebhook{"timestamp": 1460135096626,"webhookEvent": "jira:issue_updated","issue_event_type_name": "issue_generic","user": {"changelog": {"id": "12684","items": [{"field": "resolution","fieldtype": "jira","from": null,"fromString": null,"to": "1","toString": "Done"},{"field": "status","fieldtype": "jira","from": "3","fromString": "In Progress","to": "10002","toString": "Done"}]}}Apr 08, 2016 5:04:58 PM FINE com.ceilfors.jenkins.plugins.jiratrigger.webhook.JiraWebhookReceived Webhook callback from changelog. Event type: jira:issue_updatedApr 08, 2016 5:04:58 PM FINE com.ceilfors.jenkins.plugins.jiratrigger.JiraTriggerExecutorCouldn't find any projects that have JiraChangelogTrigger configured{code}But testing it in isolation in jenkins script console returns valid jobs: {code} import com.ceilfors.jenkins.plugins.jiratrigger.JiraChangelogTriggerprojects = Jenkins.instance.getAllItems(AbstractProject).findAll { it.getTrigger(JiraChangelogTrigger) }println(projects)Result[hudson.model.FreeStyleProject@44a9508c[test]] {code}
Add Comment
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
[JIRA] [jira-trigger-plugin] (JENKINS-34135) Plugin unable to find jobs configured with JiraChangelogTrigger
Title: Message Title
Luis Silva updated an issue
Jenkins / JENKINS-34135
Plugin unable to find jobs configured with JiraChangelogTrigger
Change By:
Luis Silva
Jira webhooks are working fine but the plugin doesn't seem to find any configured jobs. Check the log output below: {code} Apr 08, 2016 5:04:58 PM FINEST com.ceilfors.jenkins.plugins.jiratrigger.webhook.JiraWebhookWebhook event body:Apr 08, 2016 5:04:58 PM FINEST com.ceilfors.jenkins.plugins.jiratrigger.webhook.JiraWebhook{"timestamp": 1460135096626,"webhookEvent": "jira:issue_updated","issue_event_type_name": "issue_generic","user": {"changelog": {"id": "12684","items": [{"field": "resolution","fieldtype": "jira","from": null,"fromString": null,"to": "1","toString": "Done"},{"field": "status","fieldtype": "jira","from": "3","fromString": "In Progress","to": "10002","toString": "Done"}]}}Apr 08, 2016 5:04:58 PM FINE com.ceilfors.jenkins.plugins.jiratrigger.webhook.JiraWebhookReceived Webhook callback from changelog. Event type: jira:issue_updatedApr 08, 2016 5:04:58 PM FINE com.ceilfors.jenkins.plugins.jiratrigger.JiraTriggerExecutorCouldn't find any projects that have JiraChangelogTrigger configured {code} But testing it in isolation in jenkins script console returns valid jobs:import com.ceilfors.jenkins.plugins.jiratrigger.JiraChangelogTriggerprojects = Jenkins.instance.getAllItems(AbstractProject).findAll { it.getTrigger(JiraChangelogTrigger) }println(projects)Result[hudson.model.FreeStyleProject@44a9508c[test]]
Add Comment
This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265)
[JIRA] [jira-trigger-plugin] (JENKINS-34135) Plugin unable to find jobs configured with JiraChangelogTrigger
Title: Message Title
Luis Silva created an issue
Jenkins / JENKINS-34135
Plugin unable to find jobs configured with JiraChangelogTrigger
Issue Type:
Bug
Assignee:
Wisen Tanasa
Components:
jira-trigger-plugin
Created:
2016/Apr/08 6:37 PM
Environment:
Jenkins version: 1.642.4 Plugin version: 0.2.0 Using Jira Atlassian Cloud
Priority:
Blocker
Reporter:
Luis Silva
Jira webhooks are working fine but the plugin doesn't seem to find any configured jobs. Check the log output below:
Apr 08, 2016 5:04:58 PM FINEST com.ceilfors.jenkins.plugins.jiratrigger.webhook.JiraWebhook Webhook event body: Apr 08, 2016 5:04:58 PM FINEST com.ceilfors.jenkins.plugins.jiratrigger.webhook.JiraWebhook { "timestamp": 1460135096626, "webhookEvent": "jira:issue_updated", "issue_event_type_name": "issue_generic", "user": { "changelog": { "id": "12684", "items": [ { "field": "resolution", "fieldtype": "jira", "from": null, "fromString": null, "to": "1", "toString": "Done" }
, { "field": "status", "fieldtype": "jira", "from": "3", "fromString": "In Progress", "to": "10002", "toString": "Done" }
] } } Apr 08, 2016 5:04:58 PM FINE com.ceilfors.jenkins.plugins.jiratrigger.webhook.JiraWebhook Received Webhook callback from changelog. Event type:
