[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Oliver Gondža updated an issue Jenkins / JENKINS-41127 Multiple pipeline instances running concurrently when concurrent execution disabled Change By: Oliver Gondža Labels: concurrent lts-candidate Add Comment This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Devin Nusbaum commented on JENKINS-41127 Re: Multiple pipeline instances running concurrently when concurrent execution disabled Fixed in Jenkins 2.136. I am marking this as an LTS candidate given the impact and simplicity of the fix, although we will have to give it some time to make sure there are no regressions. Add Comment This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Devin Nusbaum updated an issue Jenkins / JENKINS-41127 Multiple pipeline instances running concurrently when concurrent execution disabled Change By: Devin Nusbaum Labels: concurrent lts-candidate Add Comment This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Oleg Nenashev updated JENKINS-41127 Jenkins / JENKINS-41127 Multiple pipeline instances running concurrently when concurrent execution disabled Change By: Oleg Nenashev Status: In Review Resolved Resolution: Fixed Released As: Jenkins 2.136 Add Comment This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Andrew Bayer updated an issue Jenkins / JENKINS-41127 Multiple pipeline instances running concurrently when concurrent execution disabled Change By: Andrew Bayer Component/s: core Component/s: workflow-job-plugin Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Sam Van Oort commented on JENKINS-41127 Re: Multiple pipeline instances running concurrently when concurrent execution disabled Andrew Bayer Could we recategorize to core on the basis of your analysis? Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Andrew Bayer commented on JENKINS-41127 Re: Multiple pipeline instances running concurrently when concurrent execution disabled fwiw, I think this likely only will happen with a flyweight task - so you could probably brew up a reproduction case with a matrix job, but I doubt you could do so with a freestyle job. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Andrew Bayer commented on JENKINS-41127 Re: Multiple pipeline instances running concurrently when concurrent execution disabled So Queue#maintain() is running twice, one immediately after the other, in some cases - probably race conditiony, not yet sure how the two are getting called. Anyway, the first run is making the first item in the queue buildable and calls makeBuildable on the item, removing said item from blockedProjects, and, via makeFlyweightTaskBuildable and createFlyWeightTaskRunnable, starting the flyweight task and adding the first item to pendings. All is well and good. But then the next run of maintain starts - and it can't find the task for the item we just started (theoretically) and put in pendings on any executor...so it removes the item from pendings. Then it gets to checking the queue again, and the new first item doesn't have anything blocking it (i.e., nothing in buildables or pending) and so...it goes through the same process as the previous item did in the previous maintain run. End result: two builds get started at the same time. So - definitely a race condition. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Andrew Bayer commented on JENKINS-41127 Re: Multiple pipeline instances running concurrently when concurrent execution disabled Got it to reproduce eventually, while I had some extra logging in Queue#getCauseOfBlockageForItem (and some other places, but that's the one that gave me something interesting). For the first few hundred jobs, everything was consistent: all the pending items would not be blocked by either Queue#getCauseOfBlockageForTask or QueueTaskDispatcher, they all were not BuildableItem, and they all had isConcurrentBuild() == false. The first item would not find its task in either buildables or pendings, and so would kick off. All the other pending items would find their tasks in pendings and so would stay queued. Yay, that's how it's supposed to be. But eventually...first item fine, many consecutive items fine, and then...one of them couldn't find its task in pendings and so kicked off too. That was followed by the rest of the queued items behaving like normal. I haven't yet navigated the Queue#maintain code enough to be sure what exactly the code path here is, but I'm fairly sure that the first item got removed from pendings before the queue processing was complete. I'm trying it again with some additional logging to try to make it more clear what's happening when. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title
Andrew Bayer commented on JENKINS-41127
Re: Multiple pipeline instances running concurrently when concurrent execution disabled
I'm running a tweaked version of that now to see what happens - had to make some changes due to serialization.
@NonCPS
def getLastNum() {
def item = Jenkins.instance.getItemByFullName("bug-reproduction/jenkins-41127")
echo "${item}"
return item.getLastSuccessfulBuild().number
}
def lastNum = getLastNum()
if(lastNum == (currentBuild.number.toInteger()-1)) {//finding previous number
def number = params.TestString.toInteger()+1
node () {
stage ('Build') {
sleep 2
build job: 'jenkins-41127', parameters: [string(name: 'TestString', value: "${number}")], wait: false
}
stage ('Again'){
sleep 2
number = number+number
build job: 'jenkins-41127', parameters: [string(name: 'TestString', value: "${number}")], wait: false
}
}
}
else {
currentBuild.result == "SUCCESS"
def RunningBuildsString = getRunStr()
build job: 'jenkins-41127-fs', parameters: [string(name: 'PreviousBuild', value: "${lastNum}"), string(name: 'CurrentBuild', value: "${currentBuild.number.toInteger()}"), string(name: 'QueueStatus', value: "${RunningBuildsString}")]
}
@NonCPS
def getRunStr() {
def RunningBuildsString = ""
Jenkins.instance.getAllItems(Job).each{
def jobBuilds=it.getBuilds()
jobBuilds.each{
if (it.isBuilding()) { RunningBuildsString = (RunningBuildsString + it.toString() + " ") }
}
}
return RunningBuildsString
}
Add Comment
This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e)
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Joe Harte edited a comment on JENKINS-41127 Re: Multiple pipeline instances running concurrently when concurrent execution disabled Saw this yet again. Haven't seen the problem in hundreds of builds since my last comment above, and today I see that Jenkins pulled the next 2 builds from the queue and ran them concurrently, event though concurrent building in explicitly disabled. Using latest version of Pipeline plugins available at time of writing, and Jenkins version 2.49 [~jglick] FYI Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Joe Harte edited a comment on JENKINS-41127 Re: Multiple pipeline instances running concurrently when concurrent execution disabled Saw this yet again. Haven't seen the problem in hundreds of builds since my last comment above, and today I see that Jenkins pulled the next 2 builds from the queue and ran them concurrently, event though concurrent building in explicitly disabled. Using latest version of Pipeline plugins available at time of writing, and Jenkins version 2.49 Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Joe Harte updated an issue Jenkins / JENKINS-41127 Multiple pipeline instances running concurrently when concurrent execution disabled Change By: Joe Harte Environment: Jenkins 2.7.1 Jenkins 2.49 Pipeline plugin 2.4 Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Joe Harte commented on JENKINS-41127 Re: Multiple pipeline instances running concurrently when concurrent execution disabled Saw this yet again. Haven't seen the problem in hundreds of builds since my last comment above, and today I see that Jenkins pulled the next 2 builds from the queue and ran them concurrently, event though concurrent building in explicitly disabled. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Joe Harte commented on JENKINS-41127 Re: Multiple pipeline instances running concurrently when concurrent execution disabled Bump. Seeing this again. My pipeline is configured to disable concurrent builds, yet I saw two instances running when they were both triggered at exactly the same time, down to the second. Jenkins version now is 2.49. Add Comment This message was sent by Atlassian JIRA (v7.3.0#73011-sha1:3c73d0e) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title
Joe Harte updated an issue
Jenkins / JENKINS-41127
Multiple pipeline instances running concurrently when concurrent execution disabled
Change By:
Joe Harte
I have configured a Jenkins pipeline to disable concurrent builds:{code}properties([disableConcurrentBuilds()]){code}However, I have noticed on some occasions multiple instances of the pipeline execute concurrently. This seems to happen when the pipeline is triggered by an upstream job at the same time that a user manually triggers it - possible race condition? Will attach logs when this occurs again, don't currently have them available.
Add Comment
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title
Joe Harte updated an issue
Jenkins / JENKINS-41127
Multiple pipeline instances running concurrently when concurrent execution disabled
Change By:
Joe Harte
I have configured a Jenkins pipeline to disabled disable concurrent builds:{code}properties([disableConcurrentBuilds()]){code}However, I have noticed on some occasions multiple instances of the pipeline execute concurrently. This seems to happen when the pipeline is triggered by an upstream job at the same time that a user manually triggers it - possible race condition?
Add Comment
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-41127) Multiple pipeline instances running concurrently when concurrent execution disabled
Title: Message Title Joe Harte created an issue Jenkins / JENKINS-41127 Multiple pipeline instances running concurrently when concurrent execution disabled Issue Type: Bug Assignee: Unassigned Components: pipeline Created: 2017/Jan/17 12:29 PM Environment: Jenkins 2.7.1 Pipeline plugin 2.4 Labels: pipeline concurrent bug Priority: Major Reporter: Joe Harte I have configured a Jenkins pipeline to disabled concurrent builds: properties([ disableConcurrentBuilds() ]) However, I have noticed on some occasions multiple instances of the pipeline execute concurrently. This seems to happen when the pipeline is triggered by an upstream job at the same time that a user manually triggers it - possible race condition?
