[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Daniel Beck resolved JENKINS-24810 as Not A Defect The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop Change By: Daniel Beck (24/Sep/14 5:29 PM) Status: Open Resolved Assignee: huybrechts Resolution: Not A Defect This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Clinton Barr
updated JENKINS-24810
The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Change By:
Clinton Barr
(24/Sep/14 3:22 PM)
Description:
We are using the ParameterFactory, "for each property file, invoke one build," to create builds for a UI testing project. Lately (behavior started last week, no change to plugins or Jenkins version), the ParameterFactory does not stop generating sets of builds. After reaching the Parameterized Trigger build step, a correct number of builds are generated (about 72 builds of one project with different parameters), but after an undetermined amount of time between 1 and 10 minutes, another set of 72 builds are generated. This continues in a loop, without stopping, with an observed maximum of over 3000 builds of the same project.More use case information on the parameterized trigger build step:* Block until triggered projects finish=false* Predefined parameters used=true* Parameters from properties file used=true* For every property file, invoke one build=true** File pattern: "test_list/current_run/*.properties"** Action to perform when no files found: Fail the build stepLogger output for "hudson.plugins.parameterizedtrigger":{quote}Sep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParametersnullSep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParametersnullSep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParametersnullSep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParametersnullSep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParametersnullSep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParametersnull{quote}(This continues for another ten minutes with the same regularity)
More information:Rather than blocking the parent project until the test project finishes, we use a system Groovy script to monitor the progress of these tests and enforce a timeout on these projects, in an attempt to curb another problem we experience, project build hangs (the project never passes the "building on [node]" point). Here is the script used:{code:java|title=Groovy|borderStyle=solid}// Monitor Script Wait for tests to finish, only continue when all tests are longer running or timeout occurs.//import hudson.model.* try {def env = Thread.currentThread().executable.parent.builds[0].properties.get("envVars");def projectname = env['CHILD_PROJECT'];def suite = env['SUITE_NAME'];def timeout = env['TIMEOUT_IN_MINUTES'].toInteger();def defaults = [ 'Custom':60, 'Core':45, 'Active':90, 'ActiveFailing':60, 'Random':45, 'All':120, 'Missing':60 ];if (timeout == 0) { timeout = defaults[suite]; } def proj = hudson.model.Hudson.instance.getItem(projectname);println(projectname + " is in the queue: " + proj.isInQueue());println(projectname + " is building: " + isJobRunning(projectname));println("Waiting for " + projectname + " to finish with a timeout of " + timeout + " minutes."); // Check if project is building every minute, print waiting message every 10 minutes def count = 0; boolean timedOut = false; while (isJobRunning(projectname) || proj.isInQueue()) { count++; Thread.sleep(6); if (count % 10 == 0) { println(projectname + " is still building and has been building for " + count + " minutes."); printActiveJobs(runningJobs(projectname), projectname); } if (count == timeout) {println(projectname + " is no longer building. Continuing with job..."); timedOut = true; break; } } if (!timedOut) { println("All " + projectname + " jobs are done building."); } else { println("TIMEOUT PERIOD REACHED: " + timeout + " minutes have passed."); } }catch (err) { println("Exception: " + err);}def isJobRunning(jobname) {runningJobs(jobname).find {if (it.toString().contains(jobname)) { return true;} } return false;}def printActiveJobs(list, name) { def logentry = "The following " + name + " builds are still running:\r\n"; list.each {logentry
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Clinton Barr commented on JENKINS-24810 The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop That's strange. After unchecking "Run on all nodes...," my original projects behaved correctly and no duplicate builds were generated. I didn't find this behavior stated in the NodeLabel Parameter plugin documentation. When would duplicate builds ever be intended? This doesn't explain why when I had upwards of 70 property files to use with the ParameterFactory, the triggered builds cascaded beyond just one extra build (the original problem was discovered by 3000+ builds in queue). Also, I believe I had this setting checked from the beginning (I've got at least a 2-month, 40-build history through development), but only just last week observed this effect. Well, I can't explain it, but thank you all for the attention and help. TL;DR (For others not expecting the behavior in the description) - If you are using a Property file ParameterFactory to trigger a child project with a node label parameter, if you check "Run on all nodes matching the label," you can expect multiple builds of the child project. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
ikedam commented on JENKINS-24810 The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop It sounds the correct behavior of nodelabelparameter plugin. You should uncheck "Run on all nodes matching the label" of NODE_LABEL. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Clinton Barr commented on JENKINS-24810 The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop I just ran the setup again with only one properties file and it built two child project jobs. As for your second question, yes - I can't believe I didn't mention this before. The child project accepts a node label-type parameter. Just now, I found that without the node label parameter, this bug does not show up. If I remove the node label parameter from the child (but still pass it in the .properties file), no build duplication occurs. If I restore the node label parameter and manually build the child project, entering an existing node, no build duplication occurs. If the parent project triggers a child project build using a .properties file (that contains an existing node label), build duplication DOES occur. By the way, thank you both for paying attention to this issue. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Daniel Beck commented on JENKINS-24810 The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop I assume for the condition to occur you need more than one properties file, so there can actually be queued builds? Does child need the node parameter? This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Clinton Barr edited a comment on JENKINS-24810 The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop New information: Ignore everything before this. Environment: 1. Parent project runs on master, child project runs on slave (allocated by node label) 2. Windows 2012 Server master, Windows 2008 R2 slave 3. Jenkins v1.581, Parameterized Trigger Plugin v2.25 4. Parent project copies one .properties file into workspace, triggers child project from it using ParameterFactory, finishes 5. See attachments for configs and properties file used to trigger child project I can condense the behavior of this issue down to the following: Each child build triggered by the parent has a chance to be built twice. The duplicate chance occurs each time a child build finishes and the executor of the slave that just built it is freed (to consume a queued child build). The parent project does not need to be running for duplicate child builds to be created. Jenkins Master reboots have no effect (we reboot our master each day) EDIT: Couldn't remove obsolete attachments. #5 above is referring to ParentProject_config.xml, ChildProject_config.xml and 0.properties. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Clinton Barr updated JENKINS-24810 The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop New information: Ignore everything before this. Environment: 1. Parent project runs on master, child project runs on slave (allocated by node label) 2. Windows 2012 Server master, Windows 2008 R2 slave 3. Jenkins v1.581, Parameterized Trigger Plugin v2.25 4. Parent project copies one .properties file into workspace, triggers child project from it using ParameterFactory, finishes 5. See attachments for configs and properties file used to trigger child project I can condense the behavior of this issue down to the following: Each child build triggered by the parent has a chance to be built twice. The duplicate chance occurs each time a child build finishes and the executor of the slave that just built it is freed (to consume a queued child build). The parent project does not need to be running for duplicate child builds to be created. Jenkins Master reboots have no effect (we reboot our master each day) Change By: Clinton Barr (23/Sep/14 9:23 PM) Attachment: 0.properties Attachment: ChildProject_config.xml Attachment: ParentProject_config.xml This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Clinton Barr commented on JENKINS-24810 The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop After re-creating a parent-child project configuration with new projects, I've found that as soon as I add slave creation (and deletion at the end) and trigger builds to run on the label given to them, I can reproduce this issue. I will continue to hone in on my configuration of projects to see where the issue may reside. Intuitively, the creation of Jenkins slaves should have no correlation to the trigger build step that causes the runaway ChildProject builds, but the runaway happens explicitly when slave creation is added. I apologize for my shotgun approach earlier. Additionally, do you have any idea when the next release will happen? I know that FileBuildParameterFactory logging would be of paramount use here. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Clinton Barr commented on JENKINS-24810 The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop I will see if I can reproduce this behavior in a simpler project. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Daniel Beck commented on JENKINS-24810 The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop Could you please try to provide a minimal example that demonstrates the problem? Preferably working on a pristine Jenkins instance with as few non-default plugins as possible? A job configuration with 20+ build steps, countless plugin dependencies, and hundreds of lines of script code isn't exactly easy to analyze... See also http://www.catb.org/esr/faqs/smart-questions.html#volume and http://stackoverflow.com/help/mcve This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Clinton Barr updated JENKINS-24810 The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop I have attached the config.xml and log in a text file, complete except for exclusions of any sensitive product information. Change By: Clinton Barr (23/Sep/14 4:34 PM) Attachment: ParameterizedTriggerBugReport_ParentProject_config.xml Attachment: ParameterizedTriggerBugReport_ParentProject_Log.txt This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Clinton Barr commented on JENKINS-24810 The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop Does that problem always occur? (every time you launch a build of that parent project?) I have about 10 consecutive builds (last 10) of the parent project that displayed this issue and needed to be aborted, so yes. Do parent builds run on slaves? Parent builds run on the master, triggers another separate project to create slaves for the child project discussed here, and then builds the child project. Does that problem occur on any slaves? Or only on a specific slave? The build trigger duplication issue happens on the master, since it is the executor that triggers the child project. The slaves that are created earlier in the parent project are only connected to the child project through a node label (so that they will accept child project builds). Do builds stop to proceed after parameterized-trigger starts? (that is, parameterized-trigger doesn't finish?) It appears as if the build trigger step does finish (as you will see in the log), since the console continues to the next build step. I suspect this doesn't matter, necessarily, since the trigger for the Child project has the "Block until triggered projects finish" setting unchecked. Config.xml and Console output? Sure. I'll go get them. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
ikedam
commented on JENKINS-24810
The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Unfortunately, FileBuildParameterFactory never outputs valid logs (not only for you case). That will be fixed in a future release.
Let me know:
Does that problem always occur? (every time you launch a build of that parent project?)
Do parent builds run on slaves?
Does that problem occur on any slaves? Or only on a specific slave?
Do builds stop to proceed after parameterized-trigger starts? (that is, parameterized-trigger doesn't finish?)
Could you attach config.xml of that project? That is placed at ${JENKINS_HOME}/jobs/(project name)/config.xml.
Could you attach whole console output of that parent build?
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Clinton Barr commented on JENKINS-24810 The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop This issue persists on Jenkins v1.581. This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators. For more information on JIRA, see: http://www.atlassian.com/software/jira -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [parameterized-trigger] (JENKINS-24810) The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Clinton Barr
created JENKINS-24810
The "invoke build for each property file" ParameterFactory Generates More than One Set of Builds, Doesn't Stop
Issue Type:
Bug
Assignee:
huybrechts
Components:
parameterized-trigger
Created:
22/Sep/14 5:11 PM
Description:
We are using the ParameterFactory, "for each property file, invoke one build," to create builds for a UI testing project. Lately (behavior started last week, no change to plugins or Jenkins version), the ParameterFactory does not stop generating sets of builds. After reaching the Parameterized Trigger build step, a correct number of builds are generated (about 72 builds of one project with different parameters), but after an undetermined amount of time between 1 and 10 minutes, another set of 72 builds are generated. This continues in a loop, without stopping, with an observed maximum of over 3000 builds of the same project.
More use case information on the parameterized trigger build step:
Block until triggered projects finish=false
Predefined parameters used=true
Parameters from properties file used=true
For every property file, invoke one build=true
File pattern: "test_list/current_run/*.properties"
Action to perform when no files found: Fail the build step
Logger output for "hudson.plugins.parameterizedtrigger":
Sep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParameters
null
Sep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParameters
null
Sep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParameters
null
Sep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParameters
null
Sep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParameters
null
Sep 22, 2014 8:55:20 AM INFO hudson.plugins.parameterizedtrigger.FileBuildParameterFactory getParameters
null
(This continues for another ten minutes with the same regularity)
More information:
Rather than blocking the parent project until the test project finishes, we use a system Groovy script to monitor the progress of these tests and enforce a timeout on these projects, in an attempt to curb another problem we experience, project build hangs (the project never passes the "building on [node]" point). Here is the script used:
Groovy
// Monitor Script
//
// Wait for tests to finish, only continue when all tests are longer running or timeout occurs.
//
import hudson.model.*
try {
def env = Thread.currentThread().executable.parent.builds[0].properties.get("envVars");
def projectname = env['CHILD_PROJECT'];
def suite = env['SUITE_NAME'];
def timeout = env['TIMEOUT_IN_MINUTES'].toInteger();
def defaults = [ 'Custom':60, 'Core':45, 'Active':90, 'ActiveFailing':60, 'Random':45, 'All':120, 'Missing':60 ];
if (timeout == 0) { timeout = defaults[suite]; }
def proj = hudson.model.Hudson.instance.getItem(projectname);
println(projectname + " is in the queue: " + proj.isInQueue());
println(projectname + " is building: " + isJobRunning(projectname));
println("Waiting for " + projectname + " to finish with a timeout of " + timeout + " minutes.");
// Check if project is building every minute, print waiting message every 10 minutes
def count = 0;
boolean timedOut = false;
while (isJobRunning(projectname) || proj.isInQueue()) {
count++;
Thread.sleep(6);
if (count % 10 == 0) {
println(projectname + " is still building and has been building for " + count + " minutes.");
printActiveJobs(runningJobs(projectname), projectname);
}
if (count == timeout) {
println(projectname + " is no longer building. Continuing with job...");
timedOut = true;
break;
}
}
if (!timedOut) { println("All " + projectname + " jobs are done building."); }
else { println("TIMEOUT PERIOD REACHED: " + timeout + " minutes have passed."); }
}
catch (err) {
println("Exception: " + err);
}
def isJobRunning(jobname) {
runningJobs(jobna
