Re: Which agents are used when a pipeline calls multiple downstream freestyle projects
Found the solution wrap the whole thing in a node() block and everything within the node block will run on the same node Cheers On Sunday, March 19, 2017 at 10:47:45 AM UTC+13, Stephen Connolly wrote: > > Downstream jobs will run on any agent matching the label they are > configured for, with preference given to running on the agent that they > most recently ran on *if idle*. > > Upstream jobs have no affect - unless they are still running at the time - > in which case they may force a *different* agent if they are occupying all > available slots. > On Sat 18 Mar 2017 at 19:22, Tony P > > wrote: > >> Does this mean I am correct that the default behavior is for downstream >> projects to run on any agents that have the same label ? This was perhaps >> the big thing I was trying to find out. >> >> Also our builds are done via "build job: " but I am struggling to find >> out how to force it to use a particular agent. >> >> My plan has been kick off the first downstream project, find out which >> agent it used and then force the remaining downstream projects to use that >> agent. Try as I may and I tried so many ideas, I can't find out how to >> determine what agent that downstream agent used. It seems next to >> impossible ? >> >> Thanks again >> >> >> On Sunday, March 19, 2017 at 12:27:03 AM UTC+13, Stephen Connolly wrote: >> >>> I think you would need to write a plugin that enforces your "same node" >>> constraint. >>> >>> It would not be general utility as eg enough people are doing builds on >>> ephemeral agents and you'd end up with stuck jobs waiting for a free >>> executor on the same "ephemeral" node that will never return. >>> >>> But I can see it being of utility for some people. Probably a >>> JobProperty that does the enforcing and some other coordination token to >>> identify which jobs you need to be on the same node as (plus a timeout if >>> you are stuck for too long waiting on that node) >>> >> On Sat 18 Mar 2017 at 10:28, Tony P wrote: >>> >> Hi, >>>> >>>> We have multiple pipelines for which each will run 3+ downstream >>>> freestyle projects. >>>> >>>> This process is defined by an automation tool that creates and >>>> initiates Jenkins builds, so using other methodologies is out :-). The >>>> problem I have is that I need all the freestyle projects to run on the >>>> same >>>> agent, because the first couple of projects clone repos and other projects >>>> then do the build or deploy - so obviously it all needs to happen in >>>> exactly the same place. >>>> >>>> I also want to initiate this process using agent/slave labels rather >>>> than agent/slave names as we will have a group of agents to handle the >>>> multiple sometimes duplicate pipeline builds. >>>> >>>> I am beginning to think that even though I am using labels Jenkins will >>>> always run all the downstream projects on the same agent. Trouble is I am >>>> having trouble finding any conclusive documentation that says I am right >>>> or >>>> I am wrong - I have spent an awful lot of time in the company of Mr Google >>>> trying to figure this out. If each of the downstream projects could >>>> potentially run on any available agent using the same label then I have a >>>> big problem and my next task is to figure out how to get them to all use >>>> the same agent. >>>> >>>> Can someone tell me whether I am correct that Jenkins will run all >>>> downstream projects on the same agent and/or where do I look for this >>>> level >>>> of documentation ? >>>> >>>> Thanks in advance >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Jenkins Users" group. >>>> >>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to jenkinsci-use...@googlegroups.com. >>> >>> >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/jenkinsci-users/e806f3a6-02c8-4fbd-b574-d29fb8dee547%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/jenkinsci-users/e806f3a6-02c8-4fbd-b574-d29fb8dee547%40googlegroups.com?utm_medium=email&utm_source=footer
Re: Which agents are used when a pipeline calls multiple downstream freestyle projects
Does this mean I am correct that the default behavior is for downstream projects to run on any agents that have the same label ? This was perhaps the big thing I was trying to find out. Also our builds are done via "build job: " but I am struggling to find out how to force it to use a particular agent. My plan has been kick off the first downstream project, find out which agent it used and then force the remaining downstream projects to use that agent. Try as I may and I tried so many ideas, I can't find out how to determine what agent that downstream agent used. It seems next to impossible ? Thanks again On Sunday, March 19, 2017 at 12:27:03 AM UTC+13, Stephen Connolly wrote: > > I think you would need to write a plugin that enforces your "same node" > constraint. > > It would not be general utility as eg enough people are doing builds on > ephemeral agents and you'd end up with stuck jobs waiting for a free > executor on the same "ephemeral" node that will never return. > > But I can see it being of utility for some people. Probably a JobProperty > that does the enforcing and some other coordination token to identify which > jobs you need to be on the same node as (plus a timeout if you are stuck > for too long waiting on that node) > On Sat 18 Mar 2017 at 10:28, Tony P > > wrote: > >> Hi, >> >> We have multiple pipelines for which each will run 3+ downstream >> freestyle projects. >> >> This process is defined by an automation tool that creates and initiates >> Jenkins builds, so using other methodologies is out :-). The problem I have >> is that I need all the freestyle projects to run on the same agent, because >> the first couple of projects clone repos and other projects then do the >> build or deploy - so obviously it all needs to happen in exactly the same >> place. >> >> I also want to initiate this process using agent/slave labels rather than >> agent/slave names as we will have a group of agents to handle the multiple >> sometimes duplicate pipeline builds. >> >> I am beginning to think that even though I am using labels Jenkins will >> always run all the downstream projects on the same agent. Trouble is I am >> having trouble finding any conclusive documentation that says I am right or >> I am wrong - I have spent an awful lot of time in the company of Mr Google >> trying to figure this out. If each of the downstream projects could >> potentially run on any available agent using the same label then I have a >> big problem and my next task is to figure out how to get them to all use >> the same agent. >> >> Can someone tell me whether I am correct that Jenkins will run all >> downstream projects on the same agent and/or where do I look for this level >> of documentation ? >> >> Thanks in advance >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Jenkins Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to jenkinsci-use...@googlegroups.com . >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/jenkinsci-users/e806f3a6-02c8-4fbd-b574-d29fb8dee547%40googlegroups.com >> >> <https://groups.google.com/d/msgid/jenkinsci-users/e806f3a6-02c8-4fbd-b574-d29fb8dee547%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > Sent from my phone > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/52b89bf8-4615-40cb-affd-51ab29792f00%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Which agents are used when a pipeline calls multiple downstream freestyle projects
Hi, We have multiple pipelines for which each will run 3+ downstream freestyle projects. This process is defined by an automation tool that creates and initiates Jenkins builds, so using other methodologies is out :-). The problem I have is that I need all the freestyle projects to run on the same agent, because the first couple of projects clone repos and other projects then do the build or deploy - so obviously it all needs to happen in exactly the same place. I also want to initiate this process using agent/slave labels rather than agent/slave names as we will have a group of agents to handle the multiple sometimes duplicate pipeline builds. I am beginning to think that even though I am using labels Jenkins will always run all the downstream projects on the same agent. Trouble is I am having trouble finding any conclusive documentation that says I am right or I am wrong - I have spent an awful lot of time in the company of Mr Google trying to figure this out. If each of the downstream projects could potentially run on any available agent using the same label then I have a big problem and my next task is to figure out how to get them to all use the same agent. Can someone tell me whether I am correct that Jenkins will run all downstream projects on the same agent and/or where do I look for this level of documentation ? Thanks in advance -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/e806f3a6-02c8-4fbd-b574-d29fb8dee547%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Problems when updating plugins manually and via Plugin Manager
Hi, I have been having quite a bit of trouble manually installing plugin updates and would like to pose a theory. I have a system behind firewalls without internet access, it was originally created in a different environment WITH internet access and copied here. When I download the latest plugins and copy them to Jenkins and *restart* a number fail to start with errors like: May 07, 2013 3:42:56 PM hudson.PluginManager$1$3$1 isDuplicate INFO: Ignoring D:\Jenkins\plugins\cobertura.hpi because D:\Jenkins\plugins\cobertura.jpi is already loaded I have read that when you download via the Internet the files are "my-plugin *.jpi*" but if I download them then it's "my-plugin*.hpi*" - I am downloading from http://updates.jenkins-ci.org/download/plugins/ For one failing plugin I deleted the existing active-directory.jpi copied over the new active-directory.hpi and voila it worked. I believe the problem is the inconsistency in the file extension is the problem. My sense of it is that people either update one way or the other but not both. I am writing up some instructions and would like to get it right. I have Googled for Africa but can't find an answer. Cheers -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Problems updating plugins manually and via "Plugin Manager"
Apologies if this is double posted. I thought I posted it but it disappeared and I still couldn't see it after half an hour so am posting again :-( I have a Jenkins system behind firewalls without internet access, it is a copy of a system that had internet access. When I try to update plugins manually by copying over the *.hpi files I get a few plugins failing with the following errors in Tomcat: May 07, 2013 3:42:56 PM hudson.PluginManager$1$3$1 isDuplicate INFO: Ignoring D:\Jenkins\plugins\cobertura.hpi because D:\Jenkins\plugins\cobertura.jpi is already loaded I am writing instructions on how to update plugins and so would like to get it right. Despite Googling for Africa I can't find a definitive answer. For example it was failing on the active directory plugin so I deleted "active-directory.jpi", copied over active-directory.hpi and voila it worked. I saw somewhere that if you update via Plugin Manager the files are *.jpi but if you download and install manually they are *.hpi My working theory is that I am correct to delete the *.jpi files and the problem is the inconsistency between *.jpi and *.hpi. I am downloading from https://updates.jenkins-ci.org/download/plugins/ Cheers -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Translation plugin won't update
Hi, The "Translation Assistance" plugin stubbornly refuses to update. I have my Jenkins master behind a firewall unable to get to the internet yet it continues to report the current version of the plugin is 1.10 and that I only have version 1.8 I have tried all manner and means including downloading the latest version but it still stays at 1.8. As I understand it this plugin is in the Jenkins war and should update automatically. I may be wrong but I think I cracked it. Far as I can tell the latest Jenkins war includes not version 1.10 of the plugin but version 1.8 - which of course explains my problem. Does anyone know if I am right or gone off the track :-) Cheers
Re: svn confusion
I never trust this type of thing, I guess I’m just a pessimist at heart J I take a different approach, I checkout my code to a structure similar to “/myjob/clean/myproject/src/” and then I mirror this to “/myjob/ build/myproject/src/” using Windows Robocopy. The mirroring means it removes files which should not exist and put in any that are missing. I elected to use Windows boxes for various reasons and on Windows I have found Robocopy is good because it doesn’t copy things that are unchanged. I am sure there are Linux equivalents. This gives me a crystal clean build structure that doesn’t take too much longer than normal and doesn’t put unnecessary strain on my SVN server and so on. I have been following this philosophy since using CruiseControl.
Re: jelly:util tag is missing
I have had the exact same problem and frustration. It is because Jenkins doesn't come with all the Jelly tag libraries including Util. I run Jenkins from Tomcat and the only slightly satisfactory way I have found it to work is to put the Util jar file in exploded Jenkins war directory /wwwroot/jenkins/WEB-INF/lib directory BEFORE starting Tomcat. The above does seem to work but I don't like it because each time you upgrade Jenkins you need to remember to do this again.
Jelly scripts and commons-jelly-tags-util-1.1.1.jar
Hi, I currently use the following in my email jelly script: Unfortunately I can't guarantee this file will exist and when it doesn't Jelly completely bombs out and virtually nothing shows up. So smart thing to do is test if the file exists and include only if it exists. Far as I can tell this can't be done using Jelly in Jenkins. When Googling for Jelly solutions to this they all come back to using the Jelly "Util" tag library which is not included in Jenkins. Reluctantly I tried putting the util library in but the only place I can get it to work is if I put it in the expanded Jenkins folder in Tomcat - sorry didn't mention I am using Tomcat 5.5. I tried putting it in the shared and common libs in Tomcat and Tomcat picks it up alright but it doesn't work for me with the emails. I tried in the email plugin directory under WEB-INF and under classes and by creating a lib directory. I even tried expanding the jar file and putting it under classes - nothing. When Tomcat is running the jar is locked but from a functional point of view it is not available when doing the emails. I don't want to put it in the expanded Jenkins directory because that means each time there is a new Jenkins release I have to remember to do this again. So not exactly a perfect solution. Whilst a solution to the included file issue is appreciated I think the bigger problem is how to use other tag libraries. I could try moving to Groovy but when I tried that it didn't work and anyways that would mean redoing my script and I would rather not have to do that. Thanks
How to use "HUDSON_BUILDS" or "JENKINS_BUILDS"
Being able to separate the build/archive directory structure from the configuration structure was discussed sometime back. A Jira was created and apparently in Jenkins 1.421. This sounded very useful. The current changelog says for release 1.421: Allow build directories and workspace directories in $JENKINS_HOME to be placed elsewhere. (issue 8446) And Issues 8446 calls this "'builds' directory in a separate location?" and is marked as resolved. I have spent sometime trying to find any docs on this and how to use it but I just can't seem to get it to do anything. I have tried setting the environment variables "HUDSON_BUILDS" and "JENKINS_BUILDS" but whatever I do nothing different happens. Does anyone know how to use these or can point me to docs on the subject.
Is there such as thing template builds
Hi, I have to say "Template" is not quite the right word but I can't currently think of a better one.. We have quite a few builds that are based on the same "template", they are all clones of a particular build I have called "Generic_Ivy_Module_Template". All these jobs are identical to the generic one aside from a handful of changes, such as the location of the source code and project name. All good so far. If I decide to make a change to the master "Template" and make this change effective for all its clones then I need to go into each and every clone and update them. Yes I know I can change the config.xml using a global search and replace - actually that is often what I do. However it would be really cool to have a better mechanism, ideally where there was some sort of master job and any changes to that we automatically used by the clones. Something like this may exist and I have missed it. Incidentally I have multiple "templates". Thanks
