RE: How to get a Second Instance of Jenkins Running with the latest release and the Backup Strategy to Incorporate
Thanks a lot Artur for your valuable inputs. Thank You -Original Message- From: jenkinsci-users@googlegroups.com [mailto:jenkinsci-users@googlegroups.com] On Behalf Of Artur Szostak Sent: Thursday, July 20, 2017 6:07 PM To: jenkinsci-users@googlegroups.com Subject: Re: How to get a Second Instance of Jenkins Running with the latest release and the Backup Strategy to Incorporate Jenkins has no built in enterprise grade backup or migration strategy. You have to deal with all of this yourself. With regards to migration, for small Jenkins instances you should be OK with: 1) Stopping the new Jenkins instance. 2) Copying the XML configuration files and job directory from $JENKINS_HOME from the old instance to the new instance. 3) Start the new instance up. For larger and more complex setups the probability of this working tends towards zero. In that case the better strategy is copying the configuration and jobs over one at a time while going through a stop, copy, start and verify cycle each time. This is extremely painful though. If you can it is better to script migration of the jobs through the REST APIs, while both instances are up and running. You will still need to consider how to verify your migration. If you do have a large setup and have not done so already you might want to take this opportunity to put in the effort and build your new instance so that the whole configuration is created automatically (ideally in a virtualised server). You will again have to use the REST APIs and tools like jenkins-job-builder. For larger setups I think this becomes a must, since the default Jenkins GUI is not scalable for administrative tasks. Try maintain >60 build nodes and >1000 jobs with the existing Jenkins GUI and you will know what I mean. With regards to backup, if you want a true and full backup strategy then I am not aware of any built in facility or plugin that can help. There are a few that will skim the XML configs of the jobs and a few other bits and pieces, but nothing that will restore a server to its original state if the whole thing burns down. When I first setup our Jenkins server a few years ago the best option I could come up with was using https://wiki.jenkins.io/display/JENKINS/Exclusive+Execution+Plugin and rsync within an exclusive job to snapshot $JENKINS_HOME (in our case excluding any job workspaces, just keeping their configs and build logs). The snapshot itself can then be backed up properly in the background while Jenkins returns to work. This worked OK, but it is not very scalable. At some point the snapshot starts taking way too long. An improved strategy I plan to implement on our next upgrade is to use Btrfs (ZFS can work also) as the underlying data store for #JENKINS_HOME. This will allow to perform copy on write snapshots which are very quick. I will also switch to using https://wiki.jenkins.io/display/JENKINS/Build+Blocker+Plugin rather than https://wiki.jenkins.io/display/JENKINS/Exclusive+Execution+Plugin. Note that I am relying on the fact that Jenkins does not do any serious I/Os while the snapshotting is taking place. Otherwise the backup will never be consistent. This is a major pitfall and until Jenkins provides a real quiescing infrastructure ,100% safe online backups will never be possible. However, I believe that with Btrfs one can mitigate this by performing 2 COW snapshots with a small interval of time between them and see if there is any difference between those snapshots. If not then there is a high probability that you have a consistent backup. If you do not care for online backups then you can of course avoid the consistency problem by shutting down Jenkins, doing the backup and then starting it up again. But our Jenkins instance takes > 40min to start, so I stick to online backups, since I want to do them regularly. Hope you find some of this useful. Cheers Artur From: jenkinsci-users@googlegroups.com on behalf of Palanilkunnathil Melemuriyil, Vinod P Sent: 17 July 2017 17:40:20 To: jenkinsci-users@googlegroups.com Subject: How to get a Second Instance of Jenkins Running with the latest release and the Backup Strategy to Incorporate Hi All, We have an old version (1.651.3) of Jenkins running on a server (RHEL) with many jobs configured to automate our day to day activities. I would like to migrate to the latest stable release of Jenkins. For that, I would want to have a second instance of Jenkins running (with the latest stable release of Jenkins) on the same server and slowly migrate all the jobs and users to the new jenkins instance after thorough testing. Can anyone guide me on how all the jobs and all the configurations which I have already in place in the old instance can be migrated in one go (maybe by copying some directories or files from the older version) to the new instance and I start testing the new instance to see if all jobs ar
RE: How to get a Second Instance of Jenkins Running with the latest release and the Backup Strategy to Incorporate
Could anyone guide me on the below query? Thank You From: jenkinsci-users@googlegroups.com [mailto:jenkinsci-users@googlegroups.com] On Behalf Of Palanilkunnathil Melemuriyil, Vinod P Sent: Monday, July 17, 2017 9:10 PM To: jenkinsci-users@googlegroups.com Subject: How to get a Second Instance of Jenkins Running with the latest release and the Backup Strategy to Incorporate Hi All, We have an old version (1.651.3) of Jenkins running on a server (RHEL) with many jobs configured to automate our day to day activities. I would like to migrate to the latest stable release of Jenkins. For that, I would want to have a second instance of Jenkins running (with the latest stable release of Jenkins) on the same server and slowly migrate all the jobs and users to the new jenkins instance after thorough testing. Can anyone guide me on how all the jobs and all the configurations which I have already in place in the old instance can be migrated in one go (maybe by copying some directories or files from the older version) to the new instance and I start testing the new instance to see if all jobs are running without any issues. What would be the best approach to migrate to the latest version of Jenkins and at the same time have the old version running till we decommission it in due course. Also right now, we don’t have a backup strategy in place for our Jenkins instance. I am wondering if by chance the Jenkins instance crashes how do I recover it? What are the directories/configuration files I need to backup (on a daily basis) so that in case Jenkins crashes, I can bring it up by having the backup files restored. Appreciate your help on this. Thank You Vinod -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com<mailto:jenkinsci-users+unsubscr...@googlegroups.com>. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0E91D17D9545B3D415CFA6415A730150BBD3E6%40CORPOWM-13<https://groups.google.com/d/msgid/jenkinsci-users/0E91D17D9545B3D415CFA6415A730150BBD3E6%40CORPOWM-13?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0E91D17D9545B3D415CFA6415A730150BC0C25%40CORPOWM-13. For more options, visit https://groups.google.com/d/optout.
How to get a Second Instance of Jenkins Running with the latest release and the Backup Strategy to Incorporate
Hi All, We have an old version (1.651.3) of Jenkins running on a server (RHEL) with many jobs configured to automate our day to day activities. I would like to migrate to the latest stable release of Jenkins. For that, I would want to have a second instance of Jenkins running (with the latest stable release of Jenkins) on the same server and slowly migrate all the jobs and users to the new jenkins instance after thorough testing. Can anyone guide me on how all the jobs and all the configurations which I have already in place in the old instance can be migrated in one go (maybe by copying some directories or files from the older version) to the new instance and I start testing the new instance to see if all jobs are running without any issues. What would be the best approach to migrate to the latest version of Jenkins and at the same time have the old version running till we decommission it in due course. Also right now, we don't have a backup strategy in place for our Jenkins instance. I am wondering if by chance the Jenkins instance crashes how do I recover it? What are the directories/configuration files I need to backup (on a daily basis) so that in case Jenkins crashes, I can bring it up by having the backup files restored. Appreciate your help on this. Thank You Vinod -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0E91D17D9545B3D415CFA6415A730150BBD3E6%40CORPOWM-13. For more options, visit https://groups.google.com/d/optout.
SimpleClearCase-Plugin with Jenkins
Hello all, After having installed this plug-in (SimpleClearCase-Plugin) to integrate Jenkins with UCM Clearcase and to poll the SCM for any changes, I am not sure what values are to be filled in for these options displayed in the job setup. I have referred this URL https://wiki.jenkins-ci.org/display/JENKINS/SimpleClearCase-Plugin Load Rule: Where are the load rules defined in Clearcase and where can we find the appropriate vales? What value should go in here. View Name: I hope this is the View (Dynamic view name in my case) which needs to be polled associated with the Integration stream where final tested changes would be delivered. Branch Name to watch: What value should go in here? Appreciate your valuable inputs (maybe with some mock example as this pertains to the values for a setup which may differ in all cases). Thank You T.S -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
RE: EXTERNAL: Clearcase integration with Jenkins to schedule jobs
Thanks a lot Jeff for your valuable guidance. Thank You From: jenkinsci-users@googlegroups.com [mailto:jenkinsci-users@googlegroups.com] On Behalf Of Ng, Jeff Sent: Wednesday, January 15, 2014 2:53 AM To: jenkinsci-users@googlegroups.com Subject: RE: EXTERNAL: Clearcase integration with Jenkins to schedule jobs I have one of my UCM projects setup to do exactly what you are requesting. Whenever a dev completes a delivery to the project integration stream, a "postop deliver_complete" trigger kicks off a build job using a URL (trigger builds remotely). Here's what you'll need: 1. Configure your Jenkins job to "Trigger builds remotely" (under Build Triggers). - I named my token "BUILD_AFTER_DELIVER_COMPLETE". 2. Make a ClearCase attribute used to enable or disable the automatic build triggering on a build stream. - Create the attribute type in the PVOB (I named mine "TriggerBuildAfterDelivery). - Scope: ordinary - Enumeration values: "ON", "OFF" - Default value: "ON" - Apply the attribute to any build stream that you wish to trigger a build. 3. Make a UCM trigger type that fires after a delivery has been completed. - Create the trigger type in the PVOB (I named mine "JENKINS_BUILD_AFTER_DELIVERY"). - all UCM object trigger - post-op deliver_complete - Executes corresponding trigger script 4. Write a trigger script, executed by the UCM trigger type, which constructs and executes the Jenkins URL. - I wrote mine in Perl and named it "jenkinsBuildAfterDelivery.tgr". - My Jenkins build job name matches the name of the UCM stream being built (you'll see why when you construct the URL). - If you use Perl, use the LWP::Simple module for the get() subroutine. - First get the value of the TriggerBuildAfterDelivery attribute (ct desc -s -attr $attr stream:$stream). - If the value == "ON", construct your URL and `get` it. - Hint: $jobName = $buildStream = $ENV{CLEARCASE_STREAM}; I also serialized UCM deliveries on these specific build streams to prevent multiple devs from delivering concurrently, thus interfering with the build triggering. I'll let you google for how to serialize UCM deliveries. It's easy and essentially requires the same things: An attribute, a trigger type and a trigger script. I've had this setup on my project for over a year now and it works great. -Jeff Ng From: jenkinsci-users@googlegroups.com [mailto:jenkinsci-users@googlegroups.com] On Behalf Of Vinod P Sent: Monday, January 13, 2014 8:50 PM To: jenkinsci-users@googlegroups.com Subject: EXTERNAL: Clearcase integration with Jenkins to schedule jobs Hello all, I am a newbie to Jenkins and Clearcase and have been going through the various options of Jenkins to facilitate CI. However, I am not sure on how to configure Jenkins with UCM Clearcase in such a way that when a developer delivers his code to the Integration Stream in Clearcase (after having tested his code in the Development Stream where he makes changes to the code), Jenkins should start the build of the application. In short, the requirement is that whenever any changes are delivered to the Integration stream in clearcase (UCM) , a build job should be automatically triggered in Jenkins. Appreciate any help/guidance on this. Regards T.S -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com<mailto:jenkinsci-users+unsubscr...@googlegroups.com>. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
RE: Clearcase integration with Jenkins to schedule jobs
Thanks Slide. With the Clearcase plug-in installed in Jenkins, wouldn't it be possible to poll SCM (i.e Clearcase) for any changes that are delivered to Integration Stream by the developers. If any changes are found in Integration Stream then Jenkins runs the particular build job. Regards T.S From: jenkinsci-users@googlegroups.com [mailto:jenkinsci-users@googlegroups.com] On Behalf Of Slide Sent: Tuesday, January 14, 2014 9:39 AM To: jenkinsci-users@googlegroups.com Subject: Re: Clearcase integration with Jenkins to schedule jobs I think for the automatic triggering, you would need to implement a trigger in Clearcase that would initiate a build via URL. There really isn't a native way for Clearcase to notify Jenkins that someone delivered something, you would need to roll your own. slide On Mon, Jan 13, 2014 at 8:49 PM, Vinod P mailto:vinodts2...@gmail.com>> wrote: Hello all, I am a newbie to Jenkins and Clearcase and have been going through the various options of Jenkins to facilitate CI. However, I am not sure on how to configure Jenkins with UCM Clearcase in such a way that when a developer delivers his code to the Integration Stream in Clearcase (after having tested his code in the Development Stream where he makes changes to the code), Jenkins should start the build of the application. In short, the requirement is that whenever any changes are delivered to the Integration stream in clearcase (UCM) , a build job should be automatically triggered in Jenkins. Appreciate any help/guidance on this. Regards T.S -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com<mailto:jenkinsci-users%2bunsubscr...@googlegroups.com>. For more options, visit https://groups.google.com/groups/opt_out. -- Website: http://earl-of-code.com -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
Clearcase integration with Jenkins to schedule jobs
Hello all, I am a newbie to Jenkins and Clearcase and have been going through the various options of Jenkins to facilitate CI. However, I am not sure on how to configure Jenkins with UCM Clearcase in such a way that when a developer delivers his code to the Integration Stream in Clearcase (after having tested his code in the Development Stream where he makes changes to the code), Jenkins should start the build of the application. In short, the requirement is that whenever any changes are delivered to the Integration stream in clearcase (UCM) , a build job should be automatically triggered in Jenkins. Appreciate any help/guidance on this. Regards T.S -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.