RE: How to get a Second Instance of Jenkins Running with the latest release and the Backup Strategy to Incorporate

[Palanilkunnathil Melemuriyil, Vinod P]

Thanks a lot Artur for your valuable inputs. 

Thank You

-Original Message-
From: jenkinsci-users@googlegroups.com 
[mailto:jenkinsci-users@googlegroups.com] On Behalf Of Artur Szostak
Sent: Thursday, July 20, 2017 6:07 PM
To: jenkinsci-users@googlegroups.com
Subject: Re: How to get a Second Instance of Jenkins Running with the latest 
release and the Backup Strategy to Incorporate

Jenkins has no built in enterprise grade backup or migration strategy. You have 
to deal with all of this yourself.

With regards to migration, for small Jenkins instances you should be OK with:
1) Stopping the new Jenkins instance.
2) Copying the XML configuration files and job directory from $JENKINS_HOME 
from the old instance to the new instance.
3) Start the new instance up.
For larger and more complex setups the probability of this working tends 
towards zero. In that case the better strategy is copying the configuration and 
jobs over one at a time while going through a stop, copy, start and verify 
cycle each time. This is extremely painful though. If you can it is better to 
script migration of the jobs through the REST APIs, while both instances are up 
and running. You will still need to consider how to verify your migration.
If you do have a large setup and have not done so already you might want to 
take this opportunity to put in the effort and build your new instance so that 
the whole configuration is created automatically (ideally in a virtualised 
server). You will again have to use the REST APIs and tools like 
jenkins-job-builder. For larger setups I think this becomes a must, since the 
default Jenkins GUI is not scalable for administrative tasks. Try maintain >60 
build nodes and >1000 jobs with the existing Jenkins GUI and you will know what 
I mean.

With regards to backup, if you want a true and full backup strategy then I am 
not aware of any built in facility or plugin that can help. There are a few 
that will skim the XML configs of the jobs and a few other bits and pieces, but 
nothing that will restore a server to its original state if the whole thing 
burns down.
When I first setup our Jenkins server a few years ago the best option I could 
come up with was using 
https://wiki.jenkins.io/display/JENKINS/Exclusive+Execution+Plugin and rsync 
within an exclusive job to snapshot $JENKINS_HOME (in our case excluding any 
job workspaces, just keeping their configs and build logs). The snapshot itself 
can then be backed up properly in the background while Jenkins returns to work.
This worked OK, but it is not very scalable. At some point the snapshot starts 
taking way too long. An improved strategy I plan to implement on our next 
upgrade is to use Btrfs (ZFS can work also) as the underlying data store for 
#JENKINS_HOME. This will allow to perform copy on write snapshots which are 
very quick.
I will also switch to using 
https://wiki.jenkins.io/display/JENKINS/Build+Blocker+Plugin rather than 
https://wiki.jenkins.io/display/JENKINS/Exclusive+Execution+Plugin.
Note that I am relying on the fact that Jenkins does not do any serious I/Os 
while the snapshotting is taking place. Otherwise the backup will never be 
consistent. This is a major pitfall and until Jenkins provides a real quiescing 
infrastructure ,100% safe online backups will never be possible. However, I 
believe that with Btrfs one can mitigate this by performing 2 COW snapshots 
with a small interval of time between them and see if there is any difference 
between those snapshots. If not then there is a high probability that you have 
a consistent backup.
If you do not care for online backups then you can of course avoid the 
consistency problem by shutting down Jenkins, doing the backup and then 
starting it up again. But our Jenkins instance takes > 40min to start, so I 
stick to online backups, since I want to do them regularly.

Hope you find some of this useful.

Cheers

Artur

From: jenkinsci-users@googlegroups.com  on 
behalf of Palanilkunnathil Melemuriyil, Vinod P 

Sent: 17 July 2017 17:40:20
To: jenkinsci-users@googlegroups.com
Subject: How to get a Second Instance of Jenkins Running with the latest 
release and the Backup Strategy to Incorporate

Hi All,

We have an old version (1.651.3) of Jenkins running on a server (RHEL) with 
many jobs configured to automate our day to day activities.
I would like to migrate to the latest stable release of Jenkins.

For that, I would want to have a second instance of Jenkins running (with the 
latest stable release of Jenkins) on the same server and slowly migrate all the 
jobs and users to the new jenkins instance after thorough testing.

Can anyone guide me on how all the jobs and all the configurations which I have 
already in place in the old instance can be migrated in one go (maybe by 
copying some directories or files from the older version) to the new instance 
and I start testing the new instance to see if all jobs ar

RE: How to get a Second Instance of Jenkins Running with the latest release and the Backup Strategy to Incorporate

[Palanilkunnathil Melemuriyil, Vinod P]

Could anyone guide me on the below query?

Thank You

From: jenkinsci-users@googlegroups.com 
[mailto:jenkinsci-users@googlegroups.com] On Behalf Of Palanilkunnathil 
Melemuriyil, Vinod P
Sent: Monday, July 17, 2017 9:10 PM
To: jenkinsci-users@googlegroups.com
Subject: How to get a Second Instance of Jenkins Running with the latest 
release and the Backup Strategy to Incorporate

Hi All,

We have an old version (1.651.3) of Jenkins running on a server (RHEL) with 
many jobs configured to automate our day to day activities.
I would like to migrate to the latest stable release of Jenkins.

For that, I would want to have a second instance of Jenkins running (with the 
latest stable release of Jenkins) on the same server and slowly migrate all the 
jobs and users to the new jenkins instance after thorough testing.

Can anyone guide me on how all the jobs and all the configurations which I have 
already in place in the old instance can be migrated in one go (maybe by 
copying some directories or files from the older version) to the new instance 
and I start testing the new instance to see if all jobs are running without any 
issues.

What would be the best approach to migrate to the latest version of Jenkins and 
at the same time have the old version running till we decommission it in due 
course.

Also right now, we don’t have a backup strategy in place for our Jenkins 
instance. I am wondering if by chance the Jenkins instance crashes how do I 
recover it?
What are the directories/configuration files I need to backup (on a daily 
basis) so that in case Jenkins crashes, I can bring it up by having the backup 
files restored.

Appreciate your help on this.


Thank You
Vinod

--
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
jenkinsci-users+unsubscr...@googlegroups.com<mailto:jenkinsci-users+unsubscr...@googlegroups.com>.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/0E91D17D9545B3D415CFA6415A730150BBD3E6%40CORPOWM-13<https://groups.google.com/d/msgid/jenkinsci-users/0E91D17D9545B3D415CFA6415A730150BBD3E6%40CORPOWM-13?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/0E91D17D9545B3D415CFA6415A730150BC0C25%40CORPOWM-13.
For more options, visit https://groups.google.com/d/optout.

How to get a Second Instance of Jenkins Running with the latest release and the Backup Strategy to Incorporate

[Palanilkunnathil Melemuriyil, Vinod P]

Hi All,

We have an old version (1.651.3) of Jenkins running on a server (RHEL) with 
many jobs configured to automate our day to day activities.
I would like to migrate to the latest stable release of Jenkins.

For that, I would want to have a second instance of Jenkins running (with the 
latest stable release of Jenkins) on the same server and slowly migrate all the 
jobs and users to the new jenkins instance after thorough testing.

Can anyone guide me on how all the jobs and all the configurations which I have 
already in place in the old instance can be migrated in one go (maybe by 
copying some directories or files from the older version) to the new instance 
and I start testing the new instance to see if all jobs are running without any 
issues.

What would be the best approach to migrate to the latest version of Jenkins and 
at the same time have the old version running till we decommission it in due 
course.

Also right now, we don't have a backup strategy in place for our Jenkins 
instance. I am wondering if by chance the Jenkins instance crashes how do I 
recover it?
What are the directories/configuration files I need to backup (on a daily 
basis) so that in case Jenkins crashes, I can bring it up by having the backup 
files restored.

Appreciate your help on this.

Thank You
Vinod

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/jenkinsci-users/0E91D17D9545B3D415CFA6415A730150BBD3E6%40CORPOWM-13.
For more options, visit https://groups.google.com/d/optout.

SimpleClearCase-Plugin with Jenkins

[Palanilkunnathil Melemuriyil, Vinod P]

Hello all,

After having installed this plug-in (SimpleClearCase-Plugin) to integrate 
Jenkins with UCM Clearcase and to poll the SCM for any changes, I am not sure 
what values are to be filled in for these options displayed in the job setup.

I have referred this URL
https://wiki.jenkins-ci.org/display/JENKINS/SimpleClearCase-Plugin

Load Rule:  Where are the load rules defined in Clearcase and where can we find 
the appropriate vales? What value should go in here.

View Name: I hope this is the View (Dynamic view name in my case) which needs 
to be polled associated with the Integration stream where final tested changes 
would be delivered.

Branch Name to watch: What value should go in here?

Appreciate your valuable inputs (maybe with some mock example as this pertains 
to the values for a setup which may differ in all cases).

Thank You
T.S


-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

RE: EXTERNAL: Clearcase integration with Jenkins to schedule jobs

[Palanilkunnathil Melemuriyil, Vinod P]

Thanks a lot Jeff for your valuable guidance.

Thank You

From: jenkinsci-users@googlegroups.com 
[mailto:jenkinsci-users@googlegroups.com] On Behalf Of Ng, Jeff
Sent: Wednesday, January 15, 2014 2:53 AM
To: jenkinsci-users@googlegroups.com
Subject: RE: EXTERNAL: Clearcase integration with Jenkins to schedule jobs

I have one of my UCM projects setup to do exactly what you are requesting.  
Whenever a dev completes a delivery to the project integration stream, a 
"postop deliver_complete" trigger kicks off a build job using a URL (trigger 
builds remotely).  Here's what you'll need:

1.  Configure your Jenkins job to "Trigger builds remotely" (under Build 
Triggers).
   - I named my token "BUILD_AFTER_DELIVER_COMPLETE".

2.  Make a ClearCase attribute used to enable or disable the automatic build 
triggering on a build stream.
   - Create the attribute type in the PVOB (I named mine 
"TriggerBuildAfterDelivery).
   - Scope:  ordinary
   - Enumeration values:  "ON", "OFF"
   - Default value:  "ON"
   - Apply the attribute to any build stream that you wish to trigger a build.

3.  Make a UCM trigger type that fires after a delivery has been completed.
   - Create the trigger type in the PVOB (I named mine 
"JENKINS_BUILD_AFTER_DELIVERY").
   - all UCM object trigger
   - post-op deliver_complete
   - Executes corresponding trigger script

4.  Write a trigger script, executed by the UCM trigger type, which constructs 
and executes the Jenkins URL.
   - I wrote mine in Perl and named it "jenkinsBuildAfterDelivery.tgr".
   - My Jenkins build job name matches the name of the UCM stream being built 
(you'll see why when you construct the URL).
   - If you use Perl, use the LWP::Simple module for the get() subroutine.
   - First get the value of the TriggerBuildAfterDelivery attribute (ct desc -s 
-attr $attr stream:$stream).
   - If the value == "ON", construct your URL and `get` it.
   - Hint:  $jobName = $buildStream = $ENV{CLEARCASE_STREAM};

I also serialized UCM deliveries on these specific build streams to prevent 
multiple devs from delivering concurrently, thus interfering with the build 
triggering.  I'll let you google for how to serialize UCM deliveries.  It's 
easy and essentially requires the same things:  An attribute, a trigger type 
and a trigger script.

I've had this setup on my project for over a year now and it works great.

-Jeff Ng

From: jenkinsci-users@googlegroups.com 
[mailto:jenkinsci-users@googlegroups.com] On Behalf Of Vinod P
Sent: Monday, January 13, 2014 8:50 PM
To: jenkinsci-users@googlegroups.com
Subject: EXTERNAL: Clearcase integration with Jenkins to schedule jobs

Hello all,

I am a newbie to Jenkins and Clearcase and have been going through the various 
options of Jenkins to facilitate CI.

However, I am not sure on how to configure Jenkins with UCM Clearcase in such a 
way that when a developer delivers his code to the Integration Stream in 
Clearcase (after having tested his code in the Development Stream where he 
makes changes to the code), Jenkins should start the build of the application.

In short, the requirement is that whenever any changes are delivered to the 
Integration stream in clearcase (UCM) , a build job should be automatically 
triggered in Jenkins.

Appreciate any help/guidance on this.

Regards
T.S
--
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

RE: Clearcase integration with Jenkins to schedule jobs

[Palanilkunnathil Melemuriyil, Vinod P]

Thanks Slide.

With the Clearcase plug-in installed in Jenkins, wouldn't it be possible to 
poll SCM (i.e Clearcase) for any changes that are delivered to Integration 
Stream by the developers.

If any changes are found in Integration Stream then Jenkins runs the particular 
build job.

Regards
T.S

From: jenkinsci-users@googlegroups.com 
[mailto:jenkinsci-users@googlegroups.com] On Behalf Of Slide
Sent: Tuesday, January 14, 2014 9:39 AM
To: jenkinsci-users@googlegroups.com
Subject: Re: Clearcase integration with Jenkins to schedule jobs

I think for the automatic triggering, you would need to implement a trigger in 
Clearcase that would initiate a build via URL. There really isn't a native way 
for Clearcase to notify Jenkins that someone delivered something, you would 
need to roll your own.

slide

On Mon, Jan 13, 2014 at 8:49 PM, Vinod P 
mailto:vinodts2...@gmail.com>> wrote:
Hello all,

I am a newbie to Jenkins and Clearcase and have been going through the various 
options of Jenkins to facilitate CI.

However, I am not sure on how to configure Jenkins with UCM Clearcase in such a 
way that when a developer delivers his code to the Integration Stream in 
Clearcase (after having tested his code in the Development Stream where he 
makes changes to the code), Jenkins should start the build of the application.

In short, the requirement is that whenever any changes are delivered to the 
Integration stream in clearcase (UCM) , a build job should be automatically 
triggered in Jenkins.

Appreciate any help/guidance on this.

Regards
T.S
--
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to 
jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.



--
Website: http://earl-of-code.com
--
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to jenkinsci-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.