Re: [gentoo-user] Clone live system as a simple backup?
On Wed, Mar 07, 2012 at 02:47:08PM -0500, Joshua Murphy wrote:
On Wed, Mar 7, 2012 at 8:55 AM, gand...@d-danks.co.uk wrote:
Hi,
I'm interested in the idea of cloning a live, complicated hardware
system onto a single external hard drive as a simple backup. I would
like this external drive to be completely bootable. What's the best
way to approach doing this? I was considering just doing a Gentoo
install from scratch but figured maybe there's a way to clone enough
of the live system to get me there less painfully?
The system I'm playing with has five 500MB hard drives with most
partitions in linked together in various forms of RAID. (1, 5 6)
That said, the total storage that this system presents KDE and the
users is about 600GB.
I have an external 1TB eSATA drive which is therefore large enough
to hold everything on this system, albeit without the reliability of
RAID which is fine for this purpose.
The system looks more or less like:
/dev/sda1 - /boot (50MB)
/dev/sdb1 - /boot copy
/dev/sdc1 - /boot copy
c2stable ~ # df
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 51612920 31862844 17128276 66% /
/dev/root 51612920 31862844 17128276 66% /
rc-svcdir 1024 92 932 9% /lib64/rc/init.d
udev 10240 476 9764 5% /dev
shm 6151284 0 6151284 0% /dev/shm
/dev/md7 389183252 350247628 19166232 95% /VirtualMachines
tmpfs 8388608 0 8388608 0% /var/tmp/portage
/dev/sda1 54416 29516 22091 58% /boot
c2stable ~ # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
[raid4]
md6 : active raid5 sdb6[1] sdc6[2] sda6[0]
494833664 blocks super 1.1 level 5, 64k chunk, algorithm 2 [3/3]
[UUU]
md7 : active raid6 sdb7[1] sdc7[2] sda7[0] sdd2[3] sde2[4]
395387904 blocks super 1.2 level 6, 16k chunk, algorithm 2 [5/5]
[U]
md3 : active raid6 sdb3[1] sdc3[2] sda3[0] sdd3[3] sde3[4]
157305168 blocks super 1.2 level 6, 16k chunk, algorithm 2 [5/5]
[U]
md126 : active raid1 sdc5[2] sda5[0] sdb5[1]
52436032 blocks [3/3] [UUU]
unused devices: none
c2stable ~ #
/dev/md3 is a second Gentoo installation that doesn't need to be
backed up at this time. md6 is an internal RAID used to back up md7
daily. It doesn't need to be backed up, but if the machine totally
failed killing all the drives that wouldn't survive so currently I
back up md126 to md6 daily, and then back up md6 weekly to an external
eSATA drive.
What I'd like to do is clone
1) /boot (sda1) including grub and everything required to make it bootable
2) back up the system portions of dev/md126 (/ )
3) Add some swap space on the external drive
4) back up /dev/md7 which is all of my VMs
5) back up /home to a separate partition on the external drive
6) back up some special things like /var/lib/portage/world and
/usr/portage/packages
My thought is that this drive is basically bootable, but over time
gets out-of-sync with the system. However should the system fail I've
got a bootable external drive with all the binary packages required to
get it running again quickly. However I can always boot the drive, do
an emerge -ek @world, and basically be back to where I am as of the
last backup.
The external drive will look something like:
/dev/sdg1 - /boot
/dev/sdg2 - swap
/dev/sdg3 - / (not including /home, /usr/portage/distfiles, etc)
/dev/sdg5 - /usr/portage/packages
/dev/sdg6 - /dev/md7
etc
I will of course have to modify grub.conf and /etc/fstab to work
from this drive but that's no big deal.
What are folks best ideas about how to approach doing something like
this?
Thanks,
Mark
Hi,
Why don't you something like bind mount the folders you want to copy
and rsync them to the eSATA disk, after creating a similar partition
layout on it. Remember to exclude system files like /proc/*, /dev/*
and /sys/* as well as the ones you want to exclude yourself from the
rsync. When you want to sync the clone again just do the same again
and rsync the changes.
Regards,
Derek
As an added note on this, rsync's --one-file-system (-x) flag is handy
for avoiding grabbing unneeded things, but will typically leave you
without the base few device nodes needed to boot the backup, those can
either be grabbed from a stage3, or created with (courtesy of Linux
From Scratch's section 6.2.1. Creating Initial Device Nodes):
mknod -m 600 ${backup}/dev/console c 5 1
mknod -m 666 ${backup}/dev/null c 1 3
The best way to copy a filesystem without any sub-mounts is to
mount-bind it to some directory and copy it from there:
mkdir /tmp/root
mount --bind / /tmp/root
rsync -a /tmp/root/ /mnt/backup/
Note that copying a rw filesystem (especially /) on a
Re: [gentoo-user] Clone live system as a simple backup?
On Mar 8, 2012 2:50 AM, Joshua Murphy poiso...@gmail.com wrote:
8 snip
As an added note on this, rsync's --one-file-system (-x) flag is handy
for avoiding grabbing unneeded things, but will typically leave you
without the base few device nodes needed to boot the backup, those can
either be grabbed from a stage3, or created with (courtesy of Linux
From Scratch's section 6.2.1. Creating Initial Device Nodes):
mknod -m 600 ${backup}/dev/console c 5 1
mknod -m 666 ${backup}/dev/null c 1 3
... or just add another rsync invocation to backup /dev ...
Rgds,
Re: [gentoo-user] Clone live system as a simple backup?
On Thu, 8 Mar 2012 23:52:53 +0700, Pandu Poluan wrote:
As an added note on this, rsync's --one-file-system (-x) flag is handy
for avoiding grabbing unneeded things, but will typically leave you
without the base few device nodes needed to boot the backup, those can
either be grabbed from a stage3, or created with (courtesy of Linux
From Scratch's section 6.2.1. Creating Initial Device Nodes):
mknod -m 600 ${backup}/dev/console c 5 1
mknod -m 666 ${backup}/dev/null c 1 3
... or just add another rsync invocation to backup /dev ...
That won't work because it will backup the full devfs mounted on /dev,
not the files that exist in the directory itself.
--
Neil Bothwick
Despite the cost of living, have you noticed how it remains so popular?
signature.asc
Description: PGP signature
Re: [gentoo-user] Clone live system as a simple backup?
On Thu, Mar 8, 2012 at 9:12 AM, Neil Bothwick n...@digimed.co.uk wrote:
On Thu, 8 Mar 2012 23:52:53 +0700, Pandu Poluan wrote:
As an added note on this, rsync's --one-file-system (-x) flag is handy
for avoiding grabbing unneeded things, but will typically leave you
without the base few device nodes needed to boot the backup, those can
either be grabbed from a stage3, or created with (courtesy of Linux
From Scratch's section 6.2.1. Creating Initial Device Nodes):
mknod -m 600 ${backup}/dev/console c 5 1
mknod -m 666 ${backup}/dev/null c 1 3
... or just add another rsync invocation to backup /dev ...
That won't work because it will backup the full devfs mounted on /dev,
not the files that exist in the directory itself.
--
Neil Bothwick
Despite the cost of living, have you noticed how it remains so popular?
Thanks for all the ideas. As I've not done this sort of thing before
every one of them has been worth my time thinking about.
I ended up going a slightly different direction, only made possible by
how little disk space costs these days. Here's what I did:
1) As a chroot, on the live system I created a new Gentoo install.
This is just a very basic single disk partition type install as per
the Gentoo install guide. No apps, no KDE, nothing. Just the basic
stuff.
2) Once that install was up running I copied the server's world
file, /etc/portage/package.*, /etc/conf.d, /etc/X/xorg.conf a few
other things into this new install and ran emerge -ek @world to get it
up and running.
3) When all of that was complete and functioning, at least in the
chroot, I then took the chroot offline and used rsync to get the
chroot installed copied to the external drive.
4) WIth the rsync complete I then took a cut at modifying /etc/fstab
to be correct for booting from the external drive. Not sure about this
step at this time as I don't know for sure how the machine will name
the external drive.
I'm now looking to get /boot copied to the external drive, get grub
installed, and then see if I can boot using that drive.
Again, thanks for the ideas and keep 'em coming if you have more.
Cheers,
Mark
Re: [gentoo-user] Clone live system as a simple backup?
On 03/06/12 at 11:51AM -0800, Mark Knecht wrote: What are folks best ideas about how to approach doing something like this? Simple answer: Just use ReaR [1], it is even provided in sunrise [2] Regards, bacce [1] http://rear.sourceforge.net/ [2] http://overlays.gentoo.org/proj/sunrise/browser/sunrise/app-backup/rear signature.asc Description: Digital signature
Re: [gentoo-user] Clone live system as a simple backup?
Hi, I'm interested in the idea of cloning a live, complicated hardware system onto a single external hard drive as a simple backup. I would like this external drive to be completely bootable. What's the best way to approach doing this? I was considering just doing a Gentoo install from scratch but figured maybe there's a way to clone enough of the live system to get me there less painfully? The system I'm playing with has five 500MB hard drives with most partitions in linked together in various forms of RAID. (1, 5 6) That said, the total storage that this system presents KDE and the users is about 600GB. I have an external 1TB eSATA drive which is therefore large enough to hold everything on this system, albeit without the reliability of RAID which is fine for this purpose. The system looks more or less like: /dev/sda1 - /boot (50MB) /dev/sdb1 - /boot copy /dev/sdc1 - /boot copy c2stable ~ # df Filesystem 1K-blocks Used Available Use% Mounted on rootfs 51612920 31862844 17128276 66% / /dev/root 51612920 31862844 17128276 66% / rc-svcdir 102492 932 9% /lib64/rc/init.d udev 10240 476 9764 5% /dev shm 6151284 0 6151284 0% /dev/shm /dev/md7 389183252 350247628 19166232 95% /VirtualMachines tmpfs8388608 0 8388608 0% /var/tmp/portage /dev/sda1 54416 29516 22091 58% /boot c2stable ~ # cat /proc/mdstat Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] md6 : active raid5 sdb6[1] sdc6[2] sda6[0] 494833664 blocks super 1.1 level 5, 64k chunk, algorithm 2 [3/3] [UUU] md7 : active raid6 sdb7[1] sdc7[2] sda7[0] sdd2[3] sde2[4] 395387904 blocks super 1.2 level 6, 16k chunk, algorithm 2 [5/5] [U] md3 : active raid6 sdb3[1] sdc3[2] sda3[0] sdd3[3] sde3[4] 157305168 blocks super 1.2 level 6, 16k chunk, algorithm 2 [5/5] [U] md126 : active raid1 sdc5[2] sda5[0] sdb5[1] 52436032 blocks [3/3] [UUU] unused devices: none c2stable ~ # /dev/md3 is a second Gentoo installation that doesn't need to be backed up at this time. md6 is an internal RAID used to back up md7 daily. It doesn't need to be backed up, but if the machine totally failed killing all the drives that wouldn't survive so currently I back up md126 to md6 daily, and then back up md6 weekly to an external eSATA drive. What I'd like to do is clone 1) /boot (sda1) including grub and everything required to make it bootable 2) back up the system portions of dev/md126 (/ ) 3) Add some swap space on the external drive 4) back up /dev/md7 which is all of my VMs 5) back up /home to a separate partition on the external drive 6) back up some special things like /var/lib/portage/world and /usr/portage/packages My thought is that this drive is basically bootable, but over time gets out-of-sync with the system. However should the system fail I've got a bootable external drive with all the binary packages required to get it running again quickly. However I can always boot the drive, do an emerge -ek @world, and basically be back to where I am as of the last backup. The external drive will look something like: /dev/sdg1 - /boot /dev/sdg2 - swap /dev/sdg3 - / (not including /home, /usr/portage/distfiles, etc) /dev/sdg5 - /usr/portage/packages /dev/sdg6 - /dev/md7 etc I will of course have to modify grub.conf and /etc/fstab to work from this drive but that's no big deal. What are folks best ideas about how to approach doing something like this? Thanks, Mark Hi, Why don't you something like bind mount the folders you want to copy and rsync them to the eSATA disk, after creating a similar partition layout on it. Remember to exclude system files like /proc/*, /dev/* and /sys/* as well as the ones you want to exclude yourself from the rsync. When you want to sync the clone again just do the same again and rsync the changes. Regards, Derek
Re: [gentoo-user] Clone live system as a simple backup?
On Wed, Mar 7, 2012 at 8:55 AM, gand...@d-danks.co.uk wrote:
Hi,
I'm interested in the idea of cloning a live, complicated hardware
system onto a single external hard drive as a simple backup. I would
like this external drive to be completely bootable. What's the best
way to approach doing this? I was considering just doing a Gentoo
install from scratch but figured maybe there's a way to clone enough
of the live system to get me there less painfully?
The system I'm playing with has five 500MB hard drives with most
partitions in linked together in various forms of RAID. (1, 5 6)
That said, the total storage that this system presents KDE and the
users is about 600GB.
I have an external 1TB eSATA drive which is therefore large enough
to hold everything on this system, albeit without the reliability of
RAID which is fine for this purpose.
The system looks more or less like:
/dev/sda1 - /boot (50MB)
/dev/sdb1 - /boot copy
/dev/sdc1 - /boot copy
c2stable ~ # df
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 51612920 31862844 17128276 66% /
/dev/root 51612920 31862844 17128276 66% /
rc-svcdir 1024 92 932 9% /lib64/rc/init.d
udev 10240 476 9764 5% /dev
shm 6151284 0 6151284 0% /dev/shm
/dev/md7 389183252 350247628 19166232 95% /VirtualMachines
tmpfs 8388608 0 8388608 0% /var/tmp/portage
/dev/sda1 54416 29516 22091 58% /boot
c2stable ~ # cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5]
[raid4]
md6 : active raid5 sdb6[1] sdc6[2] sda6[0]
494833664 blocks super 1.1 level 5, 64k chunk, algorithm 2 [3/3]
[UUU]
md7 : active raid6 sdb7[1] sdc7[2] sda7[0] sdd2[3] sde2[4]
395387904 blocks super 1.2 level 6, 16k chunk, algorithm 2 [5/5]
[U]
md3 : active raid6 sdb3[1] sdc3[2] sda3[0] sdd3[3] sde3[4]
157305168 blocks super 1.2 level 6, 16k chunk, algorithm 2 [5/5]
[U]
md126 : active raid1 sdc5[2] sda5[0] sdb5[1]
52436032 blocks [3/3] [UUU]
unused devices: none
c2stable ~ #
/dev/md3 is a second Gentoo installation that doesn't need to be
backed up at this time. md6 is an internal RAID used to back up md7
daily. It doesn't need to be backed up, but if the machine totally
failed killing all the drives that wouldn't survive so currently I
back up md126 to md6 daily, and then back up md6 weekly to an external
eSATA drive.
What I'd like to do is clone
1) /boot (sda1) including grub and everything required to make it bootable
2) back up the system portions of dev/md126 (/ )
3) Add some swap space on the external drive
4) back up /dev/md7 which is all of my VMs
5) back up /home to a separate partition on the external drive
6) back up some special things like /var/lib/portage/world and
/usr/portage/packages
My thought is that this drive is basically bootable, but over time
gets out-of-sync with the system. However should the system fail I've
got a bootable external drive with all the binary packages required to
get it running again quickly. However I can always boot the drive, do
an emerge -ek @world, and basically be back to where I am as of the
last backup.
The external drive will look something like:
/dev/sdg1 - /boot
/dev/sdg2 - swap
/dev/sdg3 - / (not including /home, /usr/portage/distfiles, etc)
/dev/sdg5 - /usr/portage/packages
/dev/sdg6 - /dev/md7
etc
I will of course have to modify grub.conf and /etc/fstab to work
from this drive but that's no big deal.
What are folks best ideas about how to approach doing something like
this?
Thanks,
Mark
Hi,
Why don't you something like bind mount the folders you want to copy
and rsync them to the eSATA disk, after creating a similar partition
layout on it. Remember to exclude system files like /proc/*, /dev/*
and /sys/* as well as the ones you want to exclude yourself from the
rsync. When you want to sync the clone again just do the same again
and rsync the changes.
Regards,
Derek
As an added note on this, rsync's --one-file-system (-x) flag is handy
for avoiding grabbing unneeded things, but will typically leave you
without the base few device nodes needed to boot the backup, those can
either be grabbed from a stage3, or created with (courtesy of Linux
From Scratch's section 6.2.1. Creating Initial Device Nodes):
mknod -m 600 ${backup}/dev/console c 5 1
mknod -m 666 ${backup}/dev/null c 1 3
--
Poison [BLX]
Joshua M. Murphy
