Re: Restore backup to KVM
Hi. On Sat, Sep 30, 2017 at 03:42:03PM +0200, solitone wrote: > This is serious hacking :^) That's what they paying me for at office ☺. > On 30/09/17 13:04, Reco wrote: > > the next thing I have to suspect is that your backup misses > > /dev directory (possibly /proc and /sys). The contents for those are > > irrelevant. You simply do not have /dev, /proc, /sys in your root > > filesystem. > > No, I don't, you're perfectly right! Now I've made /dev, /proc, /sys, as > well as /run and now it boots right. > > I still have several error messages complaining about the floppy disk (?), QEMU emulates floppy drive by default. Empty one, that is (unless you pass -fda option to QEMU). Initrd merely probes the drive (which means you have "floppy" module inside initrd), the drive says "I have no floppy". Annoying, but harmless. Reco
Re: Restore backup to KVM
This is serious hacking :^) On 30/09/17 13:04, Reco wrote: the next thing I have to suspect is that your backup misses /dev directory (possibly /proc and /sys). The contents for those are irrelevant. You simply do not have /dev, /proc, /sys in your root filesystem. No, I don't, you're perfectly right! Now I've made /dev, /proc, /sys, as well as /run and now it boots right. I still have several error messages complaining about the floppy disk (?), which slow booting, but everything seems to end well. Now I'll try to correct grub. Thanks! === Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... [1.688631] blk_update_request: I/O error, dev fd0, sector 0 [1.689678] floppy: error -5 while reading block 0 [1.699913] random: fast init done [1.760666] blk_update_request: I/O error, dev fd0, sector 0 [1.762276] floppy: error -5 while reading block 0 Begin: Waiting for suspend/resume device ... Begin: Running /scripts/local-block ... done. [2.828592] blk_update_request: I/O error, dev fd0, sector 0 [2.829812] floppy: error -5 while reading block 0 [2.13] blk_update_request: I/O error, dev fd0, sector 0 [2.890727] floppy: error -5 while reading block 0 Begin: Running /scripts/local-block ... done. [3.964558] blk_update_request: I/O error, dev fd0, sector 0 [3.966112] floppy: error -5 while reading block 0 [4.024658] blk_update_request: I/O error, dev fd0, sector 0 [4.027035] floppy: error -5 while reading block 0 [...] Begin: Running /scripts/local-block ... done. [ 32.404669] blk_update_request: I/O error, dev fd0, sector 0 [ 32.406074] floppy: error -5 while reading block 0 [ 32.464656] blk_update_request: I/O error, dev fd0, sector 0 [ 32.465880] floppy: error -5 while reading block 0 done. [ 32.536758] blk_update_request: I/O error, dev fd0, sector 0 [ 32.539140] floppy: error -5 while reading block 0 [ 32.600728] blk_update_request: I/O error, dev fd0, sector 0 [ 32.603490] floppy: error -5 while reading block 0 Gave up waiting for suspend/resume device done. Begin: Will now check root file system ... fsck from util-linux 2.29.2 [/sbin/fsck.ext4 (1) -- /dev/sda1] fsck.ext4 -a -C0 /dev/sda1 /dev/sda1: clean, 434185/5898240 files, 15163985/23592711 blocks done. [ 33.183454] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) done. Begin: Running /scripts/local-bottom ... done. Begin: Running /scripts/init-bottom ... done. bash: cannot set terminal process group (-1): Inappropriate ioctl for device bash: no job control in this shell root@(none):/# ===
Re: Restore backup to KVM
Hi. On Fri, Sep 29, 2017 at 06:30:21AM +0200, solitone wrote: > On 28/09/17 08:58, Reco wrote: > > It's initrd that first tries to mount tmpfs filesystems on /root (and > > fails), and only *then* mounts your root filesystem to /root (with the > > intention to switch to it as /). > Also, I don't understand why it needs to mount the root filesystem to /root. So, I did some tests. Results are somewhat unexpected. First things first, either I'm missing something, or MODULES=most does not work as advertised. It did not put "all harddrive drivers" into initrd for me. Second, once I convinced update-initramfs to put missing kernel modules into initrd, QEMU booted successfully with "init=/bin/bash". If I understand what's written in /usr/share/initramfs-tools/script correctly, the way it should work is the following: 1) Unpack initrd, load kernel modules, etc. 2) Mount root filesystem to /root (and that's initrd's /root!). 3) Re-mount /dev into /root/dev (that's init-bootom/udev, and that's where it fails for you). 4) Mount procfs, sysfs (and several others) into appropriate directories of /root. 5) Mount everything appropriate from /etc/fstab. 6) Execute pivot_root(8) to switch root filesystem, start init. Therefore the next thing I have to suspect is that your backup misses /dev directory (possibly /proc and /sys). The contents for those are irrelevant. You simply do not have /dev, /proc, /sys in your root filesystem. Reco
Re: Restore backup to KVM
Hi. On Fri, Sep 29, 2017 at 06:30:21AM +0200, solitone wrote: > On 28/09/17 08:58, Reco wrote: > > It's initrd that first tries to mount tmpfs filesystems on /root (and > > fails), and only *then* mounts your root filesystem to /root (with the > > intention to switch to it as /). > > Is the stock initrd supposed to work like this? When I boot my production > system, I end up with tmpfs mounted on /run: > > $ df > Filesystem 1K-blocks Used Available Use% Mounted on > udev 4027784 04027784 0% /dev > tmpfs 807952 1352 806600 1% /run > /dev/sda5 85825416 58793224 22629416 73% / These are initrd doing. > tmpfs 4039744 49724034772 1% /dev/shm > tmpfs 5120 4 5116 1% /run/lock > tmpfs 4039744 04039744 0% /sys/fs/cgroup > /dev/sda1 201633 23678 177955 12% /boot/efi > tmpfs 807948 0 807948 0% /run/user/113 > tmpfs 80794820 807928 1% /run/user/1000 These are mounted by whatever init you're using (e.g. /etc/fstab, sysvinit, systemd). > Also, I don't understand why it needs to mount the root filesystem to /root. Tomorrow I'll do some test, read the source, etc. I don't understand it too so far.
Re: Restore backup to KVM
On 28/09/17 08:58, Reco wrote: It's initrd that first tries to mount tmpfs filesystems on /root (and fails), and only *then* mounts your root filesystem to /root (with the intention to switch to it as /). Is the stock initrd supposed to work like this? When I boot my production system, I end up with tmpfs mounted on /run: $ df Filesystem 1K-blocks Used Available Use% Mounted on udev 4027784 04027784 0% /dev tmpfs 807952 1352 806600 1% /run /dev/sda5 85825416 58793224 22629416 73% / tmpfs 4039744 49724034772 1% /dev/shm tmpfs 5120 4 5116 1% /run/lock tmpfs 4039744 04039744 0% /sys/fs/cgroup /dev/sda1 201633 23678 177955 12% /boot/efi tmpfs 807948 0 807948 0% /run/user/113 tmpfs 80794820 807928 1% /run/user/1000 Also, I don't understand why it needs to mount the root filesystem to /root.
Re: Restore backup to KVM
Hi. On Wed, Sep 27, 2017 at 06:56:27PM +0200, solitone wrote: > On 27/09/17 14:01, solitone wrote: > > Although mkfs had warned me, I mistakenly formatted the entire file, > > like this: > > > > $ sudo mkfs.ext4 restore.img > > > > Now I've redone it the right way, using a loop device, and I'll see how > > it goes. > > It's a struggle! Now that I partitioned the image file, it sees /dev/sda1. > There was an inconsistency between the filesystem size and the physical size > of the device though, that I repaired with resize2fs, specifying the > physical number of blocks. And that's good. > Now it finally mounts /dev/sda1, but on /root rather than on /, as it did > before with the unpartitioned image file (/dev/sda): > > == > Begin: Will now check root file system ... fsck from util-linux 2.29.2 > [/sbin/fsck.ext4 (1) -- /dev/sda1] fsck.ext4 -a -C0 /dev/sda1 > /dev/sda1: clean, 434181/5898240 files, 15163981/23592711 blocks > done. > [ 33.088863] EXT4-fs (sda1): mounted filesystem with ordered data mode. > Opts: (null) > done. > Begin: Running /scripts/local-bottom ... done. > Begin: Running /scripts/init-bottom ... mount: mounting /dev on /root/dev > failed: No such file or directory > mount: mounting /dev on /root/dev failed: No such file or directory > done. > mount: mounting /run on /root/run failed: No such file or directory > run-init: opening console: No such file or directory > Target filesystem doesn't have requested /bin/bash. > run-init: opening console: No such file or directory > run-init: opening console: No such file or directory > run-init: opening console: No such file or directory > run-init: opening console: No such file or directory > run-init: opening console: No such file or directory > No init found. Try passing init= bootarg. > > > BusyBox v1.22.1 (Debian 1:1.22.0-19+b3) built-in shell (ash) > Enter 'help' for a list of built-in commands. > > (initramfs) df > Filesystem 1K-blocks Used Available Use% Mounted on > udev 46448 0 46448 0% /dev > tmpfs1168460 11624 1% /run > /dev/sda1 92365508 58650588 2898 67% /root > (initramfs) > == And that means I have to take my words back. You cannot use stock Debian initrd in these conditions. It's not that you did something wrong at creating filesystem this time. It's initrd that first tries to mount tmpfs filesystems on /root (and fails), and only *then* mounts your root filesystem to /root (with the intention to switch to it as /). I have this feeling that you could workaround it by using custom-built initrd, but I need to think through the details. Reco
Re: Restore backup to KVM
On 27/09/17 14:01, solitone wrote: Although mkfs had warned me, I mistakenly formatted the entire file, like this: $ sudo mkfs.ext4 restore.img Now I've redone it the right way, using a loop device, and I'll see how it goes. It's a struggle! Now that I partitioned the image file, it sees /dev/sda1. There was an inconsistency between the filesystem size and the physical size of the device though, that I repaired with resize2fs, specifying the physical number of blocks. Now it finally mounts /dev/sda1, but on /root rather than on /, as it did before with the unpartitioned image file (/dev/sda): == Begin: Will now check root file system ... fsck from util-linux 2.29.2 [/sbin/fsck.ext4 (1) -- /dev/sda1] fsck.ext4 -a -C0 /dev/sda1 /dev/sda1: clean, 434181/5898240 files, 15163981/23592711 blocks done. [ 33.088863] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null) done. Begin: Running /scripts/local-bottom ... done. Begin: Running /scripts/init-bottom ... mount: mounting /dev on /root/dev failed: No such file or directory mount: mounting /dev on /root/dev failed: No such file or directory done. mount: mounting /run on /root/run failed: No such file or directory run-init: opening console: No such file or directory Target filesystem doesn't have requested /bin/bash. run-init: opening console: No such file or directory run-init: opening console: No such file or directory run-init: opening console: No such file or directory run-init: opening console: No such file or directory run-init: opening console: No such file or directory No init found. Try passing init= bootarg. BusyBox v1.22.1 (Debian 1:1.22.0-19+b3) built-in shell (ash) Enter 'help' for a list of built-in commands. (initramfs) df Filesystem 1K-blocks Used Available Use% Mounted on udev 46448 0 46448 0% /dev tmpfs1168460 11624 1% /run /dev/sda1 92365508 58650588 2898 67% /root (initramfs) ==
Re: Restore backup to KVM
On 27/09/17 08:56, Reco wrote: I'm curious to know how you'd achieve this. Although mkfs had warned me, I mistakenly formatted the entire file, like this: $ sudo mkfs.ext4 restore.img Now I've redone it the right way, using a loop device, and I'll see how it goes. Thank you!
Re: Restore backup to KVM
Hi. On Wed, Sep 27, 2017 at 09:39:50AM +0200, Thomas Schmitt wrote: > Hi, > > solitone wrote: > > > Partition Table: loop > > > 1 0.00B 96.6GB 96.6GB ext4 > > Reco wrote: > > I'm curious to know how you'd achieve this. > > The storage device (image file in this case) is unpartitioned. Thank you, that clarifies things. Now the remaining question is - could it break stock Debian initrd? Reco
Re: Restore backup to KVM
Hi,
solitone wrote:
> > Partition Table: loop
> > 1 0.00B 96.6GB 96.6GB ext4
Reco wrote:
> I'm curious to know how you'd achieve this.
The storage device (image file in this case) is unpartitioned.
If you have a MBR partition table ("msdos"), then you may achieve this
by deleting all partitions.
On byte level you may zeroize bytes 446 to 510 of the device or
image file:
dd if=/dev/zero bs=1 seek=446 count=64 conv=notrunc of=...path...
If the partition editor stays stubbornly with msdos, delete the MBR
signature:
dd if=/dev/zero bs=1 seek=510 count=2 conv=notrunc of=...path...
If it is a GPT, then the dd will make it invalid too (by removing the
MBR partition of type 0xee), but you have to expect that a partition
editor will try to restore it from the Backup GPT at the end of the
storage device.
The backup GPT header block is supposed to be in the last block of
the device. It begins by the text "EFI PART".
One would zeroize it by
dd if=/dev/zero bs=512 seek=...byte.size.divided.by.512.minus.1... \
count=1 conv=notrunc of=...path...
(Or hope that the partition editor knows how to deface GPT if you
ask it to do so. Your milage may vary.)
Have a nice day :)
Thomas
Re: Restore backup to KVM
Hi. On Tue, Sep 26, 2017 at 09:44:57PM +0200, solitone wrote: > On 26/09/17 17:31, Reco wrote: > > > On 26/09/17 13:01, solitone wrote: > > > > It's strange, since it finds /dev/sda, i.e. the entire disk: > > > > > > > > = > > > > [ 6.438693] sd 0:0:0:0: [sda] 188743680 512-byte logical blocks: > > > > (96.6 GB/90.0 GiB) > > > > [ 6.469182] sd 0:0:0:0: [sda] Write Protect is off > > > > [ 6.482421] sd 0:0:0:0: [sda] Write cache: enabled, read cache: > > > > enabled, doesn't support DPO or FUA > > > > = > > > > > > > > However, it then complains that /dev/sda1 does not exists > > > > That's because you don't have any partitions on that disk. Partition > > that's start with sector 0 is impossible. > > Interesting. I used parted to create one single partition as big as the > entire file. Here's what parted shows now: > > = > $ /sbin/parted alan_restore.img > WARNING: You are not superuser. Watch out for permissions. > GNU Parted 3.2 > Using /media/solitone/Maxtor/vmimages/alan_restore.img > Welcome to GNU Parted! Type 'help' to view a list of commands. > > (parted) p > Model: (file) > Disk /media/solitone/Maxtor/vmimages/alan_restore.img: 96.6GB > Sector size (logical/physical): 512B/512B > Partition Table: loop > Disk Flags: > > Number Start End SizeFile system Flags > 1 0.00B 96.6GB 96.6GB ext4 > = > > I assumed that that number 1 referred to partition n. 1, but I must be > mistaken. I'm curious to know how you'd achieve this. Because best I could produce is this: $ dd if=/dev/zero of=restore.img bs=1M count=0 seek=96K 0+0 records in 0+0 records out 0 bytes copied, 0.000490565 s, 0.0 kB/s $ /sbin/parted restore.img WARNING: You are not superuser. Watch out for permissions. GNU Parted 3.2 Using /tmp/restore.img Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) mklabel msdos (parted) mkpart primary 0 -1 Warning: The resulting partition is not properly aligned for best performance. Ignore/Cancel? I (parted) print Model: (file) Disk /tmp/restore.img: 103GB Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start EndSize Type File system Flags 1 512B 103GB 103GB primary lba Note the difference at 'Partition Table' and 'Start'. Reco
Re: Restore backup to KVM
On 26/09/17 17:31, Reco wrote: On 26/09/17 13:01, solitone wrote: It's strange, since it finds /dev/sda, i.e. the entire disk: = [ 6.438693] sd 0:0:0:0: [sda] 188743680 512-byte logical blocks: (96.6 GB/90.0 GiB) [ 6.469182] sd 0:0:0:0: [sda] Write Protect is off [ 6.482421] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA = However, it then complains that /dev/sda1 does not exists That's because you don't have any partitions on that disk. Partition that's start with sector 0 is impossible. Interesting. I used parted to create one single partition as big as the entire file. Here's what parted shows now: = $ /sbin/parted alan_restore.img WARNING: You are not superuser. Watch out for permissions. GNU Parted 3.2 Using /media/solitone/Maxtor/vmimages/alan_restore.img Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) p Model: (file) Disk /media/solitone/Maxtor/vmimages/alan_restore.img: 96.6GB Sector size (logical/physical): 512B/512B Partition Table: loop Disk Flags: Number Start End SizeFile system Flags 1 0.00B 96.6GB 96.6GB ext4 = I assumed that that number 1 referred to partition n. 1, but I must be mistaken.
Re: Restore backup to KVM
On 26/09/17 17:31, Reco wrote: On Tue, Sep 26, 2017 at 01:12:52PM +0200, solitone wrote: However now it fails because it tries to mount /dev and /run on /root/dev and /root/run, rather than simply /dev and /run: = [...] Gave up waiting for suspend/resume device done. Begin: Will now check root file system ... fsck from util-linux 2.29.2 [/sbin/fsck.ext4 (1) -- /dev/sda] fsck.ext4 -a -C0 /dev/sda /dev/sda: clean, 434616/5898240 files, 15174007/23592960 blocks done. [ 40.335292] EXT4-fs (sda): mounted filesystem with ordered data mode. Opts: (null) done. Begin: Running /scripts/local-bottom ... done. Begin: Running /scripts/init-bottom ... mount: mounting /dev on /root/dev failed: No such file or directory Interesting. Do you have /root directory in your root filesystem? In the source system that I'm backing up I do, it's the home directory of the root user. Ciao!
Re: Restore backup to KVM
Hi. On Tue, Sep 26, 2017 at 01:12:52PM +0200, solitone wrote: > On 26/09/17 13:01, solitone wrote: > > It's strange, since it finds /dev/sda, i.e. the entire disk: > > > > = > > [ 6.438693] sd 0:0:0:0: [sda] 188743680 512-byte logical blocks: > > (96.6 GB/90.0 GiB) > > [ 6.469182] sd 0:0:0:0: [sda] Write Protect is off > > [ 6.482421] sd 0:0:0:0: [sda] Write cache: enabled, read cache: > > enabled, doesn't support DPO or FUA > > = > > > > However, it then complains that /dev/sda1 does not exists That's because you don't have any partitions on that disk. Partition that's start with sector 0 is impossible. > I've found out that if I specify /dev/sda like this: > > $ qemu-system-x86_64 -hda alan_restore.img > -kernel /vmlinuz > -initrd /initrd.img > -append "root=/dev/sda ro init=/bin/bash" > > then it finds, checks, and mounts the root file system. That's expected. Effectively you're using a filesystem on whole disk, so root should be /dev/sda. > However now it fails > because it tries to mount /dev and /run on /root/dev and /root/run, rather > than simply /dev and /run: > > = > [...] > Gave up waiting for suspend/resume device > done. > Begin: Will now check root file system ... fsck from util-linux 2.29.2 > [/sbin/fsck.ext4 (1) -- /dev/sda] fsck.ext4 -a -C0 /dev/sda > /dev/sda: clean, 434616/5898240 files, 15174007/23592960 blocks > done. > [ 40.335292] EXT4-fs (sda): mounted filesystem with ordered data mode. > Opts: (null) > done. > Begin: Running /scripts/local-bottom ... done. > Begin: Running /scripts/init-bottom ... mount: mounting /dev on /root/dev > failed: No such file or directory Interesting. Do you have /root directory in your root filesystem? Reco
Re: Restore backup to KVM
On 26/09/17 13:01, solitone wrote: It's strange, since it finds /dev/sda, i.e. the entire disk: = [ 6.438693] sd 0:0:0:0: [sda] 188743680 512-byte logical blocks: (96.6 GB/90.0 GiB) [ 6.469182] sd 0:0:0:0: [sda] Write Protect is off [ 6.482421] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA = However, it then complains that /dev/sda1 does not exists I've found out that if I specify /dev/sda like this: $ qemu-system-x86_64 -hda alan_restore.img -kernel /vmlinuz -initrd /initrd.img -append "root=/dev/sda ro init=/bin/bash" then it finds, checks, and mounts the root file system. However now it fails because it tries to mount /dev and /run on /root/dev and /root/run, rather than simply /dev and /run: = [...] Gave up waiting for suspend/resume device done. Begin: Will now check root file system ... fsck from util-linux 2.29.2 [/sbin/fsck.ext4 (1) -- /dev/sda] fsck.ext4 -a -C0 /dev/sda /dev/sda: clean, 434616/5898240 files, 15174007/23592960 blocks done. [ 40.335292] EXT4-fs (sda): mounted filesystem with ordered data mode. Opts: (null) done. Begin: Running /scripts/local-bottom ... done. Begin: Running /scripts/init-bottom ... mount: mounting /dev on /root/dev failed: No such file or directory mount: mounting /dev on /root/dev failed: No such file or directory done. mount: mounting /run on /root/run failed: No such file or directory run-init: opening console: No such file or directory Target filesystem doesn't have requested /bin/bash. run-init: opening console: No such file or directory run-init: opening console: No such file or directory run-init: opening console: No such file or directory run-init: opening console: No such file or directory run-init: opening console: No such file or directory No init found. Try passing init= bootarg. =
Re: Restore backup to KVM
On 26/09/17 08:46, Reco wrote: /dev/sda2 refers to the partition in QEMU disk that contains your restored root filesystem. I've got only one partition in my image file: = $ /sbin/parted alan_restore.img WARNING: You are not superuser. Watch out for permissions. GNU Parted 3.2 Using /media/solitone/Maxtor/vmimages/alan_restore.img Welcome to GNU Parted! Type 'help' to view a list of commands. (parted) p Model: (file) Disk /media/solitone/Maxtor/vmimages/alan_restore.img: 96.6GB Sector size (logical/physical): 512B/512B Partition Table: loop Disk Flags: Number Start End SizeFile system Flags 1 0.00B 96.6GB 96.6GB ext4 = I'd expect I should refer to it with /dev/sda1. However, when booting it doesn't find it. It's strange, since it finds /dev/sda, i.e. the entire disk: = [6.438693] sd 0:0:0:0: [sda] 188743680 512-byte logical blocks: (96.6 GB/90.0 GiB) [6.469182] sd 0:0:0:0: [sda] Write Protect is off [6.482421] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA = However, it then complains that /dev/sda1 does not exists: = [ 39.409892] blk_update_request: I/O error, dev fd0, sector 0 [ 39.410232] floppy: error -5 while reading block 0 [ 39.481887] blk_update_request: I/O error, dev fd0, sector 0 [ 39.482281] floppy: error -5 while reading block 0 Gave up waiting for suspend/resume device done. Begin: Waiting for root file system ... Begin: Running /scripts/local-block ... done. done. Gave up waiting for root file system device. Common problems: - Boot args (cat /proc/cmdline) - Check rootdelay= (did the system wait long enough?) - Missing modules (cat /proc/modules; ls /dev) ALERT! /dev/sda1 does not exist. Dropping to a shell! = Same thing if I try with /dev/sda2 or anything. Regards!
Re: Restore backup to KVM
Hi. On Mon, Sep 25, 2017 at 09:16:35PM +0200, solitone wrote: > On 24/09/17 11:51, Reco wrote: > > Cheat it then and run QEMU like this (I don't know what's your root > > filesystem is called, you may need to replace sda2 with something else): > > > > qemu-system-x86_64 -hda \ > > -kernel \ > > -initrd \ > > -append "root=/dev/sda2 ro init=/bin/bash" > > Hi Reco, does /dev/sda2 refer to the partition containing the root file > system of the *host*? No. /dev/sda2 refers to the partition in QEMU disk that contains your restored root filesystem. Reco
Re: Restore backup to KVM
Hi. On Mon, Sep 25, 2017 at 09:05:25PM +0200, solitone wrote: > On 24/09/17 11:51, Reco wrote: > > ACLs are easy. Even tar(1) knows them. > > It's things like these that give you headache: > > > > $ /sbin/getcap /bin/ping > > /bin/ping = cap_net_raw+ep > > > > # lsattr /etc/resolv.conf > > i-e /etc/resolv.conf > > > > # getfattr -d /var/log/messages > > # file: var/log/messages > > user.name="main system log" > > > > # ls -alZ .bashrc > > -rw-r--r--. root root system_u:object_r:admin_home_t:s0 .bashrc > > > > If you have any of these in your source system, but don't have in target > > one - your backup is invalid, consider changing tool you're using. > > Note that these are just the examples, there can be other files like > > this. > > Is there anything like this in the standard debian configuration? I haven't > set any. setcap should be there: $ grep -c setcap /var/lib/dpkg/info/*postinst | grep -v 0$ /var/lib/dpkg/info/iputils-ping.postinst:3 /var/lib/dpkg/info/iputils-tracepath.postinst:2 /var/lib/dpkg/info/wireshark-common.postinst:2 SELinux labels will be there if you have it installed, but SELinux has optional priority. 'chattr +i /etc/resolv.conf' is a popular measure at this list. setfattr is exotic. Reco
Re: Restore backup to KVM
On 24/09/17 11:51, Reco wrote: Cheat it then and run QEMU like this (I don't know what's your root filesystem is called, you may need to replace sda2 with something else): qemu-system-x86_64 -hda \ -kernel \ -initrd \ -append "root=/dev/sda2 ro init=/bin/bash" Hi Reco, does /dev/sda2 refer to the partition containing the root file system of the *host*?
Re: Restore backup to KVM
On 24/09/17 11:51, Reco wrote: ACLs are easy. Even tar(1) knows them. It's things like these that give you headache: $ /sbin/getcap /bin/ping /bin/ping = cap_net_raw+ep # lsattr /etc/resolv.conf i-e /etc/resolv.conf # getfattr -d /var/log/messages # file: var/log/messages user.name="main system log" # ls -alZ .bashrc -rw-r--r--. root root system_u:object_r:admin_home_t:s0 .bashrc If you have any of these in your source system, but don't have in target one - your backup is invalid, consider changing tool you're using. Note that these are just the examples, there can be other files like this. Is there anything like this in the standard debian configuration? I haven't set any.
Re: Restore backup to KVM
On Sun, Sep 24, 2017 at 10:58:42AM +0200, solitone wrote: > On 22/09/17 21:38, Reco wrote: > > 2) Your backup is made by rsync(1) or tar(1). > > > > Make yourself a file representing virtual machine disk. > > Apply parted/fdisk/whatever to make appropriate number of partitions > > inside it. Create filesystems. > > Mount these somewhere, invoke rsync(1)/tar(1) as needed. > > Ok, I've done all these steps. > > > Fix extended file attributes, capability labels, SELinux labels if any > > etc. By hand, that is. > > This is hard, since I'm unaware of what files have extended attributes. For > instance, I've just found out that /media/solitone has an ACL, but I would > need to extract the information for all files involved in a list and back it > up, in order to restore it later on manually. ACLs are easy. Even tar(1) knows them. It's things like these that give you headache: $ /sbin/getcap /bin/ping /bin/ping = cap_net_raw+ep # lsattr /etc/resolv.conf i-e /etc/resolv.conf # getfattr -d /var/log/messages # file: var/log/messages user.name="main system log" # ls -alZ .bashrc -rw-r--r--. root root system_u:object_r:admin_home_t:s0 .bashrc If you have any of these in your source system, but don't have in target one - your backup is invalid, consider changing tool you're using. Note that these are just the examples, there can be other files like this. > > Fix boot/grub/grub.cfg or whatever configuration file of bootloader > > you're using. > > I don't know how to do this. I have to admit I never really understand GRUB. > I've always relied on the configuration steps that debian automatically > performs. Have you got a simple document that could guide me? Cheat it then and run QEMU like this (I don't know what's your root filesystem is called, you may need to replace sda2 with something else): qemu-system-x86_64 -hda \ -kernel \ -initrd \ -append "root=/dev/sda2 ro init=/bin/bash" If it boots *and* mounts everything it should - terminate QEMU and run it like this and run update-grub from the inside: qemu-system-x86_64 -hda \ -kernel \ -initrd \ -append "root=/dev/sda2 ro" Don't forget to unmount filesystems from your host OS before running QEMU. Reco
Re: Restore backup to KVM
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sun, Sep 24, 2017 at 10:58:42AM +0200, solitone wrote: > On 22/09/17 21:38, Reco wrote: > >2) Your backup is made by rsync(1) or tar(1). > > > >Make yourself a file representing virtual machine disk. > >Apply parted/fdisk/whatever to make appropriate number of partitions > >inside it. Create filesystems. > >Mount these somewhere, invoke rsync(1)/tar(1) as needed. > > Ok, I've done all these steps. > > >Fix extended file attributes, capability labels, SELinux labels if any > >etc. By hand, that is. Note that rsync can be told to preserve extended file attributes (option - -X). Whether that succeeds will depend, of course, on whether the target file system can digest that. Likewise, tar has an --xattrs option. Capabilities and SELinux labels seem to be implemented on top of xattrs (Disclaimer: I barely know what I'm talking about, but I Looked It Up On The Internet(TM)). So all seems well here. "By hand" sounds a bit sadomasochistic. I'd use -X resp. --xattrs and sift carefully through warnings and errors. Plus perhaps check one file or three to catch some systematic error. Cheeers - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlnHd3QACgkQBcgs9XrR2kZfGQCeOSG2M9EVkRGrmc3kXVWjSvfd FkMAn3TmkPHQQC6KA5LkNfeU8rw3X/zd =RP+J -END PGP SIGNATURE-
Re: Restore backup to KVM
On 22/09/17 21:38, Reco wrote: 2) Your backup is made by rsync(1) or tar(1). Make yourself a file representing virtual machine disk. Apply parted/fdisk/whatever to make appropriate number of partitions inside it. Create filesystems. Mount these somewhere, invoke rsync(1)/tar(1) as needed. Ok, I've done all these steps. Fix extended file attributes, capability labels, SELinux labels if any etc. By hand, that is. This is hard, since I'm unaware of what files have extended attributes. For instance, I've just found out that /media/solitone has an ACL, but I would need to extract the information for all files involved in a list and back it up, in order to restore it later on manually. Fix boot/grub/grub.cfg or whatever configuration file of bootloader you're using. I don't know how to do this. I have to admit I never really understand GRUB. I've always relied on the configuration steps that debian automatically performs. Have you got a simple document that could guide me? Thanks!
Re: Restore backup to KVM
Hi. On Fri, Sep 22, 2017 at 09:10:28PM +0200, solitone wrote: > On 22/09/17 08:08, Reco wrote: > > Execute this on your source system. > > > > grep MODULES /etc/initramfs-tools/initramfs.conf > > > > If it says MODULES=most then you're in luck as it means your initrd > > contains all kernel modules for all kinds of hardware. > > And restoring from backup into QEMU-KVM means you only need to > > reconfigure the bootloader. > > > > > Is there a way to configure KVM so that it resembles my bare > > > metal, and the test is significant? > > > > That's highly unlikely. On x86-64 there are two QEMU device models > > worthy of speaking, and that's Intel i440FX and Intel Q35 motherboards. > > Chances are you have different hardware. > > > > So, it *will* have different NIC, Video adapter *and* most importantly, > > IDE/SATA/SCSI controller. > > > > Using Debian and MODULES=most you have a luxury of not to think about > > it. > > Ok, I see, and it seems I'm in luck: > > ~$ grep MODULES /etc/initramfs-tools/initramfs.conf > # MODULES: [ most | netboot | dep | list ] > MODULES=most No wonder. This is Debian default, and Debian is famous for sane defaults. > But now I don't know what to do next, since: > > > > I would install a basic debian system in KVM, and then overwrite it > > > with my backed up files. Is this approach correct? > > > > No. Some (but not all) configuration files would differ. Some (but not > > all) packages would differ. That depends on what kind of backup you have. 1) An old school case - you backup is made by dump(8). Make yourself a file representing virtual machine disk. Apply parted/fdisk/whatever to make appropriate number of partitions inside it. Create filesystems. Mount these somewhere, invoke restore(8) as needed. Fix boot/grub/grub.cfg or whatever configuration file of bootloader you're using. Dismount filesystems, try to boot in QEMU. 2) Your backup is made by rsync(1) or tar(1). Make yourself a file representing virtual machine disk. Apply parted/fdisk/whatever to make appropriate number of partitions inside it. Create filesystems. Mount these somewhere, invoke rsync(1)/tar(1) as needed. Fix extended file attributes, capability labels, SELinux labels if any etc. By hand, that is. Fix boot/grub/grub.cfg or whatever configuration file of bootloader you're using. Dismount filesystems, try to boot in QEMU. 3) You don't know what is used to make your backups, but it has an agent (amanda, bacula, etc). Grab yourself a liveCD, boot it in QEMU. Invoke appropriate backup agent. Reboot into (hopefully) restored QEMU disk. 4) You don't want to know how it's made but it's called Clonezilla. Boot Clonezilla in QEMU. Invoke restore. Watch it done. Reboot. 5) It's all complex and confusing, the backup is done by dd'ing block devices. dd it back in QEMU disk file, try to boot. It may even work. Reco
Re: Restore backup to KVM
On 22/09/17 08:08, Reco wrote: Execute this on your source system. grep MODULES /etc/initramfs-tools/initramfs.conf If it says MODULES=most then you're in luck as it means your initrd contains all kernel modules for all kinds of hardware. And restoring from backup into QEMU-KVM means you only need to reconfigure the bootloader. Is there a way to configure KVM so that it resembles my bare metal, and the test is significant? That's highly unlikely. On x86-64 there are two QEMU device models worthy of speaking, and that's Intel i440FX and Intel Q35 motherboards. Chances are you have different hardware. So, it *will* have different NIC, Video adapter *and* most importantly, IDE/SATA/SCSI controller. Using Debian and MODULES=most you have a luxury of not to think about it. Ok, I see, and it seems I'm in luck: ~$ grep MODULES /etc/initramfs-tools/initramfs.conf # MODULES: [ most | netboot | dep | list ] MODULES=most But now I don't know what to do next, since: I would install a basic debian system in KVM, and then overwrite it with my backed up files. Is this approach correct? No. Some (but not all) configuration files would differ. Some (but not all) packages would differ.
Re: Restore backup to KVM
Hi. On Fri, Sep 22, 2017 at 06:42:17AM +0200, solitone wrote: > It's time to test my backups. Apart from user files, I also back up system > files, except for the following directories that are excluded: /dev, > /lost+found, /media, /mnt, /proc, /run, /sys, /tmp. > > I would try and restore them to a virtual machine (KVM). Would it be > possible? Execute this on your source system. grep MODULES /etc/initramfs-tools/initramfs.conf If it says MODULES=most then you're in luck as it means your initrd contains all kernel modules for all kinds of hardware. And restoring from backup into QEMU-KVM means you only need to reconfigure the bootloader. If it says anything else you'll need to rebuild initrd along with the bootloader configuration. > Is there a way to configure KVM so that it resembles my bare > metal, and the test is significant? That's highly unlikely. On x86-64 there are two QEMU device models worthy of speaking, and that's Intel i440FX and Intel Q35 motherboards. Chances are you have different hardware. So, it *will* have different NIC, Video adapter *and* most importantly, IDE/SATA/SCSI controller. Using Debian and MODULES=most you have a luxury of not to think about it. > I would install a basic debian system in KVM, and then overwrite it > with my backed up files. Is this approach correct? No. Some (but not all) configuration files would differ. Some (but not all) packages would differ. Reco
