Hi,
gene heskett wrote:
> > where did the extra 19.4G's come from? Can filesystem
> > ext4's overhead account for that?
In an earlier mail:
> > > command line: rsync -a --bwlimit=10m --fsync --progress /home/
> > > /mnt/homevol
David Christensen wrote:
> Please RTFM rsync(1) to choose your options. These look
> useful:
> --archive, -a (-rlptgoD)
> --delete
> --hard-links, -H
> --one-file-system, -x
> --sparse, -S
I bet on --hard-links and --sparse as means to avoid the extra disk space
consumption. (--archive is important for other reasons, but it was
already in use as -a with your successful rsync run. --delete will be
of importance if the rsync run gets repeated on the already filled target
directory tree.)
man rsync:
-H, --hard-links
This tells rsync to look for hard-linked files in the source and
link together the corresponding files on the destination. With‐
out this option, hard-linked files in the source are treated as
though they were separate files.
[...]
-S, --sparse
Try to handle sparse files efficiently so they take up less
space on the destination. [...]
One can observe a similar inflation effect when copying the files of a
Debian installation ISO to hard disk. In the original disk directory
on the machine which created the ISO there were hardlinked kernels and
firmware packages. In the ISO these link siblings share the same file
content storage.
But when mounted, the siblings get treated as separate files with
different inode numbers. So the 8,135,584 bytes of the hardlink siblings
/install.amd/gtk/vmlinuz
/install.amd/vmlinuz
/install.amd/xen/vmlinuz
get triplicated when these three files get copied out of the ISO.
I am somewhat astonished that --hard-links is not default in rsync,
as it is quite important for backup fidelity.
(On the other hand it is some effort to find all siblings on the disk.)
Sparse files are files with large areas of 0-bytes. Many filesystems
don't store the zeros but rather an instruction to hand out the given
number of 0-bytes when requested by a reader.
If i were you, i'd let rsync make a complete new copy with --hard-links
--sparse, and --delete, but without --bwlimit= in order to get a higher
copy fidelity and also to check whether the transfer speed really was not
to blame for the appearance of the OOM killer.
Have a nice day :)
Thomas
