Re: dump reads more than restore writes?
On 1/8/07, Oliver Fromme <[EMAIL PROTECTED]> wrote: However, I think that there must be a bug in gstat when it displays 600 GB read to copy a 200 GB file system. dump(8) is inefficient, but not _that_ inefficient. When doing the dump|restore dance to copy filesystems, I make it a habit to have iostat(1) running. It is _always_ displaying a read rate twice as high as the write rate. And since the iostat/gstat numbers usually match up, eg., dd(1) numbers, I heavily doubt that it's a reporting/statistics glitch. It has to be dump(8). Uli ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: dump reads more than restore writes?
[EMAIL PROTECTED] wrote: > [...] > > Is dump reading substantially more than restore is writing? > > Quite possibly, esp. if the source disk is nowhere near full and/or > most of the files being handled are small. dump reads every inode > on the disk, including those which are unallocated, I don't think it does that. Why should it read inodes that are unallocated? It reads the directory tree of the file system, so it knows which inodes are allocated, and there's no reason to read anything beyond that. However, it is true that dump|restore is a slow way to copy a file system. It's much faster to mount it and then use something like "cd $SRC; find -d . | cpio -dump $DST", or cpdup (from ports/sysutils/cpdup), or even tar. If the file systems are the same size and nearly full, then dd(1) is probably the fastest way to copy it (but neither of them must be mounted, of course). If it's not nearly full, dd is inefficient because it simply copies the device without regard to what areas are actually unallocated to files. However, I think that there must be a bug in gstat when it displays 600 GB read to copy a 200 GB file system. dump(8) is inefficient, but not _that_ inefficient. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing Dienstleistungen mit Schwerpunkt FreeBSD: http://www.secnetix.de/bsd Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "anyone new to programming should be kept as far from C++ as possible; actually showing the stuff should be considered a criminal offence" -- Jacek Generowicz ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: dump reads more than restore writes?
> "Dan" == Dan Nelson <[EMAIL PROTECTED]> writes: Dan> If you have a lot of small files, dump may be rereading directory Dan> information. Dump has a cache option that can help, but make Dan> sure you also dump a snapshot (i.e. always use -L when using -C). Several people have suggested this, but actually it has the same behaviour when using -C (I often use -C 32). This filesystem is not mounted, so -L is not required, but I do use -L when required. But the filesystem is 95% full (aren't they all) and the vast majority of the files on it are "media" files --- ie: movies, mp3's, ISOs, etc. Very few files under a meg. Probably not too many files over 100 meg as most of the files on the disk are in rar format. ... now Azurus (used to obtain most of the files) writes holey files. One chunk at a time (512k-ish to 4meg-ish). Those could end up being not-very-contiguous, but I'd expect them to consist (by majority) of full filesystem blocks. The amazing part (to me) is how consistent it is. If this is not a reporting error of gstat, it makes dump look _very_ wasteful. If the numbers are being reported correctly, it means that dump is reading 600 gig to copy a 200 gig disk. !?! Dave. -- |David Gilbert, Independent Contractor. | Two things can be | |Mail: [EMAIL PROTECTED]| equal if and only if they | |http://daveg.ca | are precisely opposite. | =GLO ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: dump reads more than restore writes?
> Is dump reading substantially more than restore is writing? Quite possibly, esp. if the source disk is nowhere near full and/or most of the files being handled are small. dump reads every inode on the disk, including those which are unallocated, and probably reads entire data blocks -- or even entire FS blocks -- even when only part of the block contains useful data. Restore writes only what it has to. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: dump reads more than restore writes?
In the last episode (Jan 07), David Gilbert said: > I've got a command line like the following: > > dump -0af - /dev/ad1s1g | restore -rf - > > ... and I'm watching gstat. ad1s1g is not mounted. The disk on which > the restore is running is also quiet (nobody using the disk). > > And gstat says that ad1 is consistently reading 31 to 37 megabytes > per second and ad2 (the restore disk) is consistently writing 10 to > 13 megabytes per second. This is over hours --- the figures never > catch up. If you have a lot of small files, dump may be rereading directory information. Dump has a cache option that can help, but make sure you also dump a snapshot (i.e. always use -L when using -C). -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
dump reads more than restore writes?
I've got a command line like the following: dump -0af - /dev/ad1s1g | restore -rf - ... and I'm watching gstat. ad1s1g is not mounted. The disk on which the restore is running is also quiet (nobody using the disk). And gstat says that ad1 is consistently reading 31 to 37 megabytes per second and ad2 (the restore disk) is consistently writing 10 to 13 megabytes per second. This is over hours --- the figures never catch up. So... is gstat wrong? Is dump reading substantially more than restore is writing? Dave. -- |David Gilbert, Independent Contractor. | Two things can be | |Mail: [EMAIL PROTECTED]| equal if and only if they | |http://daveg.ca | are precisely opposite. | =GLO ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
