Hi,
Timothy J. Massey wrote on 26.01.2007 at 15:44:25 [Re: [BackupPC-users] Long:
How BackupPC handles pooling, and how transfer methods affect bandwidth usage]:
> Holger Parplies <[EMAIL PROTECTED]> wrote on 01/26/2007 02:48:29 PM:
> > I'm a bit confused by the terms 'host' and 'server'.
>
> [...]
> The problem is, my BackupPC targets *are* servers themselves,
> so it's *very* easy to slip up and call them that, even though in this
> context they're "hosts"...
yes, I know, and sometimes that fact is important, because putting load on *a*
server may be something to worry about, while putting load on a workstation
during a backup is probably not.
"Windows people" seem to have other habits about what they call "server" and
what "client" than the rest of us, causing further confusion. Is there a
good, agreed-upon terminology for speaking about BackupPC setups?
> > > From what I
> > > understand, there would *never* be any files in the destination path:
> > > the destination path is always a newly created directory.
> >
> > See the rsync --compare-dest and --copy-dest options. I'd guess it's
> > similar to what they do: compare with one tree and create a second.
^^^^^^^
> Ah! I was not aware of those parameters! However, that still means
> that we can only compare from a single previously-completed backup...
Actually, rsync >= 2.6.4 can compare with more than one source tree,
probably by using the checksums Craig spoke of to check several files if
necessary. Still, the pool is, generally speaking, too large to take *every
file* into consideration (well, at least it would no longer be a speedup,
except maybe over 300 baud connections ;-).
Maybe if there was something like a FallbackHost parameter, as in "for the
first backup, take files from FallbackHost's most recent full backup into
consideration for rsync transfers" ...
> > > the documentation says
> > > "As rsync runs, it checks each file in the backup to see if it is
> > > identical to an existing file from any previous backup of any PC."
> >
> > Apparently, it doesn't because it can't. You've probably read Craig's
> > reply in the other thread by now.
>
> I don't remember: it was the confusing, seemingly conflicting
> information in different threads (and the documentation) that caused me
> to do the research and write my e-mail.
I was talking about:
Craig Barratt wrote on 25.01.2007 at 22:37:20 [Re: [BackupPC-users] Avoiding
long backup times]:
C> Holger writes:
C>
C> > I like the idea of taking *any* identical file from the pool as reference
C> > though. I don't know if it is possible (i.e. whether the remote rsync will
C> > transfer one checksum covering the whole file before the local part needs
C> > to commit to selecting one file), but it certainly would be a worthwhile
C> > speedup.
C>
C> It's not currently possible. Rsync does have a --checksum option that
C> causes it to compute and send the file digest as part of the initial
C> file list. That could allow BackupPC to try to use a file which is
C> a likely match. However, rsync's and BackupPC's digests are different,
C> so getting the --checksum digest doesn't allow you to look up the file
C> in the pool.
C>
C> It's possible BackupPCd could do something like this in the future.
Les Mikesell wrote on 26.01.2007 at 15:04:43 [Re: [BackupPC-users] Long: How
BackupPC handles pooling, and how transfer methods affect bandwidth usage]:
> Timothy J. Massey wrote:
> >It seems to me, then, that the documentation is *wrong*: rsync does not
> >compare against the pool, *ever*; only against a previous backup (most
> >likely the next-highest backup level, but I have yet to find this
> >information yet).
> It is not wrong if you take into account the fact that links are just
> different names for the same file.
I think Timothy meant that in respect to *transfers* of the file. Even files
that are transfered and turn out to be already in the pool are then replaced
with a hardlink to the pool file. More explicitly: the duplicate copy with link
count 1 is first created, then detected as being a copy, deleted, and finally
replaced with a hardlink (making the same content visible under the same
name again).
And that's probably only an oversimplified (and strictly incorrect) way of
imagining the process, because BackupPC::Xfer::RsyncFileIO uses
BackupPC::PoolWrite too (which I have not looked at, but which Timothy
explained as creating the file as a new file *only if* it is not already
in the pool and as a hardlink otherwise).
So, apparently:
- rsync only takes a previous backup into consideration, not the complete
pool, for determining which files need to be transfered and how to speed
up the transfer,
- with rsync, the same logic about creating transfered files on the BackupPC
pool filesystem only if they not already exist is applied as with tar/smb.
Timothy:
> (most likely the next-highest backup level, but I have yet to find this
> information yet).
>From BackupPC-3.0.0beta3/ChangeLog:
> * Added support for multi-level incrementals. In the style of dump(1),
> the level of each incremental can be specified. Each incremental
> backups up everything since the most recent backup of a lower level
> (fulls are always level 0). Previous behavior was all incrementals
> were level 1, meaning they backed up everything since the last full
> (level 0). Default configuration is all incrementals are level 1.
> > One final consideration: does it make much sense to backup the complete
> > operating system of multiple similar servers?
>
> That was merely a fictional example. However, my answer to backup is
> very, very simple: back up everything, period. Even if you *know* you
> won't use it. Even if you *know* it's redundant. Disk space is cheap,
> forgetting that *one* file you misfiled but now vitally need is not.
Very, very true.
Regards,
Holger
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/
