Hi,
Markus Hirschmann wrote on 2016-10-10 17:54:45 +0200 [[BackupPC-users] BackupPC
hangs]:
>
> I backup a huge QNAS Storage with rsync over ssh and it always hangs
> with 100% load. I tried to find the problem with lsof:
>
> [...]
> BackupPC_ 1842 backuppc 6r REG 253,3 30938 1330743
> /var/lib/backuppc/cpool/d/6/d/d6dfb50e7648054a9c2f83537f956c2c_185
185 ... wow ...
> BackupPC_ 1842 backuppc 7r REG 253,3 77320 419524
> /var/lib/backuppc/cpool/d/6/d/d6dfb50e7648054a9c2f83537f956c2c_0
> BackupPC_ 1842 backuppc 9r REG 253,3 171056 419525
> /var/lib/backuppc/cpool/d/6/d/d6dfb50e7648054a9c2f83537f956c2c_1
> BackupPC_ 1842 backuppc 11r REG 253,3 72265 419526
> /var/lib/backuppc/cpool/d/6/d/d6dfb50e7648054a9c2f83537f956c2c_2
[and so on]
You've got a rather long hash chain here (185 candidates for a pool match).
For the current file in question, BackupPC needs to compare the contents with
all candidates (or at least those that haven't been ruled out yet ... see
the missing numbers further down:
> BackupPC_ 1842 backuppc 27r REG 253,3 100064 419572
> /var/lib/backuppc/cpool/d/6/d/d6dfb50e7648054a9c2f83537f956c2c_18
> BackupPC_ 1842 backuppc 28r REG 253,3 113544 419587
> /var/lib/backuppc/cpool/d/6/d/d6dfb50e7648054a9c2f83537f956c2c_33
). I'm guessing the file is rather large, so the comparison takes quite long.
Note also that BackupPC won't take all candidates into account at once, due
to the limit on concurrently open files, though I'm not sure what the strategy
is (it might mean the comparison needs to be repeated).
Hash collisions happen (mainly) for files with identical *length* and identical
first and last 128 KiB chunk within the first 1 MiB of data (if I remember
correctly; the details aren't important, so I won't check). Maybe you're
backing up a large database with changes just in the wrong places? Log files
would tend to grow, i.e. change length (and therefore hash), so they usually
won't cause trouble in form of hash chains.
Hint: 'ls -ali /var/lib/backuppc/cpool/d/6/d/d6dfb50e7648054a9c2f83537f956c2c_0'
(or any other file in the chain) for a quick glance at (compressed) length and
number of links (an extremely large number of copies would also tend to make
hash chains longer than they need to be as well as make comparisons tedious -
your $Conf{HardLinkMax} isn't by chance set to a ridiculously small value, is
it?). You might also want to 'BackupPC_zcat !$ | wc -c' (please expand the
!$ ;-) for an exact uncompressed file size. The inode number is included in
the ls example, in case you feel like doing a 'find $TopDir/pc -inum ...' for
locating the file(s) in question.
> [...] I should delete my pool and start new but I don't want to do that. :/
I'm not sure that would help much. If your backup strategy continues to backup
this content, you'll probably run into the same situation again in the future.
If you can and want to remove the file from (some) previous backups, search
the list for ... err ... something like BackupPC_delete - a script to safely
delete individual files from backups. BackupPC_nightly should take care of
cleaning up (i.e. chain renumbering), *BUT* *PLEASE don't run it manually*!
Of course, if you don't need to backup the file, simply excluding it would
also fix things, as the comparison wouldn't happen in the future.
Hope that helps.
Regards,
Holger
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
