Hi all -
This issue is closed.
Craig threw together his neurons and solved this super quick. He's
posted his fixes to Github.
rsync_bpc has become very memory efficient and BackupPC has backed up
well over 1 million new files with zero problems with his fixes. Thank
you Craig!
I deal with CPAs and several are moving to eCase which produces a
gazillion files. By moving away from hard links I can also move away
from block-level filesystem replication. Simplification, Yea!
Now comes the restore test.
If all goes well, dare I say this might bump the version?
Once again, thank you Craig!
Cheers!
mph
On 2017-02-11 00:41, Craig Barratt wrote:
> Another thing to try is to remove the --bpc-attrib-new option to rsync_bpc by
> commenting out these two lines in lib/BackupPC/Xfer/Rsync.pm, ie:
>
>> #'--bpc-attrib-new',
> Craig
>
> On Sat, Feb 11, 2017 at 12:20 AM, Craig Barratt
> <[email protected]> wrote:
>
> Definitely looks like a bug. Let me see if I can reproduce this.
> Interesting clue about the strange attrib paths. Does that only happen right
> before things fail?
>
> It would be helpful to run with a higher XferLogLevel (eg: 8). Please send
> the interesting (last) parts of the log file (email direct to me).
>
> Also, what is your remote rsync version? Is it running over ssh?
>
> Craig
>
> On Fri, Feb 10, 2017 at 10:29 PM, Michael Huntley <[email protected]>
> wrote:
>
> All -
>
> Been attempting to backup using latest grab from github made using the
> makeDist command. This is very similar to the issue listed on Github where
> the backup dies with code (12) after X number of files. Usually near 200k
> files, but as low as 50K from time-to-time:
>
> R bpc_sysCall_poolFileCheck(xaaaaaacsrt): no pool file at
> /var/lib/backuppc/cpool/c6/64/c765d203b57e0e748b90564c87401bdb
> R bpc_mkstemp: returning .xaaaaaacsrt.000000, fd = 3
> R bpc_write(3 (.xaaaaaacsrt.000000), buf, 100)
> R bpc_ftruncate(3 (.xaaaaaacsrt.000000), 564632)
> R bpc_close(3 (.xaaaaaacsrt.000000))
> rsync_bpc: connection unexpectedly closed (4608490 bytes received so far)
> [generator]
> G bpc_sysCall_cleanup: doneInit = 1
> G No match... adding /var/lib/backuppc/cpool/18779.0.1 to pool (digest =
> b6cc8c58c4da233e62efd1e8b3a1b160)
> G bpc_attrib_dirWrite: created new attrib file
> /var/lib/backuppc/pc/172.26.26.13/508//attrib_b6cc8c58c4da233e62efd1e8b3a1b160
> [1]
> G bpc_attrib_dirWrite: removed old attrib file
> /var/lib/backuppc/pc/172.26.26.13/508//attrib_39a7abfe0817669e521a85ad3188a923
> [2]
> G No match... adding /var/lib/backuppc/cpool/18779.1.1 to pool (digest =
> fceb91a6bba1514df057bd51c9fdc10a)
> G bpc_attrib_dirWrite: created new attrib file
> /var/lib/backuppc/pc/172.26.26.13/508/f%2fcygdrive%2fc%2fshadows%2fE%2ftest/attrib_fceb91a6bba1514df057bd51c9fdc10a
> [3]
> DoneGen: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 47782
> filesTotal, 24778100 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 83992
> inode
> Parsing done: nFilesTotal = 47782
> rsync error: error in rsync protocol data stream (code 12) at io.c(629)
> [generator=3.0.9.5]
> rsync_bpc exited with fatal status 12 (3072) (rsync error: error in rsync
> protocol data stream (code 12) at io.c(629) [generator=3.0.9.5])
> recv >f+++++++++ rw-rw-r-- 197108, 197121 100 xaaaaaacsrt
> Xfer PIDs are now
> Got fatal error during xfer (rsync error: error in rsync protocol data stream
> (code 12) at io.c(629) [generator=3.0.9.5])
> Backup aborted (rsync error: error in rsync protocol data stream (code 12) at
> io.c(629) [generator=3.0.9.5])
> BackupFailCleanup: nFilesTotal = 53691, type = full, BackupCase = 6, inPlace
> = 1, lastBkupNum =
> BackupFailCleanup: inPlace with some new files... no cleanup and marking
> partial
>
> I updated the rsync_bpc version manually to 3.0.9.5 since the configure.ac
> [4] file is still at 3.0.9.4 within the files made by makeDist. Nontheless
> the files are using the current versions updated by Craig.
>
> This last one horked at 53k files, but it usually dies near 200k files. It
> is a quick cut off. Nothing hangs, the door simply slams shut.
>
> One oddity I noticed is not all attrib files are properly logged. Look
> above:
>
> G No match... adding /var/lib/backuppc/cpool/18779.0.1 to pool (digest =
> b6cc8c58c4da233e62efd1e8b3a1b160)
> G bpc_attrib_dirWrite: created new attrib file
> /var/lib/backuppc/pc/172.26.26.13/508//attrib_b6cc8c58c4da233e62efd1e8b3a1b160
> [1] <-- double whack, not full path
>
> vs.
>
> G No match... adding /var/lib/backuppc/cpool/18779.1.1 to pool (digest =
> fceb91a6bba1514df057bd51c9fdc10a)
> G bpc_attrib_dirWrite: created new attrib file
> /var/lib/backuppc/pc/172.26.26.13/508/f%2fcygdrive%2fc%2fshadows%2fE%2ftest/attrib_fceb91a6bba1514df057bd51c9fdc10a
> [3] <-- Full path
>
> In between is an isolated attrib with no corresponding "No match... adding"
> line:
>
> G bpc_attrib_dirWrite: removed old attrib file
> /var/lib/backuppc/pc/172.26.26.13/508//attrib_39a7abfe0817669e521a85ad3188a923
> [2] <-- Again, double whack, not full path
>
> The real bummer is determining if this is a (pre-compile) configure issue, a
> build issue, resource issue, a command issue, or ........
>
> Cheers!
>
> mph
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> BackupPC-users mailing list
> [email protected]
> List: https://lists.sourceforge.net/lists/listinfo/backuppc-users [5]
> Wiki: http://backuppc.wiki.sourceforge.net [6]
> Project: http://backuppc.sourceforge.net/ [7]
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
BackupPC-users mailing list
[email protected]
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Links:
------
[1] http://172.26.26.13/508//attrib_b6cc8c58c4da233e62efd1e8b3a1b160
[2] http://172.26.26.13/508//attrib_39a7abfe0817669e521a85ad3188a923
[3]
http://172.26.26.13/508/f%2fcygdrive%2fc%2fshadows%2fE%2ftest/attrib_fceb91a6bba1514df057bd51c9fdc10a
[4] http://configure.ac
[5] https://lists.sourceforge.net/lists/listinfo/backuppc-users
[6] http://backuppc.wiki.sourceforge.net
[7] http://backuppc.sourceforge.net/
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
BackupPC-users mailing list
[email protected]
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
