Re: [BackupPC-users] Real Time View of What Is Being Copied
I use: strace -e open,openat $(ps -o lwp= -LC rsync_bpc | sed 's/^/-p/') 2>&1 | less George King wrote at about 19:54:40 + on Tuesday, March 8, 2022: > Just going back to this, as the OP's use case is backing up a Windows > machine, the strace suggestion has potential but I've I've tried it and > after > > strace: Process 12671 attached > > nothing happens although the Windows Resource Monitor does show file > access. Any other way to show the file access from the backuppc server, > or am I missing something? Thanks. This would be useful because I have > had the same problem as the OP and when the backup hangs, it requires > manual intervention to stop the backup and the log files seem to vanish > long with that backup number. > > George > > On 03/03/2022 15:19, Greg Harris wrote: > > Somewhere in the archives in the past two years or so, you’ll find a > > thread discussing this. From my notes, here’s a few things that might > > help the search process: > > > > ls -l /proc//fd (Run on machine being backed up) > > > > strace -e open -p (Requires installing strace, run > > from server) > > > > sudo watch -n0.1 'lsof -c rsync | grep 3r' (Requires installing > > lsof, run on machine being backed up) > > > > Thanks, > > > > Greg Harris > > > >> On Mar 3, 2022, at 9:27 AM, Micha Kersloot wrote: > >> > >> Hi, > >> > >> you could try to split the backup in several parts? > >> > >> Met vriendelijke groet, > >> > >> Micha Kersloot > >> <https://www.kovoks.nl/> > >> > >> <https://www.kovoks.nl/e-mail> > >> www.kovoks.nl <https://www.kovoks.nl/>|www.zimbra-nederland.nl > >> <https://www.zimbra-nederland.nl/>| Volg@kovoks > >> <http://twitter.com/kovoks>op Twitter > >> *E*mi...@kovoks.nl > >> *T*+31 (0)345 - 53 29 27/0345 - 53 29 27 > >> *A*Belle van Zuylenlaan 1-5 <https://goo.gl/maps/MrqmTv442tH2>|4105 > >> JX Culemborg <https://goo.gl/maps/MrqmTv442tH2> > >> > >> KvK 1104 > >> > >> > >> > >> > >> > >> > >> *Van:*"Akibu Flash" > >> *Aan:*"General list for user discussion, questions and support" > >> > >> *Verzonden:*Donderdag 3 maart 2022 15:23:23 > >> *Onderwerp:*Re: [BackupPC-users] Real Time View of What Is Being > >> Copied > >> > >> Just bumping this up… any suggestions for a resolution? Thanks. > >> > >> *From:*Akibu Flash > >> *Sent:*Monday, February 28, 2022 4:32 PM > >> *To:*BackupPC-users@lists.sourceforge.net > >> *Subject:*[BackupPC-users] Real Time View of What Is Being Copied > >> > >> Dear All, > >> > >> I backup a couple of Windows machines to a linux machine via > >> rsync. However, the backup for one of the Windows machines never > >> finishes and results in a partial backup. In the CGI user > >> interface Count column, the backup always seems to hang at 311941 > >> for hours and hours. Then at some point, the backup apparently > >> stops and afterwards another backup which by that time has queued > >> up starts anew. Never do I get a completed backup only a partial > >> (at least for the last year or so). > >> > >> Is there a way to start a backup from the command line and have > >> the screen show what in real time is actually being backed up? > >> If so, what would that command be? It may be that the BackupPC > >> is somehow getting stuck on a file but that is only speculation. > >> > >> Any other ideas as to what my be happening? > >> > >> I decompressed the partial backup log file (backup #884) using > >> the command BackupPC_zcat XferLOG.884.z. I have reproduced the > >> bottom portion below. > >> > >> Any assistance in helping to track down what is happening is > >> appreciated. I do see in the log that there is an error message > >> indicating an “error in the rsync protocol data stream (code 12) > >> at io.c(226). > >> > >> Thanks in advance. > >> > >>
Re: [BackupPC-users] Real Time View of What Is Being Copied
Just going back to this, as the OP's use case is backing up a Windows
machine, the strace suggestion has potential but I've I've tried it and
after
strace: Process 12671 attached
nothing happens although the Windows Resource Monitor does show file
access. Any other way to show the file access from the backuppc server,
or am I missing something? Thanks. This would be useful because I have
had the same problem as the OP and when the backup hangs, it requires
manual intervention to stop the backup and the log files seem to vanish
long with that backup number.
George
On 03/03/2022 15:19, Greg Harris wrote:
Somewhere in the archives in the past two years or so, you’ll find a
thread discussing this. From my notes, here’s a few things that might
help the search process:
ls -l /proc//fd (Run on machine being backed up)
strace -e open -p (Requires installing strace, run
from server)
sudo watch -n0.1 'lsof -c rsync | grep 3r' (Requires installing
lsof, run on machine being backed up)
Thanks,
Greg Harris
On Mar 3, 2022, at 9:27 AM, Micha Kersloot wrote:
Hi,
you could try to split the backup in several parts?
Met vriendelijke groet,
Micha Kersloot
<https://www.kovoks.nl/>
<https://www.kovoks.nl/e-mail>
www.kovoks.nl <https://www.kovoks.nl/>|www.zimbra-nederland.nl
<https://www.zimbra-nederland.nl/>| Volg@kovoks
<http://twitter.com/kovoks>op Twitter
*E*mi...@kovoks.nl
*T*+31 (0)345 - 53 29 27/0345 - 53 29 27
*A*Belle van Zuylenlaan 1-5 <https://goo.gl/maps/MrqmTv442tH2>|4105
JX Culemborg <https://goo.gl/maps/MrqmTv442tH2>
KvK 1104
*Van:*"Akibu Flash"
*Aan:*"General list for user discussion, questions and support"
*Verzonden:*Donderdag 3 maart 2022 15:23:23
*Onderwerp:*Re: [BackupPC-users] Real Time View of What Is Being
Copied
Just bumping this up… any suggestions for a resolution? Thanks.
*From:*Akibu Flash
*Sent:*Monday, February 28, 2022 4:32 PM
*To:*BackupPC-users@lists.sourceforge.net
*Subject:*[BackupPC-users] Real Time View of What Is Being Copied
Dear All,
I backup a couple of Windows machines to a linux machine via
rsync. However, the backup for one of the Windows machines never
finishes and results in a partial backup. In the CGI user
interface Count column, the backup always seems to hang at 311941
for hours and hours. Then at some point, the backup apparently
stops and afterwards another backup which by that time has queued
up starts anew. Never do I get a completed backup only a partial
(at least for the last year or so).
Is there a way to start a backup from the command line and have
the screen show what in real time is actually being backed up?
If so, what would that command be? It may be that the BackupPC
is somehow getting stuck on a file but that is only speculation.
Any other ideas as to what my be happening?
I decompressed the partial backup log file (backup #884) using
the command BackupPC_zcat XferLOG.884.z. I have reproduced the
bottom portion below.
Any assistance in helping to track down what is happening is
appreciated. I do see in the log that there is an error message
indicating an “error in the rsync protocol data stream (code 12)
at io.c(226).
Thanks in advance.
Best, Akibu
G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo.gif,
0x0, 00) -> 3
G bpc_read(3
(Downloads/Downloads/BackupPC-4.3.2/images/logo.gif), buf, 1394)
tmpFd = -1
G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif))
G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL
G bpc_lstat(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png)
G
bpc_file_checksum(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png)
G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png,
0x0, 00) -> 3
G bpc_read(3
(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png), buf,
7814) tmpFd = -1
G bpc_close(3
(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png))
G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL
rsync_bpc: connection unexpectedly closed (23961310 bytes
received so far) [receiver]
R bpc_sysCall_cleanup: doneInit = 1
Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0
filesTotal, 0 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp,
24467705 inode
Parsing done: nFilesTotal = 0
rsync error: error in rsync protocol data stream (code 12) at
io.c(226) [receiver=3.1.3.0]
rsync_bpc: connection unexpectedly closed (24012481 bytes
received so far) [generator]
G bpc_sysCall_cleanup: doneInit = 1
DoneGen: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp,
311942 filesTotal, 5694
Re: [BackupPC-users] Real Time View of What Is Being Copied
On Mon, 2022-03-07 at 13:26 +, G.W. Haywood via BackupPC-users wrote: > > > One common legitimate use case for sparse files is virtual disk images. > > Ah, good point! But nobody in his right mind would use BackupPC > to back them up directly as files, would they? I used to do that until I found out that the Veeam Backup Agent for linux was free and a stand-alone product. My Veeam setup does a daily bare metal backup, while my BackupPC setup does a daily backup of the everchanging smaller stuff in /etc/, /home and such. Stuff that I need to get back fast should the need arise. The folder containing my virtualbox disk images are however excluded for practical reasons. -- Kind regards, Sorin Srbu Find my OpenPGP public key here: https://cloud.srbu.se/index.php/s/KeEsCCDsG7PZG7N signature.asc Description: This is a digitally signed message part ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Real Time View of What Is Being Copied
Hi there, On Mon, 7 Mar 2022, Dave Sherohman wrote: Re: Real Time View of What Is Being Copied On 3/5/22 14:36, G.W. Haywood via BackupPC-users wrote: On Sat, 5 Mar 2022, Les Mikesell wrote: Unix/Linux has something calle 'sparse' files ... I can't remember the last time I saw a sparse file used ... One common legitimate use case for sparse files is virtual disk images. Ah, good point! But nobody in his right mind would use BackupPC to back them up directly as files, would they? -- 73, Ged. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Real Time View of What Is Being Copied
On 3/5/22 14:36, G.W. Haywood via BackupPC-users wrote: On Sat, 5 Mar 2022, Les Mikesell wrote: Unix/Linux has something calle 'sparse' files used by some types of databases where you can seek far into a file and write without using/allocating any space up to that point. The file as stored may not be large but most tools to copy it will act as though the empty parts were filled with nulls. I can't remember the last time I saw a sparse file used *anywhere* in 'real life', although they are occasionally found in malicious mail. One common legitimate use case for sparse files is virtual disk images. I run a fair number of virtual machines at work (I think we're currently at around 90 of them) and sparse files allow me to have a VM with a "1 TB" virtual disk that only takes up a few dozen GB of real disk space and grows as needed to hold additional data. As Les said, copying these files (with, e.g., `cp`) takes time proportional to their virtual size, not their physical size, even though the smaller physical size is preserved by the operation. Don't know whether this applies to rsync, though. Might need to try that next time I need to duplicate one, just to see whether rsync handles sparse files smarter/faster than other tools. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Real Time View of What Is Being Copied
Hi there, On Sat, 5 Mar 2022, Les Mikesell wrote: On Fri, Mar 4, 2022 G.W. Haywood via BackupPC-users wrote: > Another possibility (and the only time I've ever seen this happen) is > that there might be a large file which is causing a timeout. It was > on a Linux box here, something went off the reservation and started > writing gigabytes of messages to a file called '.xsession-errors'. Unix/Linux has something calle 'sparse' files used by some types of databases where you can seek far into a file and write without using/allocating any space up to that point. The file as stored may not be large but most tools to copy it will act as though the empty parts were filled with nulls. Rsync might handle them these days but may still take the time to send the stream of nulls. But in any case they are rarely used on Windows. I can't remember the last time I saw a sparse file used *anywhere* in 'real life', although they are occasionally found in malicious mail. Apparently they're available on NTFS: https://docs.microsoft.com/en-us/windows/win32/fileio/sparse-files so it might be another line worth investigating - but in any case, that .xsession-errors file was just one great big ordinary file full of garbage text messages about X. IIRC most of it was the same 30 or 40 lines repeated ad nauseam - and every second line said exactly the same thing. :/ -- 73, Ged. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Real Time View of What Is Being Copied
On Fri, Mar 4, 2022 at 2:39 PM Kenneth Porter wrote: > > rsync over the network should use compression. If you're doing it inside > ssh, then use ssh's compression, which is more reliable. The run of > nulls will be compressed down to a count and one null. Their transfer > will be almost instantaneous. > Yes, but it still goes through the motions of 'reading' what could potentially be many TB of data even from a small file with the filesystem delivering all those nulls. It has been a long time since I had that problem but it did in fact cause trouble at least with the rsync back then and slower CPUs. It could be that rsync tried to write them all at the other end too. -- Les Mikesell lesmikes...@gmail.com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Real Time View of What Is Being Copied
On 3/4/2022 7:21 AM, Les Mikesell wrote: Unix/Linux has something calle 'sparse' files used by some types of databases where you can seek far into a file and write without using/allocating any space up to that point. The file as stored may not be large but most tools to copy it will act as though the empty parts were filled with nulls. Rsync might handle them these days but may still take the time to send the stream of nulls. But in any case they are rarely used on Windows. rsync over the network should use compression. If you're doing it inside ssh, then use ssh's compression, which is more reliable. The run of nulls will be compressed down to a count and one null. Their transfer will be almost instantaneous. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Real Time View of What Is Being Copied
On Fri, Mar 4, 2022 at 7:16 AM G.W. Haywood via BackupPC-users wrote: > > Another possibility (and the only time I've ever seen this happen) is > that there might be a large file which is causing a timeout. It was > on a Linux box here, something went off the reservation and started > writing gigabytes of messages to a file called '.xsession-errors'. Unix/Linux has something calle 'sparse' files used by some types of databases where you can seek far into a file and write without using/allocating any space up to that point. The file as stored may not be large but most tools to copy it will act as though the empty parts were filled with nulls. Rsync might handle them these days but may still take the time to send the stream of nulls. But in any case they are rarely used on Windows. -- Les Mikesell lesmikes...@gmail.com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Real Time View of What Is Being Copied
Hi there, On Fri, 4 Mar 2022, Les Mikesell wrote: On Mon, Feb 28, 2022 at 3:34 PM Akibu Flash wrote: > > I backup a couple of Windows machines to a linux machine via rsync. > However, the backup for one of the Windows machines never finishes > and results in a partial backup. In the CGI user interface Count > column, the backup always seems to hang at 311941 for hours ... > > Is there a way to start a backup from the command line and have the > screen show what in real time is actually being backed up? Not exactly what you are asking for, but since the problem target is windows you are probably hitting a file that windows has locked, and the solution might be to use windows VSS snapshots. Another possibility (and the only time I've ever seen this happen) is that there might be a large file which is causing a timeout. It was on a Linux box here, something went off the reservation and started writing gigabytes of messages to a file called '.xsession-errors'. On Linux, by default, filenames beginning with a dot aren't displayed by the command 'ls -l' so it took longer to spot than it should have. :/ -- 73, Ged. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Real Time View of What Is Being Copied
On Mon, Feb 28, 2022 at 3:34 PM Akibu Flash wrote: > > > I backup a couple of Windows machines to a linux machine via rsync. However, > the backup for one of the Windows machines never finishes and results in a > partial backup. In the CGI user interface Count column, the backup always > seems to hang at 311941 for hours and hours. Then at some point, the backup > apparently stops and afterwards another backup which by that time has queued > up starts anew. Never do I get a completed backup only a partial (at least > for the last year or so). > > > > Is there a way to start a backup from the command line and have the screen > show what in real time is actually being backed up? Not exactly what you are asking for, but since the problem target is windows you are probably hitting a file that windows has locked, and the solution might be to use windows VSS snapshots. You can search the mail list for other discussions but there is a link here to a tool to do that. https://sourceforge.net/p/backuppc/mailman/message/37189039/ -- Les Mikesell lesmikes...@gmail.com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Real Time View of What Is Being Copied
Hi Akibu
One of two ways.
1. On the Win10 machine: set up a new shares with paths describing
parts of the drive e.g. d:\Documents and d:\not_Documents rather than
just d:\ And on backuppc create new hosts with the new share names
under 'RsyncShareName'.
2. Or leave the Win10 share the same. In backuppc duplicate the
relevant host (and give it a different but similar name). Edit one host
Xfer settings so 'BackupFilesOnly' refers to /Documents and in the other
host 'BackupFilesExclude' also refers to /Documents.
If you suspect one folder on the Windows machine is where it always
stalls (using the Resource Monitor approach I described) - try excluding
that folder and seeing if it runs to completion. Narrow the exclusion
until you have worked out what's stopping it. Mine consistent stopped
in one folder - I still don't know why. They were just tiff files and
opened fine on the native machine so I don't think there was corruption.
George
On 03/03/2022 16:52, Akibu Flash wrote:
Thanks George for responding.
With respect to your reference to: “split the host into two”, how does
one actually do that? My host references a certain computer
consisting of various drives to be backed up. How does one split that
host into two separate hosts and (presumably) back up each of those
hosts? Thanks.
Akibu
*From:* George King
*Sent:* Thursday, March 3, 2022 10:44 AM
*To:* backuppc-users@lists.sourceforge.net
*Subject:* Re: [BackupPC-users] Real Time View of What Is Being Copied
No great suggestions for resolution from me but I have some sympathy
with you having had similar issues occasionally in the past.
I have a number of rsyncd hosts set up on my main Win10 desktop -
basically one for each disk or part of to keep the backups down to a
reasonable duration. I'm running DeltaCopy Server.
When one of my backups 'stalled' after having worked for a while, it
would do as you have found and stall at the same place each time.
Other disks/hosts were not affected and completed fine, so it was
nothing intrinsically wrong with DeltaCopy or backuppc. I could
monitor the file transfer via the Win10 Task Manager>Performance Tab >
Resource Monitor > Disk tab and selecting the right rsync process for
the problematic disk. At least that gave me confidence that the
transfer was starting properly and progressing - and I could tell when
it properly stuck, and at which folder on the Win10 machine.
I couldn't find any informative messages in the backuppc logs to help
me solve it. When I manually stopped the backups, the partial backups
were generally removed or merged and the logs just showed that I had
aborted the xfer.
What seems to have helped overcome the problem is, although I fail to
understand quite why, are:
- splitting the host into two, and
- copying the files from the folder at which the transfer stalled into
another folder, and
- I probably checked the Win10 folder permissions were OK and
inherited by sub-folders.
George
On 03/03/2022 14:23, Akibu Flash wrote:
Just bumping this up… any suggestions for a resolution? Thanks.
*From:* Akibu Flash
<mailto:akibufl...@outlook.com>
*Sent:* Monday, February 28, 2022 4:32 PM
*To:* BackupPC-users@lists.sourceforge.net
*Subject:* [BackupPC-users] Real Time View of What Is Being Copied
Dear All,
I backup a couple of Windows machines to a linux machine via
rsync. However, the backup for one of the Windows machines never
finishes and results in a partial backup. In the CGI user
interface Count column, the backup always seems to hang at 311941
for hours and hours. Then at some point, the backup apparently
stops and afterwards another backup which by that time has queued
up starts anew. Never do I get a completed backup only a partial
(at least for the last year or so).
Is there a way to start a backup from the command line and have
the screen show what in real time is actually being backed up? If
so, what would that command be? It may be that the BackupPC is
somehow getting stuck on a file but that is only speculation.
Any other ideas as to what my be happening?
I decompressed the partial backup log file (backup #884) using the
command BackupPC_zcat XferLOG.884.z. I have reproduced the bottom
portion below.
Any assistance in helping to track down what is happening is
appreciated. I do see in the log that there is an error message
indicating an “error in the rsync protocol data stream (code 12)
at io.c(226).
Thanks in advance.
Best, Akibu
G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo.gif,
0x0, 00) -> 3
G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif),
buf, 1394) tmpFd = -1
G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif))
G bpc_fileDescF
Re: [BackupPC-users] Real Time View of What Is Being Copied
Thanks George for responding. With respect to your reference to: “split the host into two”, how does one actually do that? My host references a certain computer consisting of various drives to be backed up. How does one split that host into two separate hosts and (presumably) back up each of those hosts? Thanks. Akibu From: George King Sent: Thursday, March 3, 2022 10:44 AM To: backuppc-users@lists.sourceforge.net Subject: Re: [BackupPC-users] Real Time View of What Is Being Copied No great suggestions for resolution from me but I have some sympathy with you having had similar issues occasionally in the past. I have a number of rsyncd hosts set up on my main Win10 desktop - basically one for each disk or part of to keep the backups down to a reasonable duration. I'm running DeltaCopy Server. When one of my backups 'stalled' after having worked for a while, it would do as you have found and stall at the same place each time. Other disks/hosts were not affected and completed fine, so it was nothing intrinsically wrong with DeltaCopy or backuppc. I could monitor the file transfer via the Win10 Task Manager>Performance Tab > Resource Monitor > Disk tab and selecting the right rsync process for the problematic disk. At least that gave me confidence that the transfer was starting properly and progressing - and I could tell when it properly stuck, and at which folder on the Win10 machine. I couldn't find any informative messages in the backuppc logs to help me solve it. When I manually stopped the backups, the partial backups were generally removed or merged and the logs just showed that I had aborted the xfer. What seems to have helped overcome the problem is, although I fail to understand quite why, are: - splitting the host into two, and - copying the files from the folder at which the transfer stalled into another folder, and - I probably checked the Win10 folder permissions were OK and inherited by sub-folders. George On 03/03/2022 14:23, Akibu Flash wrote: Just bumping this up… any suggestions for a resolution? Thanks. From: Akibu Flash <mailto:akibufl...@outlook.com> Sent: Monday, February 28, 2022 4:32 PM To: BackupPC-users@lists.sourceforge.net<mailto:BackupPC-users@lists.sourceforge.net> Subject: [BackupPC-users] Real Time View of What Is Being Copied Dear All, I backup a couple of Windows machines to a linux machine via rsync. However, the backup for one of the Windows machines never finishes and results in a partial backup. In the CGI user interface Count column, the backup always seems to hang at 311941 for hours and hours. Then at some point, the backup apparently stops and afterwards another backup which by that time has queued up starts anew. Never do I get a completed backup only a partial (at least for the last year or so). Is there a way to start a backup from the command line and have the screen show what in real time is actually being backed up? If so, what would that command be? It may be that the BackupPC is somehow getting stuck on a file but that is only speculation. Any other ideas as to what my be happening? I decompressed the partial backup log file (backup #884) using the command BackupPC_zcat XferLOG.884.z. I have reproduced the bottom portion below. Any assistance in helping to track down what is happening is appreciated. I do see in the log that there is an error message indicating an “error in the rsync protocol data stream (code 12) at io.c(226). Thanks in advance. Best, Akibu G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo.gif, 0x0, 00) -> 3 G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif), buf, 1394) tmpFd = -1 G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif)) G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL G bpc_lstat(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) G bpc_file_checksum(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png, 0x0, 00) -> 3 G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png), buf, 7814) tmpFd = -1 G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png)) G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL rsync_bpc: connection unexpectedly closed (23961310 bytes received so far) [receiver] R bpc_sysCall_cleanup: doneInit = 1 Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 24467705 inode Parsing done: nFilesTotal = 0 rsync error: error in rsync protocol data stream (code 12) at io.c(226) [receiver=3.1.3.0] rsync_bpc: connection unexpectedly closed (24012481 bytes received so far) [generator] G bpc_sysCall_cleanup: doneInit = 1 DoneGen: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 311942 filesTotal, 569449707538 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 24467704
Re: [BackupPC-users] Real Time View of What Is Being Copied
Hi, Yes you could create several rsyncsharename sections for the different folders. Or you could explicit exclude some folders from 1 share and create a seperate share to create a backup. You can even create multiple 'hosts' with each it's own share to backup different shares. In that case you could use the ClientNameAlias to point them to the same server. Met vriendelijke groet, Micha Kersloot [ https://www.kovoks.nl/ ] [ https://www.kovoks.nl/e-mail ] [ https://www.kovoks.nl/ | www.kovoks.nl ] | [ https://www.zimbra-nederland.nl/ | www.zimbra-nederland.nl ] | Volg [ http://twitter.com/kovoks | @kovoks ] op Twitter E [ mailto:mi...@kovoks.nl | mi...@kovoks.nl ] T +31 (0)345 - 53 29 27 / 0345 - 53 29 27 A [ https://goo.gl/maps/MrqmTv442tH2 | Belle van Zuylenlaan 1-5 ] | [ https://goo.gl/maps/MrqmTv442tH2 | 4105 JX Culemborg ] KvK 1104 > Van: "Akibu Flash" > Aan: "KovoKs" , "General list for user discussion, questions > and > support" > Verzonden: Donderdag 3 maart 2022 17:47:53 > Onderwerp: RE: Real Time View of What Is Being Copied > Thanks Micha. > This may be a silly question, but how does one split the backup into several > parts? Does that involve designating certain folders from a particular drive > for backup instead of the entire drive? > From: Micha Kersloot > Sent: Thursday, March 3, 2022 9:28 AM > To: General list for user discussion, questions and support > > Subject: Re: [BackupPC-users] Real Time View of What Is Being Copied > Hi, > you could try to split the backup in several parts? > Met vriendelijke groet, > Micha Kersloot > [ https://www.kovoks.nl/ ] > [ https://www.kovoks.nl/e-mail ] > [ https://www.kovoks.nl/ | www.kovoks.nl ] | [ > https://www.zimbra-nederland.nl/ > | www.zimbra-nederland.nl ] | Volg [ http://twitter.com/kovoks | @kovoks ] op > Twitter > E [ mailto:mi...@kovoks.nl | mi...@kovoks.nl ] > T +31 (0)345 - 53 29 27 / 0345 - 53 29 27 > A [ https://goo.gl/maps/MrqmTv442tH2 | Belle van Zuylenlaan 1-5 ] | [ > https://goo.gl/maps/MrqmTv442tH2 | 4105 JX Culemborg ] > KvK 1104 >> Van: "Akibu Flash" < [ mailto:akibufl...@outlook.com | >> akibufl...@outlook.com ] >> > >> Aan: "General list for user discussion, questions and support" < [ >> mailto:backuppc-users@lists.sourceforge.net | >> backuppc-users@lists.sourceforge.net ] > >> Verzonden: Donderdag 3 maart 2022 15:23:23 >> Onderwerp: Re: [BackupPC-users] Real Time View of What Is Being Copied >> Just bumping this up… any suggestions for a resolution? Thanks. >> From: Akibu Flash < [ mailto:akibufl...@outlook.com | akibufl...@outlook.com >> ] > >> Sent: Monday, February 28, 2022 4:32 PM >> To: [ mailto:BackupPC-users@lists.sourceforge.net | >> BackupPC-users@lists.sourceforge.net ] >> Subject: [BackupPC-users] Real Time View of What Is Being Copied >> Dear All, >> I backup a couple of Windows machines to a linux machine via rsync. However, >> the >> backup for one of the Windows machines never finishes and results in a >> partial >> backup. In the CGI user interface Count column, the backup always seems to >> hang >> at 311941 for hours and hours. Then at some point, the backup apparently >> stops >> and afterwards another backup which by that time has queued up starts anew. >> Never do I get a completed backup only a partial (at least for the last year >> or >> so). >> Is there a way to start a backup from the command line and have the screen >> show >> what in real time is actually being backed up? If so, what would that command >> be? It may be that the BackupPC is somehow getting stuck on a file but that >> is >> only speculation. >> Any other ideas as to what my be happening? >> I decompressed the partial backup log file (backup #884) using the command >> BackupPC_zcat XferLOG.884.z. I have reproduced the bottom portion below. >> Any assistance in helping to track down what is happening is appreciated. I >> do >> see in the log that there is an error message indicating an “error in the >> rsync >> protocol data stream (code 12) at io.c(226). >> Thanks in advance. >> Best, Akibu >> G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo.gif, 0x0, 00) -> 3 >> G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif), buf, 1394) >> tmpFd = -1 >> G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif)) >> G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL >> G bpc_lstat(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) >> G bpc_file_checksum(Dow
Re: [BackupPC-users] Real Time View of What Is Being Copied
Thanks Micha. This may be a silly question, but how does one split the backup into several parts? Does that involve designating certain folders from a particular drive for backup instead of the entire drive? From: Micha Kersloot Sent: Thursday, March 3, 2022 9:28 AM To: General list for user discussion, questions and support Subject: Re: [BackupPC-users] Real Time View of What Is Being Copied Hi, you could try to split the backup in several parts? Met vriendelijke groet, Micha Kersloot [https://www.kovoks.nl/images/klantzimbra/kovoks.png]<https://www.kovoks.nl/> [https://www.kovoks.nl/images/klantzimbra/Zimbra_SalesCertified.png]<https://www.kovoks.nl/e-mail> www.kovoks.nl<https://www.kovoks.nl> | www.zimbra-nederland.nl<https://www.zimbra-nederland.nl> | Volg @kovoks<http://twitter.com/kovoks> op Twitter E mi...@kovoks.nl<mailto:mi...@kovoks.nl> T +31 (0)345 - 53 29 27 / 0345 - 53 29 27 A Belle van Zuylenlaan 1-5<https://goo.gl/maps/MrqmTv442tH2> | 4105 JX Culemborg<https://goo.gl/maps/MrqmTv442tH2> KvK 1104 Van: "Akibu Flash" mailto:akibufl...@outlook.com>> Aan: "General list for user discussion, questions and support" mailto:backuppc-users@lists.sourceforge.net>> Verzonden: Donderdag 3 maart 2022 15:23:23 Onderwerp: Re: [BackupPC-users] Real Time View of What Is Being Copied Just bumping this up… any suggestions for a resolution? Thanks. From: Akibu Flash mailto:akibufl...@outlook.com>> Sent: Monday, February 28, 2022 4:32 PM To: BackupPC-users@lists.sourceforge.net<mailto:BackupPC-users@lists.sourceforge.net> Subject: [BackupPC-users] Real Time View of What Is Being Copied Dear All, I backup a couple of Windows machines to a linux machine via rsync. However, the backup for one of the Windows machines never finishes and results in a partial backup. In the CGI user interface Count column, the backup always seems to hang at 311941 for hours and hours. Then at some point, the backup apparently stops and afterwards another backup which by that time has queued up starts anew. Never do I get a completed backup only a partial (at least for the last year or so). Is there a way to start a backup from the command line and have the screen show what in real time is actually being backed up? If so, what would that command be? It may be that the BackupPC is somehow getting stuck on a file but that is only speculation. Any other ideas as to what my be happening? I decompressed the partial backup log file (backup #884) using the command BackupPC_zcat XferLOG.884.z. I have reproduced the bottom portion below. Any assistance in helping to track down what is happening is appreciated. I do see in the log that there is an error message indicating an “error in the rsync protocol data stream (code 12) at io.c(226). Thanks in advance. Best, Akibu G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo.gif, 0x0, 00) -> 3 G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif), buf, 1394) tmpFd = -1 G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif)) G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL G bpc_lstat(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) G bpc_file_checksum(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png, 0x0, 00) -> 3 G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png), buf, 7814) tmpFd = -1 G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png)) G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL rsync_bpc: connection unexpectedly closed (23961310 bytes received so far) [receiver] R bpc_sysCall_cleanup: doneInit = 1 Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 24467705 inode Parsing done: nFilesTotal = 0 rsync error: error in rsync protocol data stream (code 12) at io.c(226) [receiver=3.1.3.0] rsync_bpc: connection unexpectedly closed (24012481 bytes received so far) [generator] G bpc_sysCall_cleanup: doneInit = 1 DoneGen: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 311942 filesTotal, 569449707538 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 24467704 inode Parsing done: nFilesTotal = 311942 rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0] rsync_bpc exited with fatal status 12 (3072) (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0]) Xfer PIDs are now Got fatal error during xfer (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0]) Backup aborted (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0]) BackupFailCleanup: nFilesTotal = 311942, type = full, BackupCase = 4, inPlace = 0, lastBkupNum = 8
Re: [BackupPC-users] Real Time View of What Is Being Copied
Just out of curiosity, why use 'rsyncd' rather than 'rsync' which to me seems more secure as well as easier to set up and support? George King wrote at about 15:44:03 + on Thursday, March 3, 2022: > No great suggestions for resolution from me but I have some sympathy > with you having had similar issues occasionally in the past. > > I have a number of rsyncd hosts set up on my main Win10 desktop - > basically one for each disk or part of to keep the backups down to a > reasonable duration. I'm running DeltaCopy Server. > > When one of my backups 'stalled' after having worked for a while, it > would do as you have found and stall at the same place each time. Other > disks/hosts were not affected and completed fine, so it was nothing > intrinsically wrong with DeltaCopy or backuppc. I could monitor the > file transfer via the Win10 Task Manager>Performance Tab > Resource > Monitor > Disk tab and selecting the right rsync process for the > problematic disk. At least that gave me confidence that the transfer > was starting properly and progressing - and I could tell when it > properly stuck, and at which folder on the Win10 machine. > > I couldn't find any informative messages in the backuppc logs to help me > solve it. When I manually stopped the backups, the partial backups were > generally removed or merged and the logs just showed that I had aborted > the xfer. > > What seems to have helped overcome the problem is, although I fail to > understand quite why, are: > - splitting the host into two, and > - copying the files from the folder at which the transfer stalled into > another folder, and > - I probably checked the Win10 folder permissions were OK and inherited > by sub-folders. > > George > > On 03/03/2022 14:23, Akibu Flash wrote: > > > > Just bumping this up… any suggestions for a resolution? Thanks. > > > > *From:* Akibu Flash > > *Sent:* Monday, February 28, 2022 4:32 PM > > *To:* BackupPC-users@lists.sourceforge.net > > *Subject:* [BackupPC-users] Real Time View of What Is Being Copied > > > > Dear All, > > > > I backup a couple of Windows machines to a linux machine via rsync. > > However, the backup for one of the Windows machines never finishes and > > results in a partial backup. In the CGI user interface Count column, > > the backup always seems to hang at 311941 for hours and hours. Then > > at some point, the backup apparently stops and afterwards another > > backup which by that time has queued up starts anew. Never do I get a > > completed backup only a partial (at least for the last year or so). > > > > Is there a way to start a backup from the command line and have the > > screen show what in real time is actually being backed up? If so, > > what would that command be? It may be that the BackupPC is somehow > > getting stuck on a file but that is only speculation. > > > > Any other ideas as to what my be happening? > > > > I decompressed the partial backup log file (backup #884) using the > > command BackupPC_zcat XferLOG.884.z. I have reproduced the bottom > > portion below. > > > > Any assistance in helping to track down what is happening is > > appreciated. I do see in the log that there is an error message > > indicating an “error in the rsync protocol data stream (code 12) at > > io.c(226). > > > > Thanks in advance. > > > > Best, Akibu > > > > G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo.gif, 0x0, > > 00) -> 3 > > > > G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif), > > buf, 1394) tmpFd = -1 > > > > G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif)) > > > > G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL > > > > G bpc_lstat(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) > > > > G bpc_file_checksum(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) > > > > G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png, 0x0, > > 00) -> 3 > > > > G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png), > > buf, 7814) tmpFd = -1 > > > > G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png)) > > > > G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL > > > > rsync_bpc: connection unexpectedly closed (23961310 bytes received so > > far) [receiver] > > > > R bpc_sysCall_c
Re: [BackupPC-users] Real Time View of What Is Being Copied
No great suggestions for resolution from me but I have some sympathy with you having had similar issues occasionally in the past. I have a number of rsyncd hosts set up on my main Win10 desktop - basically one for each disk or part of to keep the backups down to a reasonable duration. I'm running DeltaCopy Server. When one of my backups 'stalled' after having worked for a while, it would do as you have found and stall at the same place each time. Other disks/hosts were not affected and completed fine, so it was nothing intrinsically wrong with DeltaCopy or backuppc. I could monitor the file transfer via the Win10 Task Manager>Performance Tab > Resource Monitor > Disk tab and selecting the right rsync process for the problematic disk. At least that gave me confidence that the transfer was starting properly and progressing - and I could tell when it properly stuck, and at which folder on the Win10 machine. I couldn't find any informative messages in the backuppc logs to help me solve it. When I manually stopped the backups, the partial backups were generally removed or merged and the logs just showed that I had aborted the xfer. What seems to have helped overcome the problem is, although I fail to understand quite why, are: - splitting the host into two, and - copying the files from the folder at which the transfer stalled into another folder, and - I probably checked the Win10 folder permissions were OK and inherited by sub-folders. George On 03/03/2022 14:23, Akibu Flash wrote: Just bumping this up… any suggestions for a resolution? Thanks. *From:* Akibu Flash *Sent:* Monday, February 28, 2022 4:32 PM *To:* BackupPC-users@lists.sourceforge.net *Subject:* [BackupPC-users] Real Time View of What Is Being Copied Dear All, I backup a couple of Windows machines to a linux machine via rsync. However, the backup for one of the Windows machines never finishes and results in a partial backup. In the CGI user interface Count column, the backup always seems to hang at 311941 for hours and hours. Then at some point, the backup apparently stops and afterwards another backup which by that time has queued up starts anew. Never do I get a completed backup only a partial (at least for the last year or so). Is there a way to start a backup from the command line and have the screen show what in real time is actually being backed up? If so, what would that command be? It may be that the BackupPC is somehow getting stuck on a file but that is only speculation. Any other ideas as to what my be happening? I decompressed the partial backup log file (backup #884) using the command BackupPC_zcat XferLOG.884.z. I have reproduced the bottom portion below. Any assistance in helping to track down what is happening is appreciated. I do see in the log that there is an error message indicating an “error in the rsync protocol data stream (code 12) at io.c(226). Thanks in advance. Best, Akibu G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo.gif, 0x0, 00) -> 3 G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif), buf, 1394) tmpFd = -1 G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif)) G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL G bpc_lstat(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) G bpc_file_checksum(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png, 0x0, 00) -> 3 G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png), buf, 7814) tmpFd = -1 G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png)) G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL rsync_bpc: connection unexpectedly closed (23961310 bytes received so far) [receiver] R bpc_sysCall_cleanup: doneInit = 1 Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 24467705 inode Parsing done: nFilesTotal = 0 rsync error: error in rsync protocol data stream (code 12) at io.c(226) [receiver=3.1.3.0] rsync_bpc: connection unexpectedly closed (24012481 bytes received so far) [generator] G bpc_sysCall_cleanup: doneInit = 1 DoneGen: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 311942 filesTotal, 569449707538 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 24467704 inode Parsing done: nFilesTotal = 311942 rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0] rsync_bpc exited with fatal status 12 (3072) (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0]) Xfer PIDs are now Got fatal error during xfer (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0]) Backup aborted (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0]) Back
Re: [BackupPC-users] Real Time View of What Is Being Copied
Hi, you could try to split the backup in several parts? Met vriendelijke groet, Micha Kersloot [ https://www.kovoks.nl/ ] [ https://www.kovoks.nl/e-mail ] [ https://www.kovoks.nl/ | www.kovoks.nl ] | [ https://www.zimbra-nederland.nl/ | www.zimbra-nederland.nl ] | Volg [ http://twitter.com/kovoks | @kovoks ] op Twitter E [ mailto:mi...@kovoks.nl | mi...@kovoks.nl ] T +31 (0)345 - 53 29 27 / 0345 - 53 29 27 A [ https://goo.gl/maps/MrqmTv442tH2 | Belle van Zuylenlaan 1-5 ] | [ https://goo.gl/maps/MrqmTv442tH2 | 4105 JX Culemborg ] KvK 1104 > Van: "Akibu Flash" > Aan: "General list for user discussion, questions and support" > > Verzonden: Donderdag 3 maart 2022 15:23:23 > Onderwerp: Re: [BackupPC-users] Real Time View of What Is Being Copied > Just bumping this up… any suggestions for a resolution? Thanks. > From: Akibu Flash > Sent: Monday, February 28, 2022 4:32 PM > To: BackupPC-users@lists.sourceforge.net > Subject: [BackupPC-users] Real Time View of What Is Being Copied > Dear All, > I backup a couple of Windows machines to a linux machine via rsync. However, > the > backup for one of the Windows machines never finishes and results in a partial > backup. In the CGI user interface Count column, the backup always seems to > hang > at 311941 for hours and hours. Then at some point, the backup apparently stops > and afterwards another backup which by that time has queued up starts anew. > Never do I get a completed backup only a partial (at least for the last year > or > so). > Is there a way to start a backup from the command line and have the screen > show > what in real time is actually being backed up? If so, what would that command > be? It may be that the BackupPC is somehow getting stuck on a file but that is > only speculation. > Any other ideas as to what my be happening? > I decompressed the partial backup log file (backup #884) using the command > BackupPC_zcat XferLOG.884.z. I have reproduced the bottom portion below. > Any assistance in helping to track down what is happening is appreciated. I do > see in the log that there is an error message indicating an “error in the > rsync > protocol data stream (code 12) at io.c(226). > Thanks in advance. > Best, Akibu > G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo.gif, 0x0, 00) -> 3 > G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif), buf, 1394) > tmpFd = -1 > G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif)) > G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL > G bpc_lstat(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) > G bpc_file_checksum(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) > G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png, 0x0, 00) -> > 3 > G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png), buf, > 7814) > tmpFd = -1 > G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png)) > G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL > rsync_bpc: connection unexpectedly closed (23961310 bytes received so far) > [receiver] > R bpc_sysCall_cleanup: doneInit = 1 > Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 > sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 24467705 inode > Parsing done: nFilesTotal = 0 > rsync error: error in rsync protocol data stream (code 12) at io.c(226) > [receiver=3.1.3.0] > rsync_bpc: connection unexpectedly closed (24012481 bytes received so far) > [generator] > G bpc_sysCall_cleanup: doneInit = 1 > DoneGen: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 311942 > filesTotal, 569449707538 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, > 24467704 inode > Parsing done: nFilesTotal = 311942 > rsync error: error in rsync protocol data stream (code 12) at io.c(226) > [generator=3.1.3.0] > rsync_bpc exited with fatal status 12 (3072) (rsync error: error in rsync > protocol data stream (code 12) at io.c(226) [generator=3.1.3.0]) > Xfer PIDs are now > Got fatal error during xfer (rsync error: error in rsync protocol data stream > (code 12) at io.c(226) [generator=3.1.3.0]) > Backup aborted (rsync error: error in rsync protocol data stream (code 12) at > io.c(226) [generator=3.1.3.0]) > BackupFailCleanup: nFilesTotal = 311942, type = full, BackupCase = 4, inPlace > = > 0, lastBkupNum = 883 > Removing prior partial backup #883 > Xfer PIDs are now 38440 > BackupPC_backupDelete: removing #883 > BackupPC_backupDelete: Merge into backup 327 > mergeDir: name = attrib, path = . > bpc_attrib_backwardCompat: WriteOldStyleAttribFile = 0, KeepOldAttribFiles = 0 > mergeDir: skipping name = refCnt, path = ./refCnt > me
Re: [BackupPC-users] Real Time View of What Is Being Copied
Just bumping this up... any suggestions for a resolution? Thanks. From: Akibu Flash Sent: Monday, February 28, 2022 4:32 PM To: BackupPC-users@lists.sourceforge.net Subject: [BackupPC-users] Real Time View of What Is Being Copied Dear All, I backup a couple of Windows machines to a linux machine via rsync. However, the backup for one of the Windows machines never finishes and results in a partial backup. In the CGI user interface Count column, the backup always seems to hang at 311941 for hours and hours. Then at some point, the backup apparently stops and afterwards another backup which by that time has queued up starts anew. Never do I get a completed backup only a partial (at least for the last year or so). Is there a way to start a backup from the command line and have the screen show what in real time is actually being backed up? If so, what would that command be? It may be that the BackupPC is somehow getting stuck on a file but that is only speculation. Any other ideas as to what my be happening? I decompressed the partial backup log file (backup #884) using the command BackupPC_zcat XferLOG.884.z. I have reproduced the bottom portion below. Any assistance in helping to track down what is happening is appreciated. I do see in the log that there is an error message indicating an "error in the rsync protocol data stream (code 12) at io.c(226). Thanks in advance. Best, Akibu G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo.gif, 0x0, 00) -> 3 G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif), buf, 1394) tmpFd = -1 G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif)) G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL G bpc_lstat(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) G bpc_file_checksum(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png, 0x0, 00) -> 3 G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png), buf, 7814) tmpFd = -1 G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png)) G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL rsync_bpc: connection unexpectedly closed (23961310 bytes received so far) [receiver] R bpc_sysCall_cleanup: doneInit = 1 Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 24467705 inode Parsing done: nFilesTotal = 0 rsync error: error in rsync protocol data stream (code 12) at io.c(226) [receiver=3.1.3.0] rsync_bpc: connection unexpectedly closed (24012481 bytes received so far) [generator] G bpc_sysCall_cleanup: doneInit = 1 DoneGen: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 311942 filesTotal, 569449707538 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 24467704 inode Parsing done: nFilesTotal = 311942 rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0] rsync_bpc exited with fatal status 12 (3072) (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0]) Xfer PIDs are now Got fatal error during xfer (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0]) Backup aborted (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0]) BackupFailCleanup: nFilesTotal = 311942, type = full, BackupCase = 4, inPlace = 0, lastBkupNum = 883 Removing prior partial backup #883 Xfer PIDs are now 38440 BackupPC_backupDelete: removing #883 BackupPC_backupDelete: Merge into backup 327 mergeDir: name = attrib, path = . bpc_attrib_backwardCompat: WriteOldStyleAttribFile = 0, KeepOldAttribFiles = 0 mergeDir: skipping name = refCnt, path = ./refCnt mergeDir: skipping name = needFsck.newDir, path = ./refCnt/needFsck.newDir mergeDir: skipping name = noPoolCntOk, path = ./refCnt/noPoolCntOk mergeDir: skipping name = backupInfo, path = ./backupInfo removing remaining directory tree /var/lib/backuppc/pc/mark-desktop/883 Xfer PIDs are now 38441,38440 BackupPC_refCountUpdate: host mark-desktop #321: gotFsck = , gotDelta = , gotPoolCnt = 1, gotNoPoolCntOk = BackupPC_refCountUpdate: host mark-desktop #327: gotFsck = 1, gotDelta = , gotPoolCnt = 1, gotNoPoolCntOk = BackupPC_refCountUpdate: host mark-desktop #884: gotFsck = 1, gotDelta = , gotPoolCnt = 1, gotNoPoolCntOk = BackupPC_refCountUpdate: processing host mark-desktop #327 (fsck = 1) BackupPC_refCountUpdate: processing host mark-desktop #327 deltaFile /var/lib/backuppc/pc/mark-desktop/327/refCnt/poolCntDelta_1_0_0_38441 with 1048577 entries BackupPC_refCountUpdate: processing host mark-desktop #327 deltaFile /var/lib/backuppc/pc/mark-desktop/327/refCnt/poolCntDelta_1_0_1_38441 with 638055 entries BackupPC_refCountUpdate: processing host mark-desktop #884 (fsck = 1) BackupPC_refCountUpdate: processing host mark-desktop #884 deltaFile /var/lib/backuppc/pc/mark-desktop
[BackupPC-users] Real Time View of What Is Being Copied
Dear All, I backup a couple of Windows machines to a linux machine via rsync. However, the backup for one of the Windows machines never finishes and results in a partial backup. In the CGI user interface Count column, the backup always seems to hang at 311941 for hours and hours. Then at some point, the backup apparently stops and afterwards another backup which by that time has queued up starts anew. Never do I get a completed backup only a partial (at least for the last year or so). Is there a way to start a backup from the command line and have the screen show what in real time is actually being backed up? If so, what would that command be? It may be that the BackupPC is somehow getting stuck on a file but that is only speculation. Any other ideas as to what my be happening? I decompressed the partial backup log file (backup #884) using the command BackupPC_zcat XferLOG.884.z. I have reproduced the bottom portion below. Any assistance in helping to track down what is happening is appreciated. I do see in the log that there is an error message indicating an "error in the rsync protocol data stream (code 12) at io.c(226). Thanks in advance. Best, Akibu G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo.gif, 0x0, 00) -> 3 G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif), buf, 1394) tmpFd = -1 G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo.gif)) G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL G bpc_lstat(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) G bpc_file_checksum(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png) G bpc_open(Downloads/Downloads/BackupPC-4.3.2/images/logo320.png, 0x0, 00) -> 3 G bpc_read(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png), buf, 7814) tmpFd = -1 G bpc_close(3 (Downloads/Downloads/BackupPC-4.3.2/images/logo320.png)) G bpc_fileDescFree: fdNum = 3, tmpFd = -1, tmpFileName = NULL rsync_bpc: connection unexpectedly closed (23961310 bytes received so far) [receiver] R bpc_sysCall_cleanup: doneInit = 1 Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 24467705 inode Parsing done: nFilesTotal = 0 rsync error: error in rsync protocol data stream (code 12) at io.c(226) [receiver=3.1.3.0] rsync_bpc: connection unexpectedly closed (24012481 bytes received so far) [generator] G bpc_sysCall_cleanup: doneInit = 1 DoneGen: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 311942 filesTotal, 569449707538 sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 24467704 inode Parsing done: nFilesTotal = 311942 rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0] rsync_bpc exited with fatal status 12 (3072) (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0]) Xfer PIDs are now Got fatal error during xfer (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0]) Backup aborted (rsync error: error in rsync protocol data stream (code 12) at io.c(226) [generator=3.1.3.0]) BackupFailCleanup: nFilesTotal = 311942, type = full, BackupCase = 4, inPlace = 0, lastBkupNum = 883 Removing prior partial backup #883 Xfer PIDs are now 38440 BackupPC_backupDelete: removing #883 BackupPC_backupDelete: Merge into backup 327 mergeDir: name = attrib, path = . bpc_attrib_backwardCompat: WriteOldStyleAttribFile = 0, KeepOldAttribFiles = 0 mergeDir: skipping name = refCnt, path = ./refCnt mergeDir: skipping name = needFsck.newDir, path = ./refCnt/needFsck.newDir mergeDir: skipping name = noPoolCntOk, path = ./refCnt/noPoolCntOk mergeDir: skipping name = backupInfo, path = ./backupInfo removing remaining directory tree /var/lib/backuppc/pc/mark-desktop/883 Xfer PIDs are now 38441,38440 BackupPC_refCountUpdate: host mark-desktop #321: gotFsck = , gotDelta = , gotPoolCnt = 1, gotNoPoolCntOk = BackupPC_refCountUpdate: host mark-desktop #327: gotFsck = 1, gotDelta = , gotPoolCnt = 1, gotNoPoolCntOk = BackupPC_refCountUpdate: host mark-desktop #884: gotFsck = 1, gotDelta = , gotPoolCnt = 1, gotNoPoolCntOk = BackupPC_refCountUpdate: processing host mark-desktop #327 (fsck = 1) BackupPC_refCountUpdate: processing host mark-desktop #327 deltaFile /var/lib/backuppc/pc/mark-desktop/327/refCnt/poolCntDelta_1_0_0_38441 with 1048577 entries BackupPC_refCountUpdate: processing host mark-desktop #327 deltaFile /var/lib/backuppc/pc/mark-desktop/327/refCnt/poolCntDelta_1_0_1_38441 with 638055 entries BackupPC_refCountUpdate: processing host mark-desktop #884 (fsck = 1) BackupPC_refCountUpdate: processing host mark-desktop #884 deltaFile /var/lib/backuppc/pc/mark-desktop/884/refCnt/poolCntDelta_1_0_2_38441 with 1048577 entries BackupPC_refCountUpdate: processing host mark-desktop #884 deltaFile /var/lib/backuppc/pc/mark-desktop/884/refCnt/poolCntDelta_1_0_3_38441 with 1048577 entries BackupPC_refCountUpdate: processing host ma
