Re: [BackupPC-users] Large directory
On 2018-07-09 09:21, Bowie Bailey wrote: On 7/9/2018 12:09 PM, Les Mikesell wrote: On Mon, Jul 9, 2018 at 9:42 AM, Bowie Bailey wrote: There was still plenty of free RAM and no swap usage. I know it was still doing something because the pool filesystem was slowly growing. I could try an strace, but I'll have to research that. I've never used strace before. Be sure to check RAM/swap at both ends. You give strace a process id number and it shows every system call and its arguments that the process is making. It's usually an overwhelming stream of text but you should be able to see the open() and read()s happening as long as it is still sending files. Both ends are on the same machine for this backup. rsync still spawns two independent processes -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Large directory
On 7/9/2018 12:09 PM, Les Mikesell wrote: > On Mon, Jul 9, 2018 at 9:42 AM, Bowie Bailey wrote: >> There was still plenty of free RAM and no swap usage. I know it was >> still doing something because the pool filesystem was slowly growing. I >> could try an strace, but I'll have to research that. I've never used >> strace before. >> > Be sure to check RAM/swap at both ends. You give strace a process id > number and it shows every system call and its arguments that the > process is making. It's usually an overwhelming stream of text but > you should be able to see the open() and read()s happening as long as > it is still sending files. Both ends are on the same machine for this backup. -- Bowie -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Large directory
On Mon, Jul 9, 2018 at 9:42 AM, Bowie Bailey wrote: > > There was still plenty of free RAM and no swap usage. I know it was > still doing something because the pool filesystem was slowly growing. I > could try an strace, but I'll have to research that. I've never used > strace before. > Be sure to check RAM/swap at both ends. You give strace a process id number and it shows every system call and its arguments that the process is making. It's usually an overwhelming stream of text but you should be able to see the open() and read()s happening as long as it is still sending files. -- Les Mikesell lesmikes...@gmail.com -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Large directory
On 7/6/2018 6:18 PM, Les Mikesell wrote: > On Fri, Jul 6, 2018 at 1:38 PM, Bowie Bailey wrote: >> Right now, the only error I've seen is the error that stopped the backup: >> rsync error: error in rsync protocol data stream (code 12) at io.c(1556) >> [generator=3.0.9.12] >> >> The main annoyance is that I have no way to track progress. While the >> backup is running, I can't tell if it's about to finish, or if it's >> bogged down and is likely to take a few more hours (or days). > Rsync probably builds a table in memory to track the inodes of files > with multiple links so you may be running out of RAM or slowing down > from swapping. For a brute-force approach to see what is happening > you could strace the rsync process on the sending side. You could at > least tell if it is still opening/reading files. There was still plenty of free RAM and no swap usage. I know it was still doing something because the pool filesystem was slowly growing. I could try an strace, but I'll have to research that. I've never used strace before. -- Bowie -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Large directory
On 7/6/2018 4:14 PM, Carl W. Soderstrom wrote: > On 07/06 02:38 , Bowie Bailey wrote: >> The problem is that the original backup took 2 weeks to fail with no >> indication of problems that I could see... it was just very slow. I >> posted a previous question about it on this list while it was running. >> I could not find any bottlenecks or problems. I'm reluctant to start it >> again without some idea of what I'm looking for. How would you suggest >> I go about collecting more info? Up the log level in BPC? Make rsync >> more verbose? >> >> Right now, the only error I've seen is the error that stopped the backup: >> rsync error: error in rsync protocol data stream (code 12) at io.c(1556) >> [generator=3.0.9.12] >> >> The main annoyance is that I have no way to track progress. While the >> backup is running, I can't tell if it's about to finish, or if it's >> bogged down and is likely to take a few more hours (or days). > > Do you have a preferred tool for tracking how much bandwidth is in use, and > to which ports and hosts it's going to? > > For this I use 'iftop' (or, 'iftop -BP' to show output by bytes and to name > the ports). It's a good way to see if the backup is still moving data or if > it's hung up on something. That sounds useful. I'll give that a try. -- Bowie -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
