Re: [BackupPC-users] rsyncd full backup
Hi, You could also go from raid1 to raid10. Met vriendelijke groet, Micha Kersloot Blijf op de hoogte en ontvang de laatste tips over Zimbra/KovoKs Contact: http://twitter.com/kovoks KovoKs B.V. is ingeschreven onder KvK nummer: 1104 - Oorspronkelijk bericht - Van: Adam Goryachev mailingli...@websitemanagers.com.au Aan: General list for user discussion, questions and support backuppc-users@lists.sourceforge.net Verzonden: Donderdag 31 oktober 2013 04:13:42 Onderwerp: Re: [BackupPC-users] rsyncd full backup On 31/10/13 13:56, Sharuzzaman Ahmat Raslan wrote: Hi Adam, The low I/O is when the machine is under load. Thank you for suggesting to use SSD. I have been thinking about that as well, but currently, the storage of BackupPC is using a 1TB disk, with about 80% utilization. Changing to 1TB SSD might be a little bit overkill on the customer's budget :) Sure, 2 x 480GB SSD in linear RAID is still relatively expensive :) though it certainly is a huge performance improvement. BTW, FYI, I get 2.5GB/s read and 1.5GB/s write performance from my RAID5... Maybe I should look at bcache for Linux :) https://lwn.net/Articles/497024/ http://bcache.evilpiepirate.org/ I've seen that also, but I'm not sure it is a good (stable) solution for real use (at least, I'm not prepared to use that for a server yet, your tolerance might be different). In addition, it probably won't help the backup work load, since you need to read the entire disk, and the entire disk won't fit into the cache Regards, Adam -- Adam Goryachev Website Managers P: +61 2 8304 a...@websitemanagers.com.au F: +61 2 8304 0001 www.websitemanagers.com.au -- Adam Goryachev Website Managers www.websitemanagers.com.au -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
The du -hs /backup/pool /backup/cpool /backup/pc/* has finished. Basically I had 1 host that was taking up 6.9 TB of data with 2.8 TB in the cpool directory and most of the other hosts averaging a GB each. The 1 host was our file server (which I happen to know has a 2 TB volume (1.3 TB currently used) that is our main fileshare. I looked through the error log for this pc on backups with the most errors and found thousands of these: Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) I didn't see any of the BackupPC_link got error -4 errors. So now I'm running this command: du -hs /backup/pool /backup/cpool /backup/pc/myfileserver/* to see which backups are doing the most damage. I'll report back once that finishes. Thanks for all your help! Regards, Craig On Wed, Oct 30, 2013 at 10:24 PM, Holger Parplies wb...@parplies.de wrote: Hi, Adam Goryachev wrote on 2013-10-31 09:04:48 +1100 [Re: [BackupPC-users] Disk space used far higher than reported pool size]: On 31/10/13 07:51, Holger Parplies wrote: [...] Aside from that, I would think it might be worth the effort of determining whether all hosts are affected or not (though I can't really see why there should be a difference between hosts). If some aren't, you could at least keep their history. I suspect at least some hosts OR some backups are correct, or else OP wouldn't have anything in the pool. as I understand it, the backups from before the change from smb to rsyncd are linked into the pool. Since the change, some or all are not. Whether the change of XferMethod has anything to do with the problem or whether it coincidentally happened at about the same point in time remains to be seen. I still suspect the link to $topDir as cause, and BackupPC_link is independent of the XferMethod used (so a change in XferMethod shouldn't have any influence). [...] you might want to look at one individual host like this: du -sm /backup/pool /backup/cpool /backup/pc/host1/* This should be a *lot* quicker than the previous du command, and also should show minimal disk usage for each backup for host1. It is quicker because you are only looking at the set of files for the pool, plus one host. Just keep in mind that *incrementals* might be small even if not linked to pool files. Oh, and there is still another method that is *orders of magnitude* faster: look into the log file(s), or even at the *size* of the log files. If it happens every day, for each host, it shouldn't be hard to find. You can even write a Perl one-liner to show you which hosts it happens for (give me a sample log line and I will). If the log files show nothing, we're back to finding the problem, but I doubt that. You can't break pooling by copying, as was suggested. Yes, you get independent copies of files, and they might stay independent, but changed files should get pooled again, and your file system usage wouldn't continue growing in such a way as it seems to be. If pooling is currently broken, there's a reason for that, and there should be log messages indicating problems. PS, at this stage, you may want to look at the recent thread regarding disk caches, and caching directory entries instead of file contents. It might help with all the directory based searches you are doing to find the
Re: [BackupPC-users] Disk space used far higher than reported pool size
On Thu, Oct 31, 2013 at 7:49 AM, Craig O'Brien cobr...@fishman.com wrote: Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, What is the underlying storage here - nfs? -- Les Mikesell lesmikes...@gmail.com -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
What is the underlying storage here - nfs? Local SATA disks in a RAID 5 (5 disks, 3TB each in capacity) Regards, Craig On Thu, Oct 31, 2013 at 10:37 AM, Les Mikesell lesmikes...@gmail.comwrote: On Thu, Oct 31, 2013 at 7:49 AM, Craig O'Brien cobr...@fishman.com wrote: Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, What is the underlying storage here - nfs? -- Les Mikesell lesmikes...@gmail.com -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
On Thu, Oct 31, 2013 at 9:47 AM, Craig O'Brien cobr...@fishman.com wrote: What is the underlying storage here - nfs? Local SATA disks in a RAID 5 (5 disks, 3TB each in capacity) I think I'd force an fsck just on general principles even though it will take a long time to complete. Google turns up a few hits on similar problems, but I don't see a definitive answer. RStmp is supposed to be used to hold an uncompressed copy of the previous version of a large file with changes so rsync can seek to match up the changed block positions, so this error probably has something to do with your compressed copy being corrupted and not uncompressing properly. -- Les Mikesell lesmikes...@gmail.com -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
Craig O'Brien cobr...@fishman.com wrote on 10/31/2013 08:49:15 AM: The du -hs /backup/pool /backup/cpool /backup/pc/* has finished. Basically I had 1 host that was taking up 6.9 TB of data with 2.8 TB in the cpool directory and most of the other hosts averaging a GB each. Well, there's your problem. The 1 host was our file server (which I happen to know has a 2 TB volume (1.3 TB currently used) that is our main fileshare. I looked through the error log for this pc on backups with the most errors and found thousands of these: Just out of curiosity, why hadn't you already done that?!? Unable to read 8388608 bytes from /var/lib/BackupPC//pc/ myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0, 512,147872,1499463680,2422719488) Interesting. I'd make sure that the filesystem is OK before I went much farther... Stop BackupPC, unmount /backup and fsck /dev/whatever du -hs /backup/pool /backup/cpool /backup/pc/myfileserver/* to see which backups are doing the most damage. I'll report back once that finishes. With that, you should be able to find the bakup number(s) that are not linked. You can delete them and free up space. The big question is, though, why they aren't linking. I'd really start at the bottom of the stack (the physical drives) and work your way up. Check dmesg for any hardware errors. fsck the filesystem. Did I read correctly that this is connected vis NFSv4? I sure hope not... (I'm willing to admit it's a phobia, but there's no *WAY* I would trust my backup to work across NFS...) Tim Massey Out of the Box Solutions, Inc. Creative IT Solutions Made Simple! http://www.OutOfTheBoxSolutions.com tmas...@obscorp.com 22108 Harper Ave. St. Clair Shores, MI 48080 Office: (800)750-4OBS (4627) Cell: (586)945-8796 -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
Holger Parplies wb...@parplies.de wrote on 10/30/2013 10:24:05 PM: as I understand it, the backups from before the change from smb to rsyncd are linked into the pool. Since the change, some or all are not. Whether the change of XferMethod has anything to do with the problem or whether it coincidentally happened at about the same point in time remains to be seen. I still suspect the link to $topDir as cause, and BackupPC_link is independent of the XferMethod used (so a change in XferMethod shouldn't have any influence). To add my anecdote, I use a symbolic link for all of my BackupPC hosts: a couple dozen? And they all work fine. It's been my standard procedure for almost as long as I've been using BackupPC. Example: ls -l /var/lib lrwxrwxrwx. 1 rootroot 22 Apr 22 2013 BackupPC - /data/BackupPC/TopDir/ mount /dev/sda1 on /data type ext4 (rw) I understand phobias from earlier problems (see my earlier e-mail about my thoughts on NFS and backups...) but I do not think this one is an issue. If the log files show nothing, we're back to finding the problem, but I doubt that. You can't break pooling by copying, as was suggested. Yes, you get independent copies of files, and they might stay independent, but changed files should get pooled again, and your file system usage wouldn't continue growing in such a way as it seems to be. If pooling is currently broken, there's a reason for that, and there should be log messages indicating problems. You are 100% correct; but it depends on how you define break. Making a copy of a backup will absolutely break pooling--for the copy you just made! :) It won't prevent *future* copies from pooling, certainly. But it sure can fill up a drive: even if pooling *is* working correctly for new copies, they can still fill up the drive *and* BackupPC_nightly won't do a thing about it. Tim Massey Out of the Box Solutions, Inc. Creative IT Solutions Made Simple! http://www.OutOfTheBoxSolutions.com tmas...@obscorp.com 22108 Harper Ave. St. Clair Shores, MI 48080 Office: (800)750-4OBS (4627) Cell: (586)945-8796 -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
Hi, Example: ls -l /var/lib lrwxrwxrwx. 1 rootroot 22 Apr 22 2013 BackupPC - /data/BackupPC/TopDir/ mount /dev/sda1 on /data type ext4 (rw) out of curiosity - why don't you just configure /data/BackupPC/TopDir in config.pl as the TopDir? Regards Marcel -- Registrierter Linux User #307343 -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
On Thu, Oct 31, 2013 at 11:36 AM, Marcel Meckel mailinglist+backuppc-us...@foobar0815.de wrote: Hi, Example: ls -l /var/lib lrwxrwxrwx. 1 rootroot 22 Apr 22 2013 BackupPC - /data/BackupPC/TopDir/ mount /dev/sda1 on /data type ext4 (rw) out of curiosity - why don't you just configure /data/BackupPC/TopDir in config.pl as the TopDir? Versions earlier than 3.2 didn't allow that after the initial install - and in distribution-packaged version (rpm/deb) the location decision had already been made by the packagers. -- Les Mikesell lesmikes...@gmail.com -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
Just out of curiosity, why hadn't you already done that?!? I didn't know which host was the problem and didn't think of it. Although I'll readily admit it seems painfully obvious to me now. :) The big question is, though, why they aren't linking. I'd really start at the bottom of the stack (the physical drives) and work your way up. Check dmesg for any hardware errors. bash-4.1$ grep -i backup /var/log/dmesg* bash-4.1$ bash-4.1$ grep -i backup /var/log/messages* messages-20131006:Sep 30 13:53:24 servername kernel: BackupPC_dump[15365]: segfault at a80 ip 00310f695002 sp 7fff438c9770 error 4 in libperl.so[310f60+162000] messages-20131006:Sep 30 13:53:27 servername abrtd: Package 'BackupPC' isn't signed with proper key messages-20131020:Oct 19 01:24:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 01:24:54 servername kernel: BackupPC_dump D 0001 0 11922 10626 0x0080 messages-20131020:Oct 19 01:30:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 01:30:54 servername kernel: BackupPC_dump D 0001 0 11922 10626 0x0080 messages-20131020:Oct 19 01:32:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 01:32:54 servername kernel: BackupPC_dump D 0001 0 11922 10626 0x0080 messages-20131020:Oct 19 01:32:54 servername kernel: INFO: task BackupPC_nightl:18390 blocked for more than 120 seconds. messages-20131020:Oct 19 01:32:54 servername kernel: BackupPC_nigh D 0001 0 18390 1262 0x0080 messages-20131020:Oct 19 01:48:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 01:48:54 servername kernel: BackupPC_dump D 0003 0 11922 10626 0x0080 messages-20131020:Oct 19 01:52:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 01:52:54 servername kernel: BackupPC_dump D 0001 0 11922 10626 0x0080 messages-20131020:Oct 19 01:52:54 servername kernel: INFO: task BackupPC_nightl:18390 blocked for more than 120 seconds. messages-20131020:Oct 19 01:52:54 servername kernel: BackupPC_nigh D 0001 0 18390 1262 0x0080 messages-20131020:Oct 19 01:56:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 01:56:54 servername kernel: BackupPC_dump D 0003 0 11922 10626 0x0080 messages-20131020:Oct 19 02:10:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 02:10:54 servername kernel: BackupPC_dump D 0001 0 11922 10626 0x0080 messages-20131020:Oct 19 02:12:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 02:12:54 servername kernel: BackupPC_dump D 0001 0 11922 10626 0x0080 messages-20131027:Oct 23 09:00:02 servername abrtd: Package 'BackupPC' isn't signed with proper key fsck the filesystem. bash-4.1$ fsck /dev/sda1 fsck from util-linux-ng 2.17.2 e2fsck 1.41.12 (17-May-2010) /dev/sda1: clean, 20074506/2929688576 files, 2775975889/2929686016 blocks bash-4.1$ Did I read correctly that this is connected vis NFSv4? I sure hope not... (I'm willing to admit it's a phobia, but there's no *WAY* I would trust my backup to work across NFS...) The drives are local SATA ones that I set up in a raid 5, directly mounted. Def not NFS. I had an unrelated drive mounted via NFS, but that had nothing to do with my backup system and that's probably the source of confusion. So the du command finished, here's the result: bash-4.1$ du -hs /backup/pool /backup/cpool /backup/pc/fileserver/* 4.0K/backup/pool 2.8T/backup/cpool 350M/backup/pc/fileserver/223 361M/backup/pc/fileserver/250 373M/backup/pc/fileserver/278 325M/backup/pc/fileserver/302 329M/backup/pc/fileserver/331 330M/backup/pc/fileserver/360 335M/backup/pc/fileserver/388 338M/backup/pc/fileserver/417 345M/backup/pc/fileserver/446 346M/backup/pc/fileserver/475 350M/backup/pc/fileserver/503 450M/backup/pc/fileserver/524 437M/backup/pc/fileserver/525 437M/backup/pc/fileserver/526 437M/backup/pc/fileserver/527 2.5G/backup/pc/fileserver/528 1.4T/backup/pc/fileserver/529 438M/backup/pc/fileserver/530 467M/backup/pc/fileserver/531 438M/backup/pc/fileserver/532 438M/backup/pc/fileserver/533 1.4T/backup/pc/fileserver/534 438M/backup/pc/fileserver/535 1013M /backup/pc/fileserver/536 442M/backup/pc/fileserver/537 441M/backup/pc/fileserver/538 441M/backup/pc/fileserver/539 1.4T/backup/pc/fileserver/540 441M/backup/pc/fileserver/541 442M/backup/pc/fileserver/542 442M
Re: [BackupPC-users] Disk space used far higher than reported pool size
On Fri, Nov 1, 2013 at 1:33 AM, Craig O'Brien cobr...@fishman.com wrote: messages-20131006:Sep 30 13:53:24 servername kernel: BackupPC_dump[15365]: segfault at a80 ip 00310f695002 sp 7fff438c9770 error 4 in libperl.so[310f60+162000] This error shows BackupPC_dump segfault, and pointing to libperl.so How do you install your BackupPC ? From source or from RPM? If from RPM, which repo that you use? Thanks -- Sharuzzaman Ahmat Raslan -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
On Thu, Oct 31, 2013 at 12:33 PM, Craig O'Brien cobr...@fishman.com wrote: fsck the filesystem. bash-4.1$ fsck /dev/sda1 fsck from util-linux-ng 2.17.2 e2fsck 1.41.12 (17-May-2010) /dev/sda1: clean, 20074506/2929688576 files, 2775975889/2929686016 blocks bash-4.1$ That tells you it was unmounted cleanly last time, not that everything checks out OK. Try it with the -f option to make it do the actual checks. I don't suppose this helps give any insight to what happened? Thanks for all your help! I think it is related to that RStmp file that isn't uncompressing correctly so rsync can merge the changes - I'm not sure what happens after that error, though, or how to find the compressed file that is probably causing it. -- Les Mikesell lesmikes...@gmail.com -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
Craig O'Brien cobr...@fishman.com wrote on 10/31/2013 01:33:30 PM: Just out of curiosity, why hadn't you already done that?!? I didn't know which host was the problem and didn't think of it. Although I'll readily admit it seems painfully obvious to me now. :) Just so you're sufficiently humble... :) For everyone's future reference: ALWAYS check the server error log *and* the per-host logs... :) The big question is, though, why they aren't linking. I'd really start at the bottom of the stack (the physical drives) and work your way up. Check dmesg for any hardware errors. bash-4.1$ grep -i backup /var/log/dmesg* bash-4.1$ Nice try, but won't help: you need to be looking for the correct sd or ata device that is used. Don't bother with a grep like that. do a dmesg dmesg.txt and then vi (or whatever) dmesg.txt and look for scary errors... Look particularly for sda (or sdb or whatever), or ata0 (or 1 or whatever) messages, or possibly scsi messages (yes, SATA is SCSI to Linux) too. But if they're there, these should not be hard to find: there tends to be *LOTS* of them. bash-4.1$ grep -i backup /var/log/messages* Mine comes back with nothing. messages-20131006:Sep 30 13:53:24 servername kernel: BackupPC_dump [15365]: segfault at a80 ip 00310f695002 sp 7fff438c9770 error 4 in libperl.so[310f60+162000] messages-20131006:Sep 30 13:53:27 servername abrtd: Package 'BackupPC' isn't signed with proper key messages-20131020:Oct 19 01:24:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 01:24:54 servername kernel: BackupPC_dump D 0001 0 11922 10626 0x0080 messages-20131020:Oct 19 01:30:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 01:30:54 servername kernel: BackupPC_dump D 0001 0 11922 10626 0x0080 messages-20131020:Oct 19 01:32:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 01:32:54 servername kernel: BackupPC_dump D 0001 0 11922 10626 0x0080 messages-20131020:Oct 19 01:32:54 servername kernel: INFO: task BackupPC_nightl:18390 blocked for more than 120 seconds. messages-20131020:Oct 19 01:32:54 servername kernel: BackupPC_nigh D 0001 0 18390 1262 0x0080 messages-20131020:Oct 19 01:48:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 01:48:54 servername kernel: BackupPC_dump D 0003 0 11922 10626 0x0080 messages-20131020:Oct 19 01:52:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 01:52:54 servername kernel: BackupPC_dump D 0001 0 11922 10626 0x0080 messages-20131020:Oct 19 01:52:54 servername kernel: INFO: task BackupPC_nightl:18390 blocked for more than 120 seconds. messages-20131020:Oct 19 01:52:54 servername kernel: BackupPC_nigh D 0001 0 18390 1262 0x0080 messages-20131020:Oct 19 01:56:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 01:56:54 servername kernel: BackupPC_dump D 0003 0 11922 10626 0x0080 messages-20131020:Oct 19 02:10:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 02:10:54 servername kernel: BackupPC_dump D 0001 0 11922 10626 0x0080 messages-20131020:Oct 19 02:12:54 servername kernel: INFO: task BackupPC_dump:11922 blocked for more than 120 seconds. messages-20131020:Oct 19 02:12:54 servername kernel: BackupPC_dump D 0001 0 11922 10626 0x0080 messages-20131027:Oct 23 09:00:02 servername abrtd: Package 'BackupPC' isn't signed with proper key I'd try Googling those: they have no meaning for me (and my servers don't have them). What distro are you using? (I use CentOS/RHEL) fsck the filesystem. bash-4.1$ fsck /dev/sda1 fsck from util-linux-ng 2.17.2 e2fsck 1.41.12 (17-May-2010) /dev/sda1: clean, 20074506/2929688576 files, 2775975889/2929686016 blocks bash-4.1$ Definitely a good sign. Did I read correctly that this is connected vis NFSv4? I sure hope not... (I'm willing to admit it's a phobia, but there's no *WAY* I would trust my backup to work across NFS...) The drives are local SATA ones that I set up in a raid 5, directly mounted. Def not NFS. I had an unrelated drive mounted via NFS, but that had nothing to do with my backup system and that's probably the source of confusion. md raid5? What's the status of /dev/mdstat ? So the du command finished, here's the result: bash-4.1$ du -hs /backup/pool /backup/cpool /backup/pc/fileserver/* 1.4T/backup/pc/fileserver/529 1.4T
Re: [BackupPC-users] Disk space used far higher than reported pool size
Les Mikesell lesmikes...@gmail.com wrote on 10/31/2013 01:54:24 PM: On Thu, Oct 31, 2013 at 12:33 PM, Craig O'Brien cobr...@fishman.com wrote: fsck the filesystem. bash-4.1$ fsck /dev/sda1 fsck from util-linux-ng 2.17.2 e2fsck 1.41.12 (17-May-2010) /dev/sda1: clean, 20074506/2929688576 files, 2775975889/2929686016 blocks bash-4.1$ That tells you it was unmounted cleanly last time, not that everything checks out OK. Try it with the -f option to make it do the actual checks. Good catch! This should take a long time: 20 minutes to an hour? Maybe more: the drives are full. Tim Massey Out of the Box Solutions, Inc. Creative IT Solutions Made Simple! http://www.OutOfTheBoxSolutions.com tmas...@obscorp.com 22108 Harper Ave. St. Clair Shores, MI 48080 Office: (800)750-4OBS (4627) Cell: (586)945-8796 -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
Timothy J Massey wrote at about 11:52:35 -0400 on Thursday, October 31, 2013: Craig O'Brien cobr...@fishman.com wrote on 10/31/2013 08:49:15 AM: Just out of curiosity, why hadn't you already done that?!? Unable to read 8388608 bytes from /var/lib/BackupPC//pc/ myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0, 512,147872,1499463680,2422719488) Interesting. I'd make sure that the filesystem is OK before I went much farther... Stop BackupPC, unmount /backup and fsck /dev/whatever Or could be an NFS type error... du -hs /backup/pool /backup/cpool /backup/pc/myfileserver/* to see which backups are doing the most damage. I'll report back once that finishes. With that, you should be able to find the bakup number(s) that are not linked. You can delete them and free up space. Or you could relink the backups to the pool if you want to preserve the old backups... The big question is, though, why they aren't linking. I'd really start at the bottom of the stack (the physical drives) and work your way up. Check dmesg for any hardware errors. fsck the filesystem. Did I read correctly that this is connected vis NFSv4? I sure hope not... (I'm willing to admit it's a phobia, but there's no *WAY* I would trust my backup to work across NFS...) Well I have been using NFS on a NAS for 7 years without problems... but I probably wouldn't use it in a production, commercial environment... -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
Craig O'Brien wrote at about 08:49:15 -0400 on Thursday, October 31, 2013: The du -hs /backup/pool /backup/cpool /backup/pc/* has finished. Basically I had 1 host that was taking up 6.9 TB of data with 2.8 TB in the cpool directory and most of the other hosts averaging a GB each. The 1 host was our file server (which I happen to know has a 2 TB volume (1.3 TB currently used) that is our main fileshare. I looked through the error log for this pc on backups with the most errors and found thousands of these: Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Unable to read 8388608 bytes from /var/lib/BackupPC//pc/myfileserver/new//ffileshare/RStmp got=0, seekPosn=1501757440 (0,512,147872,1499463680,2422719488) Well, if such errors occur when looking for a matching pool file, then BackupPC either will create a duplicate pool entry (with a _N pool chain suffix) or it will fail to link. So it's possible that either some of your pc files are not linked to the pool and/or some of the are linked to duplicate (and unnecessary) pool chains elements... -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] rsyncd full backup
Sharuzzaman Ahmat Raslan sharuzza...@gmail.com wrote on 10/30/2013 10:06:18 PM: Hi Holger, Based on short session of troubleshooting, I believe the machine actually suffer from low I/O speed to the disk. Average read is about 3 MB/s, which I considered slow for a SATA disk in IDE emulation. *REAL* slow: I consider anything under 20MB/s slow. But where did that number come from? The pattern of reads will make a *huge* difference... I'm planning to suggest to the customer to have a RAID 1 setup to increase the I/O speed. I'm looking at possibilities to speed things up by not having to change the overall setup. I think you might want to have a better idea of what is going on first before you just start throwing hardware at it. If your numbers were correct but still too slow I'd say sure. But your numbers are *broken* wrong. You *might* fix your problem (by accident!) by throwing away some pieces and adding others, but you might not, too. Then you've got a client that just spent a bunch of money for nothing... Tim Massey Out of the Box Solutions, Inc. Creative IT Solutions Made Simple! http://www.OutOfTheBoxSolutions.com tmas...@obscorp.com 22108 Harper Ave. St. Clair Shores, MI 48080 Office: (800)750-4OBS (4627) Cell: (586)945-8796 -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
On Thu, Oct 31, 2013 at 12:51 PM, Sharuzzaman Ahmat Raslan sharuzza...@gmail.com wrote: On Fri, Nov 1, 2013 at 1:33 AM, Craig O'Brien cobr...@fishman.com wrote: messages-20131006:Sep 30 13:53:24 servername kernel: BackupPC_dump[15365]: segfault at a80 ip 00310f695002 sp 7fff438c9770 error 4 in libperl.so[310f60+162000] This error shows BackupPC_dump segfault, and pointing to libperl.so How do you install your BackupPC ? From source or from RPM? If from RPM, which repo that you use? Good catch - I missed that line. Bad RAM could cause segfaults too - and other very mysterious problems. I had a machine where the software raid1 mirrors would get differing contents and crash once in a while - and it took more than a weekend of running memtest to catch it. -- Les Mikesell lesmikes...@gmail.com -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] rsyncd full backup
Hi Timothy, I got the number by observing the output of iotop while file transfer is running. Also, on BackupPC host summary page, average transfer rate for full backup is also around 3MB/s It could be a network bottleneck also, as the customer is using 100Mbps switch with around 80 PC, not including network printer and servers. Inclusive should be around 100 network devices. Any idea how to properly troubleshoot network bottleneck? My skill is a little bit lacking on that area. Thanks. On Fri, Nov 1, 2013 at 2:12 AM, Timothy J Massey tmas...@obscorp.comwrote: Sharuzzaman Ahmat Raslan sharuzza...@gmail.com wrote on 10/30/2013 10:06:18 PM: Hi Holger, Based on short session of troubleshooting, I believe the machine actually suffer from low I/O speed to the disk. Average read is about 3 MB/s, which I considered slow for a SATA disk in IDE emulation. *REAL* slow: I consider anything under 20MB/s slow. But where did that number come from? The pattern of reads will make a *huge* difference... I'm planning to suggest to the customer to have a RAID 1 setup to increase the I/O speed. I'm looking at possibilities to speed things up by not having to change the overall setup. I think you might want to have a better idea of what is going on first before you just start throwing hardware at it. If your numbers were correct but still too slow I'd say sure. But your numbers are *broken* wrong. You *might* fix your problem (by accident!) by throwing away some pieces and adding others, but you might not, too. Then you've got a client that just spent a bunch of money for nothing... Tim Massey *Out of the Box Solutions, Inc.* * Creative IT Solutions Made Simple!** **http://www.OutOfTheBoxSolutions.com*http://www.outoftheboxsolutions.com/ * **tmas...@obscorp.com* tmas...@obscorp.com 22108 Harper Ave. St. Clair Shores, MI 48080 Office: (800)750-4OBS (4627) Cell: (586)945-8796 -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Sharuzzaman Ahmat Raslan -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
Les Mikesell wrote at about 10:15:42 -0500 on Thursday, October 31, 2013:
On Thu, Oct 31, 2013 at 9:47 AM, Craig O'Brien cobr...@fishman.com wrote:
What is the underlying storage here - nfs?
Local SATA disks in a RAID 5 (5 disks, 3TB each in capacity)
I think I'd force an fsck just on general principles even though it
will take a long time to complete. Google turns up a few hits on
similar problems, but I don't see a definitive answer. RStmp is
supposed to be used to hold an uncompressed copy of the previous
version of a large file with changes so rsync can seek to match up the
changed block positions, so this error probably has something to do
with your compressed copy being corrupted and not uncompressing
properly.
And this would explain why the elements are not being linked properly
to the pool -- though I would have thought the more likely result
would be a duplicate pool entry than an unlinked pool entry...
It might be interesting to look for pool chains with the same
(uncompressed) content and with links HardLinkMax (typically 31999)
to see if pool entries are being unnecessarily duplicated.
Try:
(cd /var/lib/BackupPC/cpool; find . -type f -links -3198 -name *_*
-exec md5sum {} \;) | sort | uniq -d -w32
Note this will find if there are any unnecessarily duplicated pool
chains (beyond the base one). Note to keep it fast and simple I am
skipping the elements without a suffix... with the assumption being
that if there are duplicated elements then there will probably be
whole chains of them...
--
Android is increasing in popularity, but the open development platform that
developers love is also attractive to malware creators. Download this white
paper to learn more about secure code signing practices that can help keep
Android apps secure.
http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] rsyncd full backup
On Wed, Oct 30, 2013 at 9:06 PM, Sharuzzaman Ahmat Raslan sharuzza...@gmail.com wrote: Hi Holger, Based on short session of troubleshooting, I believe the machine actually suffer from low I/O speed to the disk. Average read is about 3 MB/s, which I considered slow for a SATA disk in IDE emulation. Where are you getting that number? hdparm -tT device_partition_name should show about 30MB/s for the lower number even for old IDEs and at least 2 or 3x that for SATA with a SATA controller - even old ones. I'm planning to suggest to the customer to have a RAID 1 setup to increase the I/O speed. I'm looking at possibilities to speed things up by not having to change the overall setup. RAID1 is a good idea to protect against a single drive failure, but it won't make a lot of difference in speed. Writes go to both, reads can overlap if the software is smart. But, if you are currently running RAID5, using bigger disks in RAID1 would help. -- Les Mikesell lesmikes...@gmail.com -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] rsyncd full backup
Sharuzzaman Ahmat Raslan sharuzza...@gmail.com wrote on 10/31/2013 02:38:01 PM: Hi Timothy, I got the number by observing the output of iotop while file transfer is running. Also, on BackupPC host summary page, average transfer rate for full backup is also around 3MB/s It could be a network bottleneck also, as the customer is using 100Mbps switch with around 80 PC, not including network printer and servers. Inclusive should be around 100 network devices. For file transfers, 100Mb/s is good for 7MB/s transfer rate. Assuming a good quality switch (which is a *big* assumption), the number of computers shouldn't matter. But I would think strongly about buying a good quality Gigabit switch (I recommend the HP V1910-24G) as your backbone: Plug all of your servers (including the BackupPC server) into it, as well as each of your 100Mb/s switches (even better if they have Gb uplink ports!). That would eliminate the network as a bottleneck and only costs $300. And improve network performance across the board, though your users may not notice it if they only work with small files. Any idea how to properly troubleshoot network bottleneck? My skill is a little bit lacking on that area. Sure: Time the copying of files from one machine to another. Assuming the source and destination hard drives are faster than 7MB/s (and they very well *better* be!), then you'll saturate a 100Mb network no problem. For a more scientific approach, check out iperf. I'd be *much* more worried about checking out your *disk* performance. You can do tests in exactly the same way: copy files to and from the disk and see what happens. Here are some very simple examples: sync; time dd if=/dev/zero of=test.fil bs=1M count=1024; sync; sync; sync; sync; time dd if=test.fil of=/dev/null bs=1M The first line times the writing of a 1GB file named test.fil. The second one times the reading of the same 1GB file. Divide 1024 by the number of seconds it takes and that will give you the MB/s that you transferred. (The sync command is needed for accurate timing; the three sync commands is kind of an old UNIX graybeard joke: http://unix.stackexchange.com/questions/5260/is-there-truth-to-the-philosophy-that-you-should-sync-sync-sync-sync ) If you want more scientific disk performance information, check out iozone or iometer. Remember: always profile before you optimize. ( http://www.phatcode.net/res/224/files/html/ch37/37-02.html ) Tim Massey Out of the Box Solutions, Inc. Creative IT Solutions Made Simple! http://www.OutOfTheBoxSolutions.com tmas...@obscorp.com 22108 Harper Ave. St. Clair Shores, MI 48080 Office: (800)750-4OBS (4627) Cell: (586)945-8796 -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
Hi, I've spent far too long writing an email and trying to make it make sense and then discarding it again. Just one thought I want to rescue: the RStmp file was really *large* (something like 1.5 GB), your backup trees are really *large* (1.4 TB), your pool FS is really *full* (27.5 GB free). Running out of space during a backup is a bad idea. Both the RStmp file(s) will be truncated (though that should trigger a second error when it is *written*, just before it is read again) and the NewFileList, which would, in turn, lead to BackupPC_link missing new files it would be supposed to link into the pool (resulting in unlinked files). That doesn't explain your situation, but it still might be something to think about (and we might be seeing one problem on top of and as result of another). I agree with Jeffrey - an Unable to read ... error *without* a preceeding Can't write len=... to .../RStmp sounds like a mismatch between file length according to attrib file and result of decompression of compressed file - probably caused by corruption of the compressed file (or the attrib file, though unlikely, because the size is not way off). How many backups are/were you running in parallel? Hope that helps. Regards, Holger -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
On Thu, Oct 31, 2013 at 2:20 PM, Holger Parplies wb...@parplies.de wrote: That doesn't explain your situation, but it still might be something to think about (and we might be seeing one problem on top of and as result of another). I agree with Jeffrey - an Unable to read ... error *without* a preceeding Can't write len=... to .../RStmp sounds like a mismatch between file length according to attrib file and result of decompression of compressed file - probably caused by corruption of the compressed file (or the attrib file, though unlikely, because the size is not way off). I think that segfault in a perl process needs to be tracked down before expecting anything else to make sense. Either bad RAM or mismatching perl libs could break about anything else. -- Les Mikesell lesmikes...@gmail.com -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BUG SOLUTION: Can't call method getStats on an undefined value
Just ran into the bug described back in 2011 by Jeffrey (http://sourceforge.net/mailarchive/forum.php?thread_name=20017.43110.617411.92113%40consult.pretenderforum_name=backuppc-users). I had to reinstall BackupPC after an OS upgrade broke it (*sigh*) and my quick glance at the situation matches Jeff's. Just a heads-up. Thanks, - R. -- [__ Robert Sheldon [__ No Problem [__ Information technology support and services [__ (530) 575-0278 -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BUG SOLUTION: Can't call method getStats on
From: Rob Sheldon rob@as... - 2013-11-01 00:13 Just ran into the bug described back in 2011 by Jeffrey (http://sourceforge.net/mailarchive/forum.php?thread_name=20017.43110.617411.92113%40consult.pretenderforum_name=backuppc-users). I had to reinstall BackupPC after an OS upgrade broke it (*sigh*) and my quick glance at the situation matches Jeff's. Has the simple bug code fix that I suggested back in 2011 ever been implemented? -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BUG SOLUTION: Can't call method getStats on
backu...@kosowsky.org wrote at about 21:08:33 -0400 on Thursday, October 31,
2013:
From: Rob Sheldon rob@as... - 2013-11-01 00:13
Just ran into the bug described back in 2011 by Jeffrey
(http://sourceforge.net/mailarchive/forum.php?thread_name=20017.43110.617411.92113%40consult.pretenderforum_name=backuppc-users).
I had to reinstall BackupPC after an OS upgrade broke it (*sigh*)
and my quick glance at the situation matches Jeff's.
Has the simple bug code fix that I suggested back in 2011 ever been
implemented?
To answer my own question, no it hasn't -- at least as of
BackupPC-3.3.0
Hello Craig?
Here is the diff...
--- BackupPC_dump.orig 2013-10-31 21:51:57.269388705 -0400
+++ BackupPC_dump.new 2013-10-31 21:52:30.291387541 -0400
@@ -1156,7 +1156,7 @@
# only keep this backup if it has more files than the previous
# partial.
#
-if ( $type eq full ) {
+if ( $type eq full defined($xfer)) {
if ( $nFilesTotal == 0 $xfer-getStats-{fileCnt} == 0 ) {
#
# Xfer didn't report any files, but check in the new
--
Android is increasing in popularity, but the open development platform that
developers love is also attractive to malware creators. Download this white
paper to learn more about secure code signing practices that can help keep
Android apps secure.
http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Disk space used far higher than reported pool size
In my experience, segfault in libraries usually caused by installing it from different source. For example, when I install BackupPC for CentOS, I use the one in EPEL repo. I make sure that all the libraries (perl and others), only come from CentOS base repo, and not from other, as installing them from somewhere else might cause incompatibilities. In fact, sometime EPEL repo also provide perl library that conflict with CentOS base repo, but I just ignore it, and stick to base repo. On Fri, Nov 1, 2013 at 3:57 AM, Les Mikesell lesmikes...@gmail.com wrote: On Thu, Oct 31, 2013 at 2:20 PM, Holger Parplies wb...@parplies.de wrote: That doesn't explain your situation, but it still might be something to think about (and we might be seeing one problem on top of and as result of another). I agree with Jeffrey - an Unable to read ... error *without* a preceeding Can't write len=... to .../RStmp sounds like a mismatch between file length according to attrib file and result of decompression of compressed file - probably caused by corruption of the compressed file (or the attrib file, though unlikely, because the size is not way off). I think that segfault in a perl process needs to be tracked down before expecting anything else to make sense. Either bad RAM or mismatching perl libs could break about anything else. -- Les Mikesell lesmikes...@gmail.com -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Sharuzzaman Ahmat Raslan -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
