Re: [BackupPC-users] Backup fails after three days, possibly millions of files
On 7/16/19 4:27 PM, Adam Goryachev wrote:
On 17/7/19 4:22 am, David Koski wrote:
Regards,
David Koski
dko...@sutinen.com
On 7/8/19 6:16 PM, Adam Goryachev wrote:
On 9/7/19 10:23 am, David Koski wrote:
I am trying to back up about 24TB of data that has millions of
files. It takes a day or to before it starts backing up and then
stops with an error. I did a CLI dump and trapped the output and
can see the error message:
Can't write 32780 bytes to socket
Read EOF: Connection reset by peer
Tried again: got 0 bytes
finish: removing in-process file
Shares/Archives//COR_2630.png
Child is aborting
Done: 589666 files, 1667429241846 bytes
Got fatal error during xfer (aborted by signal=PIPE)
Backup aborted by user signal
Not saving this as a partial backup since it has fewer files than
the prior one (got 589666 and 589666 files versus 4225016)
dump failed: aborted by signal=PIPE
This backup is doing rsync over ssh. I enabled SSH keepalive but
it does not appear to be due to an idle network. It does not
appear to be a random network interruption because the time it
takes to fail is pretty consistent, about three days. I'm stumped.
Did you check:
$Conf{ClientTimeout} = 72000;
Also, what version of rsync on the client, what version of BackupPC
on the server, etc?
I think BPC v4 handles this scenario significantly better, in fact a
server I used to have trouble with on BPC3.x all the time has since
been combined with 4 other server (so 4 x the number of files and
total size of data) and BPC4 handles it easily.
Thank you all for your input. More information:
rsync version on client: 3.0.8 (Windows)
rsync version on server: 3.1.2 (Debian)
BackupPC version: 3.3.1
$(Config{ClientTimeout} = 604800
I just compared the output of two verbose BackupPC_dump runs and it
looks like the files are reported to be backed up even though they
are not. For example, this appears in logs of both backup runs:
create 644 4616/545 1085243184 /3412.zip
I checked and the file time stamp is year 2018. The log files are
full of these. I checked the real time clock on both systems and
they are correct. There are also files that have been backed up that
are not in the logs.
I suspect there are over ten million files but I don't have a good
way of telling now. Oddly, there are about 500,000 files backed
according to the log captured from BackupPC_dump and almost the same
number actually backed up and found in pc//0, but they are
different subsets of files. I have been tracking memory and swap
usage on the server and see no issues.
Is this a possible bug in BackupPC 3.3.1?
Please don't top-post if you can avoid it, at least not on mailing lists.
I just realised:
Read EOF: Connection reset by peer
This is a networking issue, not BackupPC. In other words, something
has broken the network connection (in the middle of transferring a
file, so I would presume it isn't due to some idle timeout, dropped
NAT entry, etc). BackupPC has been told by the operating system that
the connection is no longer valid, and so it has "cleaned up" by
removing the in-progress file (partial).
I just completed another backup cycle that failed in the same manner but
this time with a continuous ping with captured output. It didn't miss a
beat.
It takes a day to start (presumably reading ALL the files on the
client takes this long, you could improve disk performance, or
increase RAM on the client to improve this).
You might be right. But it's not a show stopper.
"and then stops with an error" - is that on the first file, or are
some files successfully transferred? Is that the first large file?
Does it always fail on the same file (seems not, since it previously
got many more).
Good points. Confirmed: Not the first file (over 600,000 files
transferred first), not a large file (less than 20Meg), does not always
fail on the same file or directory.
I'm thinking you need to check and/or improve network reliability,
make sure both client and server are not running out of RAM/etc
(mainly the backuppc client, the OOM might kill the rsync process),
etc. Check your system logs on both client and server, and/or watch
top output on both systems during the backup.
The network did not miss a beat and generally appears responsive. It has
been checked. The client and server RAM usage are tracked in Zabbix and
not close to running out. Only curious thing is swap is running out on
the client (Windows Server 2016) even with 10GB RAM available, but still
has about 2GB before crash. Server system logs (kern.log, syslog) show
no signs of issues.
Try backing up other systems, try backing up a smaller subset (exclude
some large directories, and then add them back in if you complete a
backup successfully).
That is a good idea. I'll try adding incrementally to the data backed up.
Overall, I would advise to upgrade to BPC v4.x, it handles backups o
Re: [BackupPC-users] Backup fails after three days, possibly millions of files
Thank you all for your input. More information:
rsync version on client: 3.0.8 (Windows)
rsync version on server: 3.1.2 (Debian)
BackupPC version: 3.3.1
$(Config{ClientTimeout} = 604800
I just compared the output of two verbose BackupPC_dump runs and it
looks like the files are reported to be backed up even though they are
not. For example, this appears in logs of both backup runs:
create 644 4616/545 1085243184 /3412.zip
I checked and the file time stamp is year 2018. The log files are full
of these. I checked the real time clock on both systems and they are
correct. There are also files that have been backed up that are not in
the logs.
I suspect there are over ten million files but I don't have a good way
of telling now. Oddly, there are about 500,000 files backed according
to the log captured from BackupPC_dump and almost the same number
actually backed up and found in pc//0, but they are different
subsets of files. I have been tracking memory and swap usage on the
server and see no issues.
Is this a possible bug in BackupPC 3.3.1?
P.S.: I notice on a new run the files are populating into pc/new folder
but not merging to pc/0, presumably because of the crash.
Regards,
David Koski
dko...@sutinen.com
On 7/8/19 6:16 PM, Adam Goryachev wrote:
On 9/7/19 10:23 am, David Koski wrote:
I am trying to back up about 24TB of data that has millions of
files. It takes a day or to before it starts backing up and then
stops with an error. I did a CLI dump and trapped the output and can
see the error message:
Can't write 32780 bytes to socket
Read EOF: Connection reset by peer
Tried again: got 0 bytes
finish: removing in-process file
Shares/Archives//COR_2630.png
Child is aborting
Done: 589666 files, 1667429241846 bytes
Got fatal error during xfer (aborted by signal=PIPE)
Backup aborted by user signal
Not saving this as a partial backup since it has fewer files than the
prior one (got 589666 and 589666 files versus 4225016)
dump failed: aborted by signal=PIPE
This backup is doing rsync over ssh. I enabled SSH keepalive but it
does not appear to be due to an idle network. It does not appear to
be a random network interruption because the time it takes to fail is
pretty consistent, about three days. I'm stumped.
Did you check:
$Conf{ClientTimeout} = 72000;
Also, what version of rsync on the client, what version of BackupPC on
the server, etc?
I think BPC v4 handles this scenario significantly better, in fact a
server I used to have trouble with on BPC3.x all the time has since
been combined with 4 other server (so 4 x the number of files and
total size of data) and BPC4 handles it easily.
Regards,
Adam
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
[BackupPC-users] Backup fails after three days, possibly millions of files
I am trying to back up about 24TB of data that has millions of files. It takes a day or to before it starts backing up and then stops with an error. I did a CLI dump and trapped the output and can see the error message: Can't write 32780 bytes to socket Read EOF: Connection reset by peer Tried again: got 0 bytes finish: removing in-process file Shares/Archives//COR_2630.png Child is aborting Done: 589666 files, 1667429241846 bytes Got fatal error during xfer (aborted by signal=PIPE) Backup aborted by user signal Not saving this as a partial backup since it has fewer files than the prior one (got 589666 and 589666 files versus 4225016) dump failed: aborted by signal=PIPE This backup is doing rsync over ssh. I enabled SSH keepalive but it does not appear to be due to an idle network. It does not appear to be a random network interruption because the time it takes to fail is pretty consistent, about three days. I'm stumped. David Koski ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] backup hangs, CPU usage high
Lately, one machine has started to hang on backup. It was a Linux machine, now it is a Windows 2003 Server machine that hangs. The state of the machine stays at backup in progress but there is no network traffic and the CPU usage is at 100 percent, with two BackupPC_dump processes, one using slightly more CPU than the other but with a total of 100 percent. I have changed the RsyncArgs and RsyncRestoreArgs options for --devices to -D. Backuppc Version: 2.1.2-6 Backuppc host OS: Debian 4.0 (Etch) kernel: 2.6.12.6-xen All packages up to date. Today I increased XferLogLevel to 6 and ran the dump manually and got the following error message in a repeating loop: create 0 / 0 attribSet(dir=fswsoft, file=) attribSet(dir=fswsoft, file=) makeSpecial(/var/lib/backuppc/pc/fp2003/new//fswsoft/, 9, ) Can't open /var/lib/backuppc/pc/fp2003/new//fswsoft/ for empty output\n Oddly, the directory /var/lib/backuppc/pc/fp2003/new did not exist but I was able to create it. I do have free inodes and I forced fsck.ext3 on the filesystems and they were clean. Any suggestions appreciated. Regards, David Koski [EMAIL PROTECTED] - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] backup keeps failing in exactly the same place
On Sunday 20 April 2008 14:39:33 Benjamin Staffin wrote: I've run into a frustrating problem that affects only one of my backup clients. This particular client is configured as though it is two hosts to backuppc, such that part of its contents go to a compressed pool and the rest goes to an uncompressed pool. Both of the backups have started failing for reasons that are compeltely eluding me. Here's a slightly-trimmed backup log from one of the failures: (kindly ignore the fact that I'm backing up lame anime videos here...) snip exiting after signal ALRM Maybe it is timing out because the file is too big. Try increasing the timeout value in config.pl (the name escapes me) but I had to do that when I had a similar problem. Regards, David Koski [EMAIL PROTECTED] - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] files already in pool are downloaded
I should have mentioned that I do use rsync and have since discovered some pool files do not appear to be downloaded during backup. Thanks, David On Monday 10 September 2007 09:00, Rob Owens wrote: My understanding is that with tar and smb, all files are downloaded (and then discarded if they're already in the pool). Rsync is smart enough, though, not to download files already in the pool. -Rob David Koski wrote: I have been trying to get a good backup with backuppc (2.1.1) but it has been taking days. I ran a dump on the command line so I could see what is going on and I see the files that are in the pool are being downloaded. For example: pool 700 511/1008039 home/daler/My Documents/DRAWINGS/Lakeport/Pics/C_03.tif This is a large file and at 750kb/s takes a while. Is this expected? I thought if they are in the pool they do not need to be downloaded. Thanks in advance, David Koski [EMAIL PROTECTED] - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/ - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/ - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] files already in pool are downloaded, Can't link..
Another wrinkle: Many of these same pool files get an error: Can't link /var/lib/backuppc/pc/bki/new/f%2f/fhome/path-and-file to /var/lib/backuppc/cpool/d/1/9/d19f21440531ec9046070a9ad79190c5 Yet, the pool file does not appear to have many links: -rw-r- 9 backuppc backuppc 38 2007-01-12 19:25 /var/lib/backuppc/cpool/d/1/9/d19f21440531ec9046070a9ad79190c5 Regards, David On Monday 10 September 2007 09:00, Rob Owens wrote: My understanding is that with tar and smb, all files are downloaded (and then discarded if they're already in the pool). Rsync is smart enough, though, not to download files already in the pool. -Rob David Koski wrote: I have been trying to get a good backup with backuppc (2.1.1) but it has been taking days. I ran a dump on the command line so I could see what is going on and I see the files that are in the pool are being downloaded. For example: pool 700 511/1008039 home/daler/My Documents/DRAWINGS/Lakeport/Pics/C_03.tif This is a large file and at 750kb/s takes a while. Is this expected? I thought if they are in the pool they do not need to be downloaded. Thanks in advance, David Koski [EMAIL PROTECTED] - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/ - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/ - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
[BackupPC-users] files already in pool are downloaded
I have been trying to get a good backup with backuppc (2.1.1) but it has been taking days. I ran a dump on the command line so I could see what is going on and I see the files that are in the pool are being downloaded. For example: pool 700 511/1008039 home/daler/My Documents/DRAWINGS/Lakeport/Pics/C_03.tif This is a large file and at 750kb/s takes a while. Is this expected? I thought if they are in the pool they do not need to be downloaded. Thanks in advance, David Koski [EMAIL PROTECTED] - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2005. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
[BackupPC-users] BackupPC_fixupBackupSummary: No info for backup number 207
I installed version 3 to be able to access old archives that had no entries in file backups. But when I run the recovery tool I get: Doing host localhost *** No info for backup number 207 *** No info for backup number 266 *** No info for backup number 322 *** No info for backup number 407 *** No info for backup number 419 *** No info for backup number 416 *** No info for backup number 350 *** No info for backup number 39 *** No info for backup number 418 *** No info for backup number 417 *** No info for backup number 393 *** No info for backup number 379 *** No info for backup number 414 *** No info for backup number 151 *** No info for backup number 415 No changes for host localhost Log files exist: # ls -l LOG* -rw-r- 1 backuppc backuppc 2599 2006-08-13 01:06 LOG -rw-r- 1 backuppc backuppc 799 2006-07-30 01:06 LOG.0.z -rw-r- 1 backuppc backuppc 994 2005-08-31 01:03 LOG.10.z -rw-r- 1 backuppc backuppc 1013 2005-07-31 01:04 LOG.11.z -rw-r- 1 backuppc backuppc 1145 2006-05-31 01:10 LOG.1.z -rw-r- 1 backuppc backuppc 1008 2006-04-30 01:07 LOG.2.z -rw-r- 1 backuppc backuppc 1094 2006-03-31 01:04 LOG.3.z -rw-r- 1 backuppc backuppc 989 2006-02-28 01:05 LOG.4.z -rw-r- 1 backuppc backuppc 1121 2006-01-31 01:20 LOG.5.z -rw-r- 1 backuppc backuppc 1022 2005-12-31 01:13 LOG.6.z -rw-r- 1 backuppc backuppc 994 2005-11-30 01:09 LOG.7.z -rw-r- 1 backuppc backuppc 1030 2005-10-31 01:06 LOG.8.z -rw-r- 1 backuppc backuppc 1170 2005-09-30 01:08 LOG.9.z The host localhost has the following backup summary: 0 fullyes 0 6/21 02:00 1.5 283.4 /var/lib/backuppc/pc/localhost/0 6 fullyes 0 8/5 01:08 2.0 238.4 /var/lib/backuppc/pc/localhost/6 8 incrno 1 8/7 01:06 0.3 236.4 /var/lib/backuppc/pc/localhost/8 9 incrno 1 8/8 01:08 0.2 235.4 /var/lib/backuppc/pc/localhost/9 10 incrno 1 8/9 01:11 0.4 234.4 /var/lib/backuppc/pc/localhost/10 11 incrno 1 8/10 01:07 0.3 233.4 /var/lib/backuppc/pc/localhost/11 12 incrno 1 8/11 01:08 0.3 232.4 /var/lib/backuppc/pc/localhost/12 13 fullyes 0 8/12 01:07 0.9 231.4 /var/lib/backuppc/pc/localhost/13 14 incrno 1 8/13 01:06 0.3 230.4 /var/lib/backuppc/pc/localhost/14 Regards, David Koski david [EMAIL PROTECTED] - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] recovering status.pl
On Thursday 11 January 2007 09:26, you wrote: David writes: It's ok to delete status.pl. Over time the status will get refreshed. It sounds like you deleted the archives file from the pc/HOST directories. There isn't a way to restore those. But they aren't used by BackupPC - they are only there for logging purposes. As it turns out, I had to copy pc/HOST/backups.backup to pc/HOST/backups without the first two lines that corresponded to the two archives that I had deleted. Until I did that I could see no archives in the backuppc application. Of course I had to delete the empty status.pl file too. Sorry, I misunderstood your terminology - I thought you were referring to BackupPC's archives (not the backups). In BackupPC 3.0.0 you can recover a missing or corrupted backups file using the BackupPC_fixupBackupSummary script. That script uses meta data that is saved in each backup's directory. That meta data is stored automatically with each new backup, and pre-3.0.0 backups have the meta data added on each Saturday night if you are already running 3.0.0beta. If you have just upgraded to 3.0.0 and run BackupPC_fixupBackupSummary, you will need to use the -l option which tries to get the necessary date by parsing the log files. Thank you Craig. I'll know next time. David Koski [EMAIL PROTECTED] - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
[BackupPC-users] recovering status.pl
I made the mistake of deleting some archives manually and now my status.pl file is zero size and backuppc will not start unless I delete status.pl but none of my archives are visible in backuppc. How can I recover from this? Thank you, David Koski [EMAIL PROTECTED] - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
[BackupPC-users] 555 errors: Unable to read 65536 bytes from
Two days in a row I got 555 lines of error that are all the same for both days: Unable to read 65536 bytes from /home/backuppc/pc/mail/new//f%2f/RStmp got=0, seekPosn=402653184 (0,4,24581,401604608,402727821) The curious thing is that the size is exactly 64k. And every line of error is the same for two days. I looked at the system logs and there were no errors. Where is this error comming from? Is it a media error? Is it on the client? The server? Is it a bug in backuppc? Thank you, David Koski - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] Unable to read 2757426 bytes from /home/backuppc/pc/mail/new
On Wednesday 06 September 2006 10:19, Les Mikesell wrote: On Wed, 2006-09-06 at 10:01 -0700, David Koski wrote: Last nights backup looks about the same with same errors. There are about 64 of these: Unable to read 2757426 bytes from /home/backuppc/pc/mail/new//f%2f/RStmp got=2287922, seekPosn=78452430 (4608,339,10001,80740352,81347343) The errors have returned. I am getting about 20 exactly like this: Unable to read 7389184 bytes from /home/backuppc/pc/mail/new//f%2f/RStmp got=6832128, seekPosn=364363776 (6656,451,22691,371195904,371762304) ..and about 80 with other numeric parameters. Any idea what might cause this? I still don't understand those, but maybe they are a symptom of: There are about 225 of these: Remote[2]: file has vanished: /var/CommuniGate/Queue/27732874.msg These are normal when backing up active file systems. It just means that the file went away between reading the directory to get the name and opening the file. I'm not sure what backuppc does with growing/shrinking files - tar takes only the length known when the header is written and pads with nulls if the file has shrunk. I have excluded the Queue directory to eliminate these. David Koski [EMAIL PROTECTED] - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] Unable to read 2757426 bytes from /home/backuppc/pc/mail/new
On Monday 18 September 2006 09:18, Les Mikesell wrote: On Mon, 2006-09-18 at 09:10 -0700, David Koski wrote: On Wed, 2006-09-06 at 10:01 -0700, David Koski wrote: Last nights backup looks about the same with same errors. There are about 64 of these: Unable to read 2757426 bytes from /home/backuppc/pc/mail/new//f%2f/RStmp got=2287922, seekPosn=78452430 (4608,339,10001,80740352,81347343) The errors have returned. I am getting about 20 exactly like this: Unable to read 7389184 bytes from /home/backuppc/pc/mail/new//f%2f/RStmp got=6832128, seekPosn=364363776 (6656,451,22691,371195904,371762304) ..and about 80 with other numeric parameters. Any idea what might cause this? I still don't understand those, but maybe they are a symptom of: There are about 225 of these: Remote[2]: file has vanished: /var/CommuniGate/Queue/27732874.msg These are normal when backing up active file systems. It just means that the file went away between reading the directory to get the name and opening the file. I'm not sure what backuppc does with growing/shrinking files - tar takes only the length known when the header is written and pads with nulls if the file has shrunk. I have excluded the Queue directory to eliminate these. Just to clarify, I am not getting file has vanished errors since I excluded the Queue directory from backup. Same problem, different place. The files are disappearing between the time the directory is read to find the name and opening the file for copying. That looks like a maildir directory where frequent changes would be expected. The Unable to read problem indicates the same seekPosn and other parameters every day for the past several days. Do you still think it is a transient file issue? Regards, David - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] Unable to read 2757426 bytes from /home/backuppc/pc/mail/new
On Monday 18 September 2006 09:48, Les Mikesell wrote: On Mon, 2006-09-18 at 09:27 -0700, David Koski wrote: Just to clarify, I am not getting file has vanished errors since I excluded the Queue directory from backup. Same problem, different place. The files are disappearing between the time the directory is read to find the name and opening the file for copying. That looks like a maildir directory where frequent changes would be expected. The Unable to read problem indicates the same seekPosn and other parameters every day for the past several days. Do you still think it is a transient file issue? That's 'not supposed' to happen on filesystems with unix semantics. That is, if a process gets a file open, it is supposed to continue to have access to it's data even if some other process removes or renames it. It could happen if another process truncates it - which doesn't seem likely for programs using maildir format. It also might happen over an NFS mount, although NFS goes through some contortions to avoid it. In any case, it is somehow related to filesystem activity and probably doesn't hurt much as long as you can restore the other files. But this error occurs in the same position day after day, increasing in number of times it is found in the log. Now for a dumb question: Does that error indicate a problem on the backuppc server or the server being backed up? I have been assuming the whole time it was the backuppc server. Thanks, David - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] Unable to read 2757426 byte s from /home/backuppc/pc/mail/new
On Monday 18 September 2006 11:19, Les Mikesell wrote: On Mon, 2006-09-18 at 09:59 -0700, David Koski wrote: But this error occurs in the same position day after day, increasing in number of times it is found in the log. Now for a dumb question: Does that error indicate a problem on the backuppc server or the server being backed up? I have been assuming the whole time it was the backuppc server. It's being reported on the backuppc copy. I was assuming it got that way because it encountered the error when reading the remote file after starting the copy but I could easily be wrong. Thank you. I have removed a drive from the mirror of the backuppc host, zeroed the superblock and readded it to get a fresh resynchronization. I'll see what happens tonight with the next backup. Regards, David Koski [EMAIL PROTECTED] - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys -- and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] Unable to read 2757426 bytes from /home/backuppc/pc/mail/new
On Wednesday 06 September 2006 10:19, Les Mikesell wrote: On Wed, 2006-09-06 at 10:01 -0700, David Koski wrote: Last nights backup looks about the same with same errors. There are about 64 of these: Unable to read 2757426 bytes from /home/backuppc/pc/mail/new//f%2f/RStmp got=2287922, seekPosn=78452430 (4608,339,10001,80740352,81347343) I still don't understand those, but maybe they are a symptom of: There are about 225 of these: Remote[2]: file has vanished: /var/CommuniGate/Queue/27732874.msg These are normal when backing up active file systems. It just means that the file went away between reading the directory to get the name and opening the file. I'm not sure what backuppc does with growing/shrinking files - tar takes only the length known when the header is written and pads with nulls if the file has shrunk. I did a backup excluting the queue directory and the problem went away. Thank you, David - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] Unable to read 2757426 bytes from /home/backuppc/pc/mail/new
On Monday 04 September 2006 12:55, Les Mikesell wrote: On Mon, 2006-09-04 at 13:25, David Koski wrote: I got several of these errors: Unable to read 2757426 bytes from /home/backuppc/pc/mail/new//f%2f/RStmp got=2287922, seekPosn=78452430 (4608,339,10001,80740352,81347343) I also see this in the kernel log: Sep 4 01:35:00 backup kernel: EXT3-fs error (device md(9,0)): ext3_free_blocks: bit already cleared for block 5871769 What do they mean? Sep 4 01:35:00 backup kernel: EXT3-fs error (device md(9,0)): ext3_free_blocks: bit already cleared for block 5871769 Sounds like filesystem corruption to me. Can you unmount the partition and run fsck on it? I did shutdown with the -F option. It reported no errors but refused to boot. I entered maintenance mode and did fsck again (fsck.ext3 -f -vy) with no errors reported. I did shutdown with -F again and it rebooted after fsck. Regards, David - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] Unable to read 2757426 bytes fr om /home/backuppc/pc/mail/new
On Tuesday 05 September 2006 11:49, Les Mikesell wrote: On Tue, 2006-09-05 at 10:26 -0700, David Koski wrote: I got several of these errors: Unable to read 2757426 bytes from /home/backuppc/pc/mail/new//f%2f/RStmp got=2287922, seekPosn=78452430 (4608,339,10001,80740352,81347343) I also see this in the kernel log: Sep 4 01:35:00 backup kernel: EXT3-fs error (device md(9,0)): ext3_free_blocks: bit already cleared for block 5871769 What do they mean? Sep 4 01:35:00 backup kernel: EXT3-fs error (device md(9,0)): ext3_free_blocks: bit already cleared for block 5871769 Sounds like filesystem corruption to me. Can you unmount the partition and run fsck on it? I did shutdown with the -F option. It reported no errors but refused to boot. I entered maintenance mode and did fsck again (fsck.ext3 -f -vy) with no errors reported. I did shutdown with -F again and it rebooted after fsck. Do you still have trouble reading that file? What kind of md device is this? On raid1 mirrors, I think you only need to read one of the underlying partitions so there is a possibility of the mirrors getting out of sync and appearing correct if you read one of them but not the other. I will watch the log but another backup is not scheduled until tonight. It is RAID1 but /proc/mdstat shows they are in sync. David - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
[BackupPC-users] Unable to read 2757426 bytes from /home/backuppc/pc/mail/new
I got several of these errors: Unable to read 2757426 bytes from /home/backuppc/pc/mail/new//f%2f/RStmp got=2287922, seekPosn=78452430 (4608,339,10001,80740352,81347343) I also see this in the kernel log: Sep 4 01:35:00 backup kernel: EXT3-fs error (device md(9,0)): ext3_free_blocks: bit already cleared for block 5871769 What do they mean? Sep 4 01:35:00 backup kernel: EXT3-fs error (device md(9,0)): ext3_free_blocks: bit already cleared for block 5871769 David Koski [EMAIL PROTECTED] - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] backup recovery
On Saturday 19 August 2006 23:15, Craig Barratt wrote: snip In 3.0.0 there is a script to recreate a missing or damaged backups file. It relies on some additional information saved in 3.0.0. However, it also has a mode (using the -l option) that allows it to do the best it can to recover pre-3.0.0 backups too. So, if you are running something prior to 2.1.2 then this is a known bug (fixed since 2.1.2), and I recommend upgrading to 3.0.0 and running: su backuppc BackupPC_fixupBackupSummary -l HOST I just upgraded from 2.1.1-2sarge2 to 2.1.2-5. I will wait for 3.0 to become available for Debian. Thank you much. David Koski [EMAIL PROTECTED] - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
[BackupPC-users] backup recovery
I made some changes to config.pl to reduce the number of backups:
$Conf{FullKeepCnt} = [2,0,0,2,0,0,2];
..whereas before I had [2,2,2,2,2,2,2];
Now I need to recover files from a year ago and they are actually still there
but do not show up in the GUI. The GUI only goes back to 7/29 but for some
reason the files still exist on the backuppc server. When I try to execute:
/usr/share/backuppc/bin/BackupPC_tarCreate -t -n 126 -h
tiikeri /home/users/dkoski/src
(all on one line) I get the error bad backup number 126 for host tiikeri.
How do I restore the files?
Thank you,
David Koski
[EMAIL PROTECTED]
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/
Re: [BackupPC-users] backup recovery [UPDATE]
On Sunday 13 August 2006 20:44, David Koski wrote:
I made some changes to config.pl to reduce the number of backups:
$Conf{FullKeepCnt} = [2,0,0,2,0,0,2];
..whereas before I had [2,2,2,2,2,2,2];
Now I need to recover files from a year ago and they are actually still
there but do not show up in the GUI. The GUI only goes back to 7/29 but for
some reason the files still exist on the backuppc server. When I try to
execute:
/usr/share/backuppc/bin/BackupPC_tarCreate -t -n 126 -h
tiikeri /home/users/dkoski/src
(all on one line) I get the error bad backup number 126 for host tiikeri.
How do I restore the files?
I also reduced $Conf{FullAgeMax} to 30 and noticed the RestoreInfo.xx files
in /var/lib/backuppc from a year ago are lost. Is it still possible to
restore? Is it still possible to get the archive from a year ago to be
visible in the user interface?
Regards,
David Koski
[EMAIL PROTECTED]
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/
[BackupPC-users] Strange error messages in log
I am getting the following error message in the error log: snip log Unable to read 621621 bytes from /home/backuppc/pc/mail/new//f%2f/RStmp got=208181, seekPosn=23909067 (1536,253,10003,24117248,24576026)Unable to read 621621 bytes from /home/backuppc/pc/mail/new//f%2f/RStmp snip log The second line is repeated about 125 times. I thought it might be a disk read error so I did dd if=/dev/hda of=/dev/null bs=1M and the same for hdc. (It is using software raid with /dev/hda and dev/hdc). It produced no errors and the system logs reveal nothing. Any clues as to what it might be? Thank you, David Koski [EMAIL PROTECTED] --- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637alloc_id=16865op=click ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
Re: [BackupPC-users] rsync? tar? rsyncd?
On Saturday 29 October 2005 04:11 pm, Rich Duzenbury wrote: The boxes are currently being backed up via ssh/rsync, but I wonder if ssh encryption is limiting the transfer rates. You might want to set ssh to use the blowfish cipher if you are on a secure network to see if you pick up some speed. David --- This SF.Net email is sponsored by the JBoss Inc. Get Certified Today * Register for a JBoss Training Course Free Certification Exam for All Training Attendees Through End of 2005 Visit http://www.jboss.com/services/certification for more information ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
[BackupPC-users] cwrsync: unable to bind any inbound sockets on port 873
When I start the service it just goes back down in a few seconds (maybe 5). Any clues? OS: XP-Pro Regards, David Koski [EMAIL PROTECTED] --- This SF.Net email is sponsored by the JBoss Inc. Get Certified Today * Register for a JBoss Training Course Free Certification Exam for All Training Attendees Through End of 2005 Visit http://www.jboss.com/services/certification for more information ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
