[BackupPC-users] backuppc 4.2.1 - large delta backup - partial backups never complete

2018-09-05 Thread Jonathan Molyneux 

Hi All,

Running backuppc 4.2.1 that's pulling a dataset using rsyncd over 
openvpn from an ADSL tail (the connection uploads at around 50kb/s).


The problem I'm having is when there was a large change in delta 
(currently rsync shows 34gb~that's going to take about 9 days to 
transfer) the backups never complete.
Due to circumstances, possible isp or line noise the connection has been 
resetting and interrupting the transfer resulting in the logs showing:


2018-08-31 04:18:01 incr backup started for directory backup
2018-09-03 05:36:47 Got fatal error during xfer (rsync error: error in 
rsync protocol data stream (code 12) at io.c(629) [generator=3.0.9.11])
2018-09-03 05:36:53 Backup aborted (rsync error: error in rsync protocol 
data stream (code 12) at io.c(629) [generator=3.0.9.11])


This has been happening for weeks resulting in many partial backups but 
no completed backups.


Have investigated / attempted the following:

* Although the partial backups are used by the next backup when 
attempted, due to the interruptions no two backups in a row are covering 
the whole delta.
* Have confirmed from the logs that rsyncd ends up sending the same 
files over and over (what files change each attempt depending on what 
the previous partial backup contains).
* Have attempted to resolve the connection stability, but due to 
circumstances there inevitably will be interruptions of connectivity 
between sites.
* The rsyncd connection is using a timeout of 10800 (3 hours) and have 
validated there are no files large enough to trigger the timeout (~1/2 
gig). The files themselves are mostly images that are about ~5mb in size 
a piece.
* Have resolved this previously by taking a copy of the data offsite and 
pointing backuppc to it and "seeding" the backup. Then incremental's 
occur organically.
However this is getting old, especially when it's often triggered by a 
reorganize of existing files (folders renamed) due to temporary data 
then being sorted.


Backuppc doesn't seem to handle this circumstance well, it transfers the 
file it already has then deduplicates it when saving into the pool (from 
what I can see, when the files at a new path).
The partial backups are working, but no two partials cover the total 
dataset needing to be transferred.


Is it possible to get backuppc to handle this use case/circumstance better ?
Am I missing something or possibly backuppc is miss configured ?

Any help or advice is appreciated.

Regards
Jonathan


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Troubleshooting assistance for Bad Transfer

2018-09-05 Thread Jaime Fenton 
Hi All,

Just to bump my original thread. Does anyone have any advice or next steps I 
can test to get my backups working.

I have two machines that won't backup, one's a MS surface that's the same build 
as several others that do work and a completely new computer which is a Zbook 
15 G5 and it's having the exact same issues.

Errors I'm seeing:


* NT_STATUS_ACCESS_DENIED

* NT_STATUS_SHARING_VIOLATION

* NT_STATUS_CONNECTION_DISCONNECTED

I'm just not sure how to troubleshoot it as it's working fine for other 
computers. And as mentioned smbclient connects to those machines perfectly fine.

It does seem to backup partially and those files are accessible, but it still 
errors out in the end.

Jaime


From: Jaime Fenton [mailto:jaime.fen...@animallogic.ca]
Sent: Wednesday, August 29, 2018 12:57 PM
To: General list for user discussion, questions and support 

Subject: Re: [BackupPC-users] Troubleshooting assistance for Bad Transfer

I can confirm that I can connect to the machine in question with smbclient and 
the backuppc account.



From: Michael Stowe [mailto:michael.st...@member.mensa.org]
Sent: Wednesday, August 29, 2018 11:49 AM
To: General list for user discussion, questions and support 
mailto:backuppc-users@lists.sourceforge.net>>
Cc: Jaime Fenton 
mailto:jaime.fen...@animallogic.ca>>
Subject: Re: [BackupPC-users] Troubleshooting assistance for Bad Transfer


On 2018-08-28 14:09, Jaime Fenton wrote:

Hi there,

On advice from craigbarrett, I'm posting this query here.

https://github.com/backuppc/backuppc/issues/218

I've gone to the XferLog file and it's gibberish. Is there a way to compile 
that file or advice on troubleshooting that file that can give me some insight 
as to why my backups are having fatal errors?

Thanks in advance,

Jaime Microsoft Office Outlook Signature - Jaime Fenton Support Engineer

>From what you posted in the bug report, you're trying to back up an

Administrative share via smb.

Once you sort out how to read logs, I'll note that various Microsoft updates 
have disabled Administrative shares, and smb is notoriously picky, and cannot 
handle things like open files.

As a general troubleshooting step, it's a good idea to ensure you can actually 
access the system via smbclient, especially for something dubious like an 
Administrative share.

--
Jaime Fenton
Support Engineer

T: +1 604 398 4800 (main)
D: +1 604 398 4813 (direct)
E: jaime.fen...@animallogic.ca

840 Cambie Street
Vancouver, BC V6B 2P6
CANADA


  [LinkedIn]   [Facebook] 
   [Twitter] 
   [Instagram] 


[Animal Logic]

www.animallogic.com

CONFIDENTIALITY AND PRIVILEGE NOTICE
This email is intended only to be read or used by the addressee. It is 
confidential and may contain privileged information. If you are not the 
intended recipient, any use, distribution, disclosure or copying of this email 
is strictly prohibited. Confidentiality and legal privilege attached to this 
communication are not waived or lost by reason of the mistaken delivery to you. 
If you have received this email in error, please delete it and notify us 
immediately by telephone or email.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Troubleshooting assistance for Bad Transfer

2018-09-05 Thread Kris Lou via BackupPC-users 
Then you might be running into some of the file-locking issues inherent to
SMB, especially if you can get a partial backup.  Consider trying a
different method, such as VSS-rsync?
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

[BackupPC-users] rsync-vshadow-winexe client

2018-09-05 Thread tbuchanan 
There used to be a rsync-vshadow-winexe client for backupc on the 
www.michaelstowe.com web site but I can not seem to access this any 
longer.  Is this no longer a recommended setup?  I'm looking to use 
vshadow and thought this was a good way to implement it.  Suggestions for 
good documentation on setup would also be appreciated.


-- 
Ted Buchanan


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Troubleshooting assistance for Bad Transfer

2018-09-05 Thread Jaime Fenton 
No dice, rsync does not work (nor did rsyncd, failed outright).

Any further advice? If it helps, the two machines failing are windows 10.

Jaime

From: Kris Lou via BackupPC-users [mailto:backuppc-users@lists.sourceforge.net]
Sent: Wednesday, September 5, 2018 12:05 PM
To: General list for user discussion, questions and support 

Cc: Kris Lou 
Subject: Re: [BackupPC-users] Troubleshooting assistance for Bad Transfer

Then you might be running into some of the file-locking issues inherent to SMB, 
especially if you can get a partial backup.  Consider trying a different 
method, such as VSS-rsync?


--
Jaime Fenton
Support Engineer

T: +1 604 398 4800 (main)
D: +1 604 398 4813 (direct)
E: jaime.fen...@animallogic.ca

840 Cambie Street
Vancouver, BC V6B 2P6
CANADA


  [LinkedIn]    [Facebook] 
   [Twitter] 
   [Instagram] 


[Animal Logic]

www.animallogic.com

CONFIDENTIALITY AND PRIVILEGE NOTICE
This email is intended only to be read or used by the addressee. It is 
confidential and may contain privileged information. If you are not the 
intended recipient, any use, distribution, disclosure or copying of this email 
is strictly prohibited. Confidentiality and legal privilege attached to this 
communication are not waived or lost by reason of the mistaken delivery to you. 
If you have received this email in error, please delete it and notify us 
immediately by telephone or email.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/