Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Marcel Bruckner]

Hi,

 

When will there be available a version of backuppc which includes the bugfix
for the stable release?

 

Kind regards,

 

Marcel Bruckner

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Ludovic Drolez]

On Tue, May 31, 2016 at 12:35:39PM +0200, Samuel Wolf wrote:
>Hi Ludovic,
> 
>  +  * Fix for Samba 4.2.x update which broke SMB based backups.
>  +Moreover $Conf{BackupZeroFilesIsFatal} = 0 should be set for SMB 
> backups,
>  +or add '-d 5' to $Conf{SmbClientFullCmd} and $Conf{SmbClientIncrCmd}
>  +in [1]config.pl. Closes: #820963
> 
>  you mean -d 5 than -d 1?
> 
>  $smbClientPath \\$host\$shareName $I_option -U $userName -E -d 1 -c tarmode\ 
> full -Tc$X_option - $fileList
>  $smbClientPath \\$host\$shareName $I_option -U $userName -E -d 5 -c tarmode\ 
> full -Tc$X_option - $fileList

Hi,

Yes, '-d 5' is needed if you have $Conf{BackupZeroFilesIsFatal}=1.
It's also needed for successful restores.

BR,

-- 
Ludovic Drolez.

http://www.aopensource.com - The Android Open Source Portal
http://www.drolez.com - Personal site - Linux and Free Software

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Samuel Wolf]

Hi Ludovic,

+  * Fix for Samba 4.2.x update which broke SMB based backups.
+Moreover $Conf{BackupZeroFilesIsFatal} = 0 should be set for SMB backups,
+or add '-d 5' to $Conf{SmbClientFullCmd} and $Conf{SmbClientIncrCmd}
+in config.pl. Closes: #820963

you mean -d 5 than -d 1?

$smbClientPath \\$host\$shareName $I_option -U $userName -E -d 1 -c
tarmode\ full -Tc$X_option - $fileList
$smbClientPath \\$host\$shareName $I_option -U $userName -E -d 5 -c
tarmode\ full -Tc$X_option - $fileList

Samuel



2016-05-27 10:40 GMT+02:00 Ludovic Drolez :

> On Thu, May 26, 2016 at 11:00:32PM +0200, Samuel Wolf wrote:
> >where I can found this version to test?
>
> Hi! I have to upload it first to 'unstable', it will take a little more
> time before it's accepted in proposed-updates.
>
> Regards,
>
> --
> Ludovic Drolez.
>
> http://www.aopensource.com - The Android Open Source Portal
> http://www.drolez.com - Personal site - Linux and Free Software
>

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Ludovic Drolez]

On Thu, May 26, 2016 at 11:00:32PM +0200, Samuel Wolf wrote:
>where I can found this version to test?

Hi! I have to upload it first to 'unstable', it will take a little more
time before it's accepted in proposed-updates.

Regards,

-- 
Ludovic Drolez.

http://www.aopensource.com - The Android Open Source Portal
http://www.drolez.com - Personal site - Linux and Free Software

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Samuel Wolf]

Hi Ludovic,

2016-05-22 11:30 GMT+02:00 Ludovic Drolez :

> Hi!
>
> Yes, I will upload it on Monday.
>

where I can found this version to test?


> I'm testing a more complete patch that I may release soon, where you
> don't need $Conf{BackupZeroFilesIsFatal} = 0;
>

This sounds great!
Thank you for maintaining BackupPC for Debian.

Samuel


>
> Regards,
>
> --
> Ludovic Drolez.
>
> http://www.aopensource.com - The Android Open Source Portal
> http://www.drolez.com - Personal site - Linux and Free Software
>

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Axel Beckert]

Hi Ludovic,

Ludovic Drolez wrote:
> Yes, I will upload it on Monday.
> I'm testing a more complete patch that I may release soon, where you
> don't need $Conf{BackupZeroFilesIsFatal} = 0;

That sounds great, thanks! Looking forward to it.

Regards, Axel
-- 
 ,''`.  |  Axel Beckert , http://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-|  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Ludovic Drolez]

Hi!

Yes, I will upload it on Monday.
I'm testing a more complete patch that I may release soon, where you
don't need $Conf{BackupZeroFilesIsFatal} = 0;

Regards,

-- 
Ludovic Drolez.

http://www.aopensource.com - The Android Open Source Portal
http://www.drolez.com - Personal site - Linux and Free Software

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Samuel Wolf]

Thank you for the patch Axel.

@ Ludovic,
is it possible to upload a patched version to Debian Jessie?

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Axel Beckert]

Hi,

Axel Beckert wrote:
> Ludovic Drolez wrote:
> > Yes, a full fix won't be easy because smbclient output has changed
> > since samba 4.2.
> 
> There seems to be a rather small patch in RedHat's Bugzilla:
> https://bugzilla.redhat.com/show_bug.cgi?id=1294761
> https://bugzilla.redhat.com/attachment.cgi?id=264
[...]

The patch applies without issues to Debian's backuppc package in
Jessie and works fine for me so far.

> Additionally, $Conf{BackupZeroFilesIsFatal} = 1 seems to be necessary.
> Untested yet, but will test it tomorrow together with the Debian
> package of BackupPC.

This is what was written on
https://github.com/backuppc/backuppc/issues/14, but according to
https://bugzilla.redhat.com/show_bug.cgi?id=1294761#c3 the opposite is
the case and it needs to be set to "0", not to "1":

  $Conf{BackupZeroFilesIsFatal} = 0;

That also makes more sense since it previously was "1", at least on
our boxes.

But before we can upload a patched package to stable-proposed-updates,
a fixed package needs to be uploaded to Unstable first.

Regards, Axel
-- 
 ,''`.  |  Axel Beckert , http://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-|  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Axel Beckert]

Control: tag -1 + patch

Hi,

Ludovic Drolez wrote:
> Yes, a full fix won't be easy because smbclient output has changed
> since samba 4.2.

There seems to be a rather small patch in RedHat's Bugzilla:
https://bugzilla.redhat.com/show_bug.cgi?id=1294761
https://bugzilla.redhat.com/attachment.cgi?id=264

To be sure, carbon copying it here:

diff --git a/lib/BackupPC/CGI/Browse.pm b/lib/BackupPC/CGI/Browse.pm
index b977f9b..6dec5fe 100644
--- a/lib/BackupPC/CGI/Browse.pm
+++ b/lib/BackupPC/CGI/Browse.pm
@@ -65,7 +65,7 @@ sub action
 #
 # default to the newest backup
 #
-if ( !defined($In{num}) && defined(@Backups) && @Backups > 0 ) {
+if ( !defined($In{num}) && @Backups > 0 ) {
 $i = @Backups - 1;
 $num = $Backups[$i]{num};
 }
diff --git a/lib/BackupPC/Xfer/Smb.pm b/lib/BackupPC/Xfer/Smb.pm
index eaf002e..29c1ccd 100644
--- a/lib/BackupPC/Xfer/Smb.pm
+++ b/lib/BackupPC/Xfer/Smb.pm
@@ -230,7 +230,9 @@ sub readOutput
 $t->{byteCnt} += $2;
 $t->{fileCnt}++;
 $t->{XferLOG}->write(\"$_\n") if ( $t->{logLevel} >= 1 );
-} elsif ( /^\s*tar: dumped \d+ files/ ) {
+} elsif ( /^\s*tar: dumped \d+ files/
+|| /Total bytes received: \d+/i
+   ) {
 $t->{xferOK} = 1;
 $t->{XferLOG}->write(\"$_\n") if ( $t->{logLevel} >= 0 );
 } elsif ( /^\s*tar: restored \d+ files/ ) {
@@ -270,6 +272,7 @@ sub readOutput
 } elsif ( /^\s*directory \\/i ) {
 $t->{XferLOG}->write(\"$_\n") if ( $t->{logLevel} >= 2 );
 } elsif ( /smb: \\>/
+|| /^tar:\d+/
 || /^\s*added interface/i
 || /^\s*tarmode is now/i
 || /^\s*Total bytes written/i

Additionally, $Conf{BackupZeroFilesIsFatal} = 1 seems to be necessary.
Untested yet, but will test it tomorrow together with the Debian
package of BackupPC.

Found via https://github.com/backuppc/backuppc/issues/14

Regards, Axel
-- 
 ,''`.  |  Axel Beckert , http://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-|  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Ludovic Drolez]

Hi !

(Debian bug URL
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=820963 )

Yes, a full fix won't be easy because smbclient output has changed
since samba 4.2.

For example in 4.1.x the following code was present in
samba/source3/client/clitar.c in do_atar(): 

  if (tar_noisy) {
   DEBUG(0, ("%12.0f (%7.1f kb/s) %s\n",
 (double)finfo.size, finfo.size / MAX(0.001, (1.024*this_time)),
 finfo.name));
  }

clitar.c has been rewritten in 4.2+...

-- 
Ludovic Drolez.

http://www.aopensource.com - The Android Open Source Portal
http://www.drolez.com - Personal site - Linux and Free Software

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Samuel Wolf]

https://bugzilla.redhat.com/show_bug.cgi?id=1294761

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Samuel Wolf]

Please ignore Message # 15, I copy/paste the wrong links, sorry. :-/

In my config.pl was no -N option (do not add it), with this setting it
fails again:
$Conf{SmbClientFullCmd} = '$smbClientPath $host\\$shareName $I_option
-U $userName -E -d 1 -c tarmode\\ full\\ verbose -Tc$X_option - $fileList';

As far I understand the output of tarmode is different.

tarExtract: Done: 0 errors, 16 filesExist, 2360211 sizeExist, 2163583
sizeExistComp, 349 filesTotal, 2438778 sizeTotal
Got fatal error during xfer (tar:711  Total bytes received: 2380691)
Backup aborted (tar:711  Total bytes received: 2380691)



2016-04-15 11:54 GMT+02:00 Ludovic Drolez :

> Hello,
>
> Please could you try to edit /etc/backupcc/config.pl and modify the 3
> following lines:
>
> 1- Find the lines:
>
> $Conf{SmbClientFullCmd} = '$smbClientPath $host\\$shareName
> $I_option -U $userName -E -N -d 1 -c tarmode\\ full -Tc$X_option -
> $fileList';
>
> {SmbClientIncrCmd} = '$smbClientPath $host\\$shareName
> $I_option -U $userName -E -N -d 1 -c tarmode\\ full -TcN$X_option
> $timeStampFile - $fileList';
>
> $Conf{SmbClientRestoreCmd} = '$smbClientPath $host\\$shareName
> $I_option -U $userName -E -N -d 1 -c tarmode\\ full -Tx -';
>
> 2- Add the verbose option after 'full' using double backslashes as
> follows:
>
> $Conf{SmbClientFullCmd} = '$smbClientPath $host\\$shareName
> $I_option -U $userName -E -N -d 1 -c tarmode\\ full\\ verbose -Tc$X_option
> -
> $fileList';
>
> {SmbClientIncrCmd} = '$smbClientPath $host\\$shareName
> $I_option -U $userName -E -N -d 1 -c tarmode\\ full\\ verbose -TcN$X_option
> $timeStampFile - $fileList';
>
> $Conf{SmbClientRestoreCmd} = '$smbClientPath $host\\$shareName
> $I_option -U $userName -E -N -d 1 -c tarmode\\ full\\ verbose -Tx -';
>
>
> Best regards,
>
> --
> Ludovic Drolez.
>
> http://www.aopensource.com - The Android Open Source Portal
> http://www.drolez.com - Personal site - Linux and Free Software
>

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Ludovic Drolez]

Hello,

Please could you try to edit /etc/backupcc/config.pl and modify the 3
following lines:

1- Find the lines:

$Conf{SmbClientFullCmd} = '$smbClientPath $host\\$shareName
$I_option -U $userName -E -N -d 1 -c tarmode\\ full -Tc$X_option -
$fileList';

{SmbClientIncrCmd} = '$smbClientPath $host\\$shareName
$I_option -U $userName -E -N -d 1 -c tarmode\\ full -TcN$X_option
$timeStampFile - $fileList';

$Conf{SmbClientRestoreCmd} = '$smbClientPath $host\\$shareName
$I_option -U $userName -E -N -d 1 -c tarmode\\ full -Tx -';

2- Add the verbose option after 'full' using double backslashes as
follows:

$Conf{SmbClientFullCmd} = '$smbClientPath $host\\$shareName
$I_option -U $userName -E -N -d 1 -c tarmode\\ full\\ verbose -Tc$X_option -
$fileList';

{SmbClientIncrCmd} = '$smbClientPath $host\\$shareName
$I_option -U $userName -E -N -d 1 -c tarmode\\ full\\ verbose -TcN$X_option
$timeStampFile - $fileList';

$Conf{SmbClientRestoreCmd} = '$smbClientPath $host\\$shareName
$I_option -U $userName -E -N -d 1 -c tarmode\\ full\\ verbose -Tx -';


Best regards,

-- 
Ludovic Drolez.

http://www.aopensource.com - The Android Open Source Portal
http://www.drolez.com - Personal site - Linux and Free Software

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Samuel Wolf]

See also:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=820963

In my eyes this is a very critical issue since BackupPC stop backup all
Windows shares.

Bug#820963: Backup with XferMethod smb and Samba 4.2 impossible in Debian Jessie

[Samuel Wolf]

Sorry, copy/past error:
http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-mailing-lists-3/backuppc-21/issue-with-samba-client-4-1-6-and-newer-125734/