Re: [BackupPC-users] [BackupPC-devel] rsync+xattr breaks backuppc

2014-08-08 Thread Kenneth Porter
--On Friday, August 08, 2014 1:17 AM +0200 Holger Parplies 
wb...@parplies.de wrote:

 Making assumptions about the syntax of a freely configurable command is a
 step in the wrong direction.

We've gotten spoiled with network protocols that do extensive negotiation 
and fall back to the best effort minimum service, such as with ssh. I guess 
rsync's protocol is less forgiving in this way.




--
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] [BackupPC-devel] rsync+xattr breaks backuppc

2014-08-08 Thread Les Mikesell
On Fri, Aug 8, 2014 at 4:35 AM, Kenneth Porter sh...@sewingwitch.com wrote:
 --On Friday, August 08, 2014 1:17 AM +0200 Holger Parplies
 wb...@parplies.de wrote:

 Making assumptions about the syntax of a freely configurable command is a
 step in the wrong direction.

 We've gotten spoiled with network protocols that do extensive negotiation
 and fall back to the best effort minimum service, such as with ssh. I guess
 rsync's protocol is less forgiving in this way.

The backuppc rsync implementation is a complete rewrite in perl to
permit comparing the local compressed copy with the remote native
rsync's view of the uncompressed version so it is sort of amazing that
it exists at all.   I think the upcoming 4.x version will use native
rsync on the server side too, but I haven't looked at the details of
how that will work or if it supports any extra features besides not
having to send the whole directory tree before starting the file
comparisons.

-- 
   Les Mikesell
 lesmikes...@gmail.com

--
Want fast and easy access to all the code in your enterprise? Index and
search up to 200,000 lines of code with a free copy of Black Duck
Code Sight - the same software that powers the world's largest code
search on Ohloh, the Black Duck Open Hub! Try it now.
http://p.sf.net/sfu/bds
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] [BackupPC-devel] rsync+xattr breaks backuppc

2014-08-07 Thread Holger Parplies
Hi,

I'm moving this to backuppc-users as it most definitely does not concern
BackupPC development.

subscr...@gmail.com wrote on 2014-08-07 17:04:07 +0200 [[BackupPC-devel] 
rsync+xattr breaks backuppc]:
 [...]
 My installation stopped working, long story short it was because I had 
 added --xattr to the rsync parameter list. The fact that it failed was 
 bad, but not as bad as the complete lack of meaningful error messages 
 which led me to think the culprit was a recent update of perl packages.
 I append the log at the end of this mail.
 
 I found an old message in the users mailing list saying that xattr/acls 
 are not supported, however they shouldn't kill backuppc. Maybe it could 
 be shown as a warning if found in the config?

firstly, config.pl *does* explicitly state that you shouldn't mess around with
$Conf {RsyncArgs} unless you understand how File::RsyncP works. It could be
more explicit in the comment to RsyncArgsExtra in that you can't just add
anything and expect things to still work. The hint Examples of additional
arguments that should work may be too subtle for some readers.

Secondly, the fact that it fails is *not* bad. It is to be expected. If you
get your configuration wrong, it won't work. If you supply an incorrect user
or host name (in the 'ssh' part), BackupPC won't be able to tell, and it also
won't be able to connect. Presumably, the host is down (or the NIS server for
user authentication), your ethernet cable broken or your firewall configuration
too restrictive. Does this analysis help finding your typo?
If you change the rsync protocol exchange through extra command line arguments,
BackupPC won't understand what the remote rsync is trying to tell it. Who knows
what parameters future rsync versions might introduce that have an effect on
the protocol exchange? If you mistype 'rsync' (perhaps leaving out the 'r', or
typing 'cpio' instead), you, likewise, won't get a working backup. You *need*
a correct command line that does what BackupPC expects. It's as simple as that.

Thirdly, what exactly should BackupPC attempt to detect? rsync parameters that
are expected not to work? Only '--xattr'? What about future parameters that
will be incompatible? Who says I can't come up with an rsync command line
where those parameters *will* work? Maybe '--xattr' is an argument to an
'--exclude'. Who says the remote end even *is* rsync? Maybe it's my secret
copy program that is able to emulate rsync where all parameters have a
completely different meaning. Maybe it's a wrapper script which discards some
parameters and adds others. It might even try to emulate some (like --xattr;
e.g. by creating a file containing the extended attribute information).

Making assumptions about the syntax of a freely configurable command is a step
in the wrong direction. Instead of a clearly definable requirement (start
rsync with a few required parameters and some optional ones from a set of
supported options), you end up with a mess including arbitrary restrictions
(... but don't include the literal texts --xattr, -z, ...). And you gain
*nothing* except a warning that might be helpful in some cases and a nuisance
in others. Backups with --xattr will still fail.

Fourthly, what is a meaningful error message, and how do you propose to
generate it?

 [...]
 Running: /usr/bin/ssh -q -x -l backuppc mailer2 nice -n 10 sudo 
 /usr/bin/rsync --server --sender --server --sender --numeric-ids --perms 
 --owner --group -D --links --hard-links --times --block-size=2048 
 --recursive --specials --checksum-seed=32761 --xattrs . /
 [...]
 Got fatal error during xfer (fileListReceive failed)
 Backup aborted (fileListReceive failed)

For quite a few people on this list, that message *is* meaningful in
combination with your command line. When BackupPC determines that the remote
end doesn't speak a version of the rsync protocol it understands, how much
analysis do you expect it to do? The message fileListReceive failed allows
anyone to look into the code (or at his command line) and determine what step
went wrong. If that needs to be translated to a list of possible causes, I'd
suggest the wiki rather than the BackupPC code.


What I could *imagine* is writing a script BackupPC_analyseConfig that reads a
host configuration and prints out a diagnostic message, perhaps something
like:

Host 'mailer2' exists in the BackupPC 'hosts' file.
Host configuration file found: /etc/backuppc/pc/mailer2.pl
You have configured 'mailer2' for rsync backups.
You seem to be connecting with 'ssh' as user 'backuppc' and using
'nice -n 10 sudo' to escalate privileges. You have enabled checksum
caching. Your extra rsync args are set to '--xattrs'. These contain
the suspicious option '--xattrs' which will probably cause backups
to fail.
Trying passwordless ssh connection to 'mailer2' as 'backuppc'. If
prompted to accept a public key, answer 'yes' (preferably after