edso has proposed merging lp:~ed.so/duplicity/lftp.ncftp.and.prefixes into lp:duplicity.
Requested reviews: duplicity-team (duplicity-team) For more details, see: https://code.launchpad.net/~ed.so/duplicity/lftp.ncftp.and.prefixes/+merge/240149 - retire --ssh-backend, --use-scp commands - introduce scheme prefixes for alternative backend selection e.g. ncftp+ftp://, see manpage - scp is now selected via scheme e.g. scp:// - added lftp fish, webdav(s), sftp support -- https://code.launchpad.net/~ed.so/duplicity/lftp.ncftp.and.prefixes/+merge/240149 Your team duplicity-team is requested to review the proposed merge of lp:~ed.so/duplicity/lftp.ncftp.and.prefixes into lp:duplicity.
=== modified file 'bin/duplicity.1' --- bin/duplicity.1 2014-10-23 11:56:13 +0000 +++ bin/duplicity.1 2014-10-30 18:01:04 +0000 @@ -68,22 +68,14 @@ .B Rackspace CloudFiles Pyrax API - http://docs.rackspace.com/sdks/guide/content/python.html .TP -.B "dpbx backend" (Dropbox) +.BR "dpbx backend" " (Dropbox)" .B Dropbox Python SDK - https://www.dropbox.com/developers/reference/sdk .TP -.B "copy backend" (Copy.com) +.BR "copy backend" " (Copy.com)" .B python-urllib3 - https://github.com/shazow/urllib3 .TP -.B "ftp backend" -.B NcFTP Client -- http://www.ncftp.com/ -.TP -.B "ftps backend" -.B LFTP Client -- http://lftp.yar.ru/ -.TP .BR "gdocs backend" " (Google Docs)" .B Google Data APIs Python Client Library - http://code.google.com/p/gdata-python-client/ @@ -95,17 +87,25 @@ .B D-Bus (dbus)- http://www.freedesktop.org/wiki/Software/dbus .TP -.B "rsync backend" -.B rsync client binary -- http://rsync.samba.org/ +.BR "lftp backend" " (needed for ftp, ftps, fish [over ssh] - also supports sftp, webdav[s])" +.B LFTP Client +- http://lftp.yar.ru/ .TP .BR "mega backend" " (mega.co.nz)" .B Python library for mega API - https://github.com/ckornacker/mega.py, ubuntu ppa - ppa:ckornacker/backup .TP +.BR "ncftp backend" " (ftp, select via ncftp+ftp://)" +.B NcFTP +- http://www.ncftp.com/ +.TP .B "Par2 Wrapper Backend" .B par2cmdline - http://parchive.sourceforge.net/ +.TP +.B "rsync backend" +.B rsync client binary +- http://rsync.samba.org/ .PP There are two .B ssh backends @@ -1067,87 +1067,169 @@ or preceded by a double slash, '//path', to represent an absolute filesystem path. .PP +.B Note: +.RS +Scheme (protocol) access may be provided by more than one backend. +In case the default backend is buggy or simply not working in a specific case it might be worth trying an alternative implementation. +Alternative backends can be selected by prefixing the scheme with the name of the alternative backend e.g. +.B ncftp+ftp:// +and are mentioned below the scheme's syntax summary. +.RE + +.PP Formats of each of the URL schemes follow: + +.PP +.BR "Cloud Files" " (Rackspace)" +.PP .RS -.PP -.BI "Rackspace Cloud Files" -.br cf+http://container_name -.br +.PP See also .B "A NOTE ON CLOUD FILES ACCESS" -.PP -.BI Dropbox -.br +.RE +.PP +.B "Copy cloud storage" +.PP +.RS +copy://user[:password]@copy.com/some_dir +.RE +.PP +.B Dropbox +.PP +.RS dpbx:///some_dir -.br +.PP Make sure to read .BR "A NOTE ON DROPBOX ACCESS" " first!" -.PP -copy://user[:password]@copy.com/some_dir -.PP -.PP +.RE +.PP +.B "Local file path" +.PP +.RS file://[relative|/absolute]/local/path -.PP +.RE +.PP +.BR "FISH" " (Files transferred over Shell protocol) over ssh" +.PP +.RS +fish://user[:password]@other.host[:port]/[relative|/absolute]_path +.RE +.PP +.B "FTP" +.PP +.RS ftp[s]://user[:password]@other.host[:port]/some_dir .PP +.B NOTE: +use lftp+, ncftp+ prefixes to enforce a specific backend, e.g. ncftp+ftp://... +.RE +.PP +.B "Google Docs" +.PP +.RS gdocs://user[:password]@other.host/some_dir -.PP -.BI "Google Cloud Storage" -.br +.RE +.PP +.B "Google Cloud Storage" +.PP +.RS gs://bucket[/prefix] -.PP +.RE +.PP +.B "HSI" +.PP +.RS hsi://user[:password]@other.host/some_dir -.PP +.RE +.PP +.B "IMAP email storage" +.PP +.RS imap[s]://user[:password]@host.com[/from_address_prefix] -.br +.PP See also .B "A NOTE ON IMAP" -.PP +.RE +.PP +.B "Mega cloud storage" +.PP +.RS mega://user[:password]@mega.co.nz/some_dir -.PP -.BI "Par2 Wrapper Backend" -.br +.RE +.PP +.B "Par2 Wrapper Backend" +.PP +.RS par2+scheme://[user[:password]@]host[:port]/[/]path -.br +.PP See also .B "A NOTE ON PAR2 WRAPPER BACKEND" -.PP -.B "using rsync daemon" -.br +.RE +.PP +.B "Rsync via daemon" +.PP +.RS rsync://user[:password]@host.com[:port]::[/]module/some_dir -.br -.B "using rsync over ssh (only key auth)" -.br +.PP +.RE +.B "Rsync over ssh (only key auth)" +.PP +.RS rsync://[email protected][:port]/[relative|/absolute]_path -.PP +.RE +.PP +.BR "S3 storage" " (Amazon)" +.PP +.RS s3://host/bucket_name[/prefix] .br s3+http://bucket_name[/prefix] -.br +.PP See also .B "A NOTE ON EUROPEAN S3 BUCKETS" -.PP -scp://.. or ssh://.. are synonymous with -.br -sftp://user[:password]@other.host[:port]/[/]some_dir +.RE +.PP +.B "SCP/SFTP access" +.PP +.RS +scp://.. or +.br +sftp://user[:password]@other.host[:port]/[relative|/absolute]_path +.PP +.BR "defaults" " are paramiko+scp:// and paramiko+sftp://"; +.br +.BR "alternatively" " try pexpect+scp://, pexpect+sftp://, lftp+sftp://"; .br See also -.BR --ssh-backend , .BR --ssh-askpass , -.BR --use-scp , .B --ssh-options and .BR "A NOTE ON SSH BACKENDS" . -.PP +.RE +.PP +.BR "Swift" " (Openstack)" +.PP +.RS swift://container_name -.br +.PP See also .B "A NOTE ON SWIFT (OPENSTACK OBJECT STORAGE) ACCESS" -.PP +.RE +.PP +.B "Tahoe-LAFS" +.PP +.RS tahoe://alias/directory -.PP +.RE +.PP +.B "WebDAV" +.PP +.RS webdav[s]://user[:password]@other.host[:port]/some_dir +.PP +.B alternatively +try lftp+webdav[s]:// .RE .SH TIME FORMATS === modified file 'duplicity/backend.py' --- duplicity/backend.py 2014-10-27 02:27:36 +0000 +++ duplicity/backend.py 2014-10-30 18:01:04 +0000 @@ -32,6 +32,7 @@ import re import getpass import gettext +import re import types import urllib import urlparse @@ -164,6 +165,11 @@ _backend_prefixes[scheme] = backend_factory +def strip_prefix(url_string, prefix_scheme): + """ + strip the prefix from a string e.g. par2+ftp://... -> ftp://... + """ + return re.sub('(?i)^'+re.escape(prefix_scheme)+'\+','',url_string) def is_backend_url(url_string): """ @@ -198,7 +204,7 @@ for prefix in _backend_prefixes: if url_string.startswith(prefix + '+'): factory = _backend_prefixes[prefix] - pu = ParsedUrl(url_string.lstrip(prefix + '+')) + pu = ParsedUrl(strip_prefix(url_string,prefix)) break if factory is None: @@ -337,11 +343,8 @@ def strip_auth_from_url(parsed_url): """Return a URL from a urlparse object without a username or password.""" - # Get a copy of the network location without the username or password. - straight_netloc = parsed_url.netloc.split('@')[-1] - - # Replace the full network location with the stripped copy. - return parsed_url.geturl().replace(parsed_url.netloc, straight_netloc, 1) + clean_url = re.sub('^([^:/]+://)(.*@)?(.*)',r'\1\3',parsed_url.geturl()) + return clean_url def _get_code_from_exception(backend, operation, e): if isinstance(e, BackendException) and e.code != log.ErrorCode.backend_error: === renamed file 'duplicity/backends/ftpbackend.py' => 'duplicity/backends/lftpbackend.py' --- duplicity/backends/ftpbackend.py 2014-10-01 20:35:16 +0000 +++ duplicity/backends/lftpbackend.py 2014-10-30 18:01:04 +0000 @@ -3,7 +3,10 @@ # Copyright 2002 Ben Escoto <[email protected]> # Copyright 2007 Kenneth Loafman <[email protected]> # Copyright 2010 Marcel Pennewiss <[email protected]> -# Copyright 2014 Moritz Maisel <[email protected]> +# Copyright 2014 Edgar Soldin +# - webdav, fish, sftp support +# - https cert verification switches +# - debug output # # This file is part of duplicity. # @@ -23,15 +26,15 @@ import os import os.path +import re import urllib -import re import duplicity.backend from duplicity import globals from duplicity import log from duplicity import tempdir -class FTPBackend(duplicity.backend.Backend): +class LFTPBackend(duplicity.backend.Backend): """Connect to remote store using File Transfer Protocol""" def __init__(self, parsed_url): duplicity.backend.Backend.__init__(self, parsed_url) @@ -54,61 +57,141 @@ self.parsed_url = parsed_url - self.url_string = duplicity.backend.strip_auth_from_url(self.parsed_url) +# self.url_string = duplicity.backend.strip_auth_from_url(self.parsed_url) +# # strip lftp+ prefix +# self.url_string = duplicity.backend.strip_prefix(self.url_string, 'lftp') + + self.scheme = duplicity.backend.strip_prefix( parsed_url.scheme, 'lftp' ).lower() + self.scheme = re.sub('^webdav','http',self.scheme) + self.url_string = self.scheme + '://' + parsed_url.hostname + if parsed_url.port : + self.url_string += ":%s" % parsed_url.port + + self.remote_path = re.sub('^/','',parsed_url.path) # Use an explicit directory name. - if self.url_string[-1] != '/': - self.url_string += '/' + if self.remote_path[-1] != '/': + self.remote_path += '/' - self.password = self.get_password() + self.authflag = '' + if self.parsed_url.username: + self.username = self.parsed_url.username + self.password = self.get_password() + self.authflag = "-u '%s,%s'" % (self.username,self.password) if globals.ftp_connection == 'regular': self.conn_opt = 'off' else: self.conn_opt = 'on' - if parsed_url.port != None and parsed_url.port != 21: - self.portflag = " -p '%s'" % (parsed_url.port) - else: - self.portflag = "" + # check for cacert file if https + self.cacert_file = globals.ssl_cacert_file + if self.scheme == 'https' and not globals.ssl_no_check_certificate: + cacert_candidates = [ "~/.duplicity/cacert.pem", \ + "~/duplicity_cacert.pem", \ + "/etc/duplicity/cacert.pem" ] + # + if not self.cacert_file: + for path in cacert_candidates : + path = os.path.expanduser(path) + if (os.path.isfile(path)): + self.cacert_file = path + break + # still no cacert file, inform user + if not self.cacert_file: + raise duplicity.errors.FatalBackendException("""For certificate verification a cacert database file is needed in one of these locations: %s +Hints: + Consult the man page, chapter 'SSL Certificate Verification'. + Consider using the options --ssl-cacert-file, --ssl-no-check-certificate .""" % ", ".join(cacert_candidates) ) self.tempfile, self.tempname = tempdir.default().mkstemp() + os.write(self.tempfile, "set ssl:verify-certificate " + ( "false" if globals.ssl_no_check_certificate else "true" ) + "\n") + if globals.ssl_cacert_file : + os.write(self.tempfile, "set ssl:ca-file '" + globals.ssl_cacert_file + "'\n") os.write(self.tempfile, "set ftp:ssl-allow true\n") os.write(self.tempfile, "set ftp:ssl-protect-data true\n") os.write(self.tempfile, "set ftp:ssl-protect-list true\n") + os.write(self.tempfile, "set http:use-propfind true\n") os.write(self.tempfile, "set net:timeout %s\n" % globals.timeout) os.write(self.tempfile, "set net:max-retries %s\n" % globals.num_retries) os.write(self.tempfile, "set ftp:passive-mode %s\n" % self.conn_opt) - os.write(self.tempfile, "open %s %s\n" % (self.portflag, self.parsed_url.hostname)) + if log.getverbosity() >= log.DEBUG : + os.write(self.tempfile, "debug\n") + os.write(self.tempfile, "open %s %s\n" % (self.authflag, self.url_string) ) +# os.write(self.tempfile, "open %s %s\n" % (self.portflag, self.parsed_url.hostname)) # allow .netrc auth by only setting user/pass when user was actually given - if self.parsed_url.username: - os.write(self.tempfile, "user %s %s\n" % (self.parsed_url.username, self.password)) +# if self.parsed_url.username: +# os.write(self.tempfile, "user %s %s\n" % (self.parsed_url.username, self.password)) os.close(self.tempfile) + if log.getverbosity() >= log.DEBUG : + f = open(self.tempname, 'r') + log.Debug("SETTINGS: \n" + "%s" % f.readlines() ) def _put(self, source_path, remote_filename): - remote_path = os.path.join(urllib.unquote(self.parsed_url.path.lstrip('/')), remote_filename).rstrip() - commandline = "lftp -c 'source %s;put \'%s\' -o \'%s\''" % \ - (self.tempname, source_path.name, remote_path) - self.subprocess_popen(commandline) + #remote_path = os.path.join(urllib.unquote(self.parsed_url.path.lstrip('/')), remote_filename).rstrip() + commandline = "lftp -c 'source \'%s\'; mkdir -p %s; put \'%s\' -o \'%s\''" % \ + (self.tempname, self.remote_path, source_path.name, self.remote_path + remote_filename) + log.Debug("CMD: %s" % commandline) + s, l, e = self.subprocess_popen(commandline) + log.Debug("STATUS: %s" % s) + log.Debug("STDERR:\n" + "%s" % (e)) + log.Debug("STDOUT:\n" + "%s" % (l)) def _get(self, remote_filename, local_path): - remote_path = os.path.join(urllib.unquote(self.parsed_url.path), remote_filename).rstrip() - commandline = "lftp -c 'source %s;get %s -o %s'" % \ - (self.tempname, remote_path.lstrip('/'), local_path.name) - self.subprocess_popen(commandline) + #remote_path = os.path.join(urllib.unquote(self.parsed_url.path), remote_filename).rstrip() + commandline = "lftp -c 'source \'%s\'; get \'%s\' -o \'%s\''" % \ + (self.tempname, self.remote_path+remote_filename, local_path.name) + log.Debug("CMD: %s" % commandline) + _, l, e = self.subprocess_popen(commandline) + log.Debug("STDERR:\n" + "%s" % (e)) + log.Debug("STDOUT:\n" + "%s" % (l)) def _list(self): # Do a long listing to avoid connection reset - remote_dir = urllib.unquote(self.parsed_url.path.lstrip('/')).rstrip() - commandline = "lftp -c 'source %s;ls \'%s\''" % (self.tempname, remote_dir) - _, l, _ = self.subprocess_popen(commandline) + #remote_dir = urllib.unquote(self.parsed_url.path.lstrip('/')).rstrip() + remote_dir = urllib.unquote(self.parsed_url.path) + #print remote_dir + commandline = "lftp -c 'source \'%s\'; cd \'%s\' || exit 0; ls'" % (self.tempname, self.remote_path) + log.Debug("CMD: %s" % commandline) + _, l, e = self.subprocess_popen(commandline) + log.Debug("STDERR:\n" + "%s" % (e)) + log.Debug("STDOUT:\n" + "%s" % (l)) + # Look for our files as the last element of a long list line return [x.split()[-1] for x in l.split('\n') if x] def _delete(self, filename): - remote_dir = urllib.unquote(self.parsed_url.path.lstrip('/')).rstrip() - commandline = "lftp -c 'source %s;cd \'%s\';rm \'%s\''" % (self.tempname, remote_dir, filename) - self.subprocess_popen(commandline) - -duplicity.backend.register_backend("ftp", FTPBackend) -duplicity.backend.register_backend("ftps", FTPBackend) + #remote_dir = urllib.unquote(self.parsed_url.path.lstrip('/')).rstrip() + commandline = "lftp -c 'source \'%s\'; cd \'%s\'; rm \'%s\''" % (self.tempname, self.remote_path, filename) + log.Debug("CMD: %s" % commandline) + _, l, e = self.subprocess_popen(commandline) + log.Debug("STDERR:\n" + "%s" % (e)) + log.Debug("STDOUT:\n" + "%s" % (l)) + +duplicity.backend.register_backend("ftp", LFTPBackend) +duplicity.backend.register_backend("ftps", LFTPBackend) +duplicity.backend.register_backend("fish", LFTPBackend) + +duplicity.backend.register_backend("lftp+ftp", LFTPBackend) +duplicity.backend.register_backend("lftp+ftps", LFTPBackend) +duplicity.backend.register_backend("lftp+fish", LFTPBackend) +duplicity.backend.register_backend("lftp+sftp", LFTPBackend) +duplicity.backend.register_backend("lftp+webdav", LFTPBackend) +duplicity.backend.register_backend("lftp+webdavs", LFTPBackend) +duplicity.backend.register_backend("lftp+http", LFTPBackend) +duplicity.backend.register_backend("lftp+https", LFTPBackend) + +duplicity.backend.uses_netloc.extend([ 'ftp', 'ftps', 'fish',\ + 'lftp+ftp', 'lftp+ftps',\ + 'lftp+fish', 'lftp+sftp',\ + 'lftp+webdav', 'lftp+webdavs',\ + 'lftp+http', 'lftp+https' ]) \ No newline at end of file === added file 'duplicity/backends/ncftpbackend.py' --- duplicity/backends/ncftpbackend.py 1970-01-01 00:00:00 +0000 +++ duplicity/backends/ncftpbackend.py 2014-10-30 18:01:04 +0000 @@ -0,0 +1,118 @@ +# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*- +# +# Copyright 2002 Ben Escoto <[email protected]> +# Copyright 2007 Kenneth Loafman <[email protected]> +# +# This file is part of duplicity. +# +# Duplicity is free software; you can redistribute it and/or modify it +# under the terms of the GNU General Public License as published by the +# Free Software Foundation; either version 2 of the License, or (at your +# option) any later version. +# +# Duplicity is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with duplicity; if not, write to the Free Software Foundation, +# Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + +import os.path +import urllib + +import duplicity.backend +from duplicity import globals +from duplicity import log +from duplicity import tempdir + +class NCFTPBackend(duplicity.backend.Backend): + """Connect to remote store using File Transfer Protocol""" + def __init__(self, parsed_url): + duplicity.backend.Backend.__init__(self, parsed_url) + + # we expect an error return, so go low-level and ignore it + try: + p = os.popen("ncftpls -v") + fout = p.read() + ret = p.close() + except Exception: + pass + # the expected error is 8 in the high-byte and some output + if ret != 0x0800 or not fout: + log.FatalError("NcFTP not found: Please install NcFTP version 3.1.9 or later", + log.ErrorCode.ftp_ncftp_missing) + + # version is the second word of the first line + version = fout.split('\n')[0].split()[1] + if version < "3.1.9": + log.FatalError("NcFTP too old: Duplicity requires NcFTP version 3.1.9," + "3.2.1 or later. Version 3.2.0 will not work properly.", + log.ErrorCode.ftp_ncftp_too_old) + elif version == "3.2.0": + log.Warn("NcFTP (ncftpput) version 3.2.0 may fail with duplicity.\n" + "see: http://www.ncftpd.com/ncftp/doc/changelog.html\n"; + "If you have trouble, please upgrade to 3.2.1 or later", + log.WarningCode.ftp_ncftp_v320) + log.Notice("NcFTP version is %s" % version) + + self.parsed_url = parsed_url + + self.url_string = duplicity.backend.strip_auth_from_url(self.parsed_url) + + # strip ncftp+ prefix + self.url_string = duplicity.backend.strip_prefix(self.url_string, 'ncftp') + + # This squelches the "file not found" result from ncftpls when + # the ftp backend looks for a collection that does not exist. + # version 3.2.2 has error code 5, 1280 is some legacy value + self.popen_breaks[ 'ncftpls' ] = [ 5, 1280 ] + + # Use an explicit directory name. + if self.url_string[-1] != '/': + self.url_string += '/' + + self.password = self.get_password() + + if globals.ftp_connection == 'regular': + self.conn_opt = '-E' + else: + self.conn_opt = '-F' + + self.tempfile, self.tempname = tempdir.default().mkstemp() + os.write(self.tempfile, "host %s\n" % self.parsed_url.hostname) + os.write(self.tempfile, "user %s\n" % self.parsed_url.username) + os.write(self.tempfile, "pass %s\n" % self.password) + os.close(self.tempfile) + self.flags = "-f %s %s -t %s -o useCLNT=0,useHELP_SITE=0 " % \ + (self.tempname, self.conn_opt, globals.timeout) + if parsed_url.port != None and parsed_url.port != 21: + self.flags += " -P '%s'" % (parsed_url.port) + + def _put(self, source_path, remote_filename): + remote_path = os.path.join(urllib.unquote(self.parsed_url.path.lstrip('/')), remote_filename).rstrip() + commandline = "ncftpput %s -m -V -C '%s' '%s'" % \ + (self.flags, source_path.name, remote_path) + self.subprocess_popen(commandline) + + def _get(self, remote_filename, local_path): + remote_path = os.path.join(urllib.unquote(self.parsed_url.path), remote_filename).rstrip() + commandline = "ncftpget %s -V -C '%s' '%s' '%s'" % \ + (self.flags, self.parsed_url.hostname, remote_path.lstrip('/'), local_path.name) + self.subprocess_popen(commandline) + + def _list(self): + # Do a long listing to avoid connection reset + commandline = "ncftpls %s -l '%s'" % (self.flags, self.url_string) + _, l, _ = self.subprocess_popen(commandline) + # Look for our files as the last element of a long list line + return [x.split()[-1] for x in l.split('\n') if x and not x.startswith("total ")] + + def _delete(self, filename): + commandline = "ncftpls %s -l -X 'DELE %s' '%s'" % \ + (self.flags, filename, self.url_string) + self.subprocess_popen(commandline) + +duplicity.backend.register_backend("ncftp+ftp", NCFTPBackend) +duplicity.backend.uses_netloc.extend([ 'ncftp+ftp' ]) \ No newline at end of file === renamed file 'duplicity/backends/_ssh_paramiko.py' => 'duplicity/backends/ssh_paramiko_backend.py' --- duplicity/backends/_ssh_paramiko.py 2014-10-27 03:02:20 +0000 +++ duplicity/backends/ssh_paramiko_backend.py 2014-10-30 18:01:04 +0000 @@ -217,8 +217,11 @@ self.config['port'],e)) self.client.get_transport().set_keepalive((int)(globals.timeout / 2)) + self.scheme = duplicity.backend.strip_prefix(parsed_url.scheme, 'paramiko') + self.use_scp = ( self.scheme == 'scp' ) + # scp or sftp? - if (globals.use_scp): + if (self.use_scp): # sanity-check the directory name if (re.search("'",self.remote_dir)): raise BackendException("cannot handle directory names with single quotes with --use-scp!") @@ -256,7 +259,7 @@ raise BackendException("sftp chdir to %s failed: %s" % (self.sftp.normalize(".")+"/"+d,e)) def _put(self, source_path, remote_filename): - if globals.use_scp: + if self.use_scp: f=file(source_path.name,'rb') try: chan=self.client.get_transport().open_session() @@ -284,7 +287,7 @@ self.sftp.put(source_path.name,remote_filename) def _get(self, remote_filename, local_path): - if globals.use_scp: + if self.use_scp: try: chan=self.client.get_transport().open_session() chan.settimeout(globals.timeout) @@ -327,7 +330,7 @@ def _list(self): # In scp mode unavoidable quoting issues will make this fail if the # directory name contains single quotes. - if globals.use_scp: + if self.use_scp: output = self.runremote("ls -1 '%s'" % self.remote_dir, False, "scp dir listing ") return output.splitlines() else: @@ -336,7 +339,7 @@ def _delete(self, filename): # In scp mode unavoidable quoting issues will cause failures if # filenames containing single quotes are encountered. - if globals.use_scp: + if self.use_scp: self.runremote("rm '%s/%s'" % (self.remote_dir, filename), False, "scp rm ") else: self.sftp.remove(filename) @@ -370,3 +373,9 @@ raise BackendException("could not load '%s', maybe corrupt?" % (file)) return sshconfig.lookup(host) + +duplicity.backend.register_backend("sftp", SSHParamikoBackend) +duplicity.backend.register_backend("scp", SSHParamikoBackend) +duplicity.backend.register_backend("paramiko+sftp", SSHParamikoBackend) +duplicity.backend.register_backend("paramiko+scp", SSHParamikoBackend) +duplicity.backend.uses_netloc.extend([ 'sftp', 'scp', 'paramiko+sftp', 'paramiko+scp' ]) === renamed file 'duplicity/backends/_ssh_pexpect.py' => 'duplicity/backends/ssh_pexpect_backend.py' --- duplicity/backends/_ssh_pexpect.py 2014-04-28 02:49:39 +0000 +++ duplicity/backends/ssh_pexpect_backend.py 2014-10-30 18:01:04 +0000 @@ -49,6 +49,9 @@ self.sftp_command = "sftp" if globals.sftp_command: self.sftp_command = globals.sftp_command + + self.scheme = duplicity.backend.strip_prefix(parsed_url.scheme, 'pexpect') + self.use_scp = ( self.scheme == 'scp' ) # host string of form [user@]hostname if parsed_url.username: @@ -212,7 +215,7 @@ raise BackendException("Error running '%s': %s" % (commandline, msg)) def _put(self, source_path, remote_filename): - if globals.use_scp: + if self.use_scp: self.put_scp(source_path, remote_filename) else: self.put_sftp(source_path, remote_filename) @@ -234,7 +237,7 @@ self.run_scp_command(commandline) def _get(self, remote_filename, local_path): - if globals.use_scp: + if self.use_scp: self.get_scp(remote_filename, local_path) else: self.get_sftp(remote_filename, local_path) @@ -280,3 +283,8 @@ commands.append("rm \"%s\"" % filename) commandline = ("%s %s %s" % (self.sftp_command, globals.ssh_options, self.host_string)) self.run_sftp_command(commandline, commands) + +duplicity.backend.register_backend("pexpect+sftp", SSHPExpectBackend) +duplicity.backend.register_backend("pexpect+scp", SSHPExpectBackend) +duplicity.backend.uses_netloc.extend([ 'pexpect+sftp', 'pexpect+scp' ]) + === modified file 'duplicity/commandline.py' --- duplicity/commandline.py 2014-10-27 02:27:36 +0000 +++ duplicity/commandline.py 2014-10-30 18:01:04 +0000 @@ -537,9 +537,6 @@ # default to batch mode using public-key encryption parser.add_option("--ssh-askpass", action = "store_true") - # allow the user to switch ssh backend - parser.add_option("--ssh-backend", metavar = _("paramiko|pexpect")) - # user added ssh options parser.add_option("--ssh-options", action = "extend", metavar = _("options")) @@ -567,8 +564,6 @@ # Whether to specify --use-agent in GnuPG options parser.add_option("--use-agent", action = "store_true") - parser.add_option("--use-scp", action = "store_true") - parser.add_option("--verbosity", "-v", type = "verbosity", metavar = "[0-9]", dest = "", action = "callback", callback = lambda o, s, v, p: log.setverbosity(v)) === modified file 'duplicity/file_naming.py' --- duplicity/file_naming.py 2014-10-27 02:27:36 +0000 +++ duplicity/file_naming.py 2014-10-30 18:01:04 +0000 @@ -393,6 +393,20 @@ else: pr.encrypted = None + def valid_extension(): + """ + plausibility check for duplicity file extension + before starting to extensively parse the filenames + """ + res = re.match(r'.*\.(g|z|gpg|gz|tar|p|part|manifest|sigtar)$', filename ) + #print filename, res + if res is None : + return False + return True + + if not valid_extension() : + return None + pr = check_full() if not pr: pr = check_inc() === modified file 'duplicity/globals.py' --- duplicity/globals.py 2014-05-12 07:09:00 +0000 +++ duplicity/globals.py 2014-10-30 18:01:04 +0000 @@ -231,15 +231,9 @@ # default to batch mode using public-key encryption ssh_askpass = False -# default ssh backend is paramiko -ssh_backend = "paramiko" - # user added ssh options ssh_options = "" -# whether to use scp for put/get, sftp is default -use_scp = False - # default cf backend is pyrax cf_backend = "pyrax"
_______________________________________________ Mailing list: https://launchpad.net/~duplicity-team Post to : [email protected] Unsubscribe : https://launchpad.net/~duplicity-team More help : https://help.launchpad.net/ListHelp
