Bug#654708: backupninja: rsync handler only respects the first rsync option
Em Sun, May 13, 2012 at 02:38:34AM +0200, intrigeri escreveu: Hi, I'm confused about the status of this specific bug: it's tagged as fixed-upstream in the Debian BTS, but last message from Paul reads With the patch it's even worse. I have just merged half a ton of rhatto's bugfixes branches into upstream master, may someone please check if the bug is still present in the resulting Git upstream master branch? I'm also confused about this bug, as it start dealing with a specific issue (upstream https://labs.riseup.net/code/issues/3702) but also refers to other problems, including: - upstream https://labs.riseup.net/code/issues/3882 which is also reported at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=656968 - issues related to http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=654192 -- rhatto at riseup.net pubkey 64E39FCA / keys.indymedia.org signature.asc Description: Digital signature
Bug#654708: backupninja: rsync handler only respects the first rsync option
On Mon, May 14, 2012 at 10:40:48AM -0300, rhatto wrote: Em Sun, May 13, 2012 at 02:38:34AM +0200, intrigeri escreveu: I have just merged half a ton of rhatto's bugfixes branches into upstream master, may someone please check if the bug is still present in the resulting Git upstream master branch? I'm also confused about this bug, as it start dealing with a specific issue (upstream https://labs.riseup.net/code/issues/3702) I can say at least, that the rsync handler I started using at some point started to respect my setting: rsync_options = -avF --delete I can see that it passes both options. The confusion is brought on by me switching to incrementally updated rsync handlers with fixes, often multiple at the same time. I am sorry. but also refers to other problems, including: - upstream https://labs.riseup.net/code/issues/3882 which is also reported at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=656968 I will comment on this on the right bug report. - issues related to http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=654192 I will also comment on this on the right bug report. Paul -- Using the Power of Debian GNU/Linux | E-mail: pau...@debian.org Jabber/GTalk: p...@luon.net | GnuPG key ID: 0x50064181 signature.asc Description: Digital signature
Bug#654708: backupninja: rsync handler only respects the first rsync option
Hi, Paul van Tilburg wrote (14 May 2012 13:54:28 GMT) : I can say at least, that the rsync handler I started using at some point started to respect my setting: rsync_options = -avF --delete I can see that it passes both options. Thank you! So this specific bug is rightfully marked as fixed-upstream. Great. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#654708: backupninja: rsync handler only respects the first rsync option
Hi, I'm confused about the status of this specific bug: it's tagged as fixed-upstream in the Debian BTS, but last message from Paul reads With the patch it's even worse. I have just merged half a ton of rhatto's bugfixes branches into upstream master, may someone please check if the bug is still present in the resulting Git upstream master branch? Cheers, -- intrigeri | GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc | OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#654708: backupninja: rsync handler only respects the first rsync option
Em Sun, Apr 15, 2012 at 10:26:27PM +0200, intrigeri escreveu: Paul van Tilburg wrote (15 Apr 2012 14:16:39 GMT) : Ok, yes. You must understand that this is already the 6th patch applied to /usr/share/backupninja/rsync and I am starting to loose track of what I am doing. rhatto, I think you'd better provide Paul with patches against the unmodified files shipped by the current Debian package, so that he can merely apt-get --reinstall install + patch. Thanks for pointing that and sorry for bothering you with so many patches. I'll try to provide a patch ASAP, feel free to ping me if takes too long. :) -- rhatto at riseup.net pubkey 64E39FCA / keys.indymedia.org signature.asc Description: Digital signature
Bug#654708: backupninja: rsync handler only respects the first rsync option
On Fri, Apr 13, 2012 at 10:09:08AM -0300, rhatto wrote: Em Fri, Apr 13, 2012 at 09:40:09AM +0200, Paul van Tilburg escreveu: On Thu, Apr 12, 2012 at 09:34:42PM -0300, rhatto wrote: Could you try the patch below? but rsync doesn't seem to use it for some reason. I see it backing up /lib, /dev, /proc, etc. And it now hangs on /proc/kcore, so I am revering the patch. Could you try the following change (replacing @SED@ by sed)? This might add single quotes to the excludes, like --exclude='/*bin'. I am unsure if the patch is meant to be on top of the previous on, or instead of the previous one. I.e. is the adjustment to the call of $nice $rsync etc. necessary or not? Cheers, Paul -- Using the Power of Debian GNU/Linux | E-mail: pau...@debian.org Jabber/GTalk: p...@luon.net | GnuPG key ID: 0x50064181 signature.asc Description: Digital signature
Bug#654708: backupninja: rsync handler only respects the first rsync option
Em Sun, Apr 15, 2012 at 04:00:02PM +0200, Paul van Tilburg escreveu: On Fri, Apr 13, 2012 at 10:09:08AM -0300, rhatto wrote: Em Fri, Apr 13, 2012 at 09:40:09AM +0200, Paul van Tilburg escreveu: On Thu, Apr 12, 2012 at 09:34:42PM -0300, rhatto wrote: Could you try the patch below? but rsync doesn't seem to use it for some reason. I see it backing up /lib, /dev, /proc, etc. And it now hangs on /proc/kcore, so I am revering the patch. Could you try the following change (replacing @SED@ by sed)? This might add single quotes to the excludes, like --exclude='/*bin'. I am unsure if the patch is meant to be on top of the previous on, or instead of the previous one. It's an incremental patch, so you apply atop of the previous. I'm using a dedicated branch to handle this issue: https://git.sarava.org/?p=backupninja.git;a=shortlog;h=refs/heads/bug/3882 I.e. is the adjustment to the call of $nice $rsync etc. necessary or not? Yes, it is. -- rhatto at riseup.net pubkey 64E39FCA / keys.indymedia.org signature.asc Description: Digital signature
Bug#654708: backupninja: rsync handler only respects the first rsync option
On Sun, Apr 15, 2012 at 11:10:01AM -0300, rhatto wrote: Em Sun, Apr 15, 2012 at 04:00:02PM +0200, Paul van Tilburg escreveu: I am unsure if the patch is meant to be on top of the previous on, or instead of the previous one. It's an incremental patch, so you apply atop of the previous. I'm using a dedicated branch to handle this issue: https://git.sarava.org/?p=backupninja.git;a=shortlog;h=refs/heads/bug/3882 Ok, yes. You must understand that this is already the 6th patch applied to /usr/share/backupninja/rsync and I am starting to loose track of what I am doing. With the patch it's even worse: Info: Syncing / on /media/Backup/tetraeder.rsync///daily.1/... Debug: nice -n 0 /usr/bin/rsync -avF --delete --numeric-ids --rsync-path=rsync root@tetraeder:/// /media/Backup/tetraeder.rsync///daily.1/ Unexpected remote arg: root@tetraeder:/// rsync error: syntax or usage error (code 1) at main.c(1222) [sender=3.0.7] Info: finished action /etc/backup.d/10.tetraeder.rsync: SUCCESS Note that the whole exclude part is missing in action. Do you need my config file to be able to test it out yourself? Paul -- Using the Power of Debian GNU/Linux | E-mail: pau...@debian.org Jabber/GTalk: p...@luon.net | GnuPG key ID: 0x50064181 signature.asc Description: Digital signature
Bug#654708: backupninja: rsync handler only respects the first rsync option
Hi, Paul van Tilburg wrote (15 Apr 2012 14:16:39 GMT) : Ok, yes. You must understand that this is already the 6th patch applied to /usr/share/backupninja/rsync and I am starting to loose track of what I am doing. rhatto, I think you'd better provide Paul with patches against the unmodified files shipped by the current Debian package, so that he can merely apt-get --reinstall install + patch. Cheers, -- intrigeri | GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc | OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#654708: backupninja: rsync handler only respects the first rsync option
On Thu, Apr 12, 2012 at 09:34:42PM -0300, rhatto wrote: Em Mon, Apr 09, 2012 at 02:13:01PM +0200, Paul van Tilburg escreveu: This partly fixes it. Indeed, the --delete gets passed on now, but still expansion happens. But the expansion still happens (irregularly), so '/*bin' is expanded to '/bin' and '/sbin', but '/home/*/.cache' isn't, and '/home/*/Music' is!? Hi Paul, thanks again for the reportback. I opened a new upstream bug as this looks like another issue, but still related with your rsync handler experience: https://labs.riseup.net/code/issues/3882 Could you try the patch below? Applied, but it didn't help. The commandline is now: Info: Syncing / on /media/Backup/tetraeder.rsync///daily.1/... Debug: nice -n 0 /usr/bin/rsync -avF --delete --numeric-ids --rsync-path=rsync --exclude=/*bin --exclude=/dev --exclude=/home/*/.cache --exclude=/home/*/.local/share/Trash --exclude=/home/*/.thumbnails --exclude=/home/*/Music --exclude=/home/*/Videos --exclude=/lib* --exclude=/media --exclude=/mnt --exclude=/proc --exclude=/selinux --exclude=/sys --exclude=/tmp --exclude=/usr --exclude=/var/cache --exclude=/var/run --exclude=/var/tmp root@tetraeder:/// /media/Backup/tetraeder.rsync///daily.1/ but rsync doesn't seem to use it for some reason. I see it backing up /lib, /dev, /proc, etc. And it now hangs on /proc/kcore, so I am revering the patch. Paul -- Using the Power of Debian GNU/Linux | E-mail: pau...@debian.org Jabber/GTalk: p...@luon.net | GnuPG key ID: 0x50064181 signature.asc Description: Digital signature
Bug#654708: backupninja: rsync handler only respects the first rsync option
Em Fri, Apr 13, 2012 at 09:40:09AM +0200, Paul van Tilburg escreveu: On Thu, Apr 12, 2012 at 09:34:42PM -0300, rhatto wrote: Em Mon, Apr 09, 2012 at 02:13:01PM +0200, Paul van Tilburg escreveu: This partly fixes it. Indeed, the --delete gets passed on now, but still expansion happens. But the expansion still happens (irregularly), so '/*bin' is expanded to '/bin' and '/sbin', but '/home/*/.cache' isn't, and '/home/*/Music' is!? Hi Paul, thanks again for the reportback. I opened a new upstream bug as this looks like another issue, but still related with your rsync handler experience: https://labs.riseup.net/code/issues/3882 Could you try the patch below? but rsync doesn't seem to use it for some reason. I see it backing up /lib, /dev, /proc, etc. And it now hangs on /proc/kcore, so I am revering the patch. Could you try the following change (replacing @SED@ by sed)? This might add single quotes to the excludes, like --exclude='/*bin'. If it this doesn't work, we might try to use a merge-file with --filter. --- From 39bb9e605f04665f8d838be442d9224745959872 Mon Sep 17 00:00:00 2001 From: Silvio Rhatto rha...@riseup.net Date: Fri, 13 Apr 2012 09:58:18 -0300 Subject: [PATCH] Trying additional quoting on excludes (upstream #3882) --- handlers/rsync.in |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/handlers/rsync.in b/handlers/rsync.in index ad8e64f..3108b37 100644 --- a/handlers/rsync.in +++ b/handlers/rsync.in @@ -277,7 +277,7 @@ function eval_config { mv=move_files fi - excludes=`echo $exclude | @SED@ 's/^/--exclude=/g' -e 's/ / --exclude=/g'` + excludes=`echo $exclude | @SED@ s/^/--exclude='/g -e s/ /' --exclude='/g -e s/$/'/` } -- 1.7.2.5 -- rhatto at riseup.net pubkey 64E39FCA / keys.indymedia.org signature.asc Description: Digital signature
Bug#654708: backupninja: rsync handler only respects the first rsync option
Em Mon, Apr 09, 2012 at 02:13:01PM +0200, Paul van Tilburg escreveu: This partly fixes it. Indeed, the --delete gets passed on now, but still expansion happens. But the expansion still happens (irregularly), so '/*bin' is expanded to '/bin' and '/sbin', but '/home/*/.cache' isn't, and '/home/*/Music' is!? Hi Paul, thanks again for the reportback. I opened a new upstream bug as this looks like another issue, but still related with your rsync handler experience: https://labs.riseup.net/code/issues/3882 Could you try the patch below? --- From aa8d0f864830036e12e52696243c05df76d678e2 Mon Sep 17 00:00:00 2001 From: Silvio Rhatto rha...@riseup.net Date: Thu, 12 Apr 2012 21:29:57 -0300 Subject: [PATCH 1/1] Quoting $exclude and $excludes and avoiding a for loop on $exclude to not expand wildcards in beforehand (upstream #3882) --- handlers/rsync.in |8 +++- 1 files changed, 3 insertions(+), 5 deletions(-) diff --git a/handlers/rsync.in b/handlers/rsync.in index e253c5c..a77da2a 100644 --- a/handlers/rsync.in +++ b/handlers/rsync.in @@ -277,9 +277,7 @@ function eval_config { mv=move_files fi - for path in $exclude; do -excludes=$excludes --exclude=$path - done + excludes=`echo $exclude | sed -e 's/^/--exclude=/g' -e 's/ / --exclude=/g'` } @@ -1113,8 +,8 @@ for SECTION in $include; do set_dest info Syncing $SECTION on $dest_path... - debug $nice $rsync ${rsync_options[@]} $filelist_flag $excludes $batch_option $orig $dest_path - $nice $rsync ${rsync_options[@]} $filelist_flag $excludes $batch_option $orig $dest_path | tee -a $log + debug $nice $rsync ${rsync_options[@]} $filelist_flag $excludes $batch_option $orig $dest_path + $nice $rsync ${rsync_options[@]} $filelist_flag $excludes $batch_option $orig $dest_path | tee -a $log if [ $? != 0 ]; then fatal Rsync error when trying to transfer $SECTION -- 1.7.2.5 -- rhatto at riseup.net pubkey 64E39FCA / keys.indymedia.org signature.asc Description: Digital signature
Bug#654708: backupninja: rsync handler only respects the first rsync option
Hi, An update about this bug. On Sun, Feb 26, 2012 at 02:11:16PM +0100, intrigeri wrote: Paul van Tilburg wrote (07 Jan 2012 11:14:54 GMT) : It applies as is onto 0.9.10, but it did not help. Given the above rsync_options, it runs: Debug: nice -n 0 /usr/bin/rsync -avF --exclude=/bin --exclude=/sbin [...] So, the --delete got dropped. Current upstream Git (at commit 1f367b4) is supposed to fix this bug for real; Paul, may you please test and confirm? https://labs.riseup.net/code/projects/backupninja/repository/revisions/master/changes/handlers/rsync.in This partly fixes it. Indeed, the --delete gets passed on now, but still expansion happens. But the expansion still happens (irregularly), so '/*bin' is expanded to '/bin' and '/sbin', but '/home/*/.cache' isn't, and '/home/*/Music' is!? Paul -- Using the Power of Debian GNU/Linux | E-mail: pau...@debian.org Jabber/GTalk: p...@luon.net | GnuPG key ID: 0x50064181 -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#654708: backupninja: rsync handler only respects the first rsync option
Hi Paul, Paul van Tilburg wrote (07 Jan 2012 11:14:54 GMT) : It applies as is onto 0.9.10, but it did not help. Given the above rsync_options, it runs: Debug: nice -n 0 /usr/bin/rsync -avF --exclude=/bin --exclude=/sbin [...] So, the --delete got dropped. Current upstream Git (at commit 1f367b4) is supposed to fix this bug for real; Paul, may you please test and confirm? https://labs.riseup.net/code/projects/backupninja/repository/revisions/master/changes/handlers/rsync.in Cheers, -- intrigeri | GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc | OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc | Did you exchange a walk on part in the war | for a lead role in the cage? -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#654708: backupninja: rsync handler only respects the first rsync option
Hello intrigeri, On Thu, Jan 05, 2012 at 03:36:23PM +0100, intrigeri wrote: Paul van Tilburg wrote (05 Jan 2012 12:43:30 GMT) : I tried to change the rsync options of the handler by setting: rsync_options = -avF --delete when I noticed that --delete didn't make it to the commandline. May you please try to apply the following patch? https://labs.riseup.net/code/projects/backupninja/repository/revisions/926152dfa1c3b076488bf768ae6cd2282cfa5497/diff/handlers/rsync.in It would help confirm this bug was fixed upstream by commit 926152. (I hope the patch applies as is onto 0.9.10, but I've not tried it myself.) It applies as is onto 0.9.10, but it did not help. Given the above rsync_options, it runs: Debug: nice -n 0 /usr/bin/rsync -avF --exclude=/bin --exclude=/sbin [...] So, the --delete got dropped. Another thing, I have excludes like: exclude = /home/*/.cache which works fine. But I also have: exclude = /home/*/Music exclude = /home/*/Videos which is expanded to --excluded=/home/user1/Music --exclude=/home/user2/Music ... but then for users on the local side, which makes no sense. Why are these expanded and the excludes with the dot dirs not? Cheers, Paul -- Using the Power of Debian GNU/Linux | E-mail: pau...@debian.org Jabber/GTalk: p...@luon.netl | GnuPG key ID: 0x50064181 -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#654708: backupninja: rsync handler only respects the first rsync option
Package: backupninja Version: 0.9.10-1 Severity: normal Hi! I tried to change the rsync options of the handler by setting: rsync_options = -avF --delete when I noticed that --delete didn't make it to the commandline. I tried to set rsync_options = -avF --delete, but that didn't work either as only -afF made it to the command line, leading to a syntax error. Then, I noticed that if I even go back to the default, which defined by line 143 of the rsync handler: getconf rsync_options -av --delete --recursive that only -av gets passed. As a result, a rotated daily.1 never gets cleaned up. (The --recursive is redundant by the way, as -r is implied by -a.) Cheers, Paul -- System Information: Debian Release: 6.0.3 APT prefers stable-updates APT policy: (500, 'stable-updates'), (500, 'stable') Architecture: amd64 (x86_64) Kernel: Linux 2.6.32-5-amd64 (SMP w/4 CPU cores) Locale: LANG=nl_NL.UTF-8, LC_CTYPE=nl_NL.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages backupninja depends on: ii bash 4.1-3 The GNU Bourne Again SHell ii bsd-mailx [mailx] 8.1.2-0.20100314cvs-1 simple mail user agent ii dialog 1.1-20100428-1Displays user-friendly dialog boxe ii gawk 1:3.1.7.dfsg-5GNU awk, a pattern scanning and pr ii mawk 1.3.3-15 a pattern scanning and text proces backupninja recommends no packages. Versions of packages backupninja suggests: ii bzip21.0.5-6 high-quality block-sorting file co pn debconf-utilsnone (no description available) pn duplicitynone (no description available) pn genisoimage none (no description available) pn hwinfo none (no description available) ii mdadm3.1.4-1+8efb9d1 tool to administer Linux MD arrays ii rdiff-backup 1.2.8-6 remote incremental backup ii rsync3.0.7-2 fast remote file copy program (lik pn subversion none (no description available) pn wodimnone (no description available) -- Configuration Files: /etc/backupninja.conf changed [not included] -- no debconf information -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#654708: backupninja: rsync handler only respects the first rsync option
Hi Paul, Paul van Tilburg wrote (05 Jan 2012 12:43:30 GMT) : I tried to change the rsync options of the handler by setting: rsync_options = -avF --delete when I noticed that --delete didn't make it to the commandline. May you please try to apply the following patch? https://labs.riseup.net/code/projects/backupninja/repository/revisions/926152dfa1c3b076488bf768ae6cd2282cfa5497/diff/handlers/rsync.in It would help confirm this bug was fixed upstream by commit 926152. (I hope the patch applies as is onto 0.9.10, but I've not tried it myself.) Cheers, -- intrigeri intrig...@boum.org | GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc | OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc | Do not be trapped by the need to achieve anything. | This way, you achieve everything. -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org