[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
>> You apparently performed a full backup on 2016-11-29 at 01:52 UTC, >> did everything work fine during this backup? Was it successful ? > Don't think so Don't think so either, because your full backup on that date is only one volume long and that volume is 120 bytes long: ls -l /home/david/Dropbox/Backups [...] ... 120 Nov 28 17:52 duplicity-full.20161129T015237Z.vol1.difftar.gz [...] FYI duplicity makes backups into sets of normally 25 MB volumes. And then there's your first incremental backup on Dec 1 which doesn't look good since there are only volumes 2, 7, 15, 16, 23, 41, 49 of that backup set in your backup repository. Where are all the other ones? Did Dropbox eat them? And the normally-accompanying signatures and manifest files are nowhere to be found either. I don't know if Déjà-Dup can recover from this situation and finish the partial incremental backup set but I wouldn't even lose time trying to be honest. I've always found the easiest way to recover from a severely failed backup is to delete the failed part (everything in your case) and start again. For good measure you can also delete the backup's cache directory: rm -r /home/david/Dropbox/Backups rm -r /home/david/.cache/deja-dup/f00ff76508010e2a830dd26b1d39db42 When you try again, you may want to test backing up a small subset of your documents instead of your whole user directory. That way you'll know quickly if it works or fails again. If it works, you can delete this test backup and start again with the whole thing. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: Undescriptive duplicity/collection-status error when the backup directory contains two volumes with different file names and same volume number in the same backup set To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
Thanks for the information. First of all, as reported by the log file you sent and the listing of your backup directory, there is an uncompressed backup volume in your volume directory, and an uncompressed file of files signatures, created or last modified on Dec 22: duplicity-full.20161129T015237Z.vol1.difftar duplicity-full-signatures.20161129T015237Z.sigtar along with their older compressed versions, created on Nov 28: duplicity-full.20161129T015237Z.vol1.difftar.gz duplicity-full-signatures.20161129T015237Z.sigtar.gz As far as I know there shouldn't be any such uncompressed files in the backup location. Did you uncompress them yourself perhaps? In any case, they're confusing duplicity (the command-line backup tool Déjà-Dup is a graphical front-end to), if I understand the log file and error message correctly: DUPLICITY: INFO 1 DUPLICITY: . Args: /usr/bin/duplicity collection-status [...] [...] DUPLICITY: DEBUG 1 DUPLICITY: . 12 files exist on backend DUPLICITY: DEBUG 1 DUPLICITY: . Extracting backup chains from list of files: [u'duplicity-full.20161129T015237Z.vol1.difftar', u'duplicity-full.20161129T015237Z.manifest', u'duplicity-full.20161129T015237Z.vol1.difftar.gz', u'duplicity-full-signatures.20161129T015237Z.sigtar.gz', u'duplicity-full-signatures.20161129T015237Z.sigtar', [...] DUPLICITY: DEBUG 1 DUPLICITY: . File duplicity-full.20161129T015237Z.vol1.difftar is not part of a known set; creating new set DUPLICITY: DEBUG 1 DUPLICITY: . File duplicity-full.20161129T015237Z.manifest is part of known set DUPLICITY: ERROR 30 AssertionError [...] DUPLICITY: . File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 105, in add_filename(self.volume_name_dict, filename) DUPLICITY: . AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}, 'duplicity-full.20161129T015237Z.vol1.difftar.gz') >From my understanding of the duplicity source code in the incriminated file, this is what happens when happens Déjà-Dup launches the collection-status action to have duplicity find out which backup chains are in the backup repository: Déjà-Dup passes the list of files in this directory to duplicity, which takes the uncompressed duplicity- full.20161129T015237Z.vol1.difftar for the start of a backup set, then tries to add the compressed duplicity- full.20161129T015237Z.vol1.difftar.gz to this set, and fails because the volume number of this file has already been added to the set. Otherwise there would be two backup volumes with the same volume number in the backup set, and duplicity can't guess which one to take into account. To fix this bug from happening in the future, we could perhaps restrict the file list that is passed from Déjà-Dup to duplicity collection- status? Perhaps to *.{diff,sig}tar.{gz,gpg} and *.manifest files only, or something like that? Or perhaps it could be fixed in duplicity, by having duplicity prefer a particular filename when there are two volumes with the same number in the same backup set? But then which one and on what grounds? Vej, your take on that? In the meantime, David, I believe it's safe for you to delete those extraneous uncompressed files (or move them outside of your backup repository). The assertion error should no longer be triggered, but you should bump into other errors, because your backup repository clearly doesn't have a complete, well-formed backup chain. More on that later. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
Nathanael using Fish ls -lR ~/.cache/deja-dup /home/david/.cache/deja-dup: total 4 drwxrwxr-x 2 david david 4096 Jan 14 12:50 f00ff76508010e2a830dd26b1d39db42/ /home/david/.cache/deja-dup/f00ff76508010e2a830dd26b1d39db42: total 8 -rw-rw-r-- 1 david david 165 Jan 14 12:50 duplicity-full.20161129T015237Z.manifest -rw-rw-r-- 1 david david 143 Jan 14 12:50 duplicity-full-signatures.20161129T015237Z.sigtar.gz again using Fish ls -l /home/david/Dropbox/Backups total 358108 -rw-rw-r-- 1 david david 165 Nov 28 17:52 duplicity-full.20161129T015237Z.manifest -rw-rw-r-- 1 david david 1024 Dec 22 16:08 duplicity-full.20161129T015237Z.vol1.difftar -rw-rw-r-- 1 david david 120 Nov 28 17:52 duplicity-full.20161129T015237Z.vol1.difftar.gz -rw-rw-r-- 1 david david10240 Dec 22 16:05 duplicity-full-signatures.20161129T015237Z.sigtar -rw-rw-r-- 1 david david 143 Nov 28 17:52 duplicity-full-signatures.20161129T015237Z.sigtar.gz -rw-rw-r-- 1 david david 52366483 Dec 1 16:39 duplicity-inc.20161129T015237Z.to.20161202T003153Z.vol15.difftar.gz -rw-rw-r-- 1 david david 52368153 Dec 1 16:40 duplicity-inc.20161129T015237Z.to.20161202T003153Z.vol16.difftar.gz -rw-rw-r-- 1 david david 52371036 Dec 1 16:42 duplicity-inc.20161129T015237Z.to.20161202T003153Z.vol23.difftar.gz -rw-rw-r-- 1 david david 52390301 Dec 1 16:33 duplicity-inc.20161129T015237Z.to.20161202T003153Z.vol2.difftar.gz -rw-rw-r-- 1 david david 52368335 Dec 1 16:47 duplicity-inc.20161129T015237Z.to.20161202T003153Z.vol41.difftar.gz -rw-rw-r-- 1 david david 52375173 Dec 1 16:49 duplicity-inc.20161129T015237Z.to.20161202T003153Z.vol49.difftar.gz -rw-rw-r-- 1 david david 52375354 Dec 1 16:36 duplicity-inc.20161129T015237Z.to.20161202T003153Z.vol7.difftar.gz You apparently performed a full backup on 2016-11-29 at 01:52 UTC, did everything work fine during this backup? Was it successful ? Don't think so When did you run first into this problem - on the first incremental backup ? As far as I'm aware it has never succeeded. Always got the message "Backup Failed" as it did just now when I ran it. I've had this problem for a long time all through the time that I ran Ubuntu 14.04 and it's just since I installed 16.04 that I have tried to chase it down. Thanks for your persistance David -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
Also, from the content of your log-file, I've got the impression that there isn't a single well-formed backup chain in your backup directory. Can you check with: ls -l /home/david/Dropbox/Backups -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
You apparently performed a full backup on 2016-11-29 at 01:52 UTC, did everything work fine during this backup? Was it successful ? When did you run first into this problem - on the first incremental backup ? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
The "status" Vej refers to is the bug status, on this web page, to switch from "incomplete" to "new". Thanks for the log. Can you please show the content of your cache directory? ls -lR ~/.cache/deja-dup -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
Nathanael Tried the commands you gave in fish and also the ones intended for bash in bash. some created the log but 3 times out of four the log was blank/empty Attached is the one that did leave a log. Notably every time start of backup produced "Backup failed" sTraceback (most recent call last): File "/usr/bin/duplicity", line 1532, in with_tempdir(main) File "/usr/bin/duplicity", line 1526, in with_tempdir fn() File "/usr/bin/duplicity", line 1380, in main do_backup(action) File "/usr/bin/duplicity", line 1405, in do_backup globals.archive_dir).set_values() File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 710, in set_values self.get_backup_chains(partials + backend_filename_list) File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 835, in get_backup_chains add_to_sets(f) File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 823, in add_to_sets if set.add_filename(filename): File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 105, in add_filename (self.volume_name_dict, filename) AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}, 'duplicity-full.20161129T015237Z.vol1.difftar.gz') Lastly I have not found the place to set "status"to "new" using the GUI version.(Thus far I've only been using GUI. Thanks David ** Attachment added: "deja-dup.log ___that did have content" https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+attachment/4804410/+files/deja-dup.log -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
It's because you're using the fish shell instead of the more common bash shell. If I understand correctly, the fish translation of Vej's command is: env DEJA_DUP_DEBUG=1 deja-dup --backup | tail -n 1000 > /tmp/deja- dup.log or: set -x DEJA_DUP_DEBUG 1 deja-dup --backup | tail -n 1000 > /tmp/deja-dup.log -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
found and attached Also I got DEJA_DUP_DEBUG=1 deja-dup --backup | tail -n 1000 > /tmp/deja-dup.log Unsupported use of '='. To run 'deja-dup' with a modified environment, please use 'env DEJA_DUP_DEBUG=1 deja-dup…' david@david-Lenovo-H535 ~> ** Attachment added: "deja-dup.gsettings" https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+attachment/4804058/+files/deja-dup.gsettings -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
Hello David. Thanks for the settings. Now we are still missing the /tmp/deja-dup.log with the output from the second command (that one, which did the run in debug mode). Best Regards Vej -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
Sorry Didn't recognize the commands as such. Tmp contents: org.gnome.DejaDup last-restore '' org.gnome.DejaDup periodic true org.gnome.DejaDup full-backup-period 90 org.gnome.DejaDup backend 'file' org.gnome.DejaDup last-run '' org.gnome.DejaDup nag-check '' org.gnome.DejaDup prompt-check '2017-01-01T20:59:40.740292Z' org.gnome.DejaDup root-prompt true org.gnome.DejaDup include-list ['$HOME'] org.gnome.DejaDup exclude-list ['$TRASH', '$DOWNLOAD'] org.gnome.DejaDup last-backup '' org.gnome.DejaDup periodic-period 7 org.gnome.DejaDup delete-after 182 org.gnome.DejaDup.S3 id '' org.gnome.DejaDup.S3 bucket '' org.gnome.DejaDup.S3 folder 'david-Lenovo-H535' org.gnome.DejaDup.OpenStack authurl '' org.gnome.DejaDup.OpenStack tenant '' org.gnome.DejaDup.OpenStack username '' org.gnome.DejaDup.OpenStack container 'david-Lenovo-H535' org.gnome.DejaDup.GCS id '' org.gnome.DejaDup.GCS bucket '' org.gnome.DejaDup.GCS folder 'david-Lenovo-H535' org.gnome.DejaDup.GDrive email '' org.gnome.DejaDup.GDrive folder '/deja-dup/david-Lenovo-H535' org.gnome.DejaDup.Rackspace username '' org.gnome.DejaDup.Rackspace container 'david-Lenovo-H535' org.gnome.DejaDup.File path 'file:///home/david/Dropbox/Backups' org.gnome.DejaDup.File short-name '' org.gnome.DejaDup.File uuid '' org.gnome.DejaDup.File icon '' org.gnome.DejaDup.File relpath @ay [] org.gnome.DejaDup.File name '' org.gnome.DejaDup.File type 'normal' Attempt to start again Failed with an unknown error. Traceback (most recent call last): File "/usr/bin/duplicity", line 1532, in with_tempdir(main) File "/usr/bin/duplicity", line 1526, in with_tempdir fn() File "/usr/bin/duplicity", line 1380, in main do_backup(action) File "/usr/bin/duplicity", line 1405, in do_backup globals.archive_dir).set_values() File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 710, in set_values self.get_backup_chains(partials + backend_filename_list) File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 835, in get_backup_chains add_to_sets(f) File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 823, in add_to_sets if set.add_filename(filename): File "/usr/lib/python2.7/dist-packages/duplicity/collections.py", line 105, in add_filename (self.volume_name_dict, filename) AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}, 'duplicity-full.20161129T015237Z.vol1.difftar.gz') Could not find setting "status" to change -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
Did you executed the two commands before? If so did they failed somehow? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
What I have not done is uninstall, purge & reinstall. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
Hello Vej my /Tmp contains no file with deja-dup.settings ** Attachment added: "/tmp deja-dup" https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+attachment/4801706/+files/Screenshot%20from%202017-01-08%2013-45-49.png -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1652410] Re: duja-dup AssertionError: ({1: 'duplicity-full.20161129T015237Z.vol1.difftar'}
Hello David, can you provide us with the following informations please: The file /tmp/deja-dup.gsettings after running the following line (you may want to scrub the file of any incriminating file names or details): gsettings list-recursively org.gnome.DejaDup > /tmp/deja-dup.gsettings The file /tmp/deja-dup.log after running the appropriate line below and replicating the problem (you may want to scrub the log of any incriminating file names or details): * If you're having problems backing up: DEJA_DUP_DEBUG=1 deja-dup --backup | tail -n 1000 > /tmp/deja-dup.log After that you can set the "Status" back to "New". Thanking you in anticipation Vej ** Changed in: deja-dup (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1652410 Title: duja-dup AssertionError: ({1: 'duplicity- full.20161129T015237Z.vol1.difftar'} To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/deja-dup/+bug/1652410/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs