[Desktop-packages] [Bug 1612554] Re: duplicity backup fails with OverflowError: size does not fit in an int
Incr backup ran successfully with the sparse file excluded. --- NOTICE 1 . --[ Backup Statistics ]-- . StartTime 1471995038.09 (Tue Aug 23 23:30:38 2016) . EndTime 1472019526.98 (Wed Aug 24 06:18:46 2016) . ElapsedTime 24488.90 (6 hours 48 minutes 8.90 seconds) . SourceFiles 307014 . SourceFileSize 934910254490 (871 GB) . NewFiles 35356 . NewFileSize 493377240055 (459 GB) . DeletedFiles 34498 . ChangedFiles 3956 . ChangedFileSize 1536875100 (1.43 GB) . ChangedDeltaSize 0 (0 bytes) . DeltaEntries 73810 . RawDeltaSize 494391455212 (460 GB) . TotalDestinationSizeChange 355318722017 (331 GB) . Errors 0 . - . -- You received this bug notification because you are a member of Desktop Packages, which is subscribed to duplicity in Ubuntu. https://bugs.launchpad.net/bugs/1612554 Title: duplicity backup fails with OverflowError: size does not fit in an int Status in Duplicity: In Progress Status in duplicity package in Ubuntu: New Bug description: Ubuntu 14.04.4 LTS duplicity 0.6.23-1ubuntu4.1 python2.7 2.7.6-8ubuntu0.2 Backup command used: duplicity --exclude-filelist "/root/.duplicity_excludes" --full-if-older-than 7D --log-file "/var/log/duplicity" --no-encryption --volsize 100 / "$S3BUCKET" Backups have been running fine for several months, but since last night it started giving following error: --- Reading filelist /root/.duplicity_excludes Sorting filelist /root/.duplicity_excludes Local and Remote metadata are synchronized, no sync needed. Last full backup date: Sun Aug 7 23:30:02 2016 Traceback (most recent call last): File "/usr/bin/duplicity", line 1494, in with_tempdir(main) File "/usr/bin/duplicity", line 1488, in with_tempdir fn() File "/usr/bin/duplicity", line 1337, in main do_backup(action) File "/usr/bin/duplicity", line 1470, in do_backup incremental_backup(sig_chain) File "/usr/bin/duplicity", line 637, in incremental_backup globals.backend) File "/usr/bin/duplicity", line 405, in write_multivol at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize) File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383, in GzipWriteFile new_block = block_iter.next() File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 518, in next result = self.process(self.input_iter.next()) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 213, in get_delta_iter (new_path, sig_path, sigTarFile)) File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 37, in check_common_error return function(*args) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 134, in get_delta_path delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp)) File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line 150, in __init__ self.maker = _librsync.new_deltamaker(sig_string) OverflowError: size does not fit in an int To manage notifications about this bug go to: https://bugs.launchpad.net/duplicity/+bug/1612554/+subscriptions -- Mailing list: https://launchpad.net/~desktop-packages Post to : desktop-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~desktop-packages More help : https://help.launchpad.net/ListHelp
[Desktop-packages] [Bug 1612554] Re: duplicity backup fails with OverflowError: size does not fit in an int
Re-ran the incremental backup, but it fails with the same error while adding /var/log/lastlog, which is a sparse file with a reported size of 36Gb, but actually only 56k on disk. Can this be the cause? --- (duplicity-venv) # ls -l /var/log/lastlog -rw-rw-r-- 1 root utmp 367219431264 Aug 23 09:57 /var/log/lastlog # du -k /var/log/lastlog 56 /var/log/lastlog --- Selecting /var/log/lastlog Comparing var/log/lastlog and var/log/lastlog Getting delta of (var/log/lastlog reg) and (var/log/lastlog reg) Releasing lockfile /root/.cache/duplicity/d485e8d753a7b2ad7aafce096f9fbb92/lockfile.lock Removing still remembered temporary file /tmp/duplicity-0VnewC-tempdir/mktemp-MvQFNN-128 Removing still remembered temporary file /tmp/duplicity-0VnewC-tempdir/mkstemp-Lc2BZc-1 Traceback (most recent call last): File "/opt/duplicity-venv/bin/duplicity", line 1546, in with_tempdir(main) File "/opt/duplicity-venv/bin/duplicity", line 1540, in with_tempdir fn() File "/opt/duplicity-venv/bin/duplicity", line 1391, in main do_backup(action) File "/opt/duplicity-venv/bin/duplicity", line 1522, in do_backup incremental_backup(sig_chain) File "/opt/duplicity-venv/bin/duplicity", line 671, in incremental_backup globals.backend) File "/opt/duplicity-venv/bin/duplicity", line 434, in write_multivol at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize) File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/gpg.py", line 401, in GzipWriteFile new_block = block_iter.next() File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py", line 523, in next result = self.process(self.input_iter.next()) File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py", line 218, in get_delta_iter (new_path, sig_path, sigTarFile)) File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/robust.py", line 38, in check_common_error return function(*args) File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py", line 139, in get_delta_path delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp)) File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/librsync.py", line 154, in __init__ self.maker = _librsync.new_deltamaker(sig_string) OverflowError: size does not fit in an int -- You received this bug notification because you are a member of Desktop Packages, which is subscribed to duplicity in Ubuntu. https://bugs.launchpad.net/bugs/1612554 Title: duplicity backup fails with OverflowError: size does not fit in an int Status in Duplicity: In Progress Status in duplicity package in Ubuntu: New Bug description: Ubuntu 14.04.4 LTS duplicity 0.6.23-1ubuntu4.1 python2.7 2.7.6-8ubuntu0.2 Backup command used: duplicity --exclude-filelist "/root/.duplicity_excludes" --full-if-older-than 7D --log-file "/var/log/duplicity" --no-encryption --volsize 100 / "$S3BUCKET" Backups have been running fine for several months, but since last night it started giving following error: --- Reading filelist /root/.duplicity_excludes Sorting filelist /root/.duplicity_excludes Local and Remote metadata are synchronized, no sync needed. Last full backup date: Sun Aug 7 23:30:02 2016 Traceback (most recent call last): File "/usr/bin/duplicity", line 1494, in with_tempdir(main) File "/usr/bin/duplicity", line 1488, in with_tempdir fn() File "/usr/bin/duplicity", line 1337, in main do_backup(action) File "/usr/bin/duplicity", line 1470, in do_backup incremental_backup(sig_chain) File "/usr/bin/duplicity", line 637, in incremental_backup globals.backend) File "/usr/bin/duplicity", line 405, in write_multivol at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize) File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383, in GzipWriteFile new_block = block_iter.next() File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 518, in next result = self.process(self.input_iter.next()) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 213, in get_delta_iter (new_path, sig_path, sigTarFile)) File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 37, in check_common_error return function(*args) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 134, in get_delta_path delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp)) File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line 150, in __init__ self.maker = _librsync.new_deltamaker(sig_string) OverflowError: size does not fit in an int To manage notifications about this bug go to: https://bugs.launchpad.net/duplicity/+bug/1612554/+subscriptions -- Mailing list: https://launchpad.net/~desktop-packages Post to : desktop-packages@lists.launchpad.net Unsubscribe : https:
[Desktop-packages] [Bug 1612554] Re: duplicity backup fails with OverflowError: size does not fit in an int
# duplicity cleanup --force --no-encryption $BUCKET Local and Remote metadata are synchronized, no sync needed. Last full backup date: Wed Aug 17 08:38:55 2016 No extraneous files found, nothing deleted in cleanup. Collection-status gives me: Found primary backup chain with matching signature chain: - Chain start time: Wed Aug 17 08:38:55 2016 Chain end time: Wed Aug 17 08:38:55 2016 Number of contained backup sets: 1 Total number of contained volumes: 1665 Type of backup set:Time: Num volumes: Full Wed Aug 17 08:38:55 2016 1665 - No orphaned or incomplete backup sets found. s3cmd shows that there's no files with a timestamp later than duplicity- full.20160817T083855Z -- You received this bug notification because you are a member of Desktop Packages, which is subscribed to duplicity in Ubuntu. https://bugs.launchpad.net/bugs/1612554 Title: duplicity backup fails with OverflowError: size does not fit in an int Status in Duplicity: In Progress Status in duplicity package in Ubuntu: New Bug description: Ubuntu 14.04.4 LTS duplicity 0.6.23-1ubuntu4.1 python2.7 2.7.6-8ubuntu0.2 Backup command used: duplicity --exclude-filelist "/root/.duplicity_excludes" --full-if-older-than 7D --log-file "/var/log/duplicity" --no-encryption --volsize 100 / "$S3BUCKET" Backups have been running fine for several months, but since last night it started giving following error: --- Reading filelist /root/.duplicity_excludes Sorting filelist /root/.duplicity_excludes Local and Remote metadata are synchronized, no sync needed. Last full backup date: Sun Aug 7 23:30:02 2016 Traceback (most recent call last): File "/usr/bin/duplicity", line 1494, in with_tempdir(main) File "/usr/bin/duplicity", line 1488, in with_tempdir fn() File "/usr/bin/duplicity", line 1337, in main do_backup(action) File "/usr/bin/duplicity", line 1470, in do_backup incremental_backup(sig_chain) File "/usr/bin/duplicity", line 637, in incremental_backup globals.backend) File "/usr/bin/duplicity", line 405, in write_multivol at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize) File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383, in GzipWriteFile new_block = block_iter.next() File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 518, in next result = self.process(self.input_iter.next()) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 213, in get_delta_iter (new_path, sig_path, sigTarFile)) File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 37, in check_common_error return function(*args) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 134, in get_delta_path delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp)) File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line 150, in __init__ self.maker = _librsync.new_deltamaker(sig_string) OverflowError: size does not fit in an int To manage notifications about this bug go to: https://bugs.launchpad.net/duplicity/+bug/1612554/+subscriptions -- Mailing list: https://launchpad.net/~desktop-packages Post to : desktop-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~desktop-packages More help : https://help.launchpad.net/ListHelp
[Desktop-packages] [Bug 1612554] Re: duplicity backup fails with OverflowError: size does not fit in an int
Package: librsync1 Version: 0.9.7-10 -- You received this bug notification because you are a member of Desktop Packages, which is subscribed to duplicity in Ubuntu. https://bugs.launchpad.net/bugs/1612554 Title: duplicity backup fails with OverflowError: size does not fit in an int Status in Duplicity: Invalid Status in duplicity package in Ubuntu: New Bug description: Ubuntu 14.04.4 LTS duplicity 0.6.23-1ubuntu4.1 python2.7 2.7.6-8ubuntu0.2 Backup command used: duplicity --exclude-filelist "/root/.duplicity_excludes" --full-if-older-than 7D --log-file "/var/log/duplicity" --no-encryption --volsize 100 / "$S3BUCKET" Backups have been running fine for several months, but since last night it started giving following error: --- Reading filelist /root/.duplicity_excludes Sorting filelist /root/.duplicity_excludes Local and Remote metadata are synchronized, no sync needed. Last full backup date: Sun Aug 7 23:30:02 2016 Traceback (most recent call last): File "/usr/bin/duplicity", line 1494, in with_tempdir(main) File "/usr/bin/duplicity", line 1488, in with_tempdir fn() File "/usr/bin/duplicity", line 1337, in main do_backup(action) File "/usr/bin/duplicity", line 1470, in do_backup incremental_backup(sig_chain) File "/usr/bin/duplicity", line 637, in incremental_backup globals.backend) File "/usr/bin/duplicity", line 405, in write_multivol at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize) File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383, in GzipWriteFile new_block = block_iter.next() File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 518, in next result = self.process(self.input_iter.next()) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 213, in get_delta_iter (new_path, sig_path, sigTarFile)) File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 37, in check_common_error return function(*args) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 134, in get_delta_path delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp)) File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line 150, in __init__ self.maker = _librsync.new_deltamaker(sig_string) OverflowError: size does not fit in an int To manage notifications about this bug go to: https://bugs.launchpad.net/duplicity/+bug/1612554/+subscriptions -- Mailing list: https://launchpad.net/~desktop-packages Post to : desktop-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~desktop-packages More help : https://help.launchpad.net/ListHelp
[Desktop-packages] [Bug 1612554] Re: duplicity backup fails with OverflowError: size does not fit in an int
I didn't install any specific version, since it was included in the duplicity tarball: $ tar tf duplicity-0.7.09.tar.gz | grep -i rsync duplicity-0.7.09/duplicity/_librsyncmodule.c duplicity-0.7.09/duplicity/librsync.py duplicity-0.7.09/duplicity/backends/rsyncbackend.py $ pip list boto (2.42.0) duplicity (0.7.9) lockfile (0.12.2) pip (8.1.2) setuptools (25.2.0) wheel (0.29.0) -- You received this bug notification because you are a member of Desktop Packages, which is subscribed to duplicity in Ubuntu. https://bugs.launchpad.net/bugs/1612554 Title: duplicity backup fails with OverflowError: size does not fit in an int Status in Duplicity: Invalid Status in duplicity package in Ubuntu: New Bug description: Ubuntu 14.04.4 LTS duplicity 0.6.23-1ubuntu4.1 python2.7 2.7.6-8ubuntu0.2 Backup command used: duplicity --exclude-filelist "/root/.duplicity_excludes" --full-if-older-than 7D --log-file "/var/log/duplicity" --no-encryption --volsize 100 / "$S3BUCKET" Backups have been running fine for several months, but since last night it started giving following error: --- Reading filelist /root/.duplicity_excludes Sorting filelist /root/.duplicity_excludes Local and Remote metadata are synchronized, no sync needed. Last full backup date: Sun Aug 7 23:30:02 2016 Traceback (most recent call last): File "/usr/bin/duplicity", line 1494, in with_tempdir(main) File "/usr/bin/duplicity", line 1488, in with_tempdir fn() File "/usr/bin/duplicity", line 1337, in main do_backup(action) File "/usr/bin/duplicity", line 1470, in do_backup incremental_backup(sig_chain) File "/usr/bin/duplicity", line 637, in incremental_backup globals.backend) File "/usr/bin/duplicity", line 405, in write_multivol at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize) File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383, in GzipWriteFile new_block = block_iter.next() File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 518, in next result = self.process(self.input_iter.next()) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 213, in get_delta_iter (new_path, sig_path, sigTarFile)) File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 37, in check_common_error return function(*args) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 134, in get_delta_path delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp)) File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line 150, in __init__ self.maker = _librsync.new_deltamaker(sig_string) OverflowError: size does not fit in an int To manage notifications about this bug go to: https://bugs.launchpad.net/duplicity/+bug/1612554/+subscriptions -- Mailing list: https://launchpad.net/~desktop-packages Post to : desktop-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~desktop-packages More help : https://help.launchpad.net/ListHelp
[Desktop-packages] [Bug 1612554] Re: duplicity backup fails with OverflowError: size does not fit in an int
Cried victory too soon, I'm afraid.
I have:
- compiled python 2.7.12 with the zlib patch
- created a virtualenv with that version
- installed duplicity in the virtualenv, along with lockfile and boto
The zlib test now succeeds:
(duplicity-venv) root@eniac:~# python -c "import zlib;zlib.crc32('a'*(1<<31 -
1))"
(duplicity-venv) root@eniac:~# python -c "import zlib;zlib.crc32('a'*(1<<31))"
(duplicity-venv) root@eniac:~#
But duplicity still throws the OverFlowError during Incremental backup,
and it wasn't anywhere near a large file. The modules mentioned in the
Traceback are all in the duplicity dir, btw.
---
DEBUG 1
. Selection: result: None from function: Command-line exclude glob:
/var/lib/elasticsearch
DEBUG 1
. Selection: + including file
DEBUG 1
. Selecting /var/log/lastlog
DEBUG 1
. Comparing var/log/lastlog and var/log/lastlog
DEBUG 1
. Getting delta of (var/log/lastlog reg) and (var/log/lastlog reg)
DEBUG 1
. Releasing lockfile
/root/.cache/duplicity/d485e8d753a7b2ad7aafce096f9fbb92/lockfile.lock
DEBUG 1
. Removing still remembered temporary file
/tmp/duplicity-WNi8WI-tempdir/mktemp-yuJbZJ-3
DEBUG 1
. Removing still remembered temporary file
/tmp/duplicity-WNi8WI-tempdir/mkstemp-hiRlXx-1
ERROR 30 OverflowError
. Traceback (most recent call last):
. File "/opt/duplicity-venv/bin/duplicity", line 1546, in
. with_tempdir(main)
. File "/opt/duplicity-venv/bin/duplicity", line 1540, in with_tempdir
. fn()
. File "/opt/duplicity-venv/bin/duplicity", line 1391, in main
. do_backup(action)
. File "/opt/duplicity-venv/bin/duplicity", line 1522, in do_backup
. incremental_backup(sig_chain)
. File "/opt/duplicity-venv/bin/duplicity", line 671, in incremental_backup
. globals.backend)
. File "/opt/duplicity-venv/bin/duplicity", line 434, in write_multivol
. at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize)
. File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/gpg.py",
line 401, in GzipWriteFile
. new_block = block_iter.next()
. File
"/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py", line
523, in next
. result = self.process(self.input_iter.next())
. File
"/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py", line
218, in get_delta_iter
. (new_path, sig_path, sigTarFile))
. File "/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/robust.py",
line 38, in check_common_error
. return function(*args)
. File
"/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/diffdir.py", line
139, in get_delta_path
. delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp))
. File
"/opt/duplicity-venv/lib/python2.7/site-packages/duplicity/librsync.py", line
154, in __init__
. self.maker = _librsync.new_deltamaker(sig_string)
. OverflowError: size does not fit in an int
--
You received this bug notification because you are a member of Desktop
Packages, which is subscribed to duplicity in Ubuntu.
https://bugs.launchpad.net/bugs/1612554
Title:
duplicity backup fails with OverflowError: size does not fit in an int
Status in Duplicity:
Invalid
Status in duplicity package in Ubuntu:
New
Bug description:
Ubuntu 14.04.4 LTS
duplicity 0.6.23-1ubuntu4.1
python2.7 2.7.6-8ubuntu0.2
Backup command used:
duplicity --exclude-filelist "/root/.duplicity_excludes" --full-if-older-than
7D --log-file "/var/log/duplicity" --no-encryption --volsize 100 / "$S3BUCKET"
Backups have been running fine for several months, but since last
night it started giving following error:
---
Reading filelist /root/.duplicity_excludes
Sorting filelist /root/.duplicity_excludes
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sun Aug 7 23:30:02 2016
Traceback (most recent call last):
File "/usr/bin/duplicity", line 1494, in
with_tempdir(main)
File "/usr/bin/duplicity", line 1488, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1337, in main
do_backup(action)
File "/usr/bin/duplicity", line 1470, in do_backup
incremental_backup(sig_chain)
File "/usr/bin/duplicity", line 637, in incremental_backup
globals.backend)
File "/usr/bin/duplicity", line 405, in write_multivol
at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name,
globals.volsize)
File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383,
in GzipWriteFile
new_block = block_iter.next()
File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line
518, in next
result = self.process(self.input_iter.next())
File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line
213, in get_delta_iter
(new_path, sig_path, sigTarFile))
File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line
37, in check_common_error
return function(*args)
File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", l
[Desktop-packages] [Bug 1612554] Re: duplicity backup fails with OverflowError: size does not fit in an int
Backup completed successfully with all >4Gb files excluded. -- You received this bug notification because you are a member of Desktop Packages, which is subscribed to duplicity in Ubuntu. https://bugs.launchpad.net/bugs/1612554 Title: duplicity backup fails with OverflowError: size does not fit in an int Status in Duplicity: New Status in duplicity package in Ubuntu: New Bug description: Ubuntu 14.04.4 LTS duplicity 0.6.23-1ubuntu4.1 python2.7 2.7.6-8ubuntu0.2 Backup command used: duplicity --exclude-filelist "/root/.duplicity_excludes" --full-if-older-than 7D --log-file "/var/log/duplicity" --no-encryption --volsize 100 / "$S3BUCKET" Backups have been running fine for several months, but since last night it started giving following error: --- Reading filelist /root/.duplicity_excludes Sorting filelist /root/.duplicity_excludes Local and Remote metadata are synchronized, no sync needed. Last full backup date: Sun Aug 7 23:30:02 2016 Traceback (most recent call last): File "/usr/bin/duplicity", line 1494, in with_tempdir(main) File "/usr/bin/duplicity", line 1488, in with_tempdir fn() File "/usr/bin/duplicity", line 1337, in main do_backup(action) File "/usr/bin/duplicity", line 1470, in do_backup incremental_backup(sig_chain) File "/usr/bin/duplicity", line 637, in incremental_backup globals.backend) File "/usr/bin/duplicity", line 405, in write_multivol at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize) File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383, in GzipWriteFile new_block = block_iter.next() File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 518, in next result = self.process(self.input_iter.next()) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 213, in get_delta_iter (new_path, sig_path, sigTarFile)) File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 37, in check_common_error return function(*args) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 134, in get_delta_path delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp)) File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line 150, in __init__ self.maker = _librsync.new_deltamaker(sig_string) OverflowError: size does not fit in an int To manage notifications about this bug go to: https://bugs.launchpad.net/duplicity/+bug/1612554/+subscriptions -- Mailing list: https://launchpad.net/~desktop-packages Post to : desktop-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~desktop-packages More help : https://help.launchpad.net/ListHelp
[Desktop-packages] [Bug 1612554] Re: duplicity backup fails with OverflowError: size does not fit in an int
Some more testing:
I installed a virtualenv with python 2.7.12 and duplicity 0.7.09, but
the problem still remains.
Then I ran the little test mentioned in
https://bugs.python.org/issue27130 which shows that zlib has a problem
zipping files > 4Gb:
When looking at the timestamps of the comments, it seems this issue was
fixed after 2.7.12 came out, so it will probably be in the next release.
I am currently running a backup with all files >4Gb excluded to see if
that completes OK.
---
$ python --version
Python 2.7.6
$ python -c "import zlib;zlib.crc32('a'*(1<<31 - 1))"
$ python -c "import zlib;zlib.crc32('a'*(1<<31))"
Traceback (most recent call last):
File "", line 1, in
OverflowError: size does not fit in an int
---
$ . venv.2.7.12/bin/activate
(venv.2.7.12) $ python --version
Python 2.7.12
(venv.2.7.12) $ python -c "import zlib;zlib.crc32('a'*(1<<31 - 1))"
(venv.2.7.12) $ python -c "import zlib;zlib.crc32('a'*(1<<31))"
Traceback (most recent call last):
File "", line 1, in
OverflowError: size does not fit in an int
** Bug watch added: Python Roundup #27130
http://bugs.python.org/issue27130
--
You received this bug notification because you are a member of Desktop
Packages, which is subscribed to duplicity in Ubuntu.
https://bugs.launchpad.net/bugs/1612554
Title:
duplicity backup fails with OverflowError: size does not fit in an int
Status in Duplicity:
New
Status in duplicity package in Ubuntu:
New
Bug description:
Ubuntu 14.04.4 LTS
duplicity 0.6.23-1ubuntu4.1
python2.7 2.7.6-8ubuntu0.2
Backup command used:
duplicity --exclude-filelist "/root/.duplicity_excludes" --full-if-older-than
7D --log-file "/var/log/duplicity" --no-encryption --volsize 100 / "$S3BUCKET"
Backups have been running fine for several months, but since last
night it started giving following error:
---
Reading filelist /root/.duplicity_excludes
Sorting filelist /root/.duplicity_excludes
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Sun Aug 7 23:30:02 2016
Traceback (most recent call last):
File "/usr/bin/duplicity", line 1494, in
with_tempdir(main)
File "/usr/bin/duplicity", line 1488, in with_tempdir
fn()
File "/usr/bin/duplicity", line 1337, in main
do_backup(action)
File "/usr/bin/duplicity", line 1470, in do_backup
incremental_backup(sig_chain)
File "/usr/bin/duplicity", line 637, in incremental_backup
globals.backend)
File "/usr/bin/duplicity", line 405, in write_multivol
at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name,
globals.volsize)
File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383,
in GzipWriteFile
new_block = block_iter.next()
File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line
518, in next
result = self.process(self.input_iter.next())
File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line
213, in get_delta_iter
(new_path, sig_path, sigTarFile))
File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line
37, in check_common_error
return function(*args)
File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line
134, in get_delta_path
delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp))
File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line
150, in __init__
self.maker = _librsync.new_deltamaker(sig_string)
OverflowError: size does not fit in an int
To manage notifications about this bug go to:
https://bugs.launchpad.net/duplicity/+bug/1612554/+subscriptions
--
Mailing list: https://launchpad.net/~desktop-packages
Post to : desktop-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~desktop-packages
More help : https://help.launchpad.net/ListHelp
[Desktop-packages] [Bug 1612554] Re: duplicity backup fails with OverflowError: size does not fit in an int
Hi, after performing these steps: - apt-get remove duplicity - add-apt-repository ppa:duplicity-team/ppa - apt-get update - apt-get install duplicity I am now running: $ duplicity --version duplicity 0.7.09 And I still get the Overflow errors: ERROR 30 OverflowError . Traceback (most recent call last): . File "/usr/bin/duplicity", line 1546, in . with_tempdir(main) . File "/usr/bin/duplicity", line 1540, in with_tempdir . fn() . File "/usr/bin/duplicity", line 1391, in main . do_backup(action) . File "/usr/bin/duplicity", line 1522, in do_backup . incremental_backup(sig_chain) . File "/usr/bin/duplicity", line 671, in incremental_backup . globals.backend) . File "/usr/bin/duplicity", line 434, in write_multivol . at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize) . File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 401, in GzipWriteFile . new_block = block_iter.next() . File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 523, in next . result = self.process(self.input_iter.next()) . File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 218, in get_delta_iter . (new_path, sig_path, sigTarFile)) . File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 38, in check_common_error . return function(*args) . File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 139, in get_delta_path . delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp)) . File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line 154, in __init__ . self.maker = _librsync.new_deltamaker(sig_string) . OverflowError: size does not fit in an int -- You received this bug notification because you are a member of Desktop Packages, which is subscribed to duplicity in Ubuntu. https://bugs.launchpad.net/bugs/1612554 Title: duplicity backup fails with OverflowError: size does not fit in an int Status in duplicity package in Ubuntu: Invalid Bug description: Ubuntu 14.04.4 LTS duplicity 0.6.23-1ubuntu4.1 python2.7 2.7.6-8ubuntu0.2 Backup command used: duplicity --exclude-filelist "/root/.duplicity_excludes" --full-if-older-than 7D --log-file "/var/log/duplicity" --no-encryption --volsize 100 / "$S3BUCKET" Backups have been running fine for several months, but since last night it started giving following error: --- Reading filelist /root/.duplicity_excludes Sorting filelist /root/.duplicity_excludes Local and Remote metadata are synchronized, no sync needed. Last full backup date: Sun Aug 7 23:30:02 2016 Traceback (most recent call last): File "/usr/bin/duplicity", line 1494, in with_tempdir(main) File "/usr/bin/duplicity", line 1488, in with_tempdir fn() File "/usr/bin/duplicity", line 1337, in main do_backup(action) File "/usr/bin/duplicity", line 1470, in do_backup incremental_backup(sig_chain) File "/usr/bin/duplicity", line 637, in incremental_backup globals.backend) File "/usr/bin/duplicity", line 405, in write_multivol at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize) File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383, in GzipWriteFile new_block = block_iter.next() File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 518, in next result = self.process(self.input_iter.next()) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 213, in get_delta_iter (new_path, sig_path, sigTarFile)) File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 37, in check_common_error return function(*args) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 134, in get_delta_path delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp)) File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line 150, in __init__ self.maker = _librsync.new_deltamaker(sig_string) OverflowError: size does not fit in an int To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/duplicity/+bug/1612554/+subscriptions -- Mailing list: https://launchpad.net/~desktop-packages Post to : desktop-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~desktop-packages More help : https://help.launchpad.net/ListHelp
[Desktop-packages] [Bug 1612554] Re: duplicity backup fails with OverflowError: size does not fit in an int
Possibly related to https://bugs.python.org/issue23306 -- You received this bug notification because you are a member of Desktop Packages, which is subscribed to duplicity in Ubuntu. https://bugs.launchpad.net/bugs/1612554 Title: duplicity backup fails with OverflowError: size does not fit in an int Status in duplicity package in Ubuntu: New Bug description: Ubuntu 14.04.4 LTS duplicity 0.6.23-1ubuntu4.1 python2.7 2.7.6-8ubuntu0.2 Backup command used: duplicity --exclude-filelist "/root/.duplicity_excludes" --full-if-older-than 7D --log-file "/var/log/duplicity" --no-encryption --volsize 100 / "$S3BUCKET" Backups have been running fine for several months, but since last night it started giving following error: --- Reading filelist /root/.duplicity_excludes Sorting filelist /root/.duplicity_excludes Local and Remote metadata are synchronized, no sync needed. Last full backup date: Sun Aug 7 23:30:02 2016 Traceback (most recent call last): File "/usr/bin/duplicity", line 1494, in with_tempdir(main) File "/usr/bin/duplicity", line 1488, in with_tempdir fn() File "/usr/bin/duplicity", line 1337, in main do_backup(action) File "/usr/bin/duplicity", line 1470, in do_backup incremental_backup(sig_chain) File "/usr/bin/duplicity", line 637, in incremental_backup globals.backend) File "/usr/bin/duplicity", line 405, in write_multivol at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize) File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383, in GzipWriteFile new_block = block_iter.next() File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 518, in next result = self.process(self.input_iter.next()) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 213, in get_delta_iter (new_path, sig_path, sigTarFile)) File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 37, in check_common_error return function(*args) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 134, in get_delta_path delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp)) File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line 150, in __init__ self.maker = _librsync.new_deltamaker(sig_string) OverflowError: size does not fit in an int To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/duplicity/+bug/1612554/+subscriptions -- Mailing list: https://launchpad.net/~desktop-packages Post to : desktop-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~desktop-packages More help : https://help.launchpad.net/ListHelp
[Desktop-packages] [Bug 1612554] [NEW] duplicity backup fails with OverflowError: size does not fit in an int
Public bug reported: Ubuntu 14.04.4 LTS duplicity 0.6.23-1ubuntu4.1 python2.7 2.7.6-8ubuntu0.2 Backup command used: duplicity --exclude-filelist "/root/.duplicity_excludes" --full-if-older-than 7D --log-file "/var/log/duplicity" --no-encryption --volsize 100 / "$S3BUCKET" Backups have been running fine for several months, but since last night it started giving following error: --- Reading filelist /root/.duplicity_excludes Sorting filelist /root/.duplicity_excludes Local and Remote metadata are synchronized, no sync needed. Last full backup date: Sun Aug 7 23:30:02 2016 Traceback (most recent call last): File "/usr/bin/duplicity", line 1494, in with_tempdir(main) File "/usr/bin/duplicity", line 1488, in with_tempdir fn() File "/usr/bin/duplicity", line 1337, in main do_backup(action) File "/usr/bin/duplicity", line 1470, in do_backup incremental_backup(sig_chain) File "/usr/bin/duplicity", line 637, in incremental_backup globals.backend) File "/usr/bin/duplicity", line 405, in write_multivol at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize) File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383, in GzipWriteFile new_block = block_iter.next() File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 518, in next result = self.process(self.input_iter.next()) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 213, in get_delta_iter (new_path, sig_path, sigTarFile)) File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 37, in check_common_error return function(*args) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 134, in get_delta_path delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp)) File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line 150, in __init__ self.maker = _librsync.new_deltamaker(sig_string) OverflowError: size does not fit in an int ** Affects: duplicity (Ubuntu) Importance: Undecided Status: New ** Bug watch added: Python Roundup #23306 http://bugs.python.org/issue23306 -- You received this bug notification because you are a member of Desktop Packages, which is subscribed to duplicity in Ubuntu. https://bugs.launchpad.net/bugs/1612554 Title: duplicity backup fails with OverflowError: size does not fit in an int Status in duplicity package in Ubuntu: New Bug description: Ubuntu 14.04.4 LTS duplicity 0.6.23-1ubuntu4.1 python2.7 2.7.6-8ubuntu0.2 Backup command used: duplicity --exclude-filelist "/root/.duplicity_excludes" --full-if-older-than 7D --log-file "/var/log/duplicity" --no-encryption --volsize 100 / "$S3BUCKET" Backups have been running fine for several months, but since last night it started giving following error: --- Reading filelist /root/.duplicity_excludes Sorting filelist /root/.duplicity_excludes Local and Remote metadata are synchronized, no sync needed. Last full backup date: Sun Aug 7 23:30:02 2016 Traceback (most recent call last): File "/usr/bin/duplicity", line 1494, in with_tempdir(main) File "/usr/bin/duplicity", line 1488, in with_tempdir fn() File "/usr/bin/duplicity", line 1337, in main do_backup(action) File "/usr/bin/duplicity", line 1470, in do_backup incremental_backup(sig_chain) File "/usr/bin/duplicity", line 637, in incremental_backup globals.backend) File "/usr/bin/duplicity", line 405, in write_multivol at_end = gpg.GzipWriteFile(tarblock_iter, tdp.name, globals.volsize) File "/usr/lib/python2.7/dist-packages/duplicity/gpg.py", line 383, in GzipWriteFile new_block = block_iter.next() File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 518, in next result = self.process(self.input_iter.next()) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 213, in get_delta_iter (new_path, sig_path, sigTarFile)) File "/usr/lib/python2.7/dist-packages/duplicity/robust.py", line 37, in check_common_error return function(*args) File "/usr/lib/python2.7/dist-packages/duplicity/diffdir.py", line 134, in get_delta_path delta_path.setfileobj(librsync.DeltaFile(old_sigfp, newfp)) File "/usr/lib/python2.7/dist-packages/duplicity/librsync.py", line 150, in __init__ self.maker = _librsync.new_deltamaker(sig_string) OverflowError: size does not fit in an int To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/duplicity/+bug/1612554/+subscriptions -- Mailing list: https://launchpad.net/~desktop-packages Post to : desktop-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~desktop-packages More help : https://help.launchpad.net/ListHelp
