Bug#831759: fixed in backup-manager 0.7.12-2
This is definitely not a file ordering problem, because the sample tarballs have a single file inside a directory inside a directory. The order for that will always be (I think) the parent directory, the directory inside the directory, and then the file. The problem here is gzip, not tar, when invoked without -n option. Attached you will find two different tarballs with the same contents, extracted from one of the cases where t15-dupes.sh failed. If you run "file" on them you will see this: [...] last modified: Wed Aug 17 12:31:46 2016, from Unix [...] last modified: Wed Aug 17 12:31:47 2016, from Unix So for the test to be successful, the tarballs have both to be created in the same one-second interval. That's why this usually succeeds on machines which are fast enough, but not always. Suggested fix: When creating the tar.gz, gzip should run with -n. This could be done with GZIP environment variable, but I think it's deprecated. It would be better if there was a way to tell tar to pass -n to gzip. Then there will be no embedded dates in the tar.gz and they will always be identical, no matter the second in which they end being created. Thanks. skywalker1-build-backup-manager-0.7.12-t-testdir.20160816.master.tar.gz Description: application/gzip skywalker1-build-backup-manager-0.7.12-t-testdir.20160817.master.tar.gz Description: application/gzip
Bug#831759: fixed in backup-manager 0.7.12-2
On Wed, Aug 17, 2016 at 11:55:03AM +0200, Maximiliano Curia wrote: > Do you still have the failling tarballs? It would be better to list the > tarballs contents to check if the file order is in fact the issue here or if > the difference lies somewhere else. I think I will be able to provide different tarballs with the same contents. You were right, maybe it's not the file ordering.
Bug#831759: fixed in backup-manager 0.7.12-2
On Wed, Aug 17, 2016 at 11:55:03AM +0200, Maximiliano Curia wrote: > > + /usr/bin/nice -n 10 /bin/tar -p -c -z -f > > ./repository/uranio1-tmp-BM-backup-manager-0.7.12-t-testdir.20160815.master.tar.gz > > /tmp/BM/backup-manager-0.7.12/t/testdir > > ++ get_md5sum > > ./repository/uranio1-tmp-BM-backup-manager-0.7.12-t-testdir.20160815.master.tar.gz > > ++ md5='ee176eaec8040e7230c18d2b8404d313 > > ./repository/uranio1-tmp-BM-backup-manager-0.7.12-t-testdir.20160815.master.tar.gz' > > > + debug '/usr/bin/nice -n 10 /bin/tar -p -c -z -f > > ./repository/uranio1-tmp-BM-backup-manager-0.7.12-t-testdir.20160816.master.tar.gz > > "/tmp/BM/backup-manager-0.7.12/t/testdir" > > > /tmp/backup-manager/bm-tarball.log.FwLAvP 2>&1' > > ++ md5='f39797a9ee7c033e291d34b4304386eb > > ./repository/uranio1-tmp-BM-backup-manager-0.7.12-t-testdir.20160816.master.tar.gz' > > Mmh, the script is creating two "identical" tarballs but they get different > md5sum (which is what's used to detect the duplicated tarballs). tar is know > to introduce some "undefined" bits in the files, that's what pristine-tar's > delta files manage. Actually, I think tar is currently deterministic if you give it the same files with the same timestamps and the files are in the same order. (But if you tar a directory, then the order is undefined and may be anything). > From the tar invocation, I would suspect that a difference might occur if by > any reason the file order in which tar processes the directory varies. This > could be the case if a filesystem reorders/rebalances its directories after > the first transversal, for example. Oh, I see. There is simply not a canonical order to traverse a directory and people should just not assume that traversing a directory two times in a row will yield the files in the same order. You don't need to imagine reorders or rebalances in the filesystem, there is simply not a guarantee anywhere that such thing will happen. So, making a tar from the "contents of a directory" and always expecting the same result is just wrong. This is already a known problem for people working in reproducible builds. Please take a look at this: https://wiki.debian.org/ReproducibleBuilds/FileOrderInTarballs There was even a fuse filesystem called "disorderfs" which intentionally made the order of files in the directories to be random. I think it was disabled because it was not stable enough, but I think it helped to catch quite a bunch of bugs like this one. So, I really believe that's the problem. Just follow the wiki page and you will have a fix. I don't see the need to perform more tests. But for completeness, I'll answer your questions: > [...] > How many times was needed to run the test to trigger a fail? What kind of > filesystem are you using? Is that filesystem using any special mounting > options? A lot, maybe 30 or 40. I'm using ext4, nothing special, and I'm not using any special mounting options. But that's not relevant, really. You have the key of the problem and the wiki page has the solution. Thanks.
Processed: Re: Bug#831759: fixed in backup-manager 0.7.12-2
Processing control commands: > tag -1 + moreinfo help Bug #831759 [src:backup-manager] backup-manager: FTBFS: Running test t15-dupes.sh: failed Added tag(s) help. -- 831759: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=831759 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Bug#831759: fixed in backup-manager 0.7.12-2
Control: tag -1 + moreinfo help ¡Hola Santiago! El 2016-08-16 a las 19:02 +0100, Santiago Vila escribió: On Tue, Aug 16, 2016 at 07:21:43PM +0200, Maximiliano Curia wrote: El 2016-08-16 a las 15:40 +0200, Santiago Vila escribió: found 831759 0.7.12-3 thanks Does not seem fixed: https://tests.reproducible-builds.org/debian/rbuild/testing/amd64/backup-manager_0.7.12-3.rbuild.log I doubt that it's the same issue. Anyway, I can't reproduce the issue here. Can you test the failling test with set -x ? Apparently, you have to test it a lot of times until it fails, like a lottery (so please don't ask me for more debugging, I don't even use this package myself). Attached you will find the output of one of the times I managed to make it fail. > + /usr/bin/nice -n 10 /bin/tar -p -c -z -f ./repository/uranio1-tmp-BM-backup-manager-0.7.12-t-testdir.20160815.master.tar.gz /tmp/BM/backup-manager-0.7.12/t/testdir > ++ get_md5sum ./repository/uranio1-tmp-BM-backup-manager-0.7.12-t-testdir.20160815.master.tar.gz > ++ md5='ee176eaec8040e7230c18d2b8404d313 ./repository/uranio1-tmp-BM-backup-manager-0.7.12-t-testdir.20160815.master.tar.gz' > + debug '/usr/bin/nice -n 10 /bin/tar -p -c -z -f ./repository/uranio1-tmp-BM-backup-manager-0.7.12-t-testdir.20160816.master.tar.gz "/tmp/BM/backup-manager-0.7.12/t/testdir" > /tmp/backup-manager/bm-tarball.log.FwLAvP 2>&1' > ++ md5='f39797a9ee7c033e291d34b4304386eb ./repository/uranio1-tmp-BM-backup-manager-0.7.12-t-testdir.20160816.master.tar.gz' Mmh, the script is creating two "identical" tarballs but they get different md5sum (which is what's used to detect the duplicated tarballs). tar is know to introduce some "undefined" bits in the files, that's what pristine-tar's delta files manage. From the tar invocation, I would suspect that a difference might occur if by any reason the file order in which tar processes the directory varies. This could be the case if a filesystem reorders/rebalances its directories after the first transversal, for example. If this is the case then, maybe, using --sort=name in the tar invocation fixes the issue. How many times was needed to run the test to trigger a fail? What kind of filesystem are you using? Is that filesystem using any special mounting options? Do you still have the failling tarballs? It would be better to list the tarballs contents to check if the file order is in fact the issue here or if the difference lies somewhere else. Happy hacking, -- "The first 90% of the code accounts for the first 90% of the development time. The remaining 10% of the code accounts for the other 90% of the development time." -- Tom Cargill Saludos /\/\ /\ >< `/ signature.asc Description: Digital signature
Bug#831759: fixed in backup-manager 0.7.12-2
On Tue, Aug 16, 2016 at 07:21:43PM +0200, Maximiliano Curia wrote: > ¡Hola Santiago! > > El 2016-08-16 a las 15:40 +0200, Santiago Vila escribió: > >found 831759 0.7.12-3 thanks > > >Does not seem fixed: > > >https://tests.reproducible-builds.org/debian/rbuild/testing/amd64/backup-manager_0.7.12-3.rbuild.log > > I doubt that it's the same issue. Anyway, I can't reproduce the issue here. > Can you test the failling test with set -x ? Apparently, you have to test it a lot of times until it fails, like a lottery (so please don't ask me for more debugging, I don't even use this package myself). Attached you will find the output of one of the times I managed to make it fail. Thanks. + set -e + source testlib.sh ++ locallib=../lib ++ libdir=../lib ++ source ../lib/externals.sh which zip +++ zip= +++ true which bzip2 +++ bzip=/bin/bzip2 which gzip +++ gzip=/bin/gzip which gpg +++ gpg=/usr/bin/gpg which lzma +++ lzma=/usr/bin/lzma which dar +++ dar=/usr/bin/dar which tar +++ tar=/bin/tar which rsync +++ rsync=/usr/bin/rsync which mkisofs +++ mkisofs= which genisoimage +++ mkisofs= +++ true which growisofs +++ growisofs= +++ true which dvd+rw-format +++ dvdrwformat= +++ true which cdrecord +++ cdrecord= which wodim +++ cdrecord= +++ true which md5sum +++ md5sum=/usr/bin/md5sum which bc +++ bc= +++ true which mysqldump +++ mysqldump= +++ true which mysql +++ mysql= +++ true which pg_dump +++ pgdump= +++ true which svnadmin +++ svnadmin= +++ true which logger +++ logger=/usr/bin/logger which nice +++ nice_bin=/usr/bin/nice which dd +++ dd=/bin/dd ++ source ../lib/gettext.sh +++ [[ -z ../lib ]] +++ libgettext=/usr/bin/gettext.sh +++ [[ ! -f /usr/bin/gettext.sh ]] +++ . ../lib/gettext-real.sh . /usr/bin/gettext.sh + test 'X\t' = 'X\t' + echo=echo + test -z '' + case "$0" in TEXTDOMAIN=backup-manager export TEXTDOMAIN ++ source ../lib/logger.sh ++ source ../lib/dialog.sh ++ source ../lib/files.sh +++ find_opt= +++ find -H /tmp +++ find_opt=-H ++ source ../lib/md5sum.sh ++ source ../lib/backup-methods.sh ++ source ../lib/upload-methods.sh ++ source ../lib/burning-methods.sh ++ source ../lib/actions.sh ++ source ../lib/dbus.sh +++ PROGRESS_INTERFACE=org.backupmanager.BackupManager.ProgressNotify +++ EVENT_INTERFACE=org.backupmanager.BackupManager.EventNotify +++ LOG_MESSAGE_INTERFACE=org.backupmanager.BackupManager.LogMessageNotify +++ SYSTEM_BUS_OBJECT=/org/backupmanager/instance/SystemInstance +++ USER_BUS_OBJECT=/org/backupmanager/instance/UserInstance/ ++ VERSION=0.7.1+svn +++ which zip ++ zip= ++ true +++ which bzip2 ++ bzip=/bin/bzip2 +++ which gzip ++ gzip=/bin/gzip +++ which gpg ++ gpg=/usr/bin/gpg +++ which lzma ++ lzma=/usr/bin/lzma +++ which dar ++ dar=/usr/bin/dar +++ which tar ++ tar=/bin/tar +++ which rsync ++ rsync=/usr/bin/rsync +++ which mkisofs ++ mkisofs= ++ true +++ which growisofs ++ growisofs= ++ true +++ which dvd+rw-format ++ dvdrwformat= ++ true +++ which cdrecord ++ cdrecord= ++ true +++ which md5sum ++ md5sum=/usr/bin/md5sum +++ which bc ++ bc= ++ true +++ which mysqldump ++ mysqldump= ++ true +++ which svnadmin ++ svnadmin= ++ true +++ which logger ++ logger=/usr/bin/logger ++ systemlockfile=/var/run/backup-manager.lock ++ userlockfile=/build/.backup-manager.lock ++ [[ 924 != 0 ]] ++ lockfile=/build/.backup-manager.lock ++ libdir=../lib ++ bmu='perl -I.. ../backup-manager-upload' ++ bmp='perl -I.. ../backup-manager-purge' ++ conffile=confs/base.conf ++ version=false ++ force=false ++ upload=false ++ burn=false ++ help=false ++ md5check=false ++ purge=false ++ warnings=false ++ verbose=false ++ bm_dbus_init ++ debug 'bm_dbus_init()' ++ [[ '' == \d\e\b\u\g ]] +++ which dbus-send ++ dbus_send= ++ true ++ '[' -x '' ']' ++ dbus_send_cmd= + source confs/base.conf ++ export BM_REPOSITORY_ROOT=./repository ++ BM_REPOSITORY_ROOT=./repository ++ export BM_REPOSITORY_USER=buildd ++ BM_REPOSITORY_USER=buildd ++ export BM_REPOSITORY_GROUP=buildd ++ BM_REPOSITORY_GROUP=buildd ++ export BM_REPOSITORY_SECURE=false ++ BM_REPOSITORY_SECURE=false ++ export BM_REPOSITORY_RECURSIVEPURGE=false ++ BM_REPOSITORY_RECURSIVEPURGE=false ++ export BM_ARCHIVE_TTL=5 ++ BM_ARCHIVE_TTL=5 ++ export BM_ARCHIVE_PURGEDUPS=true ++ BM_ARCHIVE_PURGEDUPS=true ++ export BM_ARCHIVE_PREFIX=uranio1 ++ BM_ARCHIVE_PREFIX=uranio1 ++ export BM_ARCHIVE_METHOD=tarball ++ BM_ARCHIVE_METHOD=tarball ++ export BM_ARCHIVE_STRICTPURGE=true ++ BM_ARCHIVE_STRICTPURGE=true ++ export BM_LOGGER=true ++ BM_LOGGER=true ++ export BM_LOGGER_LEVEL=warning ++ BM_LOGGER_LEVEL=warning ++ export BM_LOGGER_FACILITY=user ++ BM_LOGGER_FACILITY=user ++ export BM_PRE_BACKUP_COMMAND= ++ BM_PRE_BACKUP_COMMAND= ++ export BM_POST_BACKUP_COMMAND= ++ BM_POST_BACKUP_COMMAND= ++ export BM_TEMP_DIR=/tmp/backup-manager ++ BM_TEMP_DIR=/tmp/backup-manager ++ export BM_ARCHIVE_NICE_LEVEL=10 ++ BM_ARCHIVE_NICE_LEVEL=10 ++ export
Bug#831759: fixed in backup-manager 0.7.12-2
¡Hola Santiago! El 2016-08-16 a las 15:40 +0200, Santiago Vila escribió: found 831759 0.7.12-3 thanks Does not seem fixed: https://tests.reproducible-builds.org/debian/rbuild/testing/amd64/backup-manager_0.7.12-3.rbuild.log I doubt that it's the same issue. Anyway, I can't reproduce the issue here. Can you test the failling test with set -x ? Happy hacking, -- “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are–by definition–not smart enough to debug it.” -- Brian Kernighan Saludos /\/\ /\ >< `/ signature.asc Description: Digital signature
Processed: Re: Bug#831759: fixed in backup-manager 0.7.12-2
Processing commands for cont...@bugs.debian.org:
> found 831759 0.7.12-3
Bug #831759 {Done: Maximiliano Curia } [src:backup-manager]
backup-manager: FTBFS: Running test t15-dupes.sh: failed
Marked as found in versions backup-manager/0.7.12-3 and reopened.
> thanks
Stopping processing here.
Please contact me if you need assistance.
--
831759: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=831759
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
Bug#831759: fixed in backup-manager 0.7.12-2
found 831759 0.7.12-3 thanks Does not seem fixed: https://tests.reproducible-builds.org/debian/rbuild/testing/amd64/backup-manager_0.7.12-3.rbuild.log Thanks.
