Bug#881937: diffoscope: Doesn't show differences in .deb's control.tar.xz (regression?)
Package: diffoscope Version: 88 Severity: normal Control: found -1 78 Hi, when comparing the two attached .deb files with diffoscope, it doesn't show any differences inside the control.data.xz, despite it shows a different size of the two control.data.xz members: --- ../aptitude-robot_1.5.2-1_all.deb +++ ../aptitude-robot_1.5.3-1_all.deb ├── file list │ @@ -1,3 +1,3 @@ │ --rw-r--r-- 0004 2017-07-23 20:04:23.00 debian-binary │ --rw-r--r-- 000 1514 2017-07-23 20:04:23.00 control.tar.gz │ --rw-r--r-- 00025264 2017-07-23 20:04:23.00 data.tar.xz │ +-rw-r--r-- 0004 2017-11-16 17:48:45.00 debian-binary │ +-rw-r--r-- 000 1584 2017-11-16 17:48:45.00 control.tar.xz │ +-rw-r--r-- 00025656 2017-11-16 17:48:45.00 data.tar.xz ├── data.tar.xz │ ├── data.tar │ │ ├── file list │ │ │ @@ -1,50 +1,50 @@ │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./ │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./etc/ │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./etc/aptitude-robot/ │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./etc/aptitude-robot/options.d/ │ │ │ --rw-r--r-- 0 root (0) root (0) 308 2017-07-23 20:04:23.00 ./etc/aptitude-robot/options.d/10-remove-level-maximum │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./etc/aptitude-robot/pkglist.d/ │ │ │ --rw-r--r-- 0 root (0) root (0) 178 2017-07-23 20:04:23.00 ./etc/aptitude-robot/pkglist.d/README.txt │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./etc/aptitude-robot/triggers.post/ │ │ │ --rwxr-xr-x 0 root (0) root (0) 264 2017-07-23 20:04:23.00 ./etc/aptitude-robot/triggers.post/90-cleanup.example │ │ │ --rw-r--r-- 0 root (0) root (0) 205 2017-07-23 20:04:23.00 ./etc/aptitude-robot/triggers.post/README.txt │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./etc/aptitude-robot/triggers.pre/ │ │ │ --rw-r--r-- 0 root (0) root (0) 206 2017-07-23 20:04:23.00 ./etc/aptitude-robot/triggers.pre/README.txt │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./etc/cron.daily/ │ │ │ --rwxr-xr-x 0 root (0) root (0) 500 2015-06-30 16:20:06.00 ./etc/cron.daily/aptitude-robot │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./etc/default/ │ │ │ --rw-r--r-- 0 root (0) root (0) 1628 2015-06-30 16:20:06.00 ./etc/default/aptitude-robot │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./etc/init/ │ │ │ --rw-r--r-- 0 root (0) root (0) 168 2015-06-30 16:24:37.00 ./etc/init/aptitude-robot.conf │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./etc/init.d/ │ │ │ --rwxr-xr-x 0 root (0) root (0) 1867 2014-11-02 09:47:18.00 ./etc/init.d/aptitude-robot │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./etc/logrotate.d/ │ │ │ --rw-r--r-- 0 root (0) root (0) 85 2015-06-30 16:20:06.00 ./etc/logrotate.d/aptitude-robot │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./lib/ │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./lib/systemd/ │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./lib/systemd/system/ │ │ │ --rw-r--r-- 0 root (0) root (0) 202 2016-05-12 10:36:19.00 ./lib/systemd/system/aptitude-robot.service │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./usr/ │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./usr/sbin/ │ │ │ --rwxr-xr-x 0 root (0) root (0) 7132 2017-07-23 20:04:23.00 ./usr/sbin/aptitude-robot │ │ │ --rwxr-xr-x 0 root (0) root (0) 2465 2017-07-23 20:04:23.00 ./usr/sbin/aptitude-robot-session │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./usr/share/ │ │ │ -drwxr-xr-x 0 root (0) root (0)0 2017-07-23 20:04:23.00 ./usr/share/aptitude-robot/ │ │ │ --rwxr-xr-x 0 root (0) root (0) 1454 2017-07-23 20:04:23.00 ./usr/share/aptitude-robot/mail-log-on-error │ │ │ --rwxr-xr-x 0 root (0) root (0) 471
Processed: diffoscope: Doesn't show differences in .deb's control.tar.xz (regression?)
Processing control commands: > found -1 78 Bug #881937 [diffoscope] diffoscope: Doesn't show differences in .deb's control.tar.xz (regression?) Marked as found in versions diffoscope/78. -- 881937: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=881937 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#876901: QFINDTESTDATA uses __FILE__
On jueves, 16 de noviembre de 2017 13:22:00 -03 Ximin Luo wrote: [snip] > I pointed to the various C standards documents, as well as documentation > from multiple compilers, stating that __FILE__ is the "name of the source > file" and in no way guarantees that the expansion can later be re-used as > the path to an actual file. GCC documentation even explicitly states the > expansion is arbitrarily chosen by the implementation of the preprocessor, > and is explicitly "not [..] the input file name argument". OK, let's agree we do not agree here. > So I do consider this a bug in the QT test suite. > > The ideal solution would be to not use __FILE__ - that has numerous other > benefits as well. But if this is too complex to change, I also suggested a > 1-line addition to d/rules - which I agree would be "papering over" the > issue. However, it's a simple 1-line change so I don't understand why there > is so much resistance to it. Because it means diverging from upstream and that causes us *lots* of headaches when trying to solve unit tests issues with upstream's help. > There are several other possible solutions, all of which are low-cost and > unintrusive, and could be done in a QT build helper in one single place: > > - define a custom macro, QT_TEST_SOURCE_BASE, and set that in the test build > scripts, instead of using __FILE__ - export > BUILD_PATH_PREFIX_MAP="$BUILD_PATH_PREFIX_MAP:tests=$BASEDIR/tests" - > symlink "$srcpkg-$version" -> "." > > I would be very happy to send you the patches myself if you don't want to do > the work, since writing 1-line patches to a few QT projects, costs far less > time than patching 1800 packages across Debian. Let me propose you something: create a suitable patch for upstream that makes stuff work no matter the distro not OS. As I do think your approach is not correct you should push it yourself, see http://wiki.qt.io/Qt_Contribution_Guidelines for knowing how to contribute it. -- May the source be with you. Lisandro Damián Nicanor Pérez Meyer http://perezmeyer.com.ar/ http://perezmeyer.blogspot.com/ signature.asc Description: This is a digitally signed message part. ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#876901: QFINDTESTDATA uses __FILE__
Lisandro Damián Nicanor Pérez Meyer: > By the way: is there a macro or combination of macros which *default* value > can be replaced in the use of __FILE__ without caussing regressions? > > Because if that's the case it's easier to convince upstream people that > changing the usage goes in favor of reproducibility without using strange up- > to-the-builder definitions. > Unfortunately, not that I know of. However, let me try to explain why BUILD_PATH_PREFIX_MAP is not as strange as you might think. GCC already supports a flag -fdebug-prefix-map whose purpose is to do the same thing for debugging symbols. You can give multiple maps, to map different parts of your source tree to different other source directories. So maybe one could give a map like this: 1. "PWD/" -> "/usr/src/debug/$mypackage" 2. "PWD/tests" -> "./tests" if one wanted to generate debugging information for your main source code, to point to installed-source-code at /usr/src/debug/$mypackage; but since one doesn't install the tests anywhere, it would be convenient for developers to have the debuginfo of tests point to ./tests instead, so they can just run "gdb -d." if stuff fails. What I'm suggesting for QT tests would be exactly analogous to this already-supported use case for debuginfo. dpkg gives a default map corresponding to (1), and since QT tests use __FILE__ for other purposes, you give a second map corresponding to (2) above. X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#876901: QFINDTESTDATA uses __FILE__
Lisandro Damián Nicanor Pérez Meyer: > [..] > > Yes, we do understand that your workaround solves the issue, but we do also > understand that we should not be using this workaround in the first place. > > Guillem: the thread is long, but be sure that we Qt/KDE maintainers consider > that this change will be insta-RC I'm afraid. > Guillem: the TL;DR is that some QT tests fail (with our reproducibility GCC and dpkg patch) because they depend on __FILE__ to point to a real path, but the patched dpkg rewrites it in all packages so that a prefix $PWD changes to $srcpkg-$version, which doesn't exist. A simple fix on the dpkg side would be to instead rewrite $PWD to ".", much like -fdebug-prefix-map is done currently. However this loses some information and makes it hard to distinguish different packages that might share filetree structure. But that is the best fallback option IMO, if we decide that we don't want to auto-break QT tests for using __FILE__ (IMO, inappropriately). > Xi: you have found a *wonderful* way to find where bugs are, please try to > fix > the relevant code and not paper over it, because in the Qt case it is not a > bug on our side. > I pointed to the various C standards documents, as well as documentation from multiple compilers, stating that __FILE__ is the "name of the source file" and in no way guarantees that the expansion can later be re-used as the path to an actual file. GCC documentation even explicitly states the expansion is arbitrarily chosen by the implementation of the preprocessor, and is explicitly "not [..] the input file name argument". So I do consider this a bug in the QT test suite. The ideal solution would be to not use __FILE__ - that has numerous other benefits as well. But if this is too complex to change, I also suggested a 1-line addition to d/rules - which I agree would be "papering over" the issue. However, it's a simple 1-line change so I don't understand why there is so much resistance to it. There are several other possible solutions, all of which are low-cost and unintrusive, and could be done in a QT build helper in one single place: - define a custom macro, QT_TEST_SOURCE_BASE, and set that in the test build scripts, instead of using __FILE__ - export BUILD_PATH_PREFIX_MAP="$BUILD_PATH_PREFIX_MAP:tests=$BASEDIR/tests" - symlink "$srcpkg-$version" -> "." I would be very happy to send you the patches myself if you don't want to do the work, since writing 1-line patches to a few QT projects, costs far less time than patching 1800 packages across Debian. X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#876901: QFINDTESTDATA uses __FILE__
By the way: is there a macro or combination of macros which *default* value can be replaced in the use of __FILE__ without caussing regressions? Because if that's the case it's easier to convince upstream people that changing the usage goes in favor of reproducibility without using strange up- to-the-builder definitions. -- "In college, I cooked some hot dogs by putting metal forks in each end of the hot dog and running 120 volts through it. Hot dogs have just enough conductivity so that this works well" greenroom(576281) - on a truly geeky way to cook hot dogs. Posted in Slashdot, also found in The Open Source Cookbook for Geeks. Lisandro Damián Nicanor Pérez Meyer http://perezmeyer.com.ar/ http://perezmeyer.blogspot.com/ signature.asc Description: This is a digitally signed message part. ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Bug#876901: QFINDTESTDATA uses __FILE__
Explicitely CCing Guillem for this one. On miércoles, 15 de noviembre de 2017 23:01:00 -03 Ximin Luo wrote: [sip] > The GCC patch (neither the previous nor the planned version) does not change > the default behaviour of __FILE__, and was never intended to. Instead, it > gives users the ability to rewrite __FILE__, more specifically a prefix of > it. So it basically does not changes the default __FILE__ behavior, so neither GCC nor other projects upstream developers will have anything to complain... > > There is a separate patch to dpkg that enables this ability for all > packages, in the same way that SOURCE_DATE_EPOCH is enabled. Guillem the > dpkg maintainer has previously indicated that he's happy to take the patch, > once GCC accepts their end of it. ...except for a Debian self-inflicted change that will *only* happen while building Debian packages, but not when using it for normal developing processes. > It's a combination of these two patches that would cause these QT tests to > fail. The reason it fails, is because we specifically map $PWD to a > non-existent path. I suggested various ways around this. One of the > suggestions, was to add an extra mapping to map $PWD/tests back to > $PWD/tests (or just ./tests), overriding the earlier non-existent mapping. > I think this is the cleanest suggestion - assuming that tests reside in the > same directory, and away from the main source code. Kai Pastor over on bug > #876934 indicated that this would probably work for > openorienteering-mapper. Yes, we do understand that your workaround solves the issue, but we do also understand that we should not be using this workaround in the first place. Guillem: the thread is long, but be sure that we Qt/KDE maintainers consider that this change will be insta-RC I'm afraid. Xi: you have found a *wonderful* way to find where bugs are, please try to fix the relevant code and not paper over it, because in the Qt case it is not a bug on our side. -- “I don’t think security can solve problems. We need to teach greater respect.” Oslo Mayor Stang when asked whether Oslo needs greater security after the attacks in Norway, 07/2011. Lisandro Damián Nicanor Pérez Meyer http://perezmeyer.com.ar/ http://perezmeyer.blogspot.com/ signature.asc Description: This is a digitally signed message part. ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Please review the draft for week 133's blog post
On Thu, Nov 16, 2017 at 10:06:00AM +, Ximin Luo wrote: > Ping, are you ready? I don't see any commits from you in the log... not yet & on a very bad internet connection atm. i will publish 133 once it's ready. thanks. -- cheers, Holger signature.asc Description: PGP signature ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
Re: Please review the draft for week 133's blog post
Holger Levsen: > On Tue, Nov 14, 2017 at 01:34:00PM +, Ximin Luo wrote: >> This week's blog post draft is now available for review: >> https://reproducible.alioth.debian.org/blog/drafts/133/ > [...] >> I'll wait at least 24 hours from the time of this email for any comments, >> and if everything is good then I will publish it soon after that. > > please hold off publishing this blog for another 24h. > Ping, are you ready? I don't see any commits from you in the log... X -- GPG: ed25519/56034877E1F87C35 GPG: rsa4096/1318EFAC5FBBDBCE https://github.com/infinity0/pubkeys.git ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
