Bug#886736: diffoscope: mach-o disassembly with otool can fail in a way that fools diffoscope into dumping raw data instead

2018-01-09 Thread Mike Hommey
On Wed, Jan 10, 2018 at 10:36:02AM +0530, Chris Lamb wrote: > Mike Hommey wrote: > > > The two builds I was comparing: > > […] > > Thanks for sending this over. For some reason, I completely failed to > realise that I would need access to otool to make use of these,

Bug#886736: diffoscope: mach-o disassembly with otool can fail in a way that fools diffoscope into dumping raw data instead

2018-01-09 Thread Mike Hommey
On Wed, Jan 10, 2018 at 10:42:33AM +0900, Mike Hommey wrote: > On Wed, Jan 10, 2018 at 10:23:59AM +0900, Mike Hommey wrote: > > On Tue, Jan 09, 2018 at 01:07:58PM +, Chris Lamb wrote: > > > Hi Mike, > > > > > > > I only have a very large XUL

Bug#886736: diffoscope: mach-o disassembly with otool can fail in a way that fools diffoscope into dumping raw data instead

2018-01-09 Thread Mike Hommey
On Wed, Jan 10, 2018 at 10:23:59AM +0900, Mike Hommey wrote: > On Tue, Jan 09, 2018 at 01:07:58PM +, Chris Lamb wrote: > > Hi Mike, > > > > > I only have a very large XUL library... you probably don't want that. > > > > Probably not for the testsuite

Bug#886736: diffoscope: mach-o disassembly with otool can fail in a way that fools diffoscope into dumping raw data instead

2018-01-09 Thread Mike Hommey
On Tue, Jan 09, 2018 at 01:07:58PM +, Chris Lamb wrote: > Hi Mike, > > > I only have a very large XUL library... you probably don't want that. > > Probably not for the testsuite (!) but if you could make it available it > would help with a fix anyway... The two builds I was comparing: https:

Bug#886736: diffoscope: mach-o disassembly with otool can fail in a way that fools diffoscope into dumping raw data instead

2018-01-09 Thread Mike Hommey
On Tue, Jan 09, 2018 at 12:50:05PM +, Chris Lamb wrote: > Hi Mike, > > > > In some cases, otool can fail with: > > > > can't create x86_64 llvm disassembler > > Do you happen to have example files you could point to or upload? Not > essential of course, but would be much easier and reliab

Bug#886736: diffoscope: mach-o disassembly with otool can fail in a way that fools diffoscope into dumping raw data instead

2018-01-09 Thread Mike Hommey
Package: diffoscope Version: 90 Severity: normal Dear Maintainer, In some cases, otool can fail with: can't create x86_64 llvm disassembler (where x86_64 may be another platform, and where the message is usually preceded by the file path name and "(__TEXT,__text) section", well, in fact, anyt

Bug#879003: diffoscope: objdump --line-numbers makes diffoscope *very* slow

2017-10-20 Thread Mike Hommey
Here's a proposed workaround. Mike >From 7332267397e9055aa67b84a9fc6496548c420ea6 Mon Sep 17 00:00:00 2001 From: Mike Hommey Date: Fri, 20 Oct 2017 21:48:24 +0900 Subject: [PATCH] Add fallbacks to elf code section disassembly. Because objdump --line-numbers can be extremely slow, i

Bug#879011: diffoscope: zipinfo diff shows warning differences that are due to temporary file names

2017-10-20 Thread Mike Hommey
On Thu, Oct 19, 2017 at 07:15:35PM +0900, Mike Hommey wrote: > Attached is my work in progress. I won't finish it right now because > it's getting late and I don't have all the optional tools to run the > test suite. > > It's worth noting that this changes the o

Bug#879011: diffoscope: zipinfo diff shows warning differences that are due to temporary file names

2017-10-19 Thread Mike Hommey
On Thu, Oct 19, 2017 at 06:27:07PM +0900, Mike Hommey wrote: > On Thu, Oct 19, 2017 at 04:41:04PM +0900, Mike Hommey wrote: > > On Thu, Oct 19, 2017 at 03:57:29PM +0900, Mike Hommey wrote: > > > On Wed, Oct 18, 2017 at 09:27:29PM +0900, Mike Hommey wrote: > >

Bug#879003: diffoscope: objdump --line-numbers makes diffoscope *very* slow

2017-10-19 Thread Mike Hommey
On Thu, Oct 19, 2017 at 08:50:19AM +0900, Mike Hommey wrote: > On Wed, Oct 18, 2017 at 07:29:52PM +0900, Mike Hommey wrote: > > Package: diffoscope > > Version: 87 > > Severity: normal > > > > Dear Maintainer, > > > > Today I was comparing firefox bu

Bug#879011: diffoscope: zipinfo diff shows warning differences that are due to temporary file names

2017-10-19 Thread Mike Hommey
On Thu, Oct 19, 2017 at 04:41:04PM +0900, Mike Hommey wrote: > On Thu, Oct 19, 2017 at 03:57:29PM +0900, Mike Hommey wrote: > > On Wed, Oct 18, 2017 at 09:27:29PM +0900, Mike Hommey wrote: > > > Package: diffoscope > > > Version: 87 > > > Severity: norma

Bug#879011: diffoscope: zipinfo diff shows warning differences that are due to temporary file names

2017-10-19 Thread Mike Hommey
On Thu, Oct 19, 2017 at 03:57:29PM +0900, Mike Hommey wrote: > On Wed, Oct 18, 2017 at 09:27:29PM +0900, Mike Hommey wrote: > > Package: diffoscope > > Version: 87 > > Severity: normal > > > > While diffing firefox, zipinfo is run on omni.ja, and issues war

Bug#879011: diffoscope: zipinfo diff shows warning differences that are due to temporary file names

2017-10-19 Thread Mike Hommey
On Wed, Oct 18, 2017 at 09:27:29PM +0900, Mike Hommey wrote: > Package: diffoscope > Version: 87 > Severity: normal > > While diffing firefox, zipinfo is run on omni.ja, and issues warnings > like: > warning [/tmp/tmplgigxgm__diffoscope/0/24]: 17283883 extra bytes at

Bug#879003: diffoscope: objdump --line-numbers makes diffoscope *very* slow

2017-10-18 Thread Mike Hommey
On Wed, Oct 18, 2017 at 07:29:52PM +0900, Mike Hommey wrote: > Package: diffoscope > Version: 87 > Severity: normal > > Dear Maintainer, > > Today I was comparing firefox builds from mozilla CI with diffoscope, > and it was taking an awful lot of time. Quick inspection

Bug#879011: diffoscope: zipinfo diff shows warning differences that are due to temporary file names

2017-10-18 Thread Mike Hommey
Package: diffoscope Version: 87 Severity: normal While diffing firefox, zipinfo is run on omni.ja, and issues warnings like: warning [/tmp/tmplgigxgm__diffoscope/0/24]: 17283883 extra bytes at beginning or within zipfile Now, when both ends have the same warning, as expected, the diff still sho

Bug#879010: diffoscope: Add an option to exclude file lists differences

2017-10-18 Thread Mike Hommey
Package: diffoscope Version: 87 Severity: wishlist Dear Maintainer, It's usually more convenient to give tarballs than to extract them and pass directories to diffoscope. But then the difference between the two containers are displayed, while one might not care about those differences. It might a

Bug#879003: diffoscope: objdump --line-numbers makes diffoscope *very* slow

2017-10-18 Thread Mike Hommey
Package: diffoscope Version: 87 Severity: normal Dear Maintainer, Today I was comparing firefox builds from mozilla CI with diffoscope, and it was taking an awful lot of time. Quick inspection revealed that this is largely due to objdump --line-numbers. Remove --line-numbers from the objdump comm

[Reproducible-builds] Bug#808207: diffoscope: Filter objdump --disassemble output before diffing it

2016-01-18 Thread Mike Hommey
On Mon, Jan 18, 2016 at 07:06:14PM +0100, Jérémy Bobbio wrote: > Hi Mike! > > Mike Hommey: > > When comparing large ELF binaries, some minor differences can end up hurting > > the visibility of more important differences. > > > > Specifically, objdump --disasse

[Reproducible-builds] Bug#808267: diffoscope: Redundant information in ELF comparisons

2016-01-18 Thread Mike Hommey
On Mon, Jan 18, 2016 at 07:20:38PM +0100, Jérémy Bobbio wrote: > Hi Mike, > > Mike Hommey: > > When comparing ELF files, the following commands are used: > > - readelf --all > > - readelf --debug-dump > > - objdump --disassemble --full-contents > > >

[Reproducible-builds] Bug#808267: diffoscope: Redundant information in ELF comparisons

2015-12-17 Thread Mike Hommey
On Fri, Dec 18, 2015 at 10:10:54AM +0900, Mike Hommey wrote: > Source: diffoscope > Version: 43 > Severity: normal > > When comparing ELF files, the following commands are used: > - readelf --all > - readelf --debug-dump > - objdump --disassemble --full-contents > >

[Reproducible-builds] Bug#808267: diffoscope: Redundant information in ELF comparisons

2015-12-17 Thread Mike Hommey
Source: diffoscope Version: 43 Severity: normal When comparing ELF files, the following commands are used: - readelf --all - readelf --debug-dump - objdump --disassemble --full-contents objdump --disassemble --full-contents is actually redundant in itself. For example, it will dump both an hexdum

[Reproducible-builds] Bug#808207: diffoscope: Filter objdump --disassemble output before diffing it

2015-12-17 Thread Mike Hommey
Source: diffoscope Version: 43 Severity: wishlist When comparing large ELF binaries, some minor differences can end up hurting the visibility of more important differences. Specifically, objdump --disassemble displays symbols+offsets for addresses it derives from IP-relative addressing, like the

[Reproducible-builds] Bug#808199: diffoscope: Weird line numbering in diff output

2015-12-16 Thread Mike Hommey
On Thu, Dec 17, 2015 at 03:03:21PM +0900, Mike Hommey wrote: > However, there's another issue, that the diff is actually incomplete: > the right hand side stops 3 lines too early. > > (The patch is not actually changing anything here, I just figured > once the numbers looke

[Reproducible-builds] Bug#808199: diffoscope: Weird line numbering in diff output

2015-12-16 Thread Mike Hommey
Attached here is two files that can be compared directly, exposing the same problem. That's essentially the output of tar --full-time -tvf on both tarballs from the original bug report. The following patch fixes the numbering issue: diff --git a/diffoscope/presenters/html.py b/diffoscope/presente

[Reproducible-builds] Bug#808199: diffoscope: Weird line numbering in diff output

2015-12-16 Thread Mike Hommey
Source: diffoscope Version: 43 Severity: normal Dear Maintainer, * What led up to the situation? Comparing https://queue.taskcluster.net/v1/task/FEXzanIjQKW_dBmv3ZmBJQ/runs/0/artifacts/public/build/target.tar.bz2 with https://queue.taskcluster.net/v1/task/HqZ2DuHYRJ6kR2_P8Rc7Ig/runs/0/artifac

[Reproducible-builds] Bug#808003: Bug#808003: diffoscope: Comparing directories shouldn't care about file order

2015-12-16 Thread Mike Hommey
On Tue, Dec 15, 2015 at 11:07:24AM +0100, Holger Levsen wrote: > Hi Mike, > > thanks for the bug report! > > On Dienstag, 15. Dezember 2015, Mike Hommey wrote: > >* What was the outcome of this action? > > The output contained differences in the output for

[Reproducible-builds] Bug#808197: diffoscope: readelf --debug-dump can be brutal

2015-12-16 Thread Mike Hommey
Source: diffoscope Version: 43 Severity: wishlist Dear Maintainer, When comparing the debug symbol files for Firefox, which are rather massive, the output for readelf --debug-dump is 1.7GB compressed. Diff can't handle with that much data without OOM'ing. However, if each debug section was compa

[Reproducible-builds] Bug#808121: Bug#808121: diffoscope: HTML output is bloated

2015-12-16 Thread Mike Hommey
On Wed, Dec 16, 2015 at 11:19:02AM +0100, Jérémy Bobbio wrote: > Close: tag -1 + pending > > Mike Hommey: > > Looking at the HTML in the HTML output, one can see that it is needlessly > > large. > > > > Specifically, there appears to be a lot of e.g. > &g

[Reproducible-builds] Bug#808121: Bug#808121: diffoscope: HTML output is bloated

2015-12-16 Thread Mike Hommey
On Wed, Dec 16, 2015 at 11:19:02AM +0100, Jérémy Bobbio wrote: > Close: tag -1 + pending > > Mike Hommey: > > Looking at the HTML in the HTML output, one can see that it is needlessly > > large. > > > > Specifically, there appears to be a lot of e.g. > &g

[Reproducible-builds] Bug#808121: diffoscope: HTML output is bloated

2015-12-16 Thread Mike Hommey
Source: diffoscope Version: 43 Severity: wishlist Dear Maintainer, Looking at the HTML in the HTML output, one can see that it is needlessly large. Specifically, there appears to be a lot of e.g. following each other, without even a separation between them. This conflates the amount of memory n

[Reproducible-builds] Bug#808120: diffoscope: Should use less memory

2015-12-16 Thread Mike Hommey
Source: diffoscope Version: 43 Severity: wishlist Dear Maintainer, * What led up to the situation? Comparing http://archive.mozilla.org/pub/firefox/try-builds/m...@glandium.org-393e224a2bfe01eb5537d3d5ffa486d92006c4f0/try-linux-debug/firefox-45.0a1.en-US.linux-i686.tar.bz2 with https://queue.

[Reproducible-builds] Bug#808002: Bug#808002: diffoscope: Add support for Mozilla-optimized ZIPs

2015-12-16 Thread Mike Hommey
On Tue, Dec 15, 2015 at 04:31:46PM +0100, Jérémy Bobbio wrote: > Mike Hommey: > > It would be useful for diffoscope to output differences in omni.ja files as > > for other Zip files, instead of ending up with a diff of an hexdump. > > > > The attached patch implements

[Reproducible-builds] Bug#808103: diffoscope: Truncated symbols in ELF diffs

2015-12-15 Thread Mike Hommey
Source: diffoscope Version: 43 Severity: important Tags: patch Dear Maintainer, When there are differences in symbols-related sections of ELF files, symbols can be truncated if they are long enough. That happens often with C++ mangled symbols. The following patch fixes the issue: diff --git a/d

[Reproducible-builds] Bug#807997: Bug#807997: diffoscope: CbfsFile checks make diffoscope extra slow

2015-12-15 Thread Mike Hommey
On Tue, Dec 15, 2015 at 11:13:54AM +0100, Holger Levsen wrote: > Hi Mike, > > out of curiosity: did you have cbfstool installed? I don't. Mike ___ Reproducible-builds mailing list Reproducible-builds@lists.alioth.debian.org http://lists.alioth.debian.

[Reproducible-builds] Bug#807997: diffoscope: CbfsFile checks make diffoscope extra slow

2015-12-14 Thread Mike Hommey
On Tue, Dec 15, 2015 at 02:43:26PM +0900, Mike Hommey wrote: > Source: diffoscope > Version: ada1a1dcdc19217fb611e0a1e57bc3744399aefa > Severity: important > > Dear Maintainer, > >* What led up to the situation? > > Comparing > http://archive.mozill

[Reproducible-builds] Bug#808003: diffoscope: Comparing directories shouldn't care about file order

2015-12-14 Thread Mike Hommey
Source: diffoscope Version: ada1a1dcdc19217fb611e0a1e57bc3744399aefa Severity: wishlist Dear Maintainer, * What led up to the situation? Comparing two directories * What exactly did you do (or not do) that was effective (or ineffective)? Ran `diffoscope --html output.html a b` where

[Reproducible-builds] Bug#808002: diffoscope: Add support for Mozilla-optimized ZIPs

2015-12-14 Thread Mike Hommey
Source: diffoscope Version: ada1a1dcdc19217fb611e0a1e57bc3744399aefa Severity: wishlist Tags: patch It would be useful for diffoscope to output differences in omni.ja files as for other Zip files, instead of ending up with a diff of an hexdump. The attached patch implements a minimal support for

[Reproducible-builds] Bug#807997: diffoscope: CbfsFile checks make diffoscope extra slow

2015-12-14 Thread Mike Hommey
Source: diffoscope Version: ada1a1dcdc19217fb611e0a1e57bc3744399aefa Severity: important Dear Maintainer, * What led up to the situation? Comparing http://archive.mozilla.org/pub/firefox/try-builds/m...@glandium.org-93eecb52fcadc562b0f52cb22a8971770446a12e/try-linux64/firefox-45.0a1.en-US.lin