Processed: your mail

2016-12-26 Thread Debian Bug Tracking System 
Processing commands for cont...@bugs.debian.org:

> tags 835641 +patch
Bug #835641 [diffoscope] diffoscope: traceback when comparing dangling symlink 
to directory
Added tag(s) patch.
> thanks
Stopping processing here.

Please contact me if you need assistance.
-- 
835641: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=835641
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#849425: diffoscope: test_openssh_pub_key.test_diff fails on jessie after ssh-keygen output format change

2016-12-26 Thread Brett Smith 
Source: diffoscope
Version: git as of 011987f
Severity: minor
Tags: upstream

On jessie, test_openssh_pub_key.test_diff fails like this:

=== FAILURES ===
__ test_diff ___

differences = []

@skip_unless_tools_exist('ssh-keygen')
def test_diff(differences):
expected_diff = open(data('openssh_pub_key_expected_diff')).read()
>   assert differences[0].unified_diff == expected_diff
E   assert '@@ -1 +1 @@\...2.pub (RSA)\n' == '@@ -1 +1 @@\n...Test2 (RSA)\n'
E   @@ -1 +1 @@
E - -1024 0a:57:8d:93:be:8b:5c:47:7a:b6:5c:91:16:87:cd:1e 
/home/brett/repos/diffoscope/tests/data/test_openssh_pub_key1.pub (DSA)
E - +4096 8a:a5:52:0a:3f:af:8d:2d:76:52:72:e1:a8:0a:a2:47 
/home/brett/repos/diffoscope/tests/data/test_openssh_pub_key2.pub (RSA)
E + -1024 SHA256:v/O+0ETvi2H5TGRXky1RhQ1/WFwLlPpxch5E2Mrj6FM Test1 (DSA)
E + +4096 SHA256:9dH1CMkA6DSfPWU7vNwdPKS5/ppN4LMdvHTP60l7aSA Test2 (RSA)

tests/comparators/test_openssh_pub_key.py:47: AssertionError
== 1 failed, 3 passed in 0.14 seconds ==

This happens because, since jessie, ssh-keygen has added the -E option to
specify the fingerprint hash algorithm, and defaulted it to SHA256.  Older
versions used the colon-separated format (md5?).

I was working on a patch for this, but unfortunately the right thing to do
isn't obvious.  Older versions of ssh-keygen, as in jessie, don't support
the -E option at all.  This makes it difficult to ensure diffoscope's
output is consistent regardless of the version of ssh-keygen on the
underlying host.

We could have the comparator try to specify -E md5, and then fall back to
omitting the -E option if that fails, but that seems a little regressive
since md5 is basically deprecated.

We could have the test sniff for the host's ssh-keygen version, and expect a
different diff based on when it started outputting sha256 fingerprints by
default, but that punts on the consistent output issue.

What do the maintainers think?

-- System Information:
Debian Release: 8.6
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.9.0 (SMP w/4 CPU cores)
Locale: LANG=C, LC_CTYPE=C (charmap=UTF-8) (ignored: LC_ALL set to en_US.utf8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#835641: diffoscope: traceback when comparing dangling symlink to directory

2016-12-26 Thread Brett Smith 
Here's a patch to prevent the traceback.  It just detects the general
situation (comparing a directory with something else) and bails with a
more helpful message when they're incompatible.

I checked what diff does, and it does something slightly more nuanced:
if you compare directory D against non-directory F, it tries to compare
`/D/$(basename F)` against F.  If that directory+basename path is itself
a directory, then it bails with an error message like the one in this
patch.  Personally I felt like the path translation wasn't especially
helpful, so decided to just bail immediately.

-- 
Brett Smith

>From 7ec9f8df87ec18263a2f327951cffc08a9631fc8 Mon Sep 17 00:00:00 2001
From: Brett Smith 
Date: Mon, 26 Dec 2016 19:26:05 -0500
Subject: [PATCH] comparators: Avoid comparing a directory with non-directory.

There's no sensible way to do this comparison, so just bail with an error
message.  Previously we assumed both paths were files and ran comparator
tools on them, which led to more obscure errors.
---
 diffoscope/comparators/__init__.py | 19 ++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/diffoscope/comparators/__init__.py b/diffoscope/comparators/__init__.py
index e80b6d5..c97fc3e 100644
--- a/diffoscope/comparators/__init__.py
+++ b/diffoscope/comparators/__init__.py
@@ -121,11 +121,28 @@ def bail_if_non_existing(*paths):
 sys.stderr.write('%s: %s: No such file or directory\n' % (sys.argv[0], path))
 sys.exit(2)
 
+def bail_if_incompatible_file_types(path1, path1_isdir, path2, path2_isdir):
+if path1_isdir == path2_isdir:
+return
+fmt_args = {
+'prog': sys.argv[0],
+'path1': path1,
+'verb1': "is" if path1_isdir else "is not",
+'path2': path2,
+'verb2': "is" if path2_isdir else "is not",
+}
+errmsg = "{prog}: {path1} {verb1} a directory while {path2} {verb2}".format(**fmt_args)
+print(errmsg, file=sys.stderr)
+sys.exit(2)
+
 def compare_root_paths(path1, path2):
+path1_isdir = os.path.isdir(path1)
+path2_isdir = os.path.isdir(path2)
 if not Config().new_file:
 bail_if_non_existing(path1, path2)
-if os.path.isdir(path1) and os.path.isdir(path2):
+if path1_isdir and path2_isdir:
 return compare_directories(path1, path2)
+bail_if_incompatible_file_types(path1, path1_isdir, path2, path2_isdir)
 container1 = FilesystemDirectory(os.path.dirname(path1)).as_container
 file1 = specialize(FilesystemFile(path1, container=container1))
 container2 = FilesystemDirectory(os.path.dirname(path2)).as_container
-- 
2.1.4

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#848403: diffoscope: FTBFS randomly (Fatal Python error: deallocating None)

2016-12-26 Thread Маша Глухова 
I should add, I also had failures with diffoscope version 64 and older, but
no failures on newer versions. It seems likely that
one of the changes fixed this bug as a side effect, but I cannot yet tell
which one.
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#848403: diffoscope: FTBFS randomly (Fatal Python error: deallocating None)

2016-12-26 Thread Santiago Vila 
Hi.

I have built version 66 one hundred times in unstable.
The builds were made on 19 different autobuilders.
The number of failed builds has been zero.
(Previously it failed 10% of the time).

If you do not remember what kind of change may have fixed this,
then there must be some broken build-depends in testing which
is fixed in unstable.

I guess the right thing to do would be to find which one, reassign
this bug, and then close it, but we should ensure that the
fixed build-dependency propagates to testing.

Thanks.

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#848403: diffoscope: FTBFS randomly (Fatal Python error: deallocating None)

2016-12-26 Thread Santiago Vila 
On Mon, Dec 26, 2016 at 07:37:55PM +, Chris Lamb wrote:
> Santiago Vila wrote:
> 
> > If I do "python3 -m pytest" afterwards this is what it's shown:
> […]
> > Note: This is still diffoscope_63 in stretch, not sure if I should
> > better try the version in unstable and forget completely about this
> > version.
> 
> Please do so; the icc tests were fixed later and there have been quite
> a few other changes that makes focusing on stretch's version a poor
> return on time :)

Ok. I'm now building in unstable, many times. No failures so far.

Is it likely/possible that the changes between testing and unstable
have fixed this bug as a side effect?

Thanks.

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#849403: diffoscope: Improvesupport for Android apps

2016-12-26 Thread Reiner Herrmann 
On Mon, Dec 26, 2016 at 07:19:17PM +0200, Emanuel Bronshtein wrote:
> 2. Support parsing android resources not in APK file
>
> Support for some Android files (such as: AndroidManifest.xml & 
> resources.arsc) was added in:
> https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=3e748664a91b7ced412547a204e1473161425d4f
> but it works only if comparing APK files, not directories / when the content 
> archived in other format (non ZIP)

That's unfortunately a limitation of apktool that it only operates on
the whole apk archive, and is not able to dump already extracted files
(like the binary AndroidManifest).


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#848403: diffoscope: FTBFS randomly (Fatal Python error: deallocating None)

2016-12-26 Thread Chris Lamb 
Santiago Vila wrote:

> If I do "python3 -m pytest" afterwards this is what it's shown:
[…]
> Note: This is still diffoscope_63 in stretch, not sure if I should
> better try the version in unstable and forget completely about this
> version.

Please do so; the icc tests were fixed later and there have been quite
a few other changes that makes focusing on stretch's version a poor
return on time :)


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#848403: diffoscope: FTBFS randomly (Fatal Python error: deallocating None)

2016-12-26 Thread Santiago Vila 
On Sat, 24 Dec 2016, Ximin Luo wrote:

> For all you people that already have single-CPU KVM VMs set up, can you 
> please try to reduce your test cases that still reproduce the bug?
> 
> For example, can you still reproduce it with `debian/rules clean build`? What 
> about `python3 -m pytest`? Then, does it reproduce when you start manually 
> disabling various tests? What about when running only 1 test case?

Building the package by hand makes three tests to fail.

If I do "python3 -m pytest" afterwards this is what it's shown:

= test session starts ==
platform linux -- Python 3.5.2+, pytest-3.0.5, py-1.4.31, pluggy-0.4.0
rootdir: /build/diffoscope-63, inifile: 
plugins: cov-2.4.0
collected 248 items

tests/test_difference.py ...
tests/test_main.py .
tests/comparators/test_binary.py ...
tests/comparators/test_bzip2.py ..
tests/comparators/test_cbfs.py ss
tests/comparators/test_cpio.py ..
tests/comparators/test_deb.py ...
tests/comparators/test_debian.py ...
tests/comparators/test_dex.py 
tests/comparators/test_directory.py 
tests/comparators/test_elf.py ..
tests/comparators/test_epub.py 
tests/comparators/test_fonts.py 
tests/comparators/test_fsimage.py .sss
tests/comparators/test_gettext.py .
tests/comparators/test_git.py ...
tests/comparators/test_gzip.py ...
tests/comparators/test_haskell.py s.s
tests/comparators/test_icc.py F.FF
tests/comparators/test_image.py 
tests/comparators/test_ipk.py .
tests/comparators/test_iso9660.py ...
tests/comparators/test_java.py 
tests/comparators/test_json.py .
tests/comparators/test_macho.py ..ss
tests/comparators/test_mono.py 
tests/comparators/test_pdf.py .
tests/comparators/test_png.py 
tests/comparators/test_ppu.py 
tests/comparators/test_ps.py .
tests/comparators/test_rlib.py 
tests/comparators/test_rpm.py ..
tests/comparators/test_sqlite.py 
tests/comparators/test_squashfs.py ...
tests/comparators/test_tar.py ...
tests/comparators/test_text.py ..
tests/comparators/test_utils.py .ss.
tests/comparators/test_xz.py ..
tests/comparators/test_zip.py ..

=== FAILURES ===
_ test_identification __

icc1 = < 
/build/diffoscope-63/tests/data/test1.icc>

def test_identification(icc1):
>   assert isinstance(icc1, IccFile)
E   assert False
E+  where False = isinstance(< 
/build/diffoscope-63/tests/data/test1.icc>, IccFile)

tests/comparators/test_icc.py:32: AssertionError
__ test_diff ___

differences = []

@skip_unless_tools_exist('cd-iccdump')
def test_diff(differences):
expected_diff = open(data('icc_expected_diff')).read()
>   assert differences[0].unified_diff == expected_diff
E   IndexError: list index out of range

tests/comparators/test_icc.py:45: IndexError
__ test_compare_non_existing ___

monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fc9dec67128>
icc1 = < 
/build/diffoscope-63/tests/data/test1.icc>

@skip_unless_tools_exist('cd-iccdump')
def test_compare_non_existing(monkeypatch, icc1):
monkeypatch.setattr(Config(), 'new_file', True)
difference = icc1.compare(NonExistingFile('/nonexisting', icc1))
assert difference.source2 == '/nonexisting'
>   assert len(difference.details) > 0
E   assert 0 > 0
E+  where 0 = len([])
E+where [] = .details

tests/comparators/test_icc.py:52: AssertionError
== 3 failed, 230 passed, 15 skipped in 52.10 seconds ===


Note: This is still diffoscope_63 in stretch, not sure if I should
better try the version in unstable and forget completely about this
version.

Thanks.

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: Please review draft for week 87's blog post

2016-12-26 Thread Ximin Luo 
Ximin Luo:
> https://reproducible.alioth.debian.org/blog/drafts/87/
> 
> Late one this week, forgot about it because of the summit.
> 

^ um, ignore that, oversight in copy-paste. ^_^;;;

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Please review draft for week 87's blog post

2016-12-26 Thread Ximin Luo 
https://reproducible.alioth.debian.org/blog/drafts/87/

Late one this week, forgot about it because of the summit.

Feel free to commit fixes directly to drafts/87.mdwn in

https://anonscm.debian.org/git/reproducible/blog.git/

I will publish this in 48 hours, to give some time for the holiday season.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [diffoscope] 01/01: Use js-beautify as JavaScript code beautifier for .js files

2016-12-26 Thread Emanuel Bronshtein 
No, it's a partial fix, see my replay in #836786

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=836786#10

 

-Emanuel


Sent: Monday, December 26, 2016 at 8:20 PM
From: "Ximin Luo" 
To: "Emanuel Bronshtein" , "Mattia Rizzolo" 
Cc: diffosc...@lists.reproducible-builds.org, reproducible-builds@lists.alioth.debian.org
Subject: Re: [diffoscope] 01/01: Use js-beautify as _javascript_ code beautifier for .js files

Hey, thanks for this!

A question - do you consider #836786 closed by this patch, or is there still more work to be done on it?

(#836786 - diffoscope: Differences between long lines are missing in HTML format")

X

Emanuel Bronshtein:
> Sure, here is the test:
> https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=9bbcf2f6ce5dc711a9dc07c731cddcc5db9fa681
> js-beautify can be installed also with `pip install jsbeautifier` (from upstream https://github.com/beautify-web/js-beautify).
> is there a way to add pip packages in diffoscope as alternative way to install tools? (in case there is no packaged version in distro)
>
> Thanks,
> -Emanuel.
>
> Sent: Monday, December 19, 2016 at 4:32 PM
> From: "Mattia Rizzolo" 
> To: reproducible-builds@lists.alioth.debian.org, "Emanuel Bronshtein" 
> Cc: diffosc...@lists.reproducible-builds.org
> Subject: Re: [diffoscope] 01/01: Use js-beautify as _javascript_ code beautifier for .js files
> Hey Emanuel :)
>
> On Sun, Dec 18, 2016 at 09:55:08PM +, Emanuel Bronshtein wrote:
>> e3amn2l-guest pushed a commit to branch master
>> in repository diffoscope.
>>
>> commit 002f05444cc0a238d8e617b31dabef4111b4ecd2
>> Author: Emanuel Bronshtein 
>> Date: Sun Dec 18 23:52:51 2016 +0200
>>
>> Use js-beautify as _javascript_ code beautifier for .js files
>>
>> _javascript_ code can be minified (See #838984) or not well formatted.
>> ---
>> debian/control | 1 +
>> diffoscope/comparators/__init__.py | 1 +
>> diffoscope/comparators/_javascript_.py | 42 
>> diffoscope/exc.py | 3 +++
>> 4 files changed, 47 insertions(+)
>
>
> nice contribution (which also triggered #848609 from me), but a thing is
> missing: tests!
> We really love tests, so could you please come up with some for this
> comparator too? :)
>
> --
> regards,
> Mattia Rizzolo
>
> GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`.
> more about me: https://mapreri.org : :' :
> Launchpad user: https://launchpad.net/~mapreri[https://launchpad.net/~mapreri] `. `'`
> Debian QA page: https://qa.debian.org/developer.php?login=mattia[https://qa.debian.org/developer.php?login=mattia] `-
>
> ___
> Reproducible-builds mailing list
> Reproducible-builds@lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
>


--
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git




___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#836786: JavaScript beautifier added

2016-12-26 Thread Emanuel Bronshtein 
js-beautify is used as JavaScript code beautifier from commit:
https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?=002f05444cc0a238d8e617b31dabef4111b4ecd2
which fix the above examples, but there are more results that have partial 
results in lines, such as:

in attribute inside xml file: 
./usr/lib/x86_64-linux-gnu/ImageMagick-6.9.7/config-Q16/configure.xml
https://tests.reproducible-builds.org/debian/dbd/experimental/amd64/imagemagick_6.9.7.0+dfsg-1.diffoscope.html

in html file: 
./usr/share/doc/singular-dev-doc/html/include_2factory_2factoryconf_8h_source.html
in .h file: ./usr/include/x86_64-linux-gnu/singular/factory/factoryconf.h
https://tests.reproducible-builds.org/debian/rb-pkg/experimental/amd64/diffoscope-results/singular.html

for some of them a beautifier will help (a good idea anyway, even if not 
related to this bug), a general fix as suggested in point 2 will be better 
(expand with JavaScript code)

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#849411: diffoscope: HTML markup warnings/errors

2016-12-26 Thread Emanuel Bronshtein 
Source: diffoscope
Severity: normal

Dear Maintainer,

W3 validator report several errors/warnings for diffoscope output, for example:
https://validator.w3.org/nu/?doc=https://tests.reproducible-builds.org/debian/dbd/unstable/amd64/monotone_1.1-9.diffoscope.html
https://validator.w3.org/nu/?doc=https://tests.reproducible-builds.org/debian/dbd/unstable/amd64/r-cran-filehash_2.3-1.diffoscope.html

list of reported issues:

Error: Bad value  for attribute name on element a: An ID must not be the empty 
string.
Error: Bad value #monotone-doc_1.1-9_all.deb/file list for attribute href on 
element a: Illegal character in fragment: space is not allowed.
Error: Bad value monotone-doc_1.1-9_all.deb/file list for attribute name on 
element a: An ID must not contain whitespace.
Error: Forbidden code point U+0098.
Error: Saw U+ in stream.
Error: td start tag in table body.
Error: End tag td seen, but there were open elements.
Error: Stray end tag del.
Error: Unclosed element ins.
Error: Unclosed element del.
Error: Table column 4 established by element col has no cells beginning in it.
Error: Table column 2 established by element col has no cells beginning in it.
Error: A table row was 2 columns wide, which is less than the column count 
established using column markup (4).

Warning: The name attribute is obsolete. Consider putting an id attribute on 
the nearest container instead.
Warning: This document appears to be written in English. Consider adding 
lang="en" (or variant) to the html start tag.

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [diffoscope] 01/01: Use js-beautify as JavaScript code beautifier for .js files

2016-12-26 Thread Ximin Luo 
Hey, thanks for this!

A question - do you consider #836786 closed by this patch, or is there still 
more work to be done on it?

(#836786 - diffoscope: Differences between long lines are missing in HTML 
format")

X

Emanuel Bronshtein:
> Sure, here is the test:
> https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=9bbcf2f6ce5dc711a9dc07c731cddcc5db9fa681
> js-beautify can be installed also with `pip install jsbeautifier` (from 
> upstream https://github.com/beautify-web/js-beautify).
> is there a way to add pip packages in diffoscope as alternative way to 
> install tools? (in case there is no packaged version in distro)
> 
> Thanks,
> -Emanuel.
> 
> Sent: Monday, December 19, 2016 at 4:32 PM
> From: "Mattia Rizzolo" 
> To: reproducible-builds@lists.alioth.debian.org, "Emanuel Bronshtein" 
> 
> Cc: diffosc...@lists.reproducible-builds.org
> Subject: Re: [diffoscope] 01/01: Use js-beautify as JavaScript code 
> beautifier for .js files
> Hey Emanuel :)
> 
> On Sun, Dec 18, 2016 at 09:55:08PM +, Emanuel Bronshtein wrote:
>> e3amn2l-guest pushed a commit to branch master
>> in repository diffoscope.
>>
>> commit 002f05444cc0a238d8e617b31dabef4111b4ecd2
>> Author: Emanuel Bronshtein 
>> Date: Sun Dec 18 23:52:51 2016 +0200
>>
>> Use js-beautify as JavaScript code beautifier for .js files
>>
>> JavaScript code can be minified (See #838984) or not well formatted.
>> ---
>> debian/control | 1 +
>> diffoscope/comparators/__init__.py | 1 +
>> diffoscope/comparators/javascript.py | 42 
>> 
>> diffoscope/exc.py | 3 +++
>> 4 files changed, 47 insertions(+)
> 
> 
> nice contribution (which also triggered #848609 from me), but a thing is
> missing: tests!
> We really love tests, so could you please come up with some for this
> comparator too? :)
> 
> --
> regards,
> Mattia Rizzolo
> 
> GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`.
> more about me: https://mapreri.org : :' :
> Launchpad user: 
> https://launchpad.net/~mapreri[https://launchpad.net/~mapreri] `. `'`
> Debian QA page: 
> https://qa.debian.org/developer.php?login=mattia[https://qa.debian.org/developer.php?login=mattia]
>  `-
> 
> ___
> Reproducible-builds mailing list
> Reproducible-builds@lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds
> 


-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#849407: diffoscope: Failures/Errors/Warnings in readelf/objdump usages

2016-12-26 Thread Emanuel Bronshtein 
Source: diffoscope
Severity: normal

Dear Maintainer,

readelf/objdump return errors/warnings and sometimes fails, examples:

1. objdump: can't disassemble for architecture UNKNOWN!

1.1 objdump: section '.plt' mentioned in a -j option, but not found in 
any input file

http://37.218.242.117/com.traffar.pentago_2.apk.diffoscope.html#lib/armeabi-v7a/libSDL2.so

1.2 objdump: section '.text' mentioned in a -j option, but not found in 
any input file

http://37.218.242.117/us.achromaticmetaphor.agram_21.apk.diffoscope.html#lib/mips64/libanagram.so

2. readelf: Error: Not an ELF file - it has the wrong magic bytes at the start
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/libgpg-error.html

3. readelf: Error: the dynamic segment offset + size exceeds the size of the 
file
https://tests.reproducible-builds.org/debian/dbd/unstable/amd64/lakai_0.1-2.diffoscope.html

4. readelf: Error: no .dynamic section in the dynamic segment
https://tests.reproducible-builds.org/debian/dbd/unstable/arm64/dbmix_0.9.8-6.3.diffoscope.html

5. readelf: Warning: There is a hole [0x741d3 - 0x741fc] in .debug_loc section.
https://tests.reproducible-builds.org/debian/dbd/unstable/arm64/k3b_2.0.3a-2.diffoscope.html

6. readelf: Warning: Section '.rodata' was not dumped because it does not exist!
https://tests.reproducible-builds.org/debian/dbd/unstable/arm64/zbar_0.10+doc-10.diffoscope.html


apart from fixing the above usages, maybe consider using radare2 
https://packages.debian.org/sid/radare2
which has disassembler for many formats/architectures instead by default if 
possible (readelf/objdump combination will be used as fallback)

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#849403: diffoscope: Improvesupport for Android apps

2016-12-26 Thread Emanuel Bronshtein 
Source: diffoscope
Severity: wishlist

Dear Maintainer,

1. Better handle .dex files

currently enjarify tool is used in order to transform .dex file to .jar, this 
tool can fail (as probably any decompiler) for example:

Command `enjarify -o /tmp/tmpdp3fttiq_diffoscope/classes.jar 
/tmp/tmpw04hjmue_diffoscope/classes.dex` exited with -9. Output: 
http://37.218.242.117/com.nbossard.packlist_16.apk.diffoscope.html#classes.dex

No file format specific differences found inside, yet data differs:
http://37.218.242.117/com.ancantus.HYPNOTOAD_4.apk.diffoscope.html#classes.dex

in such cases the fallback is binary comparison, it will be better to:
* use other decompilers as fallback if enjarify fail, such as:
jadx - https://github.com/skylot/jadx
SOOT - https://sable.github.io/soot/
Dare - http://siis.cse.psu.edu/dare/index.html
Dex2Jar - https://github.com/pxb1988/dex2jar
DAD from androguard (which has Debian package 
https://packages.debian.org/sid/androguard )
more information: 
https://stackoverflow.com/questions/1249973/decompiling-dex-into-java-sourcecode#36159034

* if all available decompilers fail, use apktool in order to transform 
.dex file to smali/baksmali 

2. Support parsing android resources not in APK file

Support for some Android files (such as: AndroidManifest.xml & resources.arsc) 
was added in:
https://anonscm.debian.org/git/reproducible/diffoscope.git/commit/?id=3e748664a91b7ced412547a204e1473161425d4f
but it works only if comparing APK files, not directories / when the content 
archived in other format (non ZIP)

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#849391: strip-nondeterminism: autopkgtest failures

2016-12-26 Thread Chris Lamb 
tags 849391 + pending
thanks

Thanks for the report. I've fixed this in:

  
https://anonscm.debian.org/git/reproducible/strip-nondeterminism.git/commit/?h=debian=81d06905acb35b7a8681ff2e9bf29087cae5414e


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Processed: Re: strip-nondeterminism: autopkgtest failures

2016-12-26 Thread Debian Bug Tracking System 
Processing commands for cont...@bugs.debian.org:

> tags 849391 + pending
Bug #849391 [src:strip-nondeterminism] strip-nondeterminism: autopkgtest 
failures
Added tag(s) pending.
> thanks
Stopping processing here.

Please contact me if you need assistance.
-- 
849391: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=849391
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#849395: diffoscope: Improve support for comparing images

2016-12-26 Thread Emanuel Bronshtein 
Source: diffoscope
Severity: wishlist

Dear Maintainer,

1. compare image metadata (EXIF / XMP / size / etc..) for various images 
formats, can be done with following tools:
exiftool - http://www.sno.phy.queensu.ca/~phil/exiftool/ - 
https://packages.debian.org/sid/libimage-exiftool-perl
exiv2 - http://www.exiv2.org/ - 
https://packages.debian.org/unstable/exiv2
ImageMagick identify - https://www.imagemagick.org/script/identify.php 
- https://packages.debian.org/sid/imagemagick (by using -verbose option)

2. visual compare images in HTML output
images can be shown using data URI: 
https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs
The below tools can be used to detect difference in images:
Resemble.js - https://github.com/Huddle/Resemble.js/
PerceptualDiff - https://packages.debian.org/sid/perceptualdiff
ImageMagick - https://www.imagemagick.org/Usage/compare/ - 
https://packages.debian.org/sid/imagemagick
image-diff - https://github.com/uber/image-diff

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [PATCH] Reproducible debian: Log diffoscope profiling info if available.

2016-12-26 Thread Holger Levsen 
On Mon, Dec 26, 2016 at 03:26:54PM +, Chris Lamb wrote:
> > I still wonder :) Do you plan to use --profile on t.r-b.o or?
> Yes; that's the entire point of this thread! :p

I see, thanks.


-- 
cheers,
Holger


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#849391: strip-nondeterminism: autopkgtest failures

2016-12-26 Thread Mattia Rizzolo 
source: strip-nondeterminism
version: 0.029-1
severity: important


from 
https://ci.debian.net/data/packages/unstable/amd64/s/strip-nondeterminism/20161226_100743.autopkgtest.log.gz

8<--
1..2
not ok 1 - bin/dh_strip_nondeterminism --help returns 127

#   Failed test 'bin/dh_strip_nondeterminism --help returns 127'
#   at t/binaries.t line 42.
not ok 2 - bin/strip-nondeterminism --help returns 127

#   Failed test 'bin/strip-nondeterminism --help returns 127'
#   at t/binaries.t line 42.
# Looks like you failed 2 tests of 2.
adt-run [10:08:14]: test command1: ---]
adt-run [10:08:14]: test command1:  - - - - - - - - - - results - - - - - - - - 
- -
command1 FAIL non-zero exit status 2
>8

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [PATCH] Reproducible debian: Log diffoscope profiling info if available.

2016-12-26 Thread Chris Lamb 
Holger Levsen wrote:

> I still wonder :) Do you plan to use --profile on t.r-b.o or?

Yes; that's the entire point of this thread! :p


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [PATCH] Reproducible debian: Log diffoscope profiling info if available.

2016-12-26 Thread Holger Levsen 
On Mon, Dec 26, 2016 at 01:01:34PM +, Chris Lamb wrote:
> Holger Levsen wrote:
> 
> > > The --profile flag is new in diffoscope 65.
> > 
> > yes, and? it's not being used on t.r-b.o…?!!
> 
> What is "it" here? If referring to the flag then yes, it's not being
> used - that's the intention of this patch set. If it refers to diffoscope
> >= 65 then I am not actually sure why given:
 
it is the flag, yes.


>< h01ger > lamby: it uses that schroot, but inside that schroot
>   diffoscope from sid is installed 
> 
> … and sid has 66 so inferring that we "should" be using that, no? :) What
> am I missing?

in your mail about about the updated patch you wrote "(From an IRC
conversation I am working under the assumption that diffoscope
from unstable is used even when comparing build results from testing.)"
which made me wonder why you mentioned --profile and t.r-b.o together…

I still wonder :) Do you plan to use --profile on t.r-b.o or? (why did
you mention t.r-b.o?)


-- 
cheers,
Holger


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Bug#849386: diffoscope: Improve Java support (use decompilers by default and javap as fallback)

2016-12-26 Thread Emanuel Bronshtein 
Source: diffoscope
Severity: wishlist

Dear Maintainer,

Currently javap is used for .class files, for example:
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/velocity.html
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/aspectj.html
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/jabref.html
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/arm64/diffoscope-results/openjpa.html

it will be better (easier to read/understand) to use decompiler instead which 
return Java code from class files, and use javap as fallback if decompiler tool 
failed (aborted/exception/etc..) or fail to find difference.
in Debian there is package for 'procon-decompiler' Java decompiler:
https://packages.debian.org/unstable/procyon-decompiler

more FLOSS Java decompilers (might be available in other distros, such as: 
https://aur.archlinux.org/packages/?K=java+decompiler )

Krakatau - https://github.com/Storyyeller/Krakatau
Candle - https://github.com/bradsdavis/candle-decompiler
Fernflower - 
https://github.com/JetBrains/intellij-community/tree/master/plugins/java-decompiler/engine
 / https://github.com/fesh0r/fernflower
Jadx - https://github.com/skylot/jadx

that can be used as fallback to 'procyon-decompiler' (javap need to be used as 
last effort)

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [PATCH] Reproducible debian: Log diffoscope profiling info if available.

2016-12-26 Thread Chris Lamb 
Holger Levsen wrote:

> > The --profile flag is new in diffoscope 65.
> 
> yes, and? it's not being used on t.r-b.o…?!!

What is "it" here? If referring to the flag then yes, it's not being
used - that's the intention of this patch set. If it refers to diffoscope
>= 65 then I am not actually sure why given:

   < h01ger > lamby: it uses that schroot, but inside that schroot
  diffoscope from sid is installed 

… and sid has 66 so inferring that we "should" be using that, no? :) What
am I missing?


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [PATCH] Reproducible debian: Log diffoscope profiling info if available.

2016-12-26 Thread Holger Levsen 
On Mon, Dec 26, 2016 at 12:30:06PM +, Chris Lamb wrote:
> The previous patch I mailed did a version check before injecting this flag so
> it did not break.

On Mon, Dec 26, 2016 at 12:17:27PM +, Daniel Shahaf wrote:
> The --profile flag is new in diffoscope 65.

yes, and? it's not being used on t.r-b.o…?!!


-- 
cheers,
Holger


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [PATCH] Reproducible debian: Log diffoscope profiling info if available.

2016-12-26 Thread Chris Lamb 
Holger Levsen wrote:

> > Updated patch attached now that diffoscope 65 is in unstable.
> > 
> > (From an IRC conversation I am working under the assumption that diffoscope
> > from unstable is used even when comparing build results from testing.)
> 
> while this is true (on t.r-b.o) I wonder why this is relevant/mentioned
> here in this profiling context…?

The previous patch I mailed did a version check before injecting this flag so
it did not break.


Regards,

-- 
  ,''`.
 : :'  : Chris Lamb
 `. `'`  la...@debian.org / chris-lamb.co.uk
   `-

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [PATCH] Reproducible debian: Log diffoscope profiling info if available.

2016-12-26 Thread Daniel Shahaf 
Holger Levsen wrote on Mon, Dec 26, 2016 at 12:07:11 +:
> On Sun, Dec 25, 2016 at 05:44:50PM +, Chris Lamb wrote:
> > Chris Lamb wrote:
> > 
> > > Reproducible debian: Log diffoscope profiling info if available.
> > 
> > Updated patch attached now that diffoscope 65 is in unstable.
> > 
> > (From an IRC conversation I am working under the assumption that diffoscope
> > from unstable is used even when comparing build results from testing.)
> 
> while this is true (on t.r-b.o) I wonder why this is relevant/mentioned
> here in this profiling context…?

The --profile flag is new in diffoscope 65.

___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds

Re: [PATCH] Reproducible debian: Log diffoscope profiling info if available.

2016-12-26 Thread Holger Levsen 
On Sun, Dec 25, 2016 at 05:44:50PM +, Chris Lamb wrote:
> Chris Lamb wrote:
> 
> > Reproducible debian: Log diffoscope profiling info if available.
> 
> Updated patch attached now that diffoscope 65 is in unstable.
> 
> (From an IRC conversation I am working under the assumption that diffoscope
> from unstable is used even when comparing build results from testing.)

while this is true (on t.r-b.o) I wonder why this is relevant/mentioned
here in this profiling context…?


-- 
cheers,
Holger


signature.asc
Description: Digital signature
___
Reproducible-builds mailing list
Reproducible-builds@lists.alioth.debian.org
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/reproducible-builds