apksigcopier v1.0.0
Hi! > apksigcopier is a tool for copying APK signatures from a signed APK > to an unsigned one (in order to verify reproducible builds). It can > also be used to compare two APKs with different signatures. As apksigcopier [1] -- including the vendored copy in fdroidserver [2] -- has worked reliably ever since the release of v0.5.0 in April, I released v1.0.0 this week, which adds a "compare" subcommand to easily compare two APKs with different signatures (this requires apksigner). (Thanks to Holger, it will join v0.5.0 in Debian's NEW queue soon.) - Felix [1] https://github.com/obfusk/apksigcopier [2] https://gitlab.com/fdroid/fdroidserver
How can I contribute?
Hi! I'd like to contribute to Reproducible Builds (for Debian). I've read [1] and requested access to [2]; unfortunately, [3] is a broken link. - Felix [1] https://reproducible-builds.org/contribute/ [2] http://salsa.debian.org/reproducible-builds [3] https://reproducible-builds.org/contribute/debian/
Re: verifying reproducible APKs: apksigcopier
Hi! I just released v0.5.0 of apksigcopier, which is the same code that has now also been merged into fdroidserver master as a vendored library. I've also uploaded a Debian source package to mentors.debian.net, which is now awaiting Holger's sponsorship. - Felix
Re: verifying reproducible APKs: apksigcopier
Hi Torsten, * Torsten Grote [2021-03-31 16:58]: > On Sunday, 28 March 2021 22:02:33 -03 Felix C. Stegerman wrote: > > apksigcopier [2], a tool to copy APK signatures > > from a signed APK to an unsigned one. > > What kind of signatures does the tool support (couldn't find > this in the README)? v1, v2 and v3? Correct. I've added this information to the README :) - Felix
verifying reproducible APKs: apksigcopier
Hi! The F-Droid reproducible builds & verification effort recently led [1] to the development of apksigcopier [2], a tool to copy APK signatures from a signed APK to an unsigned one. ( I've started packaging it for Debian [3] and intend to file an ITP soon, but since I'm not a Debian developer -- yet, though I'd like to become one -- I could use some help with that. ) - Felix [1] https://gitlab.com/fdroid/fdroidserver/-/merge_requests/802 [2] https://github.com/obfusk/apksigcopier [3] https://salsa.debian.org/obfusk/apksigcopier/-/commits/debian/sid
Re: reproducible .pyc files (& python-for-android)
* "Bernhard M. Wiedemann" [2021-01-04 12:48]: > This is not a timestamp issue, though. If those are varying, they are in > the header (first 12 bytes) of the .pyc. > > > │ 00f0: 6d5a 0d62 6469 7374 5f77 696e 696e 7374 mZ.bdist_wininst > │ -0100: 5a05 6368 6563 6b5a 0675 706c 6f61 644e Z.checkZ.uploadN > │ +0100: da05 6368 6563 6b5a 0675 706c 6f61 644e ..checkZ.uploadN > > > I have seen this before and remember something about python string > reference counters being dumped into these pickle files and that varied > from ordering, so that > py_compile py1.py py2.py > produced different results than > py_compile py2.py py1.py > > One way to get reproducible results is to delete and recreate all .pyc > files with > find -type f -a -name "*.py" -print0 | > sort -z | > xargs -0 $python_binary -m py_compile > > > Maybe related: creating .pyc files on i586 and x86_64 (with identical > toolchain) always produced different results for me. I was finally able to reproduce this on another machine. This particular difference is caused by whether the system has liblzma-dev installed or not (when Python is built). - Felix
Re: reproducible .pyc files (& python-for-android)
Hi Frederik, * Frederik Rietdijk [2021-01-04 14:48]: > Recently I spent some time again as well on making the Python interpreters > in Nixpkgs build reproducibly. The following Nix expression results in > deterministic builds of the 3.x interpreters we have. Search for > `determinis` and you will see the changes we do to get there. Maybe it's of > help to you. > > https://github.com/NixOS/nixpkgs/blob/f19ad43d674e5bbaa70a000f65031ab09a32a338/pkgs/development/interpreters/python/cpython/default.nix Thanks for the link! I don't really see anything significantly different to what I'm doing though. The issue I had w/ .pyc files was probably caused by something specific to the GitHub Actions Ubuntu image. I'm still kind of curious as to what that is, but I don't think it's currently worth investigating since everything works as expected using a Debian buster container/VM. - Felix
Re: reproducible .pyc files (& python-for-android)
Hi Bernhard (& Chris), * "Bernhard M. Wiedemann" [2021-01-04 12:48]: > Am 04.01.21 um 11:23 schrieb Chris Lamb: > >> p4a compiles those with "hostpython -OO -m compileall -b -f" (where > >> hostpython is the cross-compiled Python for the target -- arm64-v8a or > >> armeabi-v7a -- which is thus definitely the same version on both > >> machines). > > > > As I understand it, recent versions Python can use SOURCE_DATE_EPOCH > > in its internal py_compile routine to ensure that .pyc files are > > reproducible. > > This is not a timestamp issue, though. If those are varying, they are in > the header (first 12 bytes) of the .pyc. Indeed. I should perhaps have mentioned that I am indeed setting SOURCE_DATE_EPOCH (though that is noted in the PR I [1] linked). For reference: the Python version is 3.9.1. > │ 00f0: 6d5a 0d62 6469 7374 5f77 696e 696e 7374 mZ.bdist_wininst > │ -0100: 5a05 6368 6563 6b5a 0675 706c 6f61 644e Z.checkZ.uploadN > │ +0100: da05 6368 6563 6b5a 0675 706c 6f61 644e ..checkZ.uploadN > > I have seen this before and remember something about python string > reference counters being dumped into these pickle files and that varied > from ordering, so that > py_compile py1.py py2.py > produced different results than > py_compile py2.py py1.py That's interesting. But p4a is running "hostpython -OO -m compileall -b -f $DIRECTORY" -- and compileall walks the directory tree in sorted order -- so I don't think that's the problem here. I have currently sidestepped the issue by using a Debian buster docker container (with usrmerge installed) in GitHub Actions (instead of the default Ubuntu image). I now get identical .apks there and on a local buster VM (with the same build path) \o/ If anyone is interested in the modifications I had to make to p4a to get it to build reproducibly: see the PR [1]. It also has comments on what does and does not (yet) work. Thanks. - Felix [1] https://github.com/kivy/python-for-android/pull/2390 P.S. @Chris as a longtime Debian sid user I would also consider that my 'home' distribution :)
