reproducible builds and python

2014-08-11 Thread bmorbach
Hi everyone!

I've been doing some work towards reproducible builds in Fedora (mostly
with various upstreams so far) and one of the elephants in the Room are
obviously Pythons .pyc and .pyo files.

As those contain the mtime of the original .py file, they might be
different for each rebuild of an srpm.
For many rpms this isn't a problem, because the files are not modified
and thus retain their timestamp from the archive. Quite a few rpms do
modify to .py files though and because of that, every build has a
different result.

I would like to propose to set the mtime of all .py files to a fixed
(for this specific srpm) time. This could be done
in /usr/lib/rpm/brp-python-bytecompile before doing the actual
byte-compilation. This would result in the same .py{c,o} files being
created for each rebuild.

The timestamp could be e.g. the mtime of the oldest file in the
buildroot (which would assume that not _all_ of the files are modified)
But if you are interested in the idea, I'd certainly be open to
suggestions.

To address the obvious question:
Why not special-case those files when comparing rpms?

It will certainly be impossible to achieve this for all packages in
Fedora, so for some files this might indeed be needed, but I think we
should avoid this where possible. The idea of reproducible builds
becomes meaningless if the amount of differences that you just ignore
gets to big.


What do you think of this proposal?

Greetings,
Benedikt

___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/python-devel

Re: reproducible builds and python

2014-08-11 Thread Nick Coghlan
On 08/12/2014 04:01 AM, bmorbach wrote:
> Hi everyone!
> 
> I've been doing some work towards reproducible builds in Fedora (mostly
> with various upstreams so far) and one of the elephants in the Room are
> obviously Pythons .pyc and .pyo files.
> 
> As those contain the mtime of the original .py file, they might be
> different for each rebuild of an srpm.
> For many rpms this isn't a problem, because the files are not modified
> and thus retain their timestamp from the archive. Quite a few rpms do
> modify to .py files though and because of that, every build has a
> different result.
> 
> I would like to propose to set the mtime of all .py files to a fixed
> (for this specific srpm) time. This could be done
> in /usr/lib/rpm/brp-python-bytecompile before doing the actual
> byte-compilation. This would result in the same .py{c,o} files being
> created for each rebuild.

This sounds like a reasonable approach to me, especially if the mtime
for the SRPM is derived from dist-git.

Regards,
Nick.

-- 
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane

HSS Provisioning Architect
___
python-devel mailing list
python-devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/python-devel