Re: [Python-Dev] Deterministic builds of the interpreter

[Freddy Rietdijk]

Hi Armin,

Thank you for your explanation.

I've now managed to build 2.7 and 3.5 deterministic by recompiling the
bytecode at the end of the build (and excluding 2to3).

Freddy


On Sun, Feb 19, 2017 at 9:30 AM, Armin Rigo  wrote:

> Hi Freddy,
>
> On 16 February 2017 at 18:03, Freddy Rietdijk 
> wrote:
> > As I mentioned, it seems only sets cause unreproducible
> > bytecode. Sets have no order. But when generating the bytecode, I would
> > expect there would still be an order since the code isn't actually
> executed,
> > right?
>
> No, the sets are built as real sets and then marshalled to .pyc files
> in a separate step.  So on CPython an essentially random order will
> end up in the .pyc file.  Even CPython 3.6 gives a deterministic order
> to dictionaries but not sets.  You could ensure sets are marshalled in
> a known order by changing the marshalling code, e.g. to emit them in
> sorted order (on Python 2.x; on 3.x it is more messy because different
> types are more often non-comparable).
>
>
> A bientôt,
>
> Armin.
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deterministic builds of the interpreter

[Armin Rigo]

Hi Freddy,

On 16 February 2017 at 18:03, Freddy Rietdijk  wrote:
> As I mentioned, it seems only sets cause unreproducible
> bytecode. Sets have no order. But when generating the bytecode, I would
> expect there would still be an order since the code isn't actually executed,
> right?

No, the sets are built as real sets and then marshalled to .pyc files
in a separate step.  So on CPython an essentially random order will
end up in the .pyc file.  Even CPython 3.6 gives a deterministic order
to dictionaries but not sets.  You could ensure sets are marshalled in
a known order by changing the marshalling code, e.g. to emit them in
sorted order (on Python 2.x; on 3.x it is more messy because different
types are more often non-comparable).


A bientôt,

Armin.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com