[issue45186] Marshal output isn't completely deterministic.

Change by Eric Snow : -- resolution: -> duplicate stage: patch review -> resolved status: open -> closed superseder: -> Reproducible pyc: FLAG_REF is not stable. ___ Python tracker

[issue45186] Marshal output isn't completely deterministic.

Eric Snow added the comment: I'm closing this in favor of bpo-34093. -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue45186] Marshal output isn't completely deterministic.

Eric Snow added the comment: Thanks, Inada-san. That's super helpful. -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue45186] Marshal output isn't completely deterministic.

Inada Naoki added the comment: FYI, This issue is duplicate of https://bugs.python.org/issue34093, and I had made two pull requests to solve the issue. -- nosy: +methane ___ Python tracker

[issue45186] Marshal output isn't completely deterministic.

Change by Eric Snow : -- pull_requests: +26794 pull_request: https://github.com/python/cpython/pull/28379 ___ Python tracker ___

[issue45186] Marshal output isn't completely deterministic.

Eric Snow added the comment: That's a good idea. It's certainly cleaner than the approach I took (optionally pass in to marshal.dumps() the list of "before" object/refcount pairs to compare in w_ref()). Adding a flag to marshal.dumps() to opt out shouldn't be too big a deal. (I expect

[issue45186] Marshal output isn't completely deterministic.

Guido van Rossum added the comment: I would propose that marshal internally make an extra pass over its input in order to determine which objects are referenced multiple times. This will speed up reading marshalled data (in addition to addressing the reproducibility issue with debug builds)

[issue45186] Marshal output isn't completely deterministic.

Eric Snow added the comment: New changeset cbeb81971057d6c382f45ecce92df2b204d4106a by Eric Snow in branch 'main': bpo-45020: Freeze some of the modules imported during startup. (gh-28335) https://github.com/python/cpython/commit/cbeb81971057d6c382f45ecce92df2b204d4106a --

[issue45186] Marshal output isn't completely deterministic.

Change by Eric Snow : -- keywords: +patch pull_requests: +26746 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/28335 ___ Python tracker

[issue45186] Marshal output isn't completely deterministic.

Eric Snow added the comment: One consequence of this is that frozen module .h files can be different for debug vs. non-debug, which causes CI (and Windows builds) to fail. -- ___ Python tracker

[issue45186] Marshal output isn't completely deterministic.

Eric Snow added the comment: FYI, I came up with a fix (for frozen modules, at least) in https://github.com/python/cpython/pull/28107. -- ___ Python tracker ___

[issue45186] Marshal output isn't completely deterministic.

New submission from Eric Snow : (See: https://github.com/python/cpython/pull/28107#issuecomment-915627148) The output from marshal (e.g. PyMarshal_WriteObjectToString(), marshal.dump()) may be different depending on if it is a debug or non-debug build. I found this while working on freezing