[Bug lto/66305] -ffat-lto-objects create unreproducible objects
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66305 Peter Wu changed: What|Removed |Added CC||peter at lekensteyn dot nl --- Comment #6 from Peter Wu --- -frandom-seed=0 did not work for me. It appears that since GCC 8 introduced a bug where zero values resulted in a new random value anyway: https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=edabe3d8b479e47a1bb3ed495f2a1d94b0ecbb2d Any of these would trigger that problematic case in get_random_seed: -frandom-seed=0 -frandom-seed=0x0 -frandom-seed=jNCWWJ (non-hexadecimal strings are hashed, see set_random_seed) -frandom-seed=1 does seem to result in a reproducible build with GCC 9.2.0, but only as long as the main source file passed to the compiler is a relative path. The main source path is hashed, so if it was absolute, then the source directory would still become part of the build environment: https://github.com/gcc-mirror/gcc/blob/e98ebda074bf8fc5f630a93085af81f52437d851/gcc/tree.c#L9609-L9626 file = LOCATION_FILE (input_location); len = strlen (file); q = (char *) alloca (9 + 19 + len + 1); memcpy (q, file, len + 1); snprintf (q + len, 9 + 19 + 1, "_%08X_" HOST_WIDE_INT_PRINT_HEX, crc32_string (0, name), get_random_seed (false)); ^^ (LOCATION_FILE is unaffected by -ffile-prefix-map, unlike fold_builtin_FILE in gcc/builtins.c for example.)
[Bug lto/66305] -ffat-lto-objects create unreproducible objects
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66305 Daniel Kahn Gillmor changed: What|Removed |Added CC||dkg at fifthhorseman dot net --- Comment #5 from Daniel Kahn Gillmor --- (In reply to lunar from comment #3) > Would using a hash over the section content work? In any cases, in the context > of Debian (this applies for FreeBSD as well), we have a canonical build path > so it would probably be fine to use it as the source of the hash. Please don't embed the full canonical build path into the solution for this. It seems possible to vary the build path successfully and produce the same output (see bug 68848 for a minor fix needed for debugging symbols), so it would be a shame to re-embed it here. > I guess one could already do this without further help by giving the > build path to `-frandom-seed=`. This only would need some Makefile trickery. using a cryptographic digest of the file contents would require roughly the same Makefile trickery without forcing a canonicalized build path.
[Bug lto/66305] -ffat-lto-objects create unreproducible objects
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66305 Richard Biener rguenth at gcc dot gnu.org changed: What|Removed |Added Keywords||lto CC||andi-gcc at firstfloor dot org --- Comment #1 from Richard Biener rguenth at gcc dot gnu.org --- I think they become deterministic with -frandom-seed=0 for example. They are not deterministic to support partial linking of LTO objects as far as I know. Should have the same issue with -fno-fat-lto-objects. Why are python packages shipping with LTO bytecode?
[Bug lto/66305] -ffat-lto-objects create unreproducible objects
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66305 --- Comment #3 from lunar at debian dot org --- Richard Biener: I think they become deterministic with -frandom-seed=0 for example. They are not deterministic to support partial linking of LTO objects as far as I know. They are indeed reproducible with `-frandom-seed=0`. But I guess there's a downside to that, right? Andi: In theory it would be possible to use the hash of the full output path name or similar, which would be a bit more deterministic, but there are still ways this could break (e.g. if someone copies object files around) Would using a hash over the section content work? In any cases, in the context of Debian (this applies for FreeBSD as well), we have a canonical build path so it would probably be fine to use it as the source of the hash. I guess one could already do this without further help by giving the build path to `-frandom-seed=`. This only would need some Makefile trickery. How about your deterministic build tools just learn to ignore that suffix? I am not sure I understand what you are talking about. We want to get byte-for-byte identical packages on each build (provided the same software versions are used), so we need a given version of GCC to always produce the same output.
[Bug lto/66305] -ffat-lto-objects create unreproducible objects
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66305 --- Comment #4 from andi at firstfloor dot org --- --- Comment #3 from lunar at debian dot org --- Richard Biener: I think they become deterministic with -frandom-seed=0 for example. They are not deterministic to support partial linking of LTO objects as far as I know. They are indeed reproducible with `-frandom-seed=0`. But I guess there's a downside to that, right? The downside is that incremential linking (ld -r) does not work. But the random seed is used for other things in gcc too, so you may have other problems too. similar, which would be a bit more deterministic, but there are still ways this could break (e.g. if someone copies object files around) Would using a hash over the section content work? In any cases, in the context of Debian (this applies for FreeBSD as well), we have a canonical build path so it would probably be fine to use it as the source of the hash. I guess one could already do this without further help by giving the build path to `-frandom-seed=`. This only would need some Makefile trickery. Yes. It would probably be easier in gcc, e.g. with a new option.
[Bug lto/66305] -ffat-lto-objects create unreproducible objects
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66305 --- Comment #2 from andi at firstfloor dot org --- On Wed, May 27, 2015 at 12:21:04PM +, rguenth at gcc dot gnu.org wrote: --- Comment #1 from Richard Biener rguenth at gcc dot gnu.org --- I think they become deterministic with -frandom-seed=0 for example. They are not deterministic to support partial linking of LTO objects as far as I know. Yes that's right. It prevents the linker from merging sections. In theory it would be possible to use the hash of the full output path name or similar, which would be a bit more deterministic, but there are still ways this could break (e.g. if someone copies object files around) How about your deterministic build tools just learn to ignore that suffix? -Andi
