Integrated: 8326685: Linux builds not reproducible if two builds configured in different build folders

2024-02-28 Thread Andrew Leonard
On Mon, 26 Feb 2024 16:00:42 GMT, Andrew Leonard wrote: > Adds Linux -fdebug-prefix-map'ing for SUPPORT_OUTPUTDIR and HOTSPOT_OUTPUTDIR > when absolute paths are not allowed in the binaries, thus making the building > of a JDK identically reproducible from different build di

Re: RFR: 8326685: Linux builds not reproducible if two builds configured in different build folders [v3]

2024-02-27 Thread Andrew Leonard
On Tue, 27 Feb 2024 13:44:15 GMT, Andrew Leonard wrote: >> Adds Linux -fdebug-prefix-map'ing for SUPPORT_OUTPUTDIR and >> HOTSPOT_OUTPUTDIR when absolute paths are not allowed in the binaries, thus >> making the building of a JDK identically reproducible from different bui

Re: RFR: 8326685: Linux builds not reproducible if two builds configured in different build folders [v3]

2024-02-27 Thread Andrew Leonard
> Adds Linux -fdebug-prefix-map'ing for SUPPORT_OUTPUTDIR and HOTSPOT_OUTPUTDIR > when absolute paths are not allowed in the binaries, thus making the building > of a JDK identically reproducible from different build directories. Andrew Leonard has updated the pull request incr

Re: RFR: 8326685: Linux builds not reproducible if two builds configured in different build folders [v2]

2024-02-27 Thread Andrew Leonard
> Adds Linux -fdebug-prefix-map'ing for SUPPORT_OUTPUTDIR and HOTSPOT_OUTPUTDIR > when absolute paths are not allowed in the binaries, thus making the building > of a JDK identically reproducible from different build directories. Andrew Leonard has updated the pull request incr

Re: RFR: 8326685: Linux builds not reproducible if two builds configured in different build folders [v2]

2024-02-27 Thread Andrew Leonard
On Tue, 27 Feb 2024 10:35:20 GMT, Magnus Ihse Bursie wrote: >> It seems correct to include the output dir in the remapping, but I have two >> objections/questions to the way you are doing it. >> 1) Why not just use OUTPUTDIR instead of the two specialized subdirs? That >> is simpler, more

Re: RFR: 8326685: Linux builds not reproducible if two builds configured in different build folders

2024-02-27 Thread Andrew Leonard
On Tue, 27 Feb 2024 10:59:54 GMT, Andrew Leonard wrote: >> Also, this mapping business is getting really convoluted. :-( I did not like >> it as it was, and this patch makes it even worse. I think we need to rewrite >> this to create some kind of dict/map, and then it

Re: RFR: 8326685: Linux builds not reproducible if two builds configured in different build folders

2024-02-27 Thread Andrew Leonard
On Tue, 27 Feb 2024 10:35:20 GMT, Magnus Ihse Bursie wrote: >> It seems correct to include the output dir in the remapping, but I have two >> objections/questions to the way you are doing it. >> 1) Why not just use OUTPUTDIR instead of the two specialized subdirs? That >> is simpler, more

Re: RFR: 8326685: Linux builds not reproducible if two builds configured in different build folders

2024-02-27 Thread Andrew Leonard
On Tue, 27 Feb 2024 08:13:07 GMT, Andrew Leonard wrote: >> Oh you are right, we should always add the OUTPUTDIR to both maps regardless >> of if it's internal to the WORKSPACE_ROOT or not. Paths relative to the >> OUTPUTDIR should be stable across builds in different

Re: RFR: 8326685: Linux builds not reproducible if two builds configured in different build folders

2024-02-27 Thread Andrew Leonard
On Mon, 26 Feb 2024 22:05:34 GMT, Erik Joelsson wrote: >> I have not thought this fully through, but just throwing out a question: >> would this work even if one build directory were inside the workspace root, >> and the other not? > > Oh you are right, we should always add the OUTPUTDIR to

Re: RFR: 8326685: Linux builds not reproducible if two builds configured in different build folders

2024-02-26 Thread Andrew Leonard
On Mon, 26 Feb 2024 16:00:42 GMT, Andrew Leonard wrote: > Adds Linux -fdebug-prefix-map'ing for SUPPORT_OUTPUTDIR and HOTSPOT_OUTPUTDIR > when absolute paths are not allowed in the binaries, thus making the building > of a JDK identically reproducible from different build directories

RFR: 8326685: Linux builds not reproducible if two builds configured in different build folders

2024-02-26 Thread Andrew Leonard
Adds Linux -fdebug-prefix-map'ing for SUPPORT_OUTPUTDIR and HOTSPOT_OUTPUTDIR when absolute paths are not allowed in the binaries, thus making the building of a JDK identically reproducible from different build directories. - Commit messages: - 8326685: Linux builds not

Integrated: 8323667: Library debug files contain non-reproducible full gcc include paths

2024-01-18 Thread Andrew Leonard
On Fri, 12 Jan 2024 14:33:01 GMT, Andrew Leonard wrote: > For gcc toolchains in ALLOW_ABSOLUTE_PATHS_IN_OUTPUT=False builds, this PR > finds the location of the gcc system include paths, and adds > -fdebug-prefix-map flags to map them to a standard location. Thus making the &g

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v4]

2024-01-17 Thread Andrew Leonard
On Wed, 17 Jan 2024 15:49:10 GMT, Andrew Leonard wrote: > > > So `$CC $CFLAGS` did not work? > > > > > > @magicus So I purposely chose SYSROOT_CFLAGS thinking that would all that > > would be needed, but I think you're correct, I should use the complete

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v6]

2024-01-17 Thread Andrew Leonard
sing DevKits in > different path locations. Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: 8323667: Library debug files contain non-reproducible full gcc include paths Signed-off-by: Andrew Leonard - Changes:

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v4]

2024-01-17 Thread Andrew Leonard
On Wed, 17 Jan 2024 13:52:39 GMT, Erik Joelsson wrote: > > > So `$CC $CFLAGS` did not work? > > > > > > @magicus So I purposely chose SYSROOT_CFLAGS thinking that would all that > > would be needed, but I think you're correct, I should use the complete > > CFLAGS in case a custom user

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v4]

2024-01-17 Thread Andrew Leonard
On Tue, 16 Jan 2024 21:35:14 GMT, Magnus Ihse Bursie wrote: > So `$CC $CFLAGS` did not work? @magicus So I purposely chose SYSROOT_CFLAGS thinking that would all that would be needed, but I think you're correct, I should use the complete CFLAGS in case a custom user extension include may

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v4]

2024-01-17 Thread Andrew Leonard
On Tue, 16 Jan 2024 20:35:01 GMT, Erik Joelsson wrote: > I think this looks ok but could you split the longer lines to make side by > side comparisons easier in the future? Yep, done, thanks Erik - PR Comment: https://git.openjdk.org/jdk/pull/17399#issuecomment-1895402691

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v5]

2024-01-17 Thread Andrew Leonard
sing DevKits in > different path locations. Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: 8323667: Library debug files contain non-reproducible full gcc include paths Signed-off-by: Andrew Leonard - Changes:

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v2]

2024-01-16 Thread Andrew Leonard
On Tue, 16 Jan 2024 12:55:47 GMT, Magnus Ihse Bursie wrote: > Have you checked with `strings` if there are any other hard-coded paths, or > are these really the only ones left? I have been doing some separate machine/location builds with DevKits located in different paths and build workspace

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v4]

2024-01-16 Thread Andrew Leonard
sing DevKits in > different path locations. Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: 8323667: Library debug files contain non-reproducible full gcc include paths Signed-off-by: Andrew Leonard - Changes:

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v2]

2024-01-16 Thread Andrew Leonard
On Tue, 16 Jan 2024 13:58:51 GMT, Andrew Leonard wrote: >> @magicus Yes, good point the condition is sort of hidden as it is. >> I've moved the DEBUG_PREFIX_MAP_GCC_INCLUDE_PATHS setting prior to the >> compiler option check, so the whole constructed DEBUG_PREFIX_CFLAGS is

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v2]

2024-01-16 Thread Andrew Leonard
On Tue, 16 Jan 2024 13:53:59 GMT, Andrew Leonard wrote: >> The logic of this is a bit strange. I think what we really want to express >> is: >> >> if ! ALLOW_ABSOLUTE_PATHS_IN_OUTPUT and -fdebug-prefix-map is a valid option >> for gcc then >> set -fdebu

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v3]

2024-01-16 Thread Andrew Leonard
sing DevKits in > different path locations. Andrew Leonard has updated the pull request incrementally with one additional commit since the last revision: 8323667: Library debug files contain non-reproducible full gcc include paths Signed-off-by: Andrew Leonard - Changes:

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v2]

2024-01-16 Thread Andrew Leonard
On Tue, 16 Jan 2024 12:52:13 GMT, Magnus Ihse Bursie wrote: >> make/autoconf/flags-cflags.m4 line 124: >> >>> 122: fi >>> 123: >>> 124: if test "x$DEBUG_PREFIX_CFLAGS" != x; then >> >> Should this test not be inside the "if $ALLOW_ABSOLUTE_PATHS_IN_OUTPUT == >> false" block? >> >>

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v2]

2024-01-16 Thread Andrew Leonard
On Tue, 16 Jan 2024 12:53:52 GMT, Magnus Ihse Bursie wrote: >> Andrew Leonard has updated the pull request with a new target base due to a >> merge or a rebase. The incremental webrev excludes the unrelated changes >> brought in by the merge/rebase. The pull request contai

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v2]

2024-01-16 Thread Andrew Leonard
On Mon, 15 Jan 2024 14:38:58 GMT, Andrew Leonard wrote: >> For gcc toolchains in ALLOW_ABSOLUTE_PATHS_IN_OUTPUT=False builds, this PR >> finds the location of the gcc system include paths, and adds >> -fdebug-prefix-map flags to map them to a standard location. Thus making

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v2]

2024-01-15 Thread Andrew Leonard
On Mon, 15 Jan 2024 14:38:58 GMT, Andrew Leonard wrote: >> For gcc toolchains in ALLOW_ABSOLUTE_PATHS_IN_OUTPUT=False builds, this PR >> finds the location of the gcc system include paths, and adds >> -fdebug-prefix-map flags to map them to a standard location. Thus making

Integrated: 8323671: DevKit build gcc libraries contain full paths to source location

2024-01-15 Thread Andrew Leonard
On Fri, 12 Jan 2024 14:47:55 GMT, Andrew Leonard wrote: > This PR adds --with-debug-prefix-map to the building of gcc in the DevKit > make file to map the top OUTPUT_ROOT folder to a standard "devkit" path, thus > ensuring DevKit builds are reproducible regardless of th

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v2]

2024-01-15 Thread Andrew Leonard
sing DevKits in > different path locations. Andrew Leonard has updated the pull request with a new target base due to a merge or a rebase. The incremental webrev excludes the unrelated changes brought in by the merge/rebase. The pull request contains four additional commits since the last revisi

Re: RFR: 8323667: Library debug files contain non-reproducible full gcc include paths [v2]

2024-01-15 Thread Andrew Leonard
On Fri, 12 Jan 2024 15:13:26 GMT, Erik Joelsson wrote: >> Andrew Leonard has updated the pull request with a new target base due to a >> merge or a rebase. The incremental webrev excludes the unrelated changes >> brought in by the merge/rebase. The pull request contai

Re: RFR: 8323671: DevKit build gcc libraries contain full paths to source location [v2]

2024-01-15 Thread Andrew Leonard
> This PR adds --with-debug-prefix-map to the building of gcc in the DevKit > make file to map the top OUTPUT_ROOT folder to a standard "devkit" path, thus > ensuring DevKit builds are reproducible regardless of the folder the DevKit > is built within. Andrew Leonar

RFR: 8323671: DevKit build gcc libraries contain full paths to source location

2024-01-12 Thread Andrew Leonard
This PR adds --with-debug-prefix-map to the building of gcc in the DevKit make file to map the top OUTPUT_ROOT folder to a standard "devkit" path, thus ensuring DevKit builds are reproducible regardless of the folder the DevKit is built within. - Commit messages: - 8323671:

RFR: 8323667: Library debug files contain non-reproducible full gcc include paths

2024-01-12 Thread Andrew Leonard
For gcc toolchains in ALLOW_ABSOLUTE_PATHS_IN_OUTPUT=False builds, this PR finds the location of the gcc system include paths, and adds -fdebug-prefix-map flags to map them to a standard location. Thus making the debuginfo and resulting libraries reproducible when using DevKits in different

Re: jdk8u: VS_DESCRIPTION_2017 (Microsoft Visual Studio 2017 - CURRENTLY NOT WORKING)

2023-10-17 Thread Andrew Leonard
the build working again until a later JDK > release. > > /Erik > > On 10/16/23 07:59, Andrew Leonard wrote: > > Does anyone know why the jdk8u toolchain_windows.m4 > > VS_DESCRIPTION_2017 description contains "CURRENTLY NOT WORKING"

jdk8u: VS_DESCRIPTION_2017 (Microsoft Visual Studio 2017 - CURRENTLY NOT WORKING)

2023-10-16 Thread Andrew Leonard
Does anyone know why the jdk8u toolchain_windows.m4 VS_DESCRIPTION_2017 description contains "CURRENTLY NOT WORKING" ? https://github.com/openjdk/jdk8u/blob/b24b03f9dbeb2c1e8b5cb0279659eee489bd9df4/common/autoconf/toolchain_windows.m4#L74 Thanks

Integrated: 8317601: Windows build on WSL broken after JDK-8317340

2023-10-09 Thread Andrew Leonard
On Fri, 6 Oct 2023 10:17:25 GMT, Andrew Leonard wrote: > [JDK-8317340](https://bugs.openjdk.org/browse/JDK-8317340) introduced > -pathmap'ing of VS_INCLUDE paths to enable reproducible Windows builds that > are not dependent on the Visual Studio tooling install location. Thi

Re: RFR: 8317601: Windows build on WSL broken after JDK-8317340 [v2]

2023-10-06 Thread Andrew Leonard
hs generated by the compiler, which > is no easy task within WSL. > > This PR excludes the VS include pathmap'ing within WSL, which will mean for > reproducible builds on WSL, the same VS install location must be used. Andrew Leonard has updated the pull request incrementally with one

Re: RFR: 8317601: Windows build on WSL broken after JDK-8317340 [v2]

2023-10-06 Thread Andrew Leonard
On Fri, 6 Oct 2023 12:56:17 GMT, Erik Joelsson wrote: > Could you add a comment on the new UTIL_FIXUP_WIN_LONG_PATH macro that it > only works in cygwin? Yes, good point. Added - PR Comment: https://git.openjdk.org/jdk/pull/16069#issuecomment-1750638935

Re: RFR: 8317601: Windows build on WSL broken after JDK-8317340

2023-10-06 Thread Andrew Leonard
On Fri, 6 Oct 2023 10:17:25 GMT, Andrew Leonard wrote: > [JDK-8317340](https://bugs.openjdk.org/browse/JDK-8317340) introduced > -pathmap'ing of VS_INCLUDE paths to enable reproducible Windows builds that > are not dependent on the Visual Studio tooling install location. Thi

RFR: 8317601: Windows build on WSL broken after JDK-8317340

2023-10-06 Thread Andrew Leonard
[JDK-8317340](https://bugs.openjdk.org/browse/JDK-8317340) introduced -pathmap'ing of VS_INCLUDE paths to enable reproducible Windows builds that are not dependent on the Visual Studio tooling install location. This however requires pathmap'ing the Windows long paths generated by the compiler,

Integrated: 8317340: Windows builds are not reproducible if MS VS compiler install path differs

2023-10-03 Thread Andrew Leonard
On Mon, 2 Oct 2023 10:31:41 GMT, Andrew Leonard wrote: > When doing a reproducible Windows build on two different machines, that > happen to have MS Visual Studio installed in different paths, the resulting > builds are not identical. This is due to the System include paths, eg: >

Re: RFR: 8317340: Windows builds are not reproducible if MS VS compiler install path differs

2023-10-03 Thread Andrew Leonard
On Mon, 2 Oct 2023 14:13:23 GMT, Erik Joelsson wrote: >>> What effect will this have on debugging? >> >> Hi @erikj79 >> So this is the same as we pathmap the openjdk workspace source just above, >> in that we only do it when ALLOW_ABSOLUTE_PATHS_IN_OUTPUT is "false", which >> is for release

Re: RFR: 8317340: Windows builds are not reproducible if MS VS compiler install path differs

2023-10-02 Thread Andrew Leonard
On Mon, 2 Oct 2023 14:20:12 GMT, Andrew Leonard wrote: > > > What effect will this have on debugging? > > > > > > Hi @erikj79 So this is the same as we pathmap the openjdk workspace source > > just above, in that we only do it when ALLOW_ABSOLUTE_PATHS

Re: RFR: 8317340: Windows builds are not reproducible if MS VS compiler install path differs

2023-10-02 Thread Andrew Leonard
On Mon, 2 Oct 2023 14:13:23 GMT, Erik Joelsson wrote: >>> What effect will this have on debugging? >> >> Hi @erikj79 >> So this is the same as we pathmap the openjdk workspace source just above, >> in that we only do it when ALLOW_ABSOLUTE_PATHS_IN_OUTPUT is "false", which >> is for release

Re: RFR: 8317340: Windows builds are not reproducible if MS VS compiler install path differs

2023-10-02 Thread Andrew Leonard
On Mon, 2 Oct 2023 14:13:23 GMT, Erik Joelsson wrote: > > > What effect will this have on debugging? > > > > > > Hi @erikj79 So this is the same as we pathmap the openjdk workspace source > > just above, in that we only do it when ALLOW_ABSOLUTE_PATHS_IN_OUTPUT is > > "false", which is for

Re: RFR: 8317340: Windows builds are not reproducible if MS VS compiler install path differs

2023-10-02 Thread Andrew Leonard
On Mon, 2 Oct 2023 10:31:41 GMT, Andrew Leonard wrote: > When doing a reproducible Windows build on two different machines, that > happen to have MS Visual Studio installed in different paths, the resulting > builds are not identical. This is due to the System include paths, eg: >

Re: RFR: 8317340: Windows builds are not reproducible if MS VS compiler install path differs

2023-10-02 Thread Andrew Leonard
On Mon, 2 Oct 2023 13:39:43 GMT, Erik Joelsson wrote: > What effect will this have on debugging? Hi @erikj79 So this is the same as we pathmap the openjdk workspace source just above, in that we only do it when ALLOW_ABSOLUTE_PATHS_IN_OUTPUT is "false", which is for release builds. For

RFR: 8317340: Windows builds are not reproducible if MS VS compiler install path differs

2023-10-02 Thread Andrew Leonard
When doing a reproducible Windows build on two different machines, that happen to have MS Visual Studio installed in different paths, the resulting builds are not identical. This is due to the System include paths, eg: C:\Program Files\Microsoft Visual

Integrated: 8316648: jrt-fs.jar classes not reproducible between standard and bootcycle builds

2023-09-25 Thread Andrew Leonard
On Fri, 22 Sep 2023 12:38:03 GMT, Andrew Leonard wrote: > jrt-fs.jar classes are currently compiled with the BootJDK being used, this > has a couple of issues: > - The classes are not reproducible between a standard and bootcycle build > image, since the first is compile

RFR: 8316648: jrt-fs.jar classes not reproducible between standard and bootcycle builds

2023-09-22 Thread Andrew Leonard
jrt-fs.jar classes are currently compiled with the BootJDK being used, this has a couple of issues: - The classes are not reproducible between a standard and bootcycle build image, since the first is compiled by jdk-N-1 and the later jdk-N - The jrt-fs classes in a standard product image are

Re: Should we build jrt-fs.jar again with the "Build JDK" ?

2023-09-21 Thread Andrew Leonard
Thanks Magnus, yeah it is still doing as it says on the "tin", so I think i'm just being paranoid! I'll put a PR together for review, cheers On Thu, Sep 21, 2023 at 10:49 AM Magnus Ihse Bursie < magnus.ihse.bur...@oracle.com> wrote: > On 2023-09-21 10:59, Andrew Leonard

Re: Should we build jrt-fs.jar again with the "Build JDK" ?

2023-09-21 Thread Andrew Leonard
ycle builds to produce their JDK binaries. > > Switching the compiler to interim would help with the reproducibility > issue. I would support that change. I don't think we can reasonably do > something about the jar tool. > > /Erik > On 9/20/23 08:12, Andrew Leonard wrote:

Re: Should we build jrt-fs.jar again with the "Build JDK" ?

2023-09-21 Thread Andrew Leonard
bility > issue. I would support that change. I don't think we can reasonably do > something about the jar tool. > > /Erik > On 9/20/23 08:12, Andrew Leonard wrote: > > Hi Magnus, > > So yes, jrt-fs.jar can be different between a normal build and a bootcycle > build,

Re: Should we build jrt-fs.jar again with the "Build JDK" ?

2023-09-20 Thread Andrew Leonard
t;test", and I suspect most vendors will either just do a standard "product-images" build, or perform their own bootcycle by doing two builds... Cheers Andrew On Wed, Sep 20, 2023 at 2:44 PM Magnus Ihse Bursie < magnus.ihse.bur...@oracle.com> wrote: > On 2023-09-20 0