Re: Supporting alternative toolchains on Windows

[erik . joelsson]

On 2022-03-11 23:17, Julian Waters wrote:

I meant in toolchain.m4, which allows gcc for macOS


Apple used to ship GCC as part of Xcode, and the original Mac port of 
OpenJDK used that GCC. This was eventually removed from Xcode and the 
OpenJDK build was changed to use Clang instead. To make the transition 
smoother, as we can't expect all OpenJDK builders to change Xcode 
versions at the same time, we left dual support in the build, and we 
never got around to removing the GCC support. There is still a GCC 
launcher in Xcode, but it's just a wrapper for Clang. AFAIK, we have 
never supported any other compiler than the one in Xcode on Macos.


So while toolchain.m4 may make it look like we support GCC on Macos, 
that isn't actually the case. We only support Xcode, and I very much 
doubt we can still build with an Xcode old enough to still include an 
actual GCC.


/Erik


best regards,
Julian

On Sat, Mar 12, 2022 at 4:32 AM Magnus Ihse Bursie <
magnus.ihse.bur...@oracle.com> wrote:


On 2022-03-11 20:02, Julian Waters wrote:

I understand the concerns, seems like I grossly underestimated the
complexity such a task would entail. Though I would say the following can
only really be known if it's already been implemented:

 Most likely there will be strange bugs with anything that requires
OS interaction (like dll loading and whatsnot)

To my knowledge the versions of said compilers that link against the
universal CRT utilize exactly the same Windows APIs (And corresponding
dlls) for OS related stuff that Visual C++ itself uses (Minus vcruntime,
which is specific to MSVC), without any POSIX compatibility layers on top.
I've tried rewiring the build system incrementally in the meantime on my
end to see what areas would be of interest and it's now failing when
hitting POSIX specific includes during make, hinting that (At least for
gcc) compatibility has been traded for full native support for Windows APIs
within the compiler, which may mitigate any issues slightly (That's also
why I suggested initially to only allow for versions that link against the
ucrt without any compatibility layers- Cygwin's toolchains which link
against newlib and old MinGW binaries which link against msvcrt would just
be an unnecessary headache). That said, whether aforementioned bugs will
actually surface should the attempt be successful I don't really know yet

 Actually, I would seriously assume that any other compiler than VS
on Windows will give much worse results.
 The VS compiler is battle-hardened. gcc and clang are only used by
a very small enthusiastic hobbyist minority.

The Windows ports of both compilers do keep the same optimizations and
code generation quality as they do on other platforms from what I know
though, it's mainly the linkers that have been modified to generate the PE
format instead of the ELF on Linux. I digress, I don't have any results to
show for this yet

I'm not sure I get the part about macOS strictly mapping to clang. Isn't
there the option to swap to using gcc for macOS?

No, Apple removed that option years ago.

/Magnus


best regards,
Julian

On Sat, Mar 12, 2022 at 1:50 AM Magnus Ihse Bursie <
magnus.ihse.bur...@oracle.com> wrote:



On 2022-03-11 14:34, Julian Waters wrote:

Darn, seems like it'll be much harder than I expected. Since multiple
toolchains are supported for macOS and Linux, I assumed a slight patch
would help get it to work on Windows. Looking through the stuff in make
though, it appears a lot of the build system implicitly expects the
compiler for Windows to always be Visual C++, which doesn't really help
that much (Though the fact that we can exclude many versions of gcc, such
as Cygwin's and old MinGW binaries helps a lot). The build process for the
newer Windows ports of gcc are surprisingly similar to Visual C++ though
(Eg rc can be swapped out for windres) so this might hopefully be something
I can try exploring in the future (Gonna look a bit harder at make and
write what I can find back to this mailing list in the meantime). It'd be
interesting if benchmarks of the JVM compiled with different compilers on
Windows can be compared side by side on the off chance this becomes a
reality though


In theory OS and compiler toolchain are separate things. In practice, in
the JDK, they are not. There is basically a 1-to-1 mapping between
toolchain and OS:
Windows <-> Visual Studio
macOS <-> XCode/clang
linux <-> gcc

(The one possible exception is that clang on linux is probably feasible.)

After years and years on this, all kinds of assumptions has been
hard-coded, even if people try to do the right thing. But sometimes it is
not clear if you are checking for toolchain or os; perhaps when linking
with a specific library, or setting some define.

Any attempt to change this is, as I said, a *huge* undertaking. *And*
there will be tons of negative consequences for the code base as a whole,
when trying to differentiate between toolchain and OS. So this will 

Re: Supporting alternative toolchains on Windows

[Julian Waters]

Attached to this thread is related information I could find within the
JVM's build system (Posted to the top level to not clutter the discussion
below)

Information currently gathered by simply disabling Visual C++ and link.exe
specific flags if gcc is detected until build failure. Also had to come up
with the temporary solution of providing --with-msvcr-dll with a fake dll
for configure to pass. Autoconf had a lot of trouble with WSL (Unable to
find headers and so on), apparently due to issues with CreateProcess, so
this was all tested with MSYS2.

Not a very good way to test it, but it'll have to do for now. Build
currently fails when POSIX header dlfcn.h is encountered, hinting a check
is required to switch to Windows includes if needed. Managed to compile
(Including adlc) until HotSpot source code was reached

Relevant log for future reference (Paths truncated):
Building target 'images' in configuration 'windows-x86_64-server-release'
Compiling 1 files for BUILD_TOOLS_HOTSPOT
Compiling 8 files for BUILD_TOOLS_LANGTOOLS
Warning: No SCM configuration present and no .src-rev
Parsing 2 properties into enum-like class for jdk.compiler
Compiling 13 properties into resource bundles for jdk.javadoc
Compiling 16 properties into resource bundles for jdk.compiler
Compiling 7 properties into resource bundles for jdk.jshell
Compiling 12 properties into resource bundles for jdk.jdeps
Creating hotspot/variant-server/tools/adlc/adlc.exe from 13 file(s)
Compiling 2 files for BUILD_JVMTI_TOOLS
Compiling 127 files for BUILD_java.compiler.interim
Compiling 403 files for BUILD_jdk.compiler.interim
Compiling 238 files for BUILD_jdk.javadoc.interim
Creating support/modules_libs/java.base/server/jvm.dll from 1076 file(s)
In file included from src\hotspot\share/utilities/globalDefinitions.hpp:35,
from src\hotspot\share/memory/allocation.hpp:29,
from src\hotspot\share/classfile/classLoaderData.hpp:28,
from src\hotspot\share\precompiled\precompiled.hpp:34:
src\hotspot\share/utilities/globalDefinitions_gcc.hpp:45:10: fatal error:
dlfcn.h: No such file or directory
   45 | #include 
  |  ^
compilation terminated.
/usr/bin/bash: line 1: -DJDK_VERSION_STRING=19-internal: command not found
make[3]: *** [lib/CompileJvm.gmk:148:
build/windows-x86_64-server-release/hotspot/variant-server/libjvm/objects/jvm.dll.res]
Error 127


Areas of interest so far:
toolchain_microsoft.m4
May need to take a better look at it later to see if it can be enhanced to
handle other toolchains (Mostly since we need ucrtbase.dll and windres).
Could be renamed to toolchain_windows or something else if we do this.

flags.m4 flags-cflags.m4 and flags-ldflags.m4
May need some changes to check for platform. Seems to mostly be correct in
using only the compiler type to figure out what flags to use though

GensrcAdlc.gmk, line 37 onwards
First instance of hard assumption that a specific platform means a specific
compiler is being used

NativeCompilation.gmk, including but not just line 1081 and beyond
This specific area also assumes that Visual C++ is used just because the
target (Note to self: Not the current build OS!) is Windows. On first
glance seems to handle manifest embedding and generation of the import
library lib for a currently unknown dll (The latter wouldn't be needed if
we're using gcc)

Flags: (Visual C++) -> (gcc)

Compiler Flags:
-nologo -> Visual C++ specific
-EHsc -> Exception handling model, use -fno-exceptions and -funwind-tables
to substitute? (Used for ADLC)
-W3 -> Warning level, doesn't seem like there's an exact gcc match
-D_CRT_SECURE_NO_WARNINGS -> Visual C++ specific

Linker Flags:
-nologo -> Not required
-manifest -> ? (ucrt based gcc documentation on manifests is sparse)
-implib -> gcc doesn't require import libraries to link dynamically to dlls

Re: Supporting alternative toolchains on Windows

[Julian Waters]

I meant in toolchain.m4, which allows gcc for macOS

best regards,
Julian

On Sat, Mar 12, 2022 at 4:32 AM Magnus Ihse Bursie <
magnus.ihse.bur...@oracle.com> wrote:

> On 2022-03-11 20:02, Julian Waters wrote:
>
> I understand the concerns, seems like I grossly underestimated the
> complexity such a task would entail. Though I would say the following can
> only really be known if it's already been implemented:
>
> Most likely there will be strange bugs with anything that requires
> OS interaction (like dll loading and whatsnot)
>
> To my knowledge the versions of said compilers that link against the
> universal CRT utilize exactly the same Windows APIs (And corresponding
> dlls) for OS related stuff that Visual C++ itself uses (Minus vcruntime,
> which is specific to MSVC), without any POSIX compatibility layers on top.
> I've tried rewiring the build system incrementally in the meantime on my
> end to see what areas would be of interest and it's now failing when
> hitting POSIX specific includes during make, hinting that (At least for
> gcc) compatibility has been traded for full native support for Windows APIs
> within the compiler, which may mitigate any issues slightly (That's also
> why I suggested initially to only allow for versions that link against the
> ucrt without any compatibility layers- Cygwin's toolchains which link
> against newlib and old MinGW binaries which link against msvcrt would just
> be an unnecessary headache). That said, whether aforementioned bugs will
> actually surface should the attempt be successful I don't really know yet
>
> Actually, I would seriously assume that any other compiler than VS
> on Windows will give much worse results.
> The VS compiler is battle-hardened. gcc and clang are only used by
> a very small enthusiastic hobbyist minority.
>
> The Windows ports of both compilers do keep the same optimizations and
> code generation quality as they do on other platforms from what I know
> though, it's mainly the linkers that have been modified to generate the PE
> format instead of the ELF on Linux. I digress, I don't have any results to
> show for this yet
>
> I'm not sure I get the part about macOS strictly mapping to clang. Isn't
> there the option to swap to using gcc for macOS?
>
> No, Apple removed that option years ago.
>
> /Magnus
>
>
> best regards,
> Julian
>
> On Sat, Mar 12, 2022 at 1:50 AM Magnus Ihse Bursie <
> magnus.ihse.bur...@oracle.com> wrote:
>
>>
>>
>> On 2022-03-11 14:34, Julian Waters wrote:
>>
>> Darn, seems like it'll be much harder than I expected. Since multiple
>> toolchains are supported for macOS and Linux, I assumed a slight patch
>> would help get it to work on Windows. Looking through the stuff in make
>> though, it appears a lot of the build system implicitly expects the
>> compiler for Windows to always be Visual C++, which doesn't really help
>> that much (Though the fact that we can exclude many versions of gcc, such
>> as Cygwin's and old MinGW binaries helps a lot). The build process for the
>> newer Windows ports of gcc are surprisingly similar to Visual C++ though
>> (Eg rc can be swapped out for windres) so this might hopefully be something
>> I can try exploring in the future (Gonna look a bit harder at make and
>> write what I can find back to this mailing list in the meantime). It'd be
>> interesting if benchmarks of the JVM compiled with different compilers on
>> Windows can be compared side by side on the off chance this becomes a
>> reality though
>>
>>
>> In theory OS and compiler toolchain are separate things. In practice, in
>> the JDK, they are not. There is basically a 1-to-1 mapping between
>> toolchain and OS:
>> Windows <-> Visual Studio
>> macOS <-> XCode/clang
>> linux <-> gcc
>>
>> (The one possible exception is that clang on linux is probably feasible.)
>>
>> After years and years on this, all kinds of assumptions has been
>> hard-coded, even if people try to do the right thing. But sometimes it is
>> not clear if you are checking for toolchain or os; perhaps when linking
>> with a specific library, or setting some define.
>>
>> Any attempt to change this is, as I said, a *huge* undertaking. *And*
>> there will be tons of negative consequences for the code base as a whole,
>> when trying to differentiate between toolchain and OS. So this will need to
>> be seriously considered.
>>
>> /Magnus
>>
>>
>> best regards,
>> Julian
>>
>> On Fri, Mar 11, 2022 at 9:16 PM Magnus Ihse Bursie <
>> magnus.ihse.bur...@oracle.com> wrote:
>>
>>> On 2022-03-11 12:55, Julian Waters wrote:
>>>
>>> > Hi all,
>>> >
>>> > How feasible would it be/much effort would it require to support
>>> compiling
>>> > with alternate toolchains on Windows besides Visual C++ (like the
>>> Windows
>>> > ports of clang and gcc) if we restrict the allowed toolchains to only
>>> those
>>> > that link against the ucrt? (Toolchains linking against the dated
>>> msvcrt
>>> > would present too many issues to work with)
>>>
>>> 

Re: Supporting alternative toolchains on Windows

[Magnus Ihse Bursie]

On 2022-03-11 20:02, Julian Waters wrote:
I understand the concerns, seems like I grossly underestimated the 
complexity such a task would entail. Though I would say the following 
can only really be known if it's already been implemented:


        Most likely there will be strange bugs with anything that 
requires OS interaction (like dll loading and whatsnot)


To my knowledge the versions of said compilers that link against the 
universal CRT utilize exactly the same Windows APIs (And corresponding 
dlls) for OS related stuff that Visual C++ itself uses (Minus 
vcruntime, which is specific to MSVC), without any POSIX compatibility 
layers on top. I've tried rewiring the build system incrementally in 
the meantime on my end to see what areas would be of interest and it's 
now failing when hitting POSIX specific includes during make, hinting 
that (At least for gcc) compatibility has been traded for full native 
support for Windows APIs within the compiler, which may mitigate any 
issues slightly (That's also why I suggested initially to only allow 
for versions that link against the ucrt without any compatibility 
layers- Cygwin's toolchains which link against newlib and old MinGW 
binaries which link against msvcrt would just be an unnecessary 
headache). That said, whether aforementioned bugs will actually 
surface should the attempt be successful I don't really know yet


        Actually, I would seriously assume that any other compiler 
than VS on Windows will give much worse results.
        The VS compiler is battle-hardened. gcc and clang are only 
used by a very small enthusiastic hobbyist minority.


The Windows ports of both compilers do keep the same optimizations and 
code generation quality as they do on other platforms from what I know 
though, it's mainly the linkers that have been modified to generate 
the PE format instead of the ELF on Linux. I digress, I don't have any 
results to show for this yet


I'm not sure I get the part about macOS strictly mapping to clang. 
Isn't there the option to swap to using gcc for macOS?

No, Apple removed that option years ago.

/Magnus


best regards,
Julian

On Sat, Mar 12, 2022 at 1:50 AM Magnus Ihse Bursie 
 wrote:




On 2022-03-11 14:34, Julian Waters wrote:

Darn, seems like it'll be much harder than I expected. Since
multiple toolchains are supported for macOS and Linux, I assumed
a slight patch would help get it to work on Windows. Looking
through the stuff in make though, it appears a lot of the build
system implicitly expects the compiler for Windows to always be
Visual C++, which doesn't really help that much (Though the fact
that we can exclude many versions of gcc, such as Cygwin's and
old MinGW binaries helps a lot). The build process for the newer
Windows ports of gcc are surprisingly similar to Visual C++
though (Eg rc can be swapped out for windres) so this might
hopefully be something I can try exploring in the future (Gonna
look a bit harder at make and write what I can find back to this
mailing list in the meantime). It'd be interesting if benchmarks
of the JVM compiled with different compilers on Windows can be
compared side by side on the off chance this becomes a reality though


In theory OS and compiler toolchain are separate things. In
practice, in the JDK, they are not. There is basically a 1-to-1
mapping between toolchain and OS:
Windows <-> Visual Studio
macOS <-> XCode/clang
linux <-> gcc

(The one possible exception is that clang on linux is probably
feasible.)

After years and years on this, all kinds of assumptions has been
hard-coded, even if people try to do the right thing. But
sometimes it is not clear if you are checking for toolchain or os;
perhaps when linking with a specific library, or setting some define.

Any attempt to change this is, as I said, a *huge* undertaking.
*And* there will be tons of negative consequences for the code
base as a whole, when trying to differentiate between toolchain
and OS. So this will need to be seriously considered.

/Magnus



best regards,
Julian

On Fri, Mar 11, 2022 at 9:16 PM Magnus Ihse Bursie
 wrote:

On 2022-03-11 12:55, Julian Waters wrote:

> Hi all,
>
> How feasible would it be/much effort would it require to
support compiling
> with alternate toolchains on Windows besides Visual C++
(like the Windows
> ports of clang and gcc) if we restrict the allowed
toolchains to only those
> that link against the ucrt? (Toolchains linking against the
dated msvcrt
> would present too many issues to work with)

That'd be a huge undertaking. And any such patch would only
be accepted
into the code base if the organization behinded appeared
trustworthy in
their long-term commitment to keeping it 

Re: Supporting alternative toolchains on Windows

[Julian Waters]

I understand the concerns, seems like I grossly underestimated the
complexity such a task would entail. Though I would say the following can
only really be known if it's already been implemented:

Most likely there will be strange bugs with anything that requires
OS interaction (like dll loading and whatsnot)

To my knowledge the versions of said compilers that link against the
universal CRT utilize exactly the same Windows APIs (And corresponding
dlls) for OS related stuff that Visual C++ itself uses (Minus vcruntime,
which is specific to MSVC), without any POSIX compatibility layers on top.
I've tried rewiring the build system incrementally in the meantime on my
end to see what areas would be of interest and it's now failing when
hitting POSIX specific includes during make, hinting that (At least for
gcc) compatibility has been traded for full native support for Windows APIs
within the compiler, which may mitigate any issues slightly (That's also
why I suggested initially to only allow for versions that link against the
ucrt without any compatibility layers- Cygwin's toolchains which link
against newlib and old MinGW binaries which link against msvcrt would just
be an unnecessary headache). That said, whether aforementioned bugs will
actually surface should the attempt be successful I don't really know yet

Actually, I would seriously assume that any other compiler than VS
on Windows will give much worse results.
The VS compiler is battle-hardened. gcc and clang are only used by
a very small enthusiastic hobbyist minority.

The Windows ports of both compilers do keep the same optimizations and code
generation quality as they do on other platforms from what I know though,
it's mainly the linkers that have been modified to generate the PE format
instead of the ELF on Linux. I digress, I don't have any results to show
for this yet

I'm not sure I get the part about macOS strictly mapping to clang. Isn't
there the option to swap to using gcc for macOS?

best regards,
Julian

On Sat, Mar 12, 2022 at 1:50 AM Magnus Ihse Bursie <
magnus.ihse.bur...@oracle.com> wrote:

>
>
> On 2022-03-11 14:34, Julian Waters wrote:
>
> Darn, seems like it'll be much harder than I expected. Since multiple
> toolchains are supported for macOS and Linux, I assumed a slight patch
> would help get it to work on Windows. Looking through the stuff in make
> though, it appears a lot of the build system implicitly expects the
> compiler for Windows to always be Visual C++, which doesn't really help
> that much (Though the fact that we can exclude many versions of gcc, such
> as Cygwin's and old MinGW binaries helps a lot). The build process for the
> newer Windows ports of gcc are surprisingly similar to Visual C++ though
> (Eg rc can be swapped out for windres) so this might hopefully be something
> I can try exploring in the future (Gonna look a bit harder at make and
> write what I can find back to this mailing list in the meantime). It'd be
> interesting if benchmarks of the JVM compiled with different compilers on
> Windows can be compared side by side on the off chance this becomes a
> reality though
>
>
> In theory OS and compiler toolchain are separate things. In practice, in
> the JDK, they are not. There is basically a 1-to-1 mapping between
> toolchain and OS:
> Windows <-> Visual Studio
> macOS <-> XCode/clang
> linux <-> gcc
>
> (The one possible exception is that clang on linux is probably feasible.)
>
> After years and years on this, all kinds of assumptions has been
> hard-coded, even if people try to do the right thing. But sometimes it is
> not clear if you are checking for toolchain or os; perhaps when linking
> with a specific library, or setting some define.
>
> Any attempt to change this is, as I said, a *huge* undertaking. *And*
> there will be tons of negative consequences for the code base as a whole,
> when trying to differentiate between toolchain and OS. So this will need to
> be seriously considered.
>
> /Magnus
>
>
> best regards,
> Julian
>
> On Fri, Mar 11, 2022 at 9:16 PM Magnus Ihse Bursie <
> magnus.ihse.bur...@oracle.com> wrote:
>
>> On 2022-03-11 12:55, Julian Waters wrote:
>>
>> > Hi all,
>> >
>> > How feasible would it be/much effort would it require to support
>> compiling
>> > with alternate toolchains on Windows besides Visual C++ (like the
>> Windows
>> > ports of clang and gcc) if we restrict the allowed toolchains to only
>> those
>> > that link against the ucrt? (Toolchains linking against the dated msvcrt
>> > would present too many issues to work with)
>>
>> That'd be a huge undertaking. And any such patch would only be accepted
>> into the code base if the organization behinded appeared trustworthy in
>> their long-term commitment to keeping it working.
>>
>> /Magnus
>>
>>
>>
>

Re: Supporting alternative toolchains on Windows

[Magnus Ihse Bursie]

On 2022-03-11 18:18, Thomas Stüfe wrote:

As Dalibor wrote, I would not expect too many performance surprises.


Actually, I would seriously assume that any other compiler than VS on 
Windows will give much worse results. The VS compiler is 
battle-hardened. Gcc and clang is only used by a very small enthusiastic 
hobbyist minority. There is no reason to believe it will be even close 
to optimized. Most likely there will be strange bugs with anything that 
requires OS interaction (like dll loading and whatsnot), even if the 
basic code generation part of the compiler should be identical for all 
OSes running on the same CPU.




That said, a more pragmatic approach may be to create a shim layer for 
visual studio compiler and linker, e.g. a fake "cl.exe" and "link.exe" 
that translate VC++ options and paths to whatever toolchain you like. 
That way you don't have to touch the OpenJDK make at all. I know it 
works in principle since I have such a thing in the past, albeit for a 
different product and a different target toolchain.


But there is oh so much to "convert" in that case. Just try grepping for 
"pragma" in hotspot, for a start. :)


I seriously do not believe this approach would be any ounce simpler, at 
least not for the JDK.


I did actually start to try to get a "hybrid" mode working, where the 
Visual Studio CL compiler would be run using wine on linux. (This was 
previously not allowed by MS licence, but they changed it some years 
ago.) That'd had allowed for linux users to cross-compile to Windows, 
and make a quick check that you did not break the build. But even this 
small step -- keeping the Windows compiler, but running in a linux 
environment -- turned out to be so full of tricky details that I gave up 
on it.


/Magnus



I would not be surprised if such a thing exists already. It seems such 
an obvious idea.


Cheers, Thomas

On Fri, Mar 11, 2022 at 2:35 PM Julian Waters 
 wrote:


Darn, seems like it'll be much harder than I expected. Since multiple
toolchains are supported for macOS and Linux, I assumed a slight patch
would help get it to work on Windows. Looking through the stuff in
make
though, it appears a lot of the build system implicitly expects the
compiler for Windows to always be Visual C++, which doesn't really
help
that much (Though the fact that we can exclude many versions of
gcc, such
as Cygwin's and old MinGW binaries helps a lot). The build process
for the
newer Windows ports of gcc are surprisingly similar to Visual C++
though
(Eg rc can be swapped out for windres) so this might hopefully be
something
I can try exploring in the future (Gonna look a bit harder at make and
write what I can find back to this mailing list in the meantime).
It'd be
interesting if benchmarks of the JVM compiled with different
compilers on
Windows can be compared side by side on the off chance this becomes a
reality though

best regards,
Julian

On Fri, Mar 11, 2022 at 9:16 PM Magnus Ihse Bursie <
magnus.ihse.bur...@oracle.com> wrote:

> On 2022-03-11 12:55, Julian Waters wrote:
>
> > Hi all,
> >
> > How feasible would it be/much effort would it require to support
> compiling
> > with alternate toolchains on Windows besides Visual C++ (like
the Windows
> > ports of clang and gcc) if we restrict the allowed toolchains
to only
> those
> > that link against the ucrt? (Toolchains linking against the
dated msvcrt
> > would present too many issues to work with)
>
> That'd be a huge undertaking. And any such patch would only be
accepted
> into the code base if the organization behinded appeared
trustworthy in
> their long-term commitment to keeping it working.
>
> /Magnus
>
>
>

Re: Supporting alternative toolchains on Windows

[Magnus Ihse Bursie]

On 2022-03-11 14:34, Julian Waters wrote:
Darn, seems like it'll be much harder than I expected. Since multiple 
toolchains are supported for macOS and Linux, I assumed a slight patch 
would help get it to work on Windows. Looking through the stuff in 
make though, it appears a lot of the build system implicitly expects 
the compiler for Windows to always be Visual C++, which doesn't really 
help that much (Though the fact that we can exclude many versions of 
gcc, such as Cygwin's and old MinGW binaries helps a lot). The build 
process for the newer Windows ports of gcc are surprisingly similar to 
Visual C++ though (Eg rc can be swapped out for windres) so this might 
hopefully be something I can try exploring in the future (Gonna look a 
bit harder at make and write what I can find back to this mailing list 
in the meantime). It'd be interesting if benchmarks of the JVM 
compiled with different compilers on Windows can be compared side by 
side on the off chance this becomes a reality though


In theory OS and compiler toolchain are separate things. In practice, in 
the JDK, they are not. There is basically a 1-to-1 mapping between 
toolchain and OS:

Windows <-> Visual Studio
macOS <-> XCode/clang
linux <-> gcc

(The one possible exception is that clang on linux is probably feasible.)

After years and years on this, all kinds of assumptions has been 
hard-coded, even if people try to do the right thing. But sometimes it 
is not clear if you are checking for toolchain or os; perhaps when 
linking with a specific library, or setting some define.


Any attempt to change this is, as I said, a *huge* undertaking. *And* 
there will be tons of negative consequences for the code base as a 
whole, when trying to differentiate between toolchain and OS. So this 
will need to be seriously considered.


/Magnus



best regards,
Julian

On Fri, Mar 11, 2022 at 9:16 PM Magnus Ihse Bursie 
 wrote:


On 2022-03-11 12:55, Julian Waters wrote:

> Hi all,
>
> How feasible would it be/much effort would it require to support
compiling
> with alternate toolchains on Windows besides Visual C++ (like
the Windows
> ports of clang and gcc) if we restrict the allowed toolchains to
only those
> that link against the ucrt? (Toolchains linking against the
dated msvcrt
> would present too many issues to work with)

That'd be a huge undertaking. And any such patch would only be
accepted
into the code base if the organization behinded appeared
trustworthy in
their long-term commitment to keeping it working.

/Magnus

Re: Supporting alternative toolchains on Windows

[Dalibor Topic]

On 11.03.2022 18:18, Thomas Stüfe wrote:

As Dalibor wrote, I would not expect too many performance surprises.

That said, a more pragmatic approach may be to create a shim layer for
visual studio compiler and linker, e.g. a fake "cl.exe" and "link.exe" that
translate VC++ options and paths to whatever toolchain you like. That way
you don't have to touch the OpenJDK make at all. I know it works in
principle since I have such a thing in the past, albeit for a different
product and a different target toolchain.

I would not be surprised if such a thing exists already. It seems such an
obvious idea.


For clang there is https://clang.llvm.org/docs/UsersManual.html#clang-cl 
but I have not tried it myself. But then again the linker step might be 
problematic.


cheers,
dalibor topic



Cheers, Thomas

On Fri, Mar 11, 2022 at 2:35 PM Julian Waters 
wrote:


Darn, seems like it'll be much harder than I expected. Since multiple
toolchains are supported for macOS and Linux, I assumed a slight patch
would help get it to work on Windows. Looking through the stuff in make
though, it appears a lot of the build system implicitly expects the
compiler for Windows to always be Visual C++, which doesn't really help
that much (Though the fact that we can exclude many versions of gcc, such
as Cygwin's and old MinGW binaries helps a lot). The build process for the
newer Windows ports of gcc are surprisingly similar to Visual C++ though
(Eg rc can be swapped out for windres) so this might hopefully be something
I can try exploring in the future (Gonna look a bit harder at make and
write what I can find back to this mailing list in the meantime). It'd be
interesting if benchmarks of the JVM compiled with different compilers on
Windows can be compared side by side on the off chance this becomes a
reality though

best regards,
Julian

On Fri, Mar 11, 2022 at 9:16 PM Magnus Ihse Bursie <
magnus.ihse.bur...@oracle.com> wrote:


On 2022-03-11 12:55, Julian Waters wrote:


Hi all,

How feasible would it be/much effort would it require to support

compiling

with alternate toolchains on Windows besides Visual C++ (like the

Windows

ports of clang and gcc) if we restrict the allowed toolchains to only

those

that link against the ucrt? (Toolchains linking against the dated

msvcrt

would present too many issues to work with)


That'd be a huge undertaking. And any such patch would only be accepted
into the code base if the organization behinded appeared trustworthy in
their long-term commitment to keeping it working.

/Magnus







--
 Dalibor Topic
Consulting Product Manager
Phone: +494089091214 , Mobile: +491737185961


Oracle Global Services Germany GmbH
Hauptverwaltung: Riesstr. 25, D-80992 München
Registergericht: Amtsgericht München, HRB 246209
Geschäftsführer: Ralf Herrmann

Re: Supporting alternative toolchains on Windows

[Thomas Stüfe]

As Dalibor wrote, I would not expect too many performance surprises.

That said, a more pragmatic approach may be to create a shim layer for
visual studio compiler and linker, e.g. a fake "cl.exe" and "link.exe" that
translate VC++ options and paths to whatever toolchain you like. That way
you don't have to touch the OpenJDK make at all. I know it works in
principle since I have such a thing in the past, albeit for a different
product and a different target toolchain.

I would not be surprised if such a thing exists already. It seems such an
obvious idea.

Cheers, Thomas

On Fri, Mar 11, 2022 at 2:35 PM Julian Waters 
wrote:

> Darn, seems like it'll be much harder than I expected. Since multiple
> toolchains are supported for macOS and Linux, I assumed a slight patch
> would help get it to work on Windows. Looking through the stuff in make
> though, it appears a lot of the build system implicitly expects the
> compiler for Windows to always be Visual C++, which doesn't really help
> that much (Though the fact that we can exclude many versions of gcc, such
> as Cygwin's and old MinGW binaries helps a lot). The build process for the
> newer Windows ports of gcc are surprisingly similar to Visual C++ though
> (Eg rc can be swapped out for windres) so this might hopefully be something
> I can try exploring in the future (Gonna look a bit harder at make and
> write what I can find back to this mailing list in the meantime). It'd be
> interesting if benchmarks of the JVM compiled with different compilers on
> Windows can be compared side by side on the off chance this becomes a
> reality though
>
> best regards,
> Julian
>
> On Fri, Mar 11, 2022 at 9:16 PM Magnus Ihse Bursie <
> magnus.ihse.bur...@oracle.com> wrote:
>
> > On 2022-03-11 12:55, Julian Waters wrote:
> >
> > > Hi all,
> > >
> > > How feasible would it be/much effort would it require to support
> > compiling
> > > with alternate toolchains on Windows besides Visual C++ (like the
> Windows
> > > ports of clang and gcc) if we restrict the allowed toolchains to only
> > those
> > > that link against the ucrt? (Toolchains linking against the dated
> msvcrt
> > > would present too many issues to work with)
> >
> > That'd be a huge undertaking. And any such patch would only be accepted
> > into the code base if the organization behinded appeared trustworthy in
> > their long-term commitment to keeping it working.
> >
> > /Magnus
> >
> >
> >
>

Re: Supporting alternative toolchains on Windows

[Dalibor Topic]

On 11.03.2022 14:34, Julian Waters wrote:

Darn, seems like it'll be much harder than I expected. Since multiple
toolchains are supported for macOS and Linux, I assumed a slight patch
would help get it to work on Windows. Looking through the stuff in make
though, it appears a lot of the build system implicitly expects the
compiler for Windows to always be Visual C++, which doesn't really help
that much (Though the fact that we can exclude many versions of gcc, such
as Cygwin's and old MinGW binaries helps a lot). The build process for the
newer Windows ports of gcc are surprisingly similar to Visual C++ though
(Eg rc can be swapped out for windres) so this might hopefully be something
I can try exploring in the future (Gonna look a bit harder at make and
write what I can find back to this mailing list in the meantime). It'd be
interesting if benchmarks of the JVM compiled with different compilers on
Windows can be compared side by side on the off chance this becomes a
reality though


I wouldn't necessarily expect large differences between builds compiled 
with different C/C++ compilers on industry benchmarks like SpecJBB, 
since the JVM will spend most of its time executing code it JIT-compiled 
itself.


I think more interesting from a developer/CI perspective would be 
potential differences in compilation speed, but I would naively assume 
those to be dominated by the linker, rather than the compiler itself.


cheers,
dalibor topic



best regards,
Julian

On Fri, Mar 11, 2022 at 9:16 PM Magnus Ihse Bursie <
magnus.ihse.bur...@oracle.com> wrote:


On 2022-03-11 12:55, Julian Waters wrote:


Hi all,

How feasible would it be/much effort would it require to support

compiling

with alternate toolchains on Windows besides Visual C++ (like the Windows
ports of clang and gcc) if we restrict the allowed toolchains to only

those

that link against the ucrt? (Toolchains linking against the dated msvcrt
would present too many issues to work with)


That'd be a huge undertaking. And any such patch would only be accepted
into the code base if the organization behinded appeared trustworthy in
their long-term commitment to keeping it working.

/Magnus





--
 Dalibor Topic
Consulting Product Manager
Phone: +494089091214 , Mobile: +491737185961


Oracle Global Services Germany GmbH
Hauptverwaltung: Riesstr. 25, D-80992 München
Registergericht: Amtsgericht München, HRB 246209
Geschäftsführer: Ralf Herrmann

Re: Supporting alternative toolchains on Windows

[Julian Waters]

Darn, seems like it'll be much harder than I expected. Since multiple
toolchains are supported for macOS and Linux, I assumed a slight patch
would help get it to work on Windows. Looking through the stuff in make
though, it appears a lot of the build system implicitly expects the
compiler for Windows to always be Visual C++, which doesn't really help
that much (Though the fact that we can exclude many versions of gcc, such
as Cygwin's and old MinGW binaries helps a lot). The build process for the
newer Windows ports of gcc are surprisingly similar to Visual C++ though
(Eg rc can be swapped out for windres) so this might hopefully be something
I can try exploring in the future (Gonna look a bit harder at make and
write what I can find back to this mailing list in the meantime). It'd be
interesting if benchmarks of the JVM compiled with different compilers on
Windows can be compared side by side on the off chance this becomes a
reality though

best regards,
Julian

On Fri, Mar 11, 2022 at 9:16 PM Magnus Ihse Bursie <
magnus.ihse.bur...@oracle.com> wrote:

> On 2022-03-11 12:55, Julian Waters wrote:
>
> > Hi all,
> >
> > How feasible would it be/much effort would it require to support
> compiling
> > with alternate toolchains on Windows besides Visual C++ (like the Windows
> > ports of clang and gcc) if we restrict the allowed toolchains to only
> those
> > that link against the ucrt? (Toolchains linking against the dated msvcrt
> > would present too many issues to work with)
>
> That'd be a huge undertaking. And any such patch would only be accepted
> into the code base if the organization behinded appeared trustworthy in
> their long-term commitment to keeping it working.
>
> /Magnus
>
>
>

Re: Supporting alternative toolchains on Windows

[Magnus Ihse Bursie]

On 2022-03-11 12:55, Julian Waters wrote:


Hi all,

How feasible would it be/much effort would it require to support compiling
with alternate toolchains on Windows besides Visual C++ (like the Windows
ports of clang and gcc) if we restrict the allowed toolchains to only those
that link against the ucrt? (Toolchains linking against the dated msvcrt
would present too many issues to work with)


That'd be a huge undertaking. And any such patch would only be accepted 
into the code base if the organization behinded appeared trustworthy in 
their long-term commitment to keeping it working.


/Magnus