Re: Cannot configure on Windows in Chinese Environment

[吴国璋]

Thanks Magnus for your opionion. Just one thing to point out: reaching out to Microsoft is not needed. Here is the reason:

If Visual Studio is installed with both English and another language pack, cl.exe will present itself in English while running configure.

If Visual Studio is installed without English language pack, cl.exe will present itself in an installed language.

Therefore now the issue is that the documentation needs to be improved.

Before I sent the original email, I tried to find out whether JDK welcomes other languages, and I found in the code that JDK did pay some effort for supporting them, so I assumed that an en-us environment is not required. Since it is not the case, then the documentation needs improvement.

In details, the documentation needs the following modification:

- It needs to advice developers to install Visual Studio English language pack, if they speak another language as their mother tongue;

- Switching the Windows system to English is not required (I tried it several times, just installing the language pack can solve)

2023年11月6日 21:00，Magnus Ihse Bursie <magnus.ihse.bur...@oracle.com>写道：

Let me expand a bit on what Erik says, and also somewhat contradict him. :-)

There is an open bug for documenting that en-US locale is needed to build the JDK on Windows: https://bugs.openjdk.org/browse/JDK-8264425

Unfortunately I have never gotten around to actually write this down in the docs. I'll try to prioritize it, since it's a simple fix and can help others in your position.

To expand on what Erik says: Yes, we have no principal opinion discriminating against non en-US locale support, and in general we accept patches that help users build in different scenarios. For non-Windows platforms, all user locales are supported, since we can set LC_ALL=C in the build and run with an international locale.

Unfortunately, this or any other method of temporarily changing the locale is not supported on Windows. :-( There have been several attempts over the year to overcome this problem, none of which has been successful. (You can search the archives of this mailing list for examples.) Therefore the conclusion, after the last such effort, was that we can only ever support building on en-US on Windows.

The problem on supporting the JDK on a different locale is that there are so many small things that can go wrong. Just to give an example on the top of my mind: a few weeks ago, the code that set the en-US locale to jtreg testing went AWOL. This caused some jtreg test runs to fail on a Swiss (iirc) locale, since that used a quote (´) as thousands separator, which caused parse errors.

Getting stuck on the version parsing of the compiler is just the very first steps on a road filled with pain and suffering.

So, to contradict with what Erik says: No, I don't think we should accept a patch that changes version determination for cl.exe from string parsing to compiling macros. 

There are several reasons: compiling code in configure is always tricky. This would be done before we have even determined what compiler we have or what version it is. We used to have a binary "fixpath" tool that was compiled early on, it was a constant source of trouble, and have now been removed due to those problems.

This would also add complexity to the configure script, since a trivial method (read and parse the output of running with --version or similar) that is shared by all compilers, now need to be replaced with a different method for cl.exe only.

If this was guaranteed to be the only things needed to make the JDK build and test on non-en-US locales, then I could probably consider it. But it is highly unlikely to be. And even if the build passes without error, I would be pretty wary about assuming that the build is actually identical to one built on the "official" locale.

I encourage you to get in touch with Microsoft and request them to add a solution similar to LC_ALL, so processes can run in a different locale than the default user locale. I realize a single voice does not convince them, but if the message is repeated over and over from all kinds of developers, it might have some effect.

/Magnus

On 2023-11-04 13:12, 吴 国璋 wrote:

If making the build work on different locales is accepted, then we can further discuss on this topic.

 

I would like to implement this with C macros instead of parsing the output, because the MSVC reference includes the C predefined macros, but does not include the output format.

 

In fact, Visual Studio supports 14 language packs, and I only know how cl.exe presents itself in English and Chinese, maybe also French. Maybe the sentence structure is also different in Korean or Japanese, I am not sure. With C macros the implementation will be less impacted by locale and more stable.

 

From: erik.joels...@oracle.com
Sent: 2023年11月3日 21:04
To: 吴 国璋; David Holmes; build-dev@openjdk.org
Subject: Re: Cannot configure on Windows in Chinese Environment

 

On 11/2/23 22:18, 吴 国璋 wrote:

If OpenJDK requires en-us environment, then nothing needs to be changed. Please ignore this thread.

I should clarify this a bit. We aren't against making the build work on different locales, but most of us are unable to verify that it keeps working on anything by US-English. If you are willing to put in the work to make it work in a Chinese environment, and the set of changes required seem reasonable, we would accept that contribution. Just be prepared to maintain that support over time, as it's quite likely that future changes may break it.

>> 

>> 1. Does JDK welcome localized Visual Studio?
>> I read the file `make/autoconf/toolchains.m4` and found the following comment:
>> >   elif test  "x$TOOLCHAIN_TYPE" = xmicrosoft; then
>> >     # There is no specific version flag, but all output starts with a version string.
>> >     # First line typically looks something like:
>> >     # Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.40219.01 for 80x86
>> >     # but the compiler name may vary depending on locale.
>> >     COMPILER_VERSION_OUTPUT=`$COMPILER 2>&1 1>/dev/null | $HEAD -n 1 | $TR -d '\r'`
>>
>> Therefore, it can be inferred that JDK knows that there are different localizations of Visual Studio and is ready for them. JDK thinks that maybe there will be different names of "Optimizing Compiler" or others. However, in some languages, the whole structure  of the sentence is completely different, not just the names.

So far we have only received contributions for different western type languages, so the current parsing logic has only been adapted for that.

>> 2. How can the problem be solved?
>> One solution is to change the way to parse the output of `cl.exe`. For example, JDK treat the last word separated by a blank as the target CPU, which is "x64" in English environment but "版" in Chinese environment. (`make/autoconf/toolchain.m4`, Lines 983  to 997.) We may use `grep` command to search for "x64" directly, and
> then the issue can be solved.
>> However, this solution is not good enough, because it is also based on parsing the output, which is intended to be read by human, not by scripts. (What if "x64" is changed into "64-bit" or "64 位" in a future version?)

If the output changes, we adapt the regex. We need explicit changes to support a new major version of Visual Studio anyway, so it would be done as part of those changes. The likelihood of it changing in a minor update is basically non existent.

>> 3. What is the best solution?
>> According to MSVC reference, a solid way to get the MSVC version and the target CPU is via predefined macros.
>> To get the MSVC version, we can use `_MSC_VER`. When Visual Studio 2019 is used, the macro evaluates between 1920 and 1929. When Visual Studio 2022 is used, the macro evaluates above 1930.
>> To get the target CPU, we can use `_M_X64`, `_M_IX86`, `_M_ARM` and `_M_ARM64`. For example, if the target CPU is x64, `_M_X64` will be evaluated to 100, and the other three macros are undefined.

Using the C preprocessor may work, but as Robbin pointed out, we would prefer if you used one of the Autoconf macros for generating the input files and running it if you were to go that route. However, I think I would prefer if you could just adapt the current logic for parsing the compiler version output.

/Erik