Re: Is anyone able to build on Win 7
This sounds very interesting. I have no experience with MinGW/MSYS, but will most definitely start looking into it asap and see how hard it would be to get the new build compatible with it. /Erik On 2012-03-08 19:00, Volker Simonis wrote: This thread will probably never end (Windows 2046 :) So I did more test.. - I wanted to compare with MKS and the first thing I hit on was a bug in MKS's 9.4 version of cpio (CFS# 32408--- cpio can not handle files which are ReadOnly). And it's expensive and installation and license handling is PITA if you use several virtual machiines.. - Still couldn't find the reason why the build hangs with Cygwin 1.7.10 Finally I decided to try something new - MinGW/MSYS. And indeed - it worked, it's nearly as fast as MKS, it can use the default make which comes with the MinGW/Installation. Read the glory details at: http://mail.openjdk.java.net/pipermail/build-dev/2012-March/005729.html Please feel free to test, review and (hopfully) submit it. The changes are intentionally against the old, traditional build system to fix the mentioned Cygwin problems and simplify the Windows build just now. As next steps I see the following points: - integrate MinGW/MSYS with the new build system - completely remove nmake from the HotSpot build and use prallel GNU make like on Linux (I know this works and that it's faster - just have to build a OpenJDK patch) Any comments? Volker
Re: Is anyone able to build on Win 7
Volker, Thank you for amazing work! Just a curious: 1. Do you have any performance numbers for hotspot built with MinGW? Is it as fast as one built with MS VC? 2. Did you see any issues with third-party JNI code? Thank you! -Dmitry On 2012-03-08 22:00, Volker Simonis wrote: This thread will probably never end (Windows 2046 :) So I did more test.. - I wanted to compare with MKS and the first thing I hit on was a bug in MKS's 9.4 version of cpio (CFS# 32408--- cpio can not handle files which are ReadOnly). And it's expensive and installation and license handling is PITA if you use several virtual machiines.. - Still couldn't find the reason why the build hangs with Cygwin 1.7.10 Finally I decided to try something new - MinGW/MSYS. And indeed - it worked, it's nearly as fast as MKS, it can use the default make which comes with the MinGW/Installation. Read the glory details at: http://mail.openjdk.java.net/pipermail/build-dev/2012-March/005729.html Please feel free to test, review and (hopfully) submit it. The changes are intentionally against the old, traditional build system to fix the mentioned Cygwin problems and simplify the Windows build just now. As next steps I see the following points: - integrate MinGW/MSYS with the new build system - completely remove nmake from the HotSpot build and use prallel GNU make like on Linux (I know this works and that it's faster - just have to build a OpenJDK patch) Any comments? Volker On Wed, Feb 15, 2012 at 1:10 PM, Fredrik Öhrström fredrik.ohrst...@oracle.com wrote: - kelly.oh...@oracle.com skrev: So I'm with you on the stat() theory, makes a great deal of sense. The stat theory is very interesting, but it is unclear to me if it explains all of the problem. I setup a quadruple boot x86_64 machine with 4GB of ram and 4 cores: Winxp 32bit Win7 64bit Solaris 64bit Ubuntu 64bit And tested the build times on the different OS:es. Ubuntu Fastest by far. Solaris, slower, but this is only because of bad CC performance. Winxp, even slower but still ok. Win7, ridiculously slow. The configure script prints one line per second! Clearly, just running a bash script in cygwin/win7/64bit is problematic. If we get 10% speedup from dash, then that is not going to help because the slowdown is a factor 10. Could someone try out the difference between a 32bit win7 clean install and a 64 bit win7 clean install when running the latest cygwin and just the build-infra/jdk8/common/autoconf/configure script? (My patience for installing many OSes into the same box, just ran out. And virtualization testing can give a hint, but cannot be entirely trusted.) //Fredrik -- Dmitry Samersoff Java Hotspot development team, SPB04 * There will come soft rains ...
Re: Is anyone able to build on Win 7
Hi Dimitry, I I haven't build the HotSpot with MinGW - that would be a completely different project. I've just used the MinGW/MSYS environment (i.e. shell, make, various Unix-tools) to build the OpenJDK. The build still uses MSVC as compiler. Building the HotSpot with the GCC from MinGW should be possible with a combination of the current Windows makefiles and the gcc compiler settings from the linux makefiles although it would require quiet a bit of hacking. I'm not so sure however if the whole JDK could be build with GCC from MinGW because I'm not sure if MinGW has include files and wrappers for all the required Windows libraries - but honestly speaking I havn't looked into it until now. Regards, Volker On Fri, Mar 9, 2012 at 10:06 AM, Dmitry Samersoff dmitry.samers...@oracle.com wrote: Volker, Thank you for amazing work! Just a curious: 1. Do you have any performance numbers for hotspot built with MinGW? Is it as fast as one built with MS VC? 2. Did you see any issues with third-party JNI code? Thank you! -Dmitry On 2012-03-08 22:00, Volker Simonis wrote: This thread will probably never end (Windows 2046 :) So I did more test.. - I wanted to compare with MKS and the first thing I hit on was a bug in MKS's 9.4 version of cpio (CFS# 32408--- cpio can not handle files which are ReadOnly). And it's expensive and installation and license handling is PITA if you use several virtual machiines.. - Still couldn't find the reason why the build hangs with Cygwin 1.7.10 Finally I decided to try something new - MinGW/MSYS. And indeed - it worked, it's nearly as fast as MKS, it can use the default make which comes with the MinGW/Installation. Read the glory details at: http://mail.openjdk.java.net/pipermail/build-dev/2012-March/005729.html Please feel free to test, review and (hopfully) submit it. The changes are intentionally against the old, traditional build system to fix the mentioned Cygwin problems and simplify the Windows build just now. As next steps I see the following points: - integrate MinGW/MSYS with the new build system - completely remove nmake from the HotSpot build and use prallel GNU make like on Linux (I know this works and that it's faster - just have to build a OpenJDK patch) Any comments? Volker On Wed, Feb 15, 2012 at 1:10 PM, Fredrik Öhrström fredrik.ohrst...@oracle.com wrote: - kelly.oh...@oracle.com skrev: So I'm with you on the stat() theory, makes a great deal of sense. The stat theory is very interesting, but it is unclear to me if it explains all of the problem. I setup a quadruple boot x86_64 machine with 4GB of ram and 4 cores: Winxp 32bit Win7 64bit Solaris 64bit Ubuntu 64bit And tested the build times on the different OS:es. Ubuntu Fastest by far. Solaris, slower, but this is only because of bad CC performance. Winxp, even slower but still ok. Win7, ridiculously slow. The configure script prints one line per second! Clearly, just running a bash script in cygwin/win7/64bit is problematic. If we get 10% speedup from dash, then that is not going to help because the slowdown is a factor 10. Could someone try out the difference between a 32bit win7 clean install and a 64 bit win7 clean install when running the latest cygwin and just the build-infra/jdk8/common/autoconf/configure script? (My patience for installing many OSes into the same box, just ran out. And virtualization testing can give a hint, but cannot be entirely trusted.) //Fredrik -- Dmitry Samersoff Java Hotspot development team, SPB04 * There will come soft rains ...
First push of build-infra to JDK8 coming up
Hi all, The build-infra project is getting near it's first milestone. We have a build system that is workable, and where most of the code is converted to the new system. Not all of the old build functionality is available, and not all platforms work equally well. But we have enough of well-functioning code that we want to expose it to a wider audience. We have already started to prepare for a push to JDK8. Kelly has merged the source-drop removal of JAXP and JAXWS into the build forest. Erik and Fredrik has pushed a series of patches to the Hotspot Makefiles preparing them for working with both the old and the new build system. We still have a few more of such patches (one for each of the Hotspot, Corba, Langtools and JDK repositories), but when they are pushed, we plan to make the first major push to JDK8. This will be in the form of added code in new, separate subdirectories. This push will in no way affect the old build, but will only make the new build system available for those eager to try it out. The plan is to do this push no later than March 21, but hopefully earlier. We will present more information about this push shortly, including a WebRev. Stay tuned! /Magnus
Re: RFR (M): Enable OpenJDK builds on Windows with MinGW/MSYS
On Thu, Mar 8, 2012 at 8:58 PM, Kelly O'Hair kelly.oh...@oracle.com wrote: On Mar 8, 2012, at 9:39 AM, Volker Simonis wrote: Hi, please review the following changes which make it possible to use MinGW/MSYS as underlying build environment for Windows. Because the changes touch several repositories I prepared three webrevs, all relative to http://hg.openjdk.java.net/jdk8/build For the main repository: http://cr.openjdk.java.net/~simonis/MinGW_MSYS.v1/ I had originally purged all mention of MKS from this README because we wanted to stop using it in favor of CYGWIN, and only maintain one way to build on Windows. That never happened internally, many kept using MKS because it was so much faster. So as much as I hate seeing the MKS instructions being added, I understand why you felt compelled to add them, and see no obvious inaccuracies We didn't have any plans to continue to support MKS with the new build infrastructure work. It creates a complication that we wanted to avoid, but at the same time we are running into the CYGWIN is too slow, especially on 64bit Windows problem. So we will need to discuss this on the build-infra-dev alias. Since this change is just README, and I see nothing wrong with regards to your changes, it seems ok, with the caveat that if we end up really abandoning MKS, we may need to purge the MKS references. I just don't know what will happen here right now. For the HotSpot repository: http://cr.openjdk.java.net/~simonis/MinGW_MSYS_hotspot.v1/ Seems fine. This change could probably proceed regardless of the others I think. But someone from the hotspot team may need to help here or maybe I could do it... Was the /option to -option a style issue, or was it required somehow? Just curious, I have no objection to making it consistent. Yes, it is required because MinGW tries to be smart when calling non-MinGW executables and interprets slash-options as paths (e.g. /I may end up as I:\) See http://mingw.org/wiki/Posix_path_conversion for more information. Also, the quotes on the paths.. any particular reason why? This was very tricky to get right on all three environments and the quoted version basically just worked. The problem again is MinGWs automatic path conversion: if you give the top-level make something like ALT_BOOTDIR=C:/Java/jdk1.7.0_02 it implicitly converts it into ALT_BOOTDIR=/c/Java/jdk1.7.0_02 for the next recursive make call. Later, when nmake will be called, it converts it back into C:/Java/jdk1.7.0_02 which fools nmake. But if the path is quoted, all works well. For the JDK repository http://cr.openjdk.java.net/~simonis/MinGW_MSYS_jdk.v1/ Seems fine but unfortunately, we have other closed repos that might be impacted here. The cpio change concerns me, not that I'm in love with cpio, I tried to get rid of it a long time ago, but it was so much faster than tar for some reason. So I'll need to make sure the entire Oracle JDK build process is ok with these changes before this could go in. I didn't realized any performance problems and tar is already used in some places for copying files so I thought it will be ok. Hope your internal repos work fine with the change, otherwise please let me know. Right now I'm not so concerned about using MinGW/MSYS as making sure the existing builds are not broken. But I am looking forward to using MinGW/MSYS soon. I may need to enlist someone to do some image comparisons and double check that the end result has not changed too. Absolutely. Please keep me informed if you encounter any problems. Please be so kind and also create three bug ids for the corresponding changes. Created: 7152336: Enable builds on Windows with MinGW/MSYS I think we can use the same CR for all repos. Unless you really want 3 CRs. That's your playground:) I'm happy either way. The changes are minimal in the sense that I did just enough changes to be able to build on Windows with either one of the three Unix emulation environments (MKS, Cygwin, MinGW/MSYS). The main benefits of the new MinGW/MSYS based Windows build are: - no problems with make - one can simply use the make tool which comes with MinGW/MSYS - with the current MinGW/MSYS version the build is known to succeed, while the newest Cygwin 1.7.10 version is known to break the OpenJDK build (see: http://old.nabble.com/Is-anyone-able-to-build-on-Win-7-td33196055.html) - speed - the MinGW/MSYS build times are comparable to the MKS build times and is at least 50% faster than the Cygwin build times (see numbers below) That 50% faster is great news. The changes are intentionally against the old, traditional build system to fix the mentioned Cygwin problems and simplify the Windows build just now. I don't expect any problems in adapting the new build infrastructure to MinGW/MSYS as well in a follow-up step. More information on each change can be found in the corresponding
Re: RFR (M): Enable OpenJDK builds on Windows with MinGW/MSYS
On Fri, Mar 9, 2012 at 12:14 AM, David Holmes david.hol...@oracle.com wrote: Hi Volker, On 9/03/2012 3:39 AM, Volker Simonis wrote: please review the following changes which make it possible to use MinGW/MSYS as underlying build environment for Windows. Because the changes touch several repositories I prepared three webrevs, all relative to http://hg.openjdk.java.net/jdk8/build For the main repository: http://cr.openjdk.java.net/~simonis/MinGW_MSYS.v1/ Saw one typo: fastes fixed For the HotSpot repository: http://cr.openjdk.java.net/~simonis/MinGW_MSYS_hotspot.v1/ I'm mainly looking at this from the does it break anything perspective, but note that in make/windows/makefiles/defs.make SYSTEM_UNAME is only set when doing a full build (JDK/control). If you just try to build hotspot this will not be set, so I expect you will want to change this. good catch! fixed. I also found another problem if building HotSpot standalone and the output path does not exist. In that case, the computation of ABS_OUTPUTDIR for MinGW and MKS will fail for the first invocation because the directory is only created after the second, recursive invocation: MinGW: ABS_OUTPUTDIR := $(shell $(CD) $(OUTPUTDIR);$(PWD)) MKS: ABS_OUTPUTDIR := $(subst /,\\,$(shell $(CD) $(OUTPUTDIR);$(PWD))) So I changed these to MinGW: ABS_OUTPUTDIR := $(shell mkdir -p $(OUTPUTDIR); $(CD) $(OUTPUTDIR);$(PWD)) MKS: ABS_OUTPUTDIR := $(subst /,\\,$(shell mkdir -p $(OUTPUTDIR); $(CD) $(OUTPUTDIR);$(PWD))) which is more robust. I was also wondering about the / to - changes but see in the README-builds.html changes that MSYS might otherwise misinterpret them. make/windows/makefiles/rules.make As Kelly said: why the quotes? Does MinGW not handle spaces in paths the way Cygwin does? make/windows/makefiles/sa.make Please see my answer to Kellys mail. So changes from \ to / . Not sure about this. Seems odd that we had a mix of \ and / to start with so I can't help but wonder if there was some Cygwin issue that this was addressing. I guess as long as it works then it is okay in that sense. This was exactly my approach here:) I'll collect the other comments and prepare an updated webrev next week. Thanks, Volker Cheers, David --- For the JDK repository http://cr.openjdk.java.net/~simonis/MinGW_MSYS_jdk.v1/ Please be so kind and also create three bug ids for the corresponding changes. The changes are minimal in the sense that I did just enough changes to be able to build on Windows with either one of the three Unix emulation environments (MKS, Cygwin, MinGW/MSYS). The main benefits of the new MinGW/MSYS based Windows build are: - no problems with make - one can simply use the make tool which comes with MinGW/MSYS - with the current MinGW/MSYS version the build is known to succeed, while the newest Cygwin 1.7.10 version is known to break the OpenJDK build (see: http://old.nabble.com/Is-anyone-able-to-build-on-Win-7-td33196055.html) - speed - the MinGW/MSYS build times are comparable to the MKS build times and is at least 50% faster than the Cygwin build times (see numbers below) The changes are intentionally against the old, traditional build system to fix the mentioned Cygwin problems and simplify the Windows build just now. I don't expect any problems in adapting the new build infrastructure to MinGW/MSYS as well in a follow-up step. More information on each change can be found in the corresponding webrevs. Timing results for the different build environments (MKS ver. 9.4, Cygwin ver. 1.7.9, MinGW/MSYS 1.0.17) for a full JDK 8 product build on a freshly booted, dual core i7 notebook with 8GB RAM, HDD and 64-bit Windows 7:/p MKS -- Build times -- Target all_product_build Start 2012-02-29 19:19:26 End 2012-02-29 20:07:44 00:03:10 corba 00:03:44 hotspot 00:00:30 jaxp 00:00:37 jaxws 00:39:39 jdk 00:00:36 langtools 00:48:18 TOTAL - MinGW/MSYS -- Build times -- Target all_product_build Start 2012-02-29 18:16:31 End 2012-02-29 19:10:59 00:02:57 corba 00:03:04 hotspot 00:00:32 jaxp 00:00:40 jaxws 00:46:36 jdk 00:00:36 langtools 00:54:28 TOTAL - Cygwin -- Build times -- Target all_product_build Start 2012-02-29 14:36:50 End 2012-02-29 16:05:50 00:05:47 corba 00:03:24 hotspot 00:00:33 jaxp 00:00:43 jaxws 01:17:45 jdk 00:00:39 langtools 01:29:00 TOTAL - Regards, Volker
Re: Is anyone able to build on Win 7
On Thu, Mar 8, 2012 at 8:00 PM, Kelly O'Hair kelly.oh...@oracle.com wrote: On Mar 8, 2012, at 10:00 AM, Volker Simonis wrote: This thread will probably never end (Windows 2046 :) So I did more test.. - I wanted to compare with MKS and the first thing I hit on was a bug in MKS's 9.4 version of cpio (CFS# 32408--- cpio can not handle files which are ReadOnly). And it's expensive and installation and license handling is PITA if you use several virtual machiines.. MKS 9.4 is seriously broken for us. I use 9.0p3 or 9.0p4. I filed a ticket with MKS on this issue months ago and have never heard back from them, and we have a support contract with them too. :^( - Still couldn't find the reason why the build hangs with Cygwin 1.7.10 that's a new one for me. When both MKS and CYGWIN are installed on the same system it can be tricky. After I install MKS I usually go in and take MKS out of the default PATH, and change SHELL to be just /usr/bin/sh (which appears to be more of a universal keyword than a path to a shell). Then I go shut down and disable all MKS services. Then when I want an MKS shell started up, I have some hacky PATH setting and exec of the MKS shell. I could send you the formula if you would like. I've just kept wishing MKS could go away for us... someday... And you have provide a light at the end of the tunnel. ;^) Thanks! Finally I decided to try something new - MinGW/MSYS. And indeed - it worked, it's nearly as fast as MKS, it can use the default make which comes with the MinGW/Installation. Read the glory details at: http://mail.openjdk.java.net/pipermail/build-dev/2012-March/005729.html Please feel free to test, review and (hopfully) submit it. The changes are intentionally against the old, traditional build system to fix the mentioned Cygwin problems and simplify the Windows build just now. As next steps I see the following points: - integrate MinGW/MSYS with the new build system - completely remove nmake from the HotSpot build and use prallel GNU make like on Linux (I know this works and that it's faster - just have to build a OpenJDK patch) Any comments? Fantastic stuff. I'll work on getting it in place. On replacing NMAKE, I agree with you however, I think NMAKE may be in cahoots with the VS compiler with regards to licensing checks or pre-compiled headers, the build is pretty fast. In my crude attempts in the past, I could never get anywhere close to the NMAKE build speed. Never completely understood why. :^( I don't understand the licensing checks problem you mention? Do you mean 'cl' can only be called from nmake? I havn't seen this problem, although our internal build without 'nmake' I was mentioning runs with the commercial and not the Express version of MSVS. For the precompiled headers issue we found a solution. I'll just have to port it to the OpenJDK. -kto Volker On Wed, Feb 15, 2012 at 1:10 PM, Fredrik Öhrström fredrik.ohrst...@oracle.com wrote: - kelly.oh...@oracle.com skrev: So I'm with you on the stat() theory, makes a great deal of sense. The stat theory is very interesting, but it is unclear to me if it explains all of the problem. I setup a quadruple boot x86_64 machine with 4GB of ram and 4 cores: Winxp 32bit Win7 64bit Solaris 64bit Ubuntu 64bit And tested the build times on the different OS:es. Ubuntu Fastest by far. Solaris, slower, but this is only because of bad CC performance. Winxp, even slower but still ok. Win7, ridiculously slow. The configure script prints one line per second! Clearly, just running a bash script in cygwin/win7/64bit is problematic. If we get 10% speedup from dash, then that is not going to help because the slowdown is a factor 10. Could someone try out the difference between a 32bit win7 clean install and a 64 bit win7 clean install when running the latest cygwin and just the build-infra/jdk8/common/autoconf/configure script? (My patience for installing many OSes into the same box, just ran out. And virtualization testing can give a hint, but cannot be entirely trusted.) //Fredrik
