Re: Project Proposal: Build Infrastructure Changes
Hi Steve, The initial proposal was aimed to address issues in the existing build process that we felt could be improved in the short term. The idea was to share some ideas and experiments we have done and invite discussion on these approaches. Starting the project will give us a set of repos to do these experiments. The list that Kelly presented could be done in one or two short projects. We want to do this in a relatively short time, since these could potentially affect almost every makefile and build script. We want to minimize the impact that it will have on other projects, for example porting. If the use cases you present can be done in the same time as the list in the proposal, we might want to consider them for this project. If not we should be careful to not make choices that will make it difficult to support these use cases as possible next steps. This will be clearer when we discuss the approaches to solving the project goals. Mohan On May 3, 2011, at 11:18 AM, Steve Poole wrote: On 30/04/11 00:05, Kelly O'Hair wrote: On Apr 29, 2011, at 1:31 PM, Steve Poole wrote: On 26/04/11 15:54, Kelly O'Hair wrote: On Apr 26, 2011, at 12:59 AM, Steve Poole wrote: * Allow for use of more portable build tools (compilers etc.) where possible Can I add support for alternative JVM's ? Seems a bit out of scope to me. Sorry, it was a bit of a flippant one liner, I owe you more details. There are three usecases I see that require the OpenJDK build process to be modified to accommodate: The first is bootstrapping a build. I'd like to be able to build OpenJDK on a new platform without the need for a previous SDK build to be present. In this usecase it's possible that an simple interpreter based JVM would be sufficient (ie Zero) (or even maybe a cross compiling mode) The second is getting OpenJDK to build on a platform where a hotspot JVM doesn't exist and may never exist. As you guess I'm thinking of IBM platforms specifically. I'm don't expect to port Hotspot to AIX so I need to be able to make the OpenJDK build work with J9. The third (a variant of the 2nd) is where another JVM vendor wants to get OpenJDK working with their JVM - regardless of the availability of a Hotspot JVM on the target platform. To be clear. I'm not suggesting that this project step up to defining the interfaces between JVM and classes. This is simple pragmatics. The Hotspot JVM is the starting point for the mould and I would expect to make J9 (or any new JVM) fit into it as much as possible. However there will be changes needed. These are mostly simple, like parameterising JVM command line options, to the more complicated like separating out JVM intrinsic classes such as String.java, Object.java, Thread.java etc so that the right versions get build and packaged. I certainly can understand these needs, but it is still seems beyond the initial scope of this project. Maybe in a phase 2? Hmm, maybe doing all of them in one go may be a stretch :-) The 2nd usecase is fundamental though to supporting IBM platforms, something you mentioned you wanted to do. Is there any reason why realising that usecase is beyond phase 1? -kto
Re: Project Proposal: Build Infrastructure Changes
Timeframes... Humm... Initially I would like to start with a set of jdk7 repositories, go as far as we can with that, hopefully show some major improvements in overall build time, then set it aside for potential inclusion into jdk7u2 (that would require lots of verifications that the end results are the same), and then move it into jdk8 work and setting up for module building. -kto On Apr 29, 2011, at 9:30 PM, Georges Saab wrote: Hey Kelly, One thing that might be useful is to understand the kind of timeframe you are thinking about for this project? /GES On 29 apr 2011, at 16.05, Kelly O'Hair wrote: On Apr 29, 2011, at 1:31 PM, Steve Poole wrote: On 26/04/11 15:54, Kelly O'Hair wrote: On Apr 26, 2011, at 12:59 AM, Steve Poole wrote: * Allow for use of more portable build tools (compilers etc.) where possible Can I add support for alternative JVM's ? Seems a bit out of scope to me. Sorry, it was a bit of a flippant one liner, I owe you more details. There are three usecases I see that require the OpenJDK build process to be modified to accommodate: The first is bootstrapping a build. I'd like to be able to build OpenJDK on a new platform without the need for a previous SDK build to be present. In this usecase it's possible that an simple interpreter based JVM would be sufficient (ie Zero) (or even maybe a cross compiling mode) The second is getting OpenJDK to build on a platform where a hotspot JVM doesn't exist and may never exist. As you guess I'm thinking of IBM platforms specifically. I'm don't expect to port Hotspot to AIX so I need to be able to make the OpenJDK build work with J9. The third (a variant of the 2nd) is where another JVM vendor wants to get OpenJDK working with their JVM - regardless of the availability of a Hotspot JVM on the target platform. To be clear. I'm not suggesting that this project step up to defining the interfaces between JVM and classes. This is simple pragmatics. The Hotspot JVM is the starting point for the mould and I would expect to make J9 (or any new JVM) fit into it as much as possible. However there will be changes needed. These are mostly simple, like parameterising JVM command line options, to the more complicated like separating out JVM intrinsic classes such as String.java, Object.java, Thread.java etc so that the right versions get build and packaged. I certainly can understand these needs, but it is still seems beyond the initial scope of this project. Maybe in a phase 2? -kto
Re: Project Proposal: Build Infrastructure Changes
On May 3, 2011, at 2:18 AM, Steve Poole wrote: On 30/04/11 00:05, Kelly O'Hair wrote: On Apr 29, 2011, at 1:31 PM, Steve Poole wrote: On 26/04/11 15:54, Kelly O'Hair wrote: On Apr 26, 2011, at 12:59 AM, Steve Poole wrote: * Allow for use of more portable build tools (compilers etc.) where possible Can I add support for alternative JVM's ? Seems a bit out of scope to me. Sorry, it was a bit of a flippant one liner, I owe you more details. There are three usecases I see that require the OpenJDK build process to be modified to accommodate: The first is bootstrapping a build. I'd like to be able to build OpenJDK on a new platform without the need for a previous SDK build to be present. In this usecase it's possible that an simple interpreter based JVM would be sufficient (ie Zero) (or even maybe a cross compiling mode) The second is getting OpenJDK to build on a platform where a hotspot JVM doesn't exist and may never exist. As you guess I'm thinking of IBM platforms specifically. I'm don't expect to port Hotspot to AIX so I need to be able to make the OpenJDK build work with J9. The third (a variant of the 2nd) is where another JVM vendor wants to get OpenJDK working with their JVM - regardless of the availability of a Hotspot JVM on the target platform. To be clear. I'm not suggesting that this project step up to defining the interfaces between JVM and classes. This is simple pragmatics. The Hotspot JVM is the starting point for the mould and I would expect to make J9 (or any new JVM) fit into it as much as possible. However there will be changes needed. These are mostly simple, like parameterising JVM command line options, to the more complicated like separating out JVM intrinsic classes such as String.java, Object.java, Thread.java etc so that the right versions get build and packaged. I certainly can understand these needs, but it is still seems beyond the initial scope of this project. Maybe in a phase 2? Hmm, maybe doing all of them in one go may be a stretch :-) The 2nd usecase is fundamental though to supporting IBM platforms, something you mentioned you wanted to do. Is there any reason why realising that usecase is beyond phase 1? The contract or plumbing design between the JVM implementation and the JDK itself, in my opinion, belongs with the JVM teams. If there was a detailed specification with regards to how a JVM plumbs into a JDK, we could certainly use that as part of the build process, Right now, it's just a list of files that get built by the hotspot repository build, and placed in specific locations of the JDK install image. The export/import process defined in the makefiles now, rather haphazardly, perhaps we can clean it up. But as I understand it, there is no formal spec here. The actual native 'extern' symbol contracts between the JDK and JVM might be fairly well-defined, but I'm not sure where, that's a deeper micro plumbing issue beyond building. When the langtools, corba, jaxp, and jaxws sources were split out of the original j2se Teamware workspace, I tried to define a simple delivery mechanism for these repositories, such that after they built, the files in their dist/ directory would be the delivery into the overall JDK image. A classes.zip and a src.zip in the simpliest cases (this will likely become a bundle of modules in the jdk8 timeframe). With hotspot, it wasn't as simple, and it currently constructs a sparse JDK-like install image for delivery into the overall JDK build process. So we know what hotspot contributes to the JDK build, it's just not very well documented. We can certainly try to clean that up and provide some documentation, assuming we can enforce updating the documents when developers change things. I was going to say that building hotspot does NOT require a Boot JDK to build, but I would be wrong, it does, but I agree, it probably should not. As I recall, there is some XML processing, the stupid gamma launcher Queens use, and building the extra java code for the Serviceability Agent and something else I forget the name of. I would prefer that building hotspot not rely on a JDK at all, but I can't promise anything there. As we get more into cross compilation for embedded building, we will need to address these issues cleanly, and I suspect that what you want may fall out of that work. So I would like to say that anything we do should be an improvement for J9 plugging in. But predictions of the future have always been a problem for me, I'm trying Tarot cards, no luck so far. -kto -kto
Re: Request for review: 7036525 Disable alternative source mechanism for OPENJDK builds
David Holmes (david.hol...@oracle.com) wrote: Hi John, Fresh webrev at: http://cr.openjdk.java.net/~dholmes/7036525/webrev.1/ Looks good to me. -John David Holmes said the following on 04/30/11 10:09: Thanks John. Currently sitting here shaking my head is disbelief. I'll try this again. David John Coomes said the following on 04/30/11 03:47: David Holmes (david.hol...@oracle.com) wrote: http://cr.openjdk.java.net/~dholmes/7036525/webrev/ Simple but crude. If OPENJDK is defined then the Hotspot alternative source mechanism is effectively disabled by checking for a non-existent path. This allows people using the alt-src mechanism to select which type of build they want in a way that is consistent with how builds of OPENJDK are done in the rest of the JDK. Tested by checking the errorReporter.cpp location in builds with/without OPENJDK set, and with/without src/closed present. This will be pushed into hotspot-rt/hotspot for hs21-b11 Hi David, 38 ifneq ($(OPENJDK),true) 39 HS_ALT_SRC_REL=src/closed 40 else 41 HS_ALT_SRC=NO_SUCH_PATH 42 endif 43 HS_COMMON_SRC=$(GAMMADIR)/$(HS_COMMON_SRC_REL) 44 HS_ALT_SRC=$(GAMMADIR)/$(HS_ALT_SRC_REL) The 'if' block sets HS_ALT_SRC_*REL*, but the else block sets HS_ALT_SRC (no *REL*), and that is overwritten on line 44. I think it works because after line 44, HS_ALT_SRC == $(GAMMADIR)/, but I doubt that was intended. You could change line 41 to HS_ALT_SRC_REL=$(HS_COMMON_SRC_REL) Then when OPENJDK=true, HS_ALT_SRC==HS_COMMON_SRC, and you don't have to rely on NO_SUCH_PATH. -John
Re: Request for review: 7036525 Disable alternative source mechanism for OPENJDK builds
David Holmes (david.hol...@oracle.com) wrote: Hi Keith, Keith McGuigan said the following on 05/03/11 02:13: On Apr 29, 2011, at 1:47 PM, John Coomes wrote: David Holmes (david.hol...@oracle.com) wrote: http://cr.openjdk.java.net/~dholmes/7036525/webrev/ Simple but crude. If OPENJDK is defined then the Hotspot alternative source mechanism is effectively disabled by checking for a non-existent path. This allows people using the alt-src mechanism to select which type of build they want in a way that is consistent with how builds of OPENJDK are done in the rest of the JDK. Tested by checking the errorReporter.cpp location in builds with/without OPENJDK set, and with/without src/closed present. This will be pushed into hotspot-rt/hotspot for hs21-b11 Hi David, 38 ifneq ($(OPENJDK),true) 39 HS_ALT_SRC_REL=src/closed 40 else 41 HS_ALT_SRC=NO_SUCH_PATH 42 endif 43 HS_COMMON_SRC=$(GAMMADIR)/$(HS_COMMON_SRC_REL) 44 HS_ALT_SRC=$(GAMMADIR)/$(HS_ALT_SRC_REL) The 'if' block sets HS_ALT_SRC_*REL*, but the else block sets HS_ALT_SRC (no *REL*), and that is overwritten on line 44. I think it works because after line 44, HS_ALT_SRC == $(GAMMADIR)/, but I doubt that was intended. You could change line 41 to HS_ALT_SRC_REL=$(HS_COMMON_SRC_REL) Then when OPENJDK=true, HS_ALT_SRC==HS_COMMON_SRC, and you don't have to rely on NO_SUCH_PATH. I'd prefer to leave it as NO_SUCH_PATH. That way tests for the existence of that directory (or files in those directories) will correctly fail and won't mistakenly indicates that alt-srcs exist. I believe there are a number of tests like that already, but I'll bet it currently works fine (unintentionally) if HS_ALT_SRC == HS_COMMON_SRC. I've just made the change as John suggested and to be honest I don't know why I didn't think of that myself. I do see your point though, by setting it the same the build will always use the ALT_SRC in the OpenJDK case - but this will be fine because it is the same as COMMON_SRC. This is only used to generate the Makefiles during the buildtree phase so I don't think it is really a concern either way. FWIW, I prefer the change you've made, but don't feel that strongly about it. To be honest I'm doubting the whole rationale for this change as it means that an OPENJDK build will never use the alt-src mechanism, when according to the comments alt-src was also intended to be used by others for introducing alternative code into their builds/distributions. In those cases you may well want both alt-src and OPENJDK (given that OPENJDK could be being set at the top-level JDK makefile). IMHO, better if an OPENJDK build doesn't use alt-src, at least by default. And I suspect you can override HS_ALT_SRC_REL from the gmake command line, even when OPENJDK==true (haven't tried it, though). -John
Re: JDK8 Preliminary Repository Layout
On Apr 28, 2011, at 2:48 AM, Fredrik Öhrström wrote: 2011/3/11 Kelly O'Hair kelly.oh...@oracle.com in the repository. If there are frequent pushes going on, either from too much activity or too many developers, someone may experience a: hg push# fails because you need to do a pull too many heads message hg pull -u hg merge hg commit -m Merge# Or hg fetch hg push # fails because you took too long and someone else pushed a new one hg pull -u hg merge hg commit -m Merge# Or hg fetch hg push # fails because you took too long and someone else pushed a new one I suppose this is related to the fact that mercurial has developed over time. But today, using merge to solve this problem would be ill advised. I think it is much better to use the rebase extension to do hg pull --rebase This will move your outgoing changes to the tip and avoid the creation of an unnecessary merge node. This rebase extension (which is an extension that needs to be enabled) has not always existed, it's a nice addition and will be very useful. But it does come with come cautions, effectively a rebase will regenerate changesets, and anything that regenerates changesets needs to be used with great care, just like rollbacks. And hidden in the rebase, is a merge, potentially many merges, potentially merges where file changes actually need to be resolved. With rebase, the merges just get folded into a new changeset. There are dumb merge changesets, where no files were in conflict, which the rebase can easily fix, but when there are file conflicts, you may run a risk here of mis-merging if you are not careful. And once rebased, your new changeset might not be the same changeset that you originally created, so great care is always advised with a 'hg pull', regardless of how you manage the merge. Depending on how much you pulled over, re-builds and re-tests may be important. Assuming that you have a large number of committers at work at the same time, a simple solution is to have the committers add themselves to a queue, then they get a message (IM,mail or otherwise) when they have exclusive access. When they are done, they relinquish their exclusive acces or it will be revoked automatically after 5 minutes. In my opinion, a collision between 2 developers in a small window of time (say 3mins) should be rare, and it's easily resolved. If it happens frequently, I would tend to question what is going on. That many changesets being pushed in at the same time may be a sign of something seriously wrong with development, the number of changesets per day should not be in the 1,000's. Yes, I see John mentioned the stuffed animal Teamware tokens, representing locks on workspaces, I remember those. ;^) With the old Teamware workspaces, the equivalent pull/merge was often very very slow, 30mins sometimes 60mins. But with Mercurial, the time to push is measured in minutes, sometimes seconds. I just don't see a need for locks. More important in my mind, is making sure we get good changeset pushes. -kto Fredrik Öhrström
Re: Project Proposal: Build Infrastructure Changes
On May 3, 2011, at 12:05 PM, Kelly O'Hair wrote: I was going to say that building hotspot does NOT require a Boot JDK to build, but I would be wrong, it does, but I agree, it probably should not. As I recall, there is some XML processing, the stupid gamma launcher Queens use, Whoops... little typo here. Shouldn't say stupid gamma launcher, it should be, incredibly-useful-for-debugging-the-JVM gamma launcher. Easy mistake to make, though: the keys are, like, right next to each other. :) -- - Keith
Re: Project Proposal: Build Infrastructure Changes
On May 3, 2011, at 10:16 AM, Keith McGuigan wrote: On May 3, 2011, at 12:05 PM, Kelly O'Hair wrote: I was going to say that building hotspot does NOT require a Boot JDK to build, but I would be wrong, it does, but I agree, it probably should not. As I recall, there is some XML processing, the stupid gamma launcher Queens use, Whoops... little typo here. Shouldn't say stupid gamma launcher, it should be, incredibly-useful-for-debugging-the-JVM gamma launcher. Easy mistake to make, though: the keys are, like, right next to each other. :) No doubt it's been useful, but seriously, you just built a hotspot for jdk7, with a completely different C++ compiler, and a different C++ runtime dependency, and you plop it down into a jdk6 image (that was built with a different C++ compiler, and maybe using a different C++ runtime) and you guys' expect it to work reliably? It doesn't, and likely won't run in a cross compile environment at all. At a minimum, you should be plopping it into a jdk7 image, a jdk7 image built for the target arch you are building and can run, and when it comes down to it, isn't this actually a test of your build? The number of issues with that gamma/Queens makefile logic is too high, and it's not something we ship anyway. In my opinion, we should be restricting ourselves to building what we ship as part of the product, or what we need to make that build happen. So in my opinion, the whole thing needs to be moved to the hotspot/test area. In addition, the gamma launcher code is riddled with troublesome/buggy code that gets little attention, mostly because it isn't shipped as part of the product. If you guys want this code as part of the build, it needs some attention. If it is just a test case, then isolate it and make it one. We need to do a better job of separating out the building logic from the testing logic. Just my 2 cents... ;^) -kto -- - Keith
hg: jdk7/build/jdk: 8 new changesets
Changeset: 1c31b35e9408 Author:ogino Date: 2011-04-26 21:46 -0700 URL: http://hg.openjdk.java.net/jdk7/build/jdk/rev/1c31b35e9408 7036955: Japanese man pages in linux should be in utf-8 encoding Reviewed-by: ohair, mfang ! make/common/Defs-linux.gmk ! make/common/Release.gmk Changeset: 00e485efd9e0 Author:mfang Date: 2011-04-27 23:11 -0700 URL: http://hg.openjdk.java.net/jdk7/build/jdk/rev/00e485efd9e0 6501385: ColorChooser demo - two elemets have same mnemonic in it locale, GTK LF Reviewed-by: yhuang ! src/share/classes/com/sun/java/swing/plaf/gtk/resources/gtk_de.properties ! src/share/classes/com/sun/java/swing/plaf/gtk/resources/gtk_es.properties ! src/share/classes/com/sun/java/swing/plaf/gtk/resources/gtk_fr.properties ! src/share/classes/com/sun/java/swing/plaf/gtk/resources/gtk_it.properties ! src/share/classes/com/sun/java/swing/plaf/gtk/resources/gtk_pt_BR.properties ! src/share/classes/com/sun/java/swing/plaf/gtk/resources/gtk_sv.properties ! src/share/classes/com/sun/swing/internal/plaf/basic/resources/basic_sv.properties Changeset: 4fcbaf9fb837 Author:mfang Date: 2011-04-27 23:18 -0700 URL: http://hg.openjdk.java.net/jdk7/build/jdk/rev/4fcbaf9fb837 7038803: [CCJK] Incorrect mnemonic key (0) is displayed on cancel button on messagedialog of JOptionPane Reviewed-by: yhuang ! src/share/classes/com/sun/swing/internal/plaf/basic/resources/basic.properties ! src/share/classes/com/sun/swing/internal/plaf/basic/resources/basic_ja.properties ! src/share/classes/com/sun/swing/internal/plaf/basic/resources/basic_ko.properties ! src/share/classes/com/sun/swing/internal/plaf/basic/resources/basic_zh_CN.properties ! src/share/classes/com/sun/swing/internal/plaf/basic/resources/basic_zh_TW.properties Changeset: 4b197be687dc Author:mfang Date: 2011-04-27 23:28 -0700 URL: http://hg.openjdk.java.net/jdk7/build/jdk/rev/4b197be687dc Merge Changeset: fddcc29ed8f8 Author:mfang Date: 2011-04-28 14:30 -0700 URL: http://hg.openjdk.java.net/jdk7/build/jdk/rev/fddcc29ed8f8 Merge ! make/common/Defs-linux.gmk ! make/common/Release.gmk - src/share/classes/sun/security/ssl/DefaultSSLContextImpl.java - src/share/native/sun/font/layout/Features.h - test/javax/swing/text/GlyphView/6539700/bug6539700.java Changeset: 5f0455522852 Author:mfang Date: 2011-04-28 20:15 -0700 URL: http://hg.openjdk.java.net/jdk7/build/jdk/rev/5f0455522852 7040228: [zh_TW] extra (C) on cancel button on File Chooser dialog Reviewed-by: yhuang ! src/share/classes/com/sun/java/swing/plaf/gtk/resources/gtk.properties ! src/share/classes/com/sun/java/swing/plaf/gtk/resources/gtk_ja.properties ! src/share/classes/com/sun/java/swing/plaf/gtk/resources/gtk_ko.properties ! src/share/classes/com/sun/java/swing/plaf/gtk/resources/gtk_zh_CN.properties ! src/share/classes/com/sun/java/swing/plaf/gtk/resources/gtk_zh_TW.properties ! src/share/classes/com/sun/swing/internal/plaf/basic/resources/basic.properties ! src/share/classes/com/sun/swing/internal/plaf/basic/resources/basic_ja.properties ! src/share/classes/com/sun/swing/internal/plaf/basic/resources/basic_ko.properties ! src/share/classes/com/sun/swing/internal/plaf/basic/resources/basic_zh_CN.properties ! src/share/classes/com/sun/swing/internal/plaf/basic/resources/basic_zh_TW.properties Changeset: a604668a7a64 Author:mfang Date: 2011-04-28 21:43 -0700 URL: http://hg.openjdk.java.net/jdk7/build/jdk/rev/a604668a7a64 7040257: [pt_BR,fr] Print dialog has duplicate mnemonic key. Reviewed-by: psun ! src/share/classes/sun/print/resources/serviceui_fr.properties ! src/share/classes/sun/print/resources/serviceui_pt_BR.properties Changeset: 5b7d35a6e1b3 Author:mfang Date: 2011-05-02 13:55 -0700 URL: http://hg.openjdk.java.net/jdk7/build/jdk/rev/5b7d35a6e1b3 Merge
Re: Project Proposal: Build Infrastructure Changes
On May 3, 2011, at 11:27 AM, Kelly O'Hair wrote: On May 3, 2011, at 10:16 AM, Keith McGuigan wrote: On May 3, 2011, at 12:05 PM, Kelly O'Hair wrote: I was going to say that building hotspot does NOT require a Boot JDK to build, but I would be wrong, it does, but I agree, it probably should not. As I recall, there is some XML processing, the stupid gamma launcher Queens use, Whoops... little typo here. Shouldn't say stupid gamma launcher, it should be, incredibly-useful-for-debugging-the-JVM gamma launcher. Easy mistake to make, though: the keys are, like, right next to each other. :) No doubt it's been useful, but seriously, you just built a hotspot for jdk7, with a completely different C++ compiler, and a different C++ runtime dependency, and you plop it down into a jdk6 image (that was built with a different C++ compiler, and maybe using a different C++ runtime) and you guys' expect it to work reliably? It doesn't, and likely won't run in a cross compile environment at all. At a minimum, you should be plopping it into a jdk7 image, a jdk7 image built for the target arch you are building and can run, and when it comes down to it, isn't this actually a test of your build? The number of issues with that gamma/Queens makefile logic is too high, and it's not something we ship anyway. In my opinion, we should be restricting ourselves to building what we ship as part of the product, or what we need to make that build happen. So in my opinion, the whole thing needs to be moved to the hotspot/test area. In addition, the gamma launcher code is riddled with troublesome/buggy code that gets little attention, mostly because it isn't shipped as part of the product. If you guys want this code as part of the build, it needs some attention. If it is just a test case, then isolate it and make it one. We need to do a better job of separating out the building logic from the testing logic. Just my 2 cents... ;^) Here here! In the macosx-port we explicitly added a hook to ignore the gamma test because cross compiling with the Apple Java SE 6 (which is how _everyone_ gets off the ground) is impossible otherwise. We've also had to workaround these things in our internal builds for years and years. If these tests were properly factored into a /tests directory, we could have removed some really ugly hacks to get the product to build itself. Cheers, Mike Swingler Java Engineering Apple Inc.
Re: Project Proposal: Build Infrastructure Changes
On 2011-05-03 22:27, Kelly O'Hair wrote: No doubt it's been useful, but seriously, you just built a hotspot for jdk7, with a completely different C++ compiler, and a different C++ runtime dependency, and you plop it down into a jdk6 image (that was built with a different C++ compiler, and maybe using a different C++ runtime) and you guys' expect it to work reliably? It doesn't, and likely won't run in a cross compile environment at all. We did it for years for jdk 5 - HS reference platform for jdk5 is different from JDK one. The number of issues with that gamma/Queens makefile logic is too high, and it's not something we ship anyway. In my opinion, we should be restricting ourselves to building what we ship as part of the product, or what we need to make that build happen. So in my opinion, the whole thing needs to be moved to the hotspot/test area. Generally I agree that gamma launcher should be in hotspot test but minimal smoke test at the end of the build is quite useful. The build system not always handle correctly incremental build and Queens test usually catch it at the earliest possible point. So I would like to: a) make gamma test JDK independent b) move it to hotspot/test c) keep it as part of build process -Dmitry -- Dmitry Samersoff Java Hotspot development team, SPB04 * There will come soft rains ...
Re: Project Proposal: Build Infrastructure Changes
On 05/03/2011 04:13 PM, Mike Swingler wrote: On May 3, 2011, at 11:27 AM, Kelly O'Hair wrote: On May 3, 2011, at 10:16 AM, Keith McGuigan wrote: The number of issues with that gamma/Queens makefile logic is too high, and it's not something we ship anyway. In my opinion, we should be restricting ourselves to building what we ship as part of the product, or what we need to make that build happen. So in my opinion, the whole thing needs to be moved to the hotspot/test area. In addition, the gamma launcher code is riddled with troublesome/buggy code that gets little attention, mostly because it isn't shipped as part of the product. If you guys want this code as part of the build, it needs some attention. If it is just a test case, then isolate it and make it one. We need to do a better job of separating out the building logic from the testing logic. Just my 2 cents... ;^) Here here! In the macosx-port we explicitly added a hook to ignore the gamma test because cross compiling with the Apple Java SE 6 (which is how _everyone_ gets off the ground) is impossible otherwise. We have the same problem in IcedTea. We have a patch that disables the gamma test so we can bootstrap openjdk with gcj. A proper way to skip the gamma test will be very useful. Cheers, Omair
Re: Project Proposal: Build Infrastructure Changes
On May 3, 2011, at 2:08 PM, Dmitry Samersoff wrote: Generally I agree that gamma launcher should be in hotspot test but minimal smoke test at the end of the build is quite useful. The build system not always handle correctly incremental build and Queens test usually catch it at the earliest possible point. So I would like to: a) make gamma test JDK independent b) move it to hotspot/test c) keep it as part of build process -Dmitry I have a hard time with c) but I understand you want some heartbeat test or indication the hotspot build is sane. But when that 'smoke' test starts impacting everyone else, then it needs to be isolated out of the default build. I know that having tests (heartbeat and smoke) run during the build is a somewhat common practice, but it really needs to go. And Queens is a fine little test, but it's really not enough. I would like to see a 'cd hotspot/make make make test' where the 'test' target runs your Queens test and maybe even a few other quick tests that the developers find helpful. Let the default 'make' be just a build, and running tests is a trivial 'make test'. I'd like to make that a general rule: #1 Do not run tests as part of the build process :^) -kto
Re: Project Proposal: Build Infrastructure Changes
On May 3, 2011, at 3:31 PM, Kelly O'Hair wrote: On May 3, 2011, at 2:08 PM, Dmitry Samersoff wrote: Generally I agree that gamma launcher should be in hotspot test but minimal smoke test at the end of the build is quite useful. The build system not always handle correctly incremental build and Queens test usually catch it at the earliest possible point. So I would like to: a) make gamma test JDK independent b) move it to hotspot/test c) keep it as part of build process -Dmitry I have a hard time with c) but I understand you want some heartbeat test or indication the hotspot build is sane. But when that 'smoke' test starts impacting everyone else, then it needs to be isolated out of the default build. I know that having tests (heartbeat and smoke) run during the build is a somewhat common practice, but it really needs to go. And Queens is a fine little test, but it's really not enough. I would like to see a 'cd hotspot/make make make test' where the 'test' target runs your Queens test and maybe even a few other quick tests that the developers find helpful. Let the default 'make' be just a build, and running tests is a trivial 'make test'. I'd like to make that a general rule: #1 Do not run tests as part of the build process :^) -kto Agreed. Can this be done within the JDK7 timeframe? Curious, Mike Swingler Java Engineering Apple Inc.
Re: Project Proposal: Build Infrastructure Changes
On May 3, 2011, at 3:40 PM, Mike Swingler wrote: I'd like to make that a general rule: #1 Do not run tests as part of the build process :^) -kto Agreed. Can this be done within the JDK7 timeframe? That will be tricky. JDK7 changes are now restricted, and I doubt any changes like this would be acceptable now. As far as the Build Infrastructure Changes if we start with a jdk7 repository set, it might be possible to integrate the changes into jdk7u2, but that may be a pipe dream on my part. There are multiple events of interest for this, and I have no idea when they will all happen: * When we get something working (and hotspot is not first) * When the jdk8 repositories become available, we may want to shift focus at that time * When and where the mac-port changes get integrated in jdk7uN? or jdk8? - Maybe we piggyback any Build Infrastructure Changes for jdk7uN with the mac-port changes? ;^) * When the jigsaw module changes get integrated into jdk8 (a major event) - This will likely be a big merge effort, could be a horse race as to who gets into jdk8 first. ;^) My latest Tarot card reading says I'm in for a positive experience, so far so good. ;^) -kto Curious, Mike Swingler Java Engineering Apple Inc.
Re: Request for review: 7036525 Disable alternative source mechanism for OPENJDK builds
John Coomes said the following on 05/04/11 02:37: David Holmes (david.hol...@oracle.com) wrote: I've just made the change as John suggested and to be honest I don't know why I didn't think of that myself. I do see your point though, by setting it the same the build will always use the ALT_SRC in the OpenJDK case - but this will be fine because it is the same as COMMON_SRC. This is only used to generate the Makefiles during the buildtree phase so I don't think it is really a concern either way. FWIW, I prefer the change you've made, but don't feel that strongly about it. To be honest I'm doubting the whole rationale for this change as it means that an OPENJDK build will never use the alt-src mechanism, when according to the comments alt-src was also intended to be used by others for introducing alternative code into their builds/distributions. In those cases you may well want both alt-src and OPENJDK (given that OPENJDK could be being set at the top-level JDK makefile). IMHO, better if an OPENJDK build doesn't use alt-src, at least by default. And I suspect you can override HS_ALT_SRC_REL from the gmake command line, even when OPENJDK==true (haven't tried it, though). No. Unless you use -e a variable's value from the environment will be overridden by an explicit assignment in the Makefile. Which means that the better fix here is: + 36 ifndef HS_ALT_SRC_REL 37 ifneq ($(OPENJDK),true) 38 # This needs to be changed to a more generic location, but we keep it as this 39 # for now for compatibility 40 41 HS_ALT_SRC_REL=src/closed 42 else 43 HS_ALT_SRC_REL=$(HS_COMMON_SRC_REL) 44 endif + 45 endif David
