Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On Jun 17, 2013, at 5:21 PM, Stuart Marks wrote: > Hi Kelly! You still read this stuff here? :-) I read anything that looks entertaining from entertaining people. ;^) -kto
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 6/19/13 1:01 AM, Erik Joelsson wrote: Currently, configure checks that the found boot jdk is 7 or 8. Do we really want to actively prevent using 8 all together? I could agree to printing a big warning in the summary at the end of configure to discourage it, but I do believe it necessary to have the ability to build with 8 for tracking down certain bugs. Right. This is mainly about preventing mistakes, such as a run of configure that picks up a JDK 8 that might happen to be in one's path, or someone who's not aware of the rule I just clarified in the README. :-) But of course it should be possible to configure JDK 8 as the boot for JDK 8 if necessary. I'm not sure a warning is sufficient. To paraphrase Kelly's message from earlier, Rule #1 Nobody pays attention to any warnings Rule #2 When things go wrong, claim that you didn't see the warning I looked in boot-jdk.m4 and I see that it disallows JDK 6 entirely (makes sense) and allows 7 and 8. Maybe it could disallow JDK 8 by default, unless an additional option is provided (like Max suggested). Of course, all of these numbers will need to be rolled forward when we get to JDK 9. s'marks Regarding the rearrangement of corba/jaxp/jaxws to use the fresh JDK instead of the boot JDK. At least we know they build, because the boot cycle build builds them successfully. (At least, I think it does.) Now, I don't think the artifacts produced from a boot cycle build are actually tested or are delivered anywhere in a bundle. So, while it seems quite unlikely, some bugs could have been introduced by building with a newer JDK version. Now ... circular dependencies ... urk ... I *knew* there was something that would make this complicated. Well, maybe these will need to be refactored away somehow. Or maybe some kind of GenStubs technique can be used to deal with the circularity. You introduced yet another point as well, which is the relationship between the repository organization and the build structure. As I understand things, each repository has its own build support and builds in a separate step from the others. In principle I think that the repository structure ought to be orthogonal to the build structure. At least, if we move to a more modular build structure, that shouldn't imply that we need to have each module in its own repository. In fact I'd like to see fewer repositories. To me, the only compelling reason to have a separate repo is if the source code in it is a snapshot of an upstream source base -- as seems to be the case for jaxws. Having all the stuff in fewer repos makes it easier to bisect to find failures, and it reduces the need for careful management of coordinated, cross-repo changes. My preferred solution would be to fold in the repos that aren't upstream projects into jdk and just have them compile with the rest there. I much like the idea of reducing the number of repos. If that isn't possible, we can just add those source directories to the main javac invocation in jdk too. /Erik
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 6/19/2013 4:01 PM, Erik Joelsson wrote: Now ... circular dependencies ... urk ... I *knew* there was something that would make this complicated. Well, maybe these will need to be refactored away somehow. Or maybe some kind of GenStubs technique can be used to deal with the circularity. We can create skeleton classes to deal with circular dependencies. It's like .h files for Java. --Max
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
I'm not sure how big a warning needs to be to make people aware of it. Is it possible to create another configure option like --yes-i-do-want-to-use-n that you must add to set boot jdk to 8? --Max On 6/19/2013 5:23 PM, Chris Hegarty wrote: On 19/06/2013 09:01, Erik Joelsson wrote: On 2013-06-19 03:10, Stuart Marks wrote: -- I have half a mind to look at the Configure changes myself in my spare time (ha!), but I have no spare time, and I don't have the expertise in this area anyway. So anyone is welcome to pick this up. In principle it should be fairly simple, and I think it's fairly important. This isn't the first time someone's been bitten by having the wrong boot JDK version, and it won't be the last. Currently, configure checks that the found boot jdk is 7 or 8. Do we really want to actively prevent using 8 all together? I could agree to printing a big warning in the summary at the end of configure to discourage it, but I do believe it necessary to have the ability to build with 8 for tracking down certain bugs. +1 -Chris.
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 19/06/2013 09:01, Erik Joelsson wrote: : My preferred solution would be to fold in the repos that aren't upstream projects into jdk and just have them compile with the rest there. I much like the idea of reducing the number of repos. If that isn't possible, we can just add those source directories to the main javac invocation in jdk too. I think the repo structure does need to be re-examined. While I don't think we want a repository per module, I do think we should at least consider the implication of having the code organized by modules. Also one of the benefits of the ongoing efforts to remove dependencies from the "core" on other areas is that it opens up the possibility of other structures (I've heard Jon suggest a "core" repository for the core language and runtime for example). In any case, I think a first step is to move the build of the jaxws repository to later in the build. It's an upstream project and nothing in the JDK should have any dependencies on the code in that repository. Note to self: test moving JAX-WS/JAXB/SAAJ/JAF from rt.jar to the extensions directory and see what comes out of the woodwork. I'm not sure what to say about the corba repository. There is sad history and what we have in OpenJDK is essentially a fork of the CORBA code that is in Glassfish. There have been suggestions over the years to bring the code together but I don't know whether this is possible now (or even it is worth it). -Alan
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 19/06/2013 09:01, Erik Joelsson wrote: On 2013-06-19 03:10, Stuart Marks wrote: -- I have half a mind to look at the Configure changes myself in my spare time (ha!), but I have no spare time, and I don't have the expertise in this area anyway. So anyone is welcome to pick this up. In principle it should be fairly simple, and I think it's fairly important. This isn't the first time someone's been bitten by having the wrong boot JDK version, and it won't be the last. Currently, configure checks that the found boot jdk is 7 or 8. Do we really want to actively prevent using 8 all together? I could agree to printing a big warning in the summary at the end of configure to discourage it, but I do believe it necessary to have the ability to build with 8 for tracking down certain bugs. +1 -Chris.
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 2013-06-19 03:10, Stuart Marks wrote: -- I have half a mind to look at the Configure changes myself in my spare time (ha!), but I have no spare time, and I don't have the expertise in this area anyway. So anyone is welcome to pick this up. In principle it should be fairly simple, and I think it's fairly important. This isn't the first time someone's been bitten by having the wrong boot JDK version, and it won't be the last. Currently, configure checks that the found boot jdk is 7 or 8. Do we really want to actively prevent using 8 all together? I could agree to printing a big warning in the summary at the end of configure to discourage it, but I do believe it necessary to have the ability to build with 8 for tracking down certain bugs. Regarding the rearrangement of corba/jaxp/jaxws to use the fresh JDK instead of the boot JDK. At least we know they build, because the boot cycle build builds them successfully. (At least, I think it does.) Now, I don't think the artifacts produced from a boot cycle build are actually tested or are delivered anywhere in a bundle. So, while it seems quite unlikely, some bugs could have been introduced by building with a newer JDK version. Now ... circular dependencies ... urk ... I *knew* there was something that would make this complicated. Well, maybe these will need to be refactored away somehow. Or maybe some kind of GenStubs technique can be used to deal with the circularity. You introduced yet another point as well, which is the relationship between the repository organization and the build structure. As I understand things, each repository has its own build support and builds in a separate step from the others. In principle I think that the repository structure ought to be orthogonal to the build structure. At least, if we move to a more modular build structure, that shouldn't imply that we need to have each module in its own repository. In fact I'd like to see fewer repositories. To me, the only compelling reason to have a separate repo is if the source code in it is a snapshot of an upstream source base -- as seems to be the case for jaxws. Having all the stuff in fewer repos makes it easier to bisect to find failures, and it reduces the need for careful management of coordinated, cross-repo changes. My preferred solution would be to fold in the repos that aren't upstream projects into jdk and just have them compile with the rest there. I much like the idea of reducing the number of repos. If that isn't possible, we can just add those source directories to the main javac invocation in jdk too. /Erik
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 6/18/13 2:25 AM, Alan Bateman wrote: On 18/06/2013 08:42, Stuart Marks wrote: 4) Could jaxp, jaxws, and corba be built with the current JDK, not the boot JDK? Sure, probably.[...] My understanding is that the new build is just following the old build[...] As least for the jaxws repository then I don't see any reason that it has to be built by the boot JDK. I actually don't see any reason why it needs to be rt.jar either, it could be built to jaxws.jar and dropped into the extensions directory (but that's another matter). That said, even if the jaxws repository was built later then it's unlikely that code there will start making use of new language features or APIs. That is because it's really an upstream project that still creates standalone releases to run on jdk7. The corba and jaxp are trickier because there are circular dependencies. These circular dependencies arises because we have a few places in the jdk repository that need to parse XML and there are few cases where CORBA APIs are used (the JMX remote API supports IIOP, and the JNDI CosNaming provider are two). [...] The jaxp repository is a clear case where we should be able to new new language features and APIs. Aside from the build then the only thing holding things back is periodically calls to keep the jdk7u and jdk8 code in sync. I don't know how long that can last and we will need to break from the past at some point. So I think this is a great topic to discuss. I think it has to be discussed in the context of where the repositories are going with modules. For the JDK modularization then we've long talked about restructuring the source code so that it's organized by modules, we didn't get to considering whether this would have an impact on the repositories. I think it's also worth discussing whether the "compilation unit" is the code in a single repository or whether the code in multiple repositories could be compiled together. The other extreme is where we get to the point where individual modules can be compiled on their own. OK, I've pushed the change to README-builds.html. I've filed an RFE that proposes changes to Configure to do version checking: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=8016918 Brad W filed a bug to cover changes in the way jaxp, jaxws, and corba are built: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=8016909 [links won't work until the externally-facing bug database is refreshed] -- I have half a mind to look at the Configure changes myself in my spare time (ha!), but I have no spare time, and I don't have the expertise in this area anyway. So anyone is welcome to pick this up. In principle it should be fairly simple, and I think it's fairly important. This isn't the first time someone's been bitten by having the wrong boot JDK version, and it won't be the last. Regarding the rearrangement of corba/jaxp/jaxws to use the fresh JDK instead of the boot JDK. At least we know they build, because the boot cycle build builds them successfully. (At least, I think it does.) Now, I don't think the artifacts produced from a boot cycle build are actually tested or are delivered anywhere in a bundle. So, while it seems quite unlikely, some bugs could have been introduced by building with a newer JDK version. Now ... circular dependencies ... urk ... I *knew* there was something that would make this complicated. Well, maybe these will need to be refactored away somehow. Or maybe some kind of GenStubs technique can be used to deal with the circularity. You introduced yet another point as well, which is the relationship between the repository organization and the build structure. As I understand things, each repository has its own build support and builds in a separate step from the others. In principle I think that the repository structure ought to be orthogonal to the build structure. At least, if we move to a more modular build structure, that shouldn't imply that we need to have each module in its own repository. In fact I'd like to see fewer repositories. To me, the only compelling reason to have a separate repo is if the source code in it is a snapshot of an upstream source base -- as seems to be the case for jaxws. Having all the stuff in fewer repos makes it easier to bisect to find failures, and it reduces the need for careful management of coordinated, cross-repo changes. s'marks
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 6/18/13 2:16 AM, Chris Hegarty wrote: On 06/18/2013 10:02 AM, David Holmes wrote: I don't think we should simply say "Do not use a build of JDK 8 as the boot JDK for building JDK 8." as that doesn't explain what the issue is. If I'm building the JDK for my own use I can use JDK8. So how about: "JDK 8 developers should not use JDK 8 as the boot JDK, to ensure that code changes are compatible with building using JDK 7." This suggested wording, along with Stuarts other additions to the README, look fine. I think this should be pushed, and the other issues handled separately. Guys, thanks for the review and comments. I've just pushed a change to README-builds.html that adopts a modified version of David's suggestion, the essence of which is changing "Do not use..." to "JDK 8 developers should not use" s'marks
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 18/06/2013 08:42, Stuart Marks wrote: : 4) Could jaxp, jaxws, and corba be built with the current JDK, not the boot JDK? Sure, probably. I spoke with Jon G on this topic the other day and we didn't come up with any really good reasons why they need to be built with the boot JDK. Historically they were upstream repositories so they were usually based on a backrev JDK anyway, so there was no real need for them to use the latest features. Also, using the boot JDK was probably an incidental outcome of the way the old build system was put together. (Old-timers lurking -- or not :-) -- on this list will certainly know better than me.) The new build system does the same, since one of its requirements is that it slavishly match the output of the old build system. In principle I don't see any reason why these libraries couldn't be built with the current JDK instead of the boot JDK. This might be a fairly large restructuring of the build system, though. My understanding is that the new build is just following the old build (a while back, I did ask about the same issue and I remember Kelly or someone pointing out that the new build had to follow the same build sequence as the old in order to generate equivalent bits). Although you're anxious to get the README updates then it is a great topic and now might be a great time to discuss it. As least for the jaxws repository then I don't see any reason that it has to be built by the boot JDK. I actually don't see any reason why it needs to be rt.jar either, it could be built to jaxws.jar and dropped into the extensions directory (but that's another matter). That said, even if the jaxws repository was built later then it's unlikely that code there will start making use of new language features or APIs. That is because it's really an upstream project that still creates standalone releases to run on jdk7. The corba and jaxp are trickier because there are circular dependencies. These circular dependencies arises because we have a few places in the jdk repository that need to parse XML and there are few cases where CORBA APIs are used (the JMX remote API supports IIOP, and the JNDI CosNaming provider are two). As least for the CORBA APIs then we've already done significant refactoring to support modularization and it might not be big job to move the CORBA dependencies into the corba repo. That said, CORBA is mostly just bug fixes these days so there isn't a queue at the door to use new language features or APIs. On the other hand there is still a case to building it with the newly built jdk because of sun.* and other implementation mess. We have several examples in the last year where building the corba repository with the boot JDK has been a problem (a duplicate shared secrets mechanism had to be added for example). The jaxp repository is a clear case where we should be able to new new language features and APIs. Aside from the build then the only thing holding things back is periodically calls to keep the jdk7u and jdk8 code in sync. I don't know how long that can last and we will need to break from the past at some point. So I think this is a great topic to discuss. I think it has to be discussed in the context of where the repositories are going with modules. For the JDK modularization then we've long talked about restructuring the source code so that it's organized by modules, we didn't get to considering whether this would have an impact on the repositories. I think it's also worth discussing whether the "compilation unit" is the code in a single repository or whether the code in multiple repositories could be compiled together. The other extreme is where we get to the point where individual modules can be compiled on their own. -Alan.
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 06/18/2013 10:02 AM, David Holmes wrote: Hi Stuart, > I would like people to review the README change as well. Thanks. I don't think we should simply say "Do not use a build of JDK 8 as the boot JDK for building JDK 8." as that doesn't explain what the issue is. If I'm building the JDK for my own use I can use JDK8. So how about: "JDK 8 developers should not use JDK 8 as the boot JDK, to ensure that code changes are compatible with building using JDK 7." This suggested wording, along with Stuarts other additions to the README, look fine. I think this should be pushed, and the other issues handled separately. -Chris. ? David On 18/06/2013 5:42 PM, Stuart Marks wrote: Hi folks, Looks like I generated a bit of discussion here. Let's try to tease apart some of the issues. 1) I think we need a better articulation of the rule about the boot JDK being N-1, thus my proposed change to the README. I don't mean to ever prohibit anybody from ever trying to build JDK N with a boot JDK N, but that should be a special case. If the wording I proposed isn't satisfactory, please suggest alternatives. 2) Boot cycle build. Yes, after building JDK N, we must support a boot cycle build where the just-built JDK N is used as the boot JDK to build itself again. This is really a special case within the build system, however. At configure time, the boot JDK always should be N-1, and when the build gets to the boot cycle phase, it either knows or can be taught to treat this phase differently from the first phase. 3) Rules enforcement / mistake avoidance. I think it makes the most sense for configure to ensure that the configured boot JDK is N-1. (Naturally the configured boot JDK will be overridden during the boot cycle phase. It could also be overridden by specifying an additional option explicitly.) The main point here is to avoid mistakes, so that someone who happens to have a JDK 8 in their path doesn't accidentally have it picked up by configure. Unfortunately the -source 7 -target 7 approach is insufficient, because while it rolls back the language level and classfile version, it does *not* roll back the API version. To get the old API version, one has to construct a bootclasspath with the old class libraries, and at that point one might as well just use the N-1 JDK. #include 4) Could jaxp, jaxws, and corba be built with the current JDK, not the boot JDK? Sure, probably. I spoke with Jon G on this topic the other day and we didn't come up with any really good reasons why they need to be built with the boot JDK. Historically they were upstream repositories so they were usually based on a backrev JDK anyway, so there was no real need for them to use the latest features. Also, using the boot JDK was probably an incidental outcome of the way the old build system was put together. (Old-timers lurking -- or not :-) -- on this list will certainly know better than me.) The new build system does the same, since one of its requirements is that it slavishly match the output of the old build system. In principle I don't see any reason why these libraries couldn't be built with the current JDK instead of the boot JDK. This might be a fairly large restructuring of the build system, though. Of course, langtools still needs to be built with the JDK N-1 boot JDK. I'm not sure about the Java files in hotspot. Can someone enlighten us? -- If people think it's a good idea I could file RFEs for #3 and #4. I would like people to review the README change as well. Thanks. s'marks On 6/17/13 11:57 PM, Daniel Fuchs wrote: On 6/18/13 8:28 AM, David Holmes wrote: On 18/06/2013 4:02 PM, Jonathan Gibbons wrote: The only problem with using N is that you don't know whether you have broken building with N-1. Therefore the general recommendation for most people should be to always use N-1. I think Stuart is just searching for ways to make people aware that using N-1 is "the right thing to do". There was certainly an issue here that caused a problem because the code used a JDK8 API that was not available when the source was compiled with JDK7. And sure compiling with 7u boot JDK would have caught that. But we have lots of code that only compiles with JDK8 and that is the way we want it, else JDK8 could not take advantage of any new language features or APIs in JDK8. The real problem here was that the code in question is code that is not built in a way that allows it to use the latest language features or APIs. In which case perhaps the real fix is to use build commands that enforce this restriction ie by using -source 7 -target 7 ? David Hi David, In the case of Jaxp - I'm not sure why exactly is Jaxp compiled with the boot JDK. It comes early in the build cycle - at a time when the N JDK hasn't been compiled yet. But is this a mere convenience - or is there a good technical reason for this? Because personally - I would love to be able to use JDK N feature in the JAXP source code. One could argue that using N feat
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
Hi Erik, On 18/06/2013 6:06 PM, Erik Joelsson wrote: On 2013-06-18 08:57, Daniel Fuchs wrote: On 6/18/13 8:28 AM, David Holmes wrote: On 18/06/2013 4:02 PM, Jonathan Gibbons wrote: The only problem with using N is that you don't know whether you have broken building with N-1. Therefore the general recommendation for most people should be to always use N-1. I think Stuart is just searching for ways to make people aware that using N-1 is "the right thing to do". There was certainly an issue here that caused a problem because the code used a JDK8 API that was not available when the source was compiled with JDK7. And sure compiling with 7u boot JDK would have caught that. But we have lots of code that only compiles with JDK8 and that is the way we want it, else JDK8 could not take advantage of any new language features or APIs in JDK8. The real problem here was that the code in question is code that is not built in a way that allows it to use the latest language features or APIs. In which case perhaps the real fix is to use build commands that enforce this restriction ie by using -source 7 -target 7 ? David Hi David, In the case of Jaxp - I'm not sure why exactly is Jaxp compiled with the boot JDK. It comes early in the build cycle - at a time when the N JDK hasn't been compiled yet. But is this a mere convenience - or is there a good technical reason for this? Because personally - I would love to be able to use JDK N feature in the JAXP source code. One could argue that using N features impairs the ability to port the fixes to N-1, but this is already the case today anyway: for many of the fixes I ported from 8 to 7 I had to modify my patches because I was using N-1 features in N, and therefore had to convert the code to only use N-2 features when porting from N to N-1. So if that is possible (and it may not be - I'm not a build expert) - I would argue to remove the restriction for Jaxp - rather than enforce it. After langtools has been built (which is first), the rest, including jaxp, is compiled using the bootstrap javac, which is a special version of the newly built javac that runs on the boot jdk but acts as the JDK N javac. This means JDK N language features are available, but not library features, since there is no access to the rest of the JDK N classes. Thanks for clarifying that. So to use new APIs you either have to be part of the build that constructs that API, or else your component has to be built after the new classes destined for rt.jar are available? But anyway this is getting a little off-track :) David -- This separation is just convenience as far as I can tell. It's technically possible to work around, but the gains will have to be weighed against the added complexity. Easiest solution would be to get rid of the separate jaxp repository and just put the source back with the jdk. /Erik best regards, -- daniel -- Jon On 06/17/2013 10:04 PM, David Holmes wrote: I thought the only rule was "must be buildable by N-1", not that you must not try to use N! Can the problem preventing a build using JDK8 as the boot JDK not be corrected? I'm assuming it is one of the more unusual parts of the build where we mess with bootclasspath etc? David On 18/06/2013 10:21 AM, Stuart Marks wrote: On 6/17/13 4:02 PM, Kelly O'Hair wrote: Rule #1 Nobody reads the README Rule #2 When things go wrong, blame the README I of course have no objection to the change, however, I'm not convinced it will help much the next time someone runs into this. :^( Hi Kelly! You still read this stuff here? :-) Yeah, I have no illusions that changing the README will prevent many, if any, future occurrences of this problem. However, we had an internal discussion on this incident where the N-1 rule was asserted. There was no dispute about the rule, but I went off to find where it was documented, and found only the fairly weak statement in the README. So, at the very least, that ought to be fixed. A stronger step would be to modify configure to check the version of the boot JDK and to complain if it doesn't match N-1. Or perhaps even N-1 and update >= 7. What do you think? I was considering filing an RFE. A restriction in configure would probably be more effective at preventing these kinds of errors. s'marks
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
Hi Stuart, > I would like people to review the README change as well. Thanks. I don't think we should simply say "Do not use a build of JDK 8 as the boot JDK for building JDK 8." as that doesn't explain what the issue is. If I'm building the JDK for my own use I can use JDK8. So how about: "JDK 8 developers should not use JDK 8 as the boot JDK, to ensure that code changes are compatible with building using JDK 7." ? David On 18/06/2013 5:42 PM, Stuart Marks wrote: Hi folks, Looks like I generated a bit of discussion here. Let's try to tease apart some of the issues. 1) I think we need a better articulation of the rule about the boot JDK being N-1, thus my proposed change to the README. I don't mean to ever prohibit anybody from ever trying to build JDK N with a boot JDK N, but that should be a special case. If the wording I proposed isn't satisfactory, please suggest alternatives. 2) Boot cycle build. Yes, after building JDK N, we must support a boot cycle build where the just-built JDK N is used as the boot JDK to build itself again. This is really a special case within the build system, however. At configure time, the boot JDK always should be N-1, and when the build gets to the boot cycle phase, it either knows or can be taught to treat this phase differently from the first phase. 3) Rules enforcement / mistake avoidance. I think it makes the most sense for configure to ensure that the configured boot JDK is N-1. (Naturally the configured boot JDK will be overridden during the boot cycle phase. It could also be overridden by specifying an additional option explicitly.) The main point here is to avoid mistakes, so that someone who happens to have a JDK 8 in their path doesn't accidentally have it picked up by configure. Unfortunately the -source 7 -target 7 approach is insufficient, because while it rolls back the language level and classfile version, it does *not* roll back the API version. To get the old API version, one has to construct a bootclasspath with the old class libraries, and at that point one might as well just use the N-1 JDK. #include 4) Could jaxp, jaxws, and corba be built with the current JDK, not the boot JDK? Sure, probably. I spoke with Jon G on this topic the other day and we didn't come up with any really good reasons why they need to be built with the boot JDK. Historically they were upstream repositories so they were usually based on a backrev JDK anyway, so there was no real need for them to use the latest features. Also, using the boot JDK was probably an incidental outcome of the way the old build system was put together. (Old-timers lurking -- or not :-) -- on this list will certainly know better than me.) The new build system does the same, since one of its requirements is that it slavishly match the output of the old build system. In principle I don't see any reason why these libraries couldn't be built with the current JDK instead of the boot JDK. This might be a fairly large restructuring of the build system, though. Of course, langtools still needs to be built with the JDK N-1 boot JDK. I'm not sure about the Java files in hotspot. Can someone enlighten us? -- If people think it's a good idea I could file RFEs for #3 and #4. I would like people to review the README change as well. Thanks. s'marks On 6/17/13 11:57 PM, Daniel Fuchs wrote: On 6/18/13 8:28 AM, David Holmes wrote: On 18/06/2013 4:02 PM, Jonathan Gibbons wrote: The only problem with using N is that you don't know whether you have broken building with N-1. Therefore the general recommendation for most people should be to always use N-1. I think Stuart is just searching for ways to make people aware that using N-1 is "the right thing to do". There was certainly an issue here that caused a problem because the code used a JDK8 API that was not available when the source was compiled with JDK7. And sure compiling with 7u boot JDK would have caught that. But we have lots of code that only compiles with JDK8 and that is the way we want it, else JDK8 could not take advantage of any new language features or APIs in JDK8. The real problem here was that the code in question is code that is not built in a way that allows it to use the latest language features or APIs. In which case perhaps the real fix is to use build commands that enforce this restriction ie by using -source 7 -target 7 ? David Hi David, In the case of Jaxp - I'm not sure why exactly is Jaxp compiled with the boot JDK. It comes early in the build cycle - at a time when the N JDK hasn't been compiled yet. But is this a mere convenience - or is there a good technical reason for this? Because personally - I would love to be able to use JDK N feature in the JAXP source code. One could argue that using N features impairs the ability to port the fixes to N-1, but this is already the case today anyway: for many of the fixes I ported from 8 to 7 I had to modify my patches because I was using N-1 features in N, and therefo
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 2013-06-18 08:57, Daniel Fuchs wrote: On 6/18/13 8:28 AM, David Holmes wrote: On 18/06/2013 4:02 PM, Jonathan Gibbons wrote: The only problem with using N is that you don't know whether you have broken building with N-1. Therefore the general recommendation for most people should be to always use N-1. I think Stuart is just searching for ways to make people aware that using N-1 is "the right thing to do". There was certainly an issue here that caused a problem because the code used a JDK8 API that was not available when the source was compiled with JDK7. And sure compiling with 7u boot JDK would have caught that. But we have lots of code that only compiles with JDK8 and that is the way we want it, else JDK8 could not take advantage of any new language features or APIs in JDK8. The real problem here was that the code in question is code that is not built in a way that allows it to use the latest language features or APIs. In which case perhaps the real fix is to use build commands that enforce this restriction ie by using -source 7 -target 7 ? David Hi David, In the case of Jaxp - I'm not sure why exactly is Jaxp compiled with the boot JDK. It comes early in the build cycle - at a time when the N JDK hasn't been compiled yet. But is this a mere convenience - or is there a good technical reason for this? Because personally - I would love to be able to use JDK N feature in the JAXP source code. One could argue that using N features impairs the ability to port the fixes to N-1, but this is already the case today anyway: for many of the fixes I ported from 8 to 7 I had to modify my patches because I was using N-1 features in N, and therefore had to convert the code to only use N-2 features when porting from N to N-1. So if that is possible (and it may not be - I'm not a build expert) - I would argue to remove the restriction for Jaxp - rather than enforce it. After langtools has been built (which is first), the rest, including jaxp, is compiled using the bootstrap javac, which is a special version of the newly built javac that runs on the boot jdk but acts as the JDK N javac. This means JDK N language features are available, but not library features, since there is no access to the rest of the JDK N classes. This separation is just convenience as far as I can tell. It's technically possible to work around, but the gains will have to be weighed against the added complexity. Easiest solution would be to get rid of the separate jaxp repository and just put the source back with the jdk. /Erik best regards, -- daniel -- Jon On 06/17/2013 10:04 PM, David Holmes wrote: I thought the only rule was "must be buildable by N-1", not that you must not try to use N! Can the problem preventing a build using JDK8 as the boot JDK not be corrected? I'm assuming it is one of the more unusual parts of the build where we mess with bootclasspath etc? David On 18/06/2013 10:21 AM, Stuart Marks wrote: On 6/17/13 4:02 PM, Kelly O'Hair wrote: Rule #1 Nobody reads the README Rule #2 When things go wrong, blame the README I of course have no objection to the change, however, I'm not convinced it will help much the next time someone runs into this. :^( Hi Kelly! You still read this stuff here? :-) Yeah, I have no illusions that changing the README will prevent many, if any, future occurrences of this problem. However, we had an internal discussion on this incident where the N-1 rule was asserted. There was no dispute about the rule, but I went off to find where it was documented, and found only the fairly weak statement in the README. So, at the very least, that ought to be fixed. A stronger step would be to modify configure to check the version of the boot JDK and to complain if it doesn't match N-1. Or perhaps even N-1 and update >= 7. What do you think? I was considering filing an RFE. A restriction in configure would probably be more effective at preventing these kinds of errors. s'marks
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
Hi folks, Looks like I generated a bit of discussion here. Let's try to tease apart some of the issues. 1) I think we need a better articulation of the rule about the boot JDK being N-1, thus my proposed change to the README. I don't mean to ever prohibit anybody from ever trying to build JDK N with a boot JDK N, but that should be a special case. If the wording I proposed isn't satisfactory, please suggest alternatives. 2) Boot cycle build. Yes, after building JDK N, we must support a boot cycle build where the just-built JDK N is used as the boot JDK to build itself again. This is really a special case within the build system, however. At configure time, the boot JDK always should be N-1, and when the build gets to the boot cycle phase, it either knows or can be taught to treat this phase differently from the first phase. 3) Rules enforcement / mistake avoidance. I think it makes the most sense for configure to ensure that the configured boot JDK is N-1. (Naturally the configured boot JDK will be overridden during the boot cycle phase. It could also be overridden by specifying an additional option explicitly.) The main point here is to avoid mistakes, so that someone who happens to have a JDK 8 in their path doesn't accidentally have it picked up by configure. Unfortunately the -source 7 -target 7 approach is insufficient, because while it rolls back the language level and classfile version, it does *not* roll back the API version. To get the old API version, one has to construct a bootclasspath with the old class libraries, and at that point one might as well just use the N-1 JDK. #include 4) Could jaxp, jaxws, and corba be built with the current JDK, not the boot JDK? Sure, probably. I spoke with Jon G on this topic the other day and we didn't come up with any really good reasons why they need to be built with the boot JDK. Historically they were upstream repositories so they were usually based on a backrev JDK anyway, so there was no real need for them to use the latest features. Also, using the boot JDK was probably an incidental outcome of the way the old build system was put together. (Old-timers lurking -- or not :-) -- on this list will certainly know better than me.) The new build system does the same, since one of its requirements is that it slavishly match the output of the old build system. In principle I don't see any reason why these libraries couldn't be built with the current JDK instead of the boot JDK. This might be a fairly large restructuring of the build system, though. Of course, langtools still needs to be built with the JDK N-1 boot JDK. I'm not sure about the Java files in hotspot. Can someone enlighten us? -- If people think it's a good idea I could file RFEs for #3 and #4. I would like people to review the README change as well. Thanks. s'marks On 6/17/13 11:57 PM, Daniel Fuchs wrote: On 6/18/13 8:28 AM, David Holmes wrote: On 18/06/2013 4:02 PM, Jonathan Gibbons wrote: The only problem with using N is that you don't know whether you have broken building with N-1. Therefore the general recommendation for most people should be to always use N-1. I think Stuart is just searching for ways to make people aware that using N-1 is "the right thing to do". There was certainly an issue here that caused a problem because the code used a JDK8 API that was not available when the source was compiled with JDK7. And sure compiling with 7u boot JDK would have caught that. But we have lots of code that only compiles with JDK8 and that is the way we want it, else JDK8 could not take advantage of any new language features or APIs in JDK8. The real problem here was that the code in question is code that is not built in a way that allows it to use the latest language features or APIs. In which case perhaps the real fix is to use build commands that enforce this restriction ie by using -source 7 -target 7 ? David Hi David, In the case of Jaxp - I'm not sure why exactly is Jaxp compiled with the boot JDK. It comes early in the build cycle - at a time when the N JDK hasn't been compiled yet. But is this a mere convenience - or is there a good technical reason for this? Because personally - I would love to be able to use JDK N feature in the JAXP source code. One could argue that using N features impairs the ability to port the fixes to N-1, but this is already the case today anyway: for many of the fixes I ported from 8 to 7 I had to modify my patches because I was using N-1 features in N, and therefore had to convert the code to only use N-2 features when porting from N to N-1. So if that is possible (and it may not be - I'm not a build expert) - I would argue to remove the restriction for Jaxp - rather than enforce it. best regards, -- daniel -- Jon On 06/17/2013 10:04 PM, David Holmes wrote: I thought the only rule was "must be buildable by N-1", not that you must not try to use N! Can the problem preventing a build usi
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 6/18/13 8:28 AM, David Holmes wrote: On 18/06/2013 4:02 PM, Jonathan Gibbons wrote: The only problem with using N is that you don't know whether you have broken building with N-1. Therefore the general recommendation for most people should be to always use N-1. I think Stuart is just searching for ways to make people aware that using N-1 is "the right thing to do". There was certainly an issue here that caused a problem because the code used a JDK8 API that was not available when the source was compiled with JDK7. And sure compiling with 7u boot JDK would have caught that. But we have lots of code that only compiles with JDK8 and that is the way we want it, else JDK8 could not take advantage of any new language features or APIs in JDK8. The real problem here was that the code in question is code that is not built in a way that allows it to use the latest language features or APIs. In which case perhaps the real fix is to use build commands that enforce this restriction ie by using -source 7 -target 7 ? David Hi David, In the case of Jaxp - I'm not sure why exactly is Jaxp compiled with the boot JDK. It comes early in the build cycle - at a time when the N JDK hasn't been compiled yet. But is this a mere convenience - or is there a good technical reason for this? Because personally - I would love to be able to use JDK N feature in the JAXP source code. One could argue that using N features impairs the ability to port the fixes to N-1, but this is already the case today anyway: for many of the fixes I ported from 8 to 7 I had to modify my patches because I was using N-1 features in N, and therefore had to convert the code to only use N-2 features when porting from N to N-1. So if that is possible (and it may not be - I'm not a build expert) - I would argue to remove the restriction for Jaxp - rather than enforce it. best regards, -- daniel -- Jon On 06/17/2013 10:04 PM, David Holmes wrote: I thought the only rule was "must be buildable by N-1", not that you must not try to use N! Can the problem preventing a build using JDK8 as the boot JDK not be corrected? I'm assuming it is one of the more unusual parts of the build where we mess with bootclasspath etc? David On 18/06/2013 10:21 AM, Stuart Marks wrote: On 6/17/13 4:02 PM, Kelly O'Hair wrote: Rule #1 Nobody reads the README Rule #2 When things go wrong, blame the README I of course have no objection to the change, however, I'm not convinced it will help much the next time someone runs into this. :^( Hi Kelly! You still read this stuff here? :-) Yeah, I have no illusions that changing the README will prevent many, if any, future occurrences of this problem. However, we had an internal discussion on this incident where the N-1 rule was asserted. There was no dispute about the rule, but I went off to find where it was documented, and found only the fairly weak statement in the README. So, at the very least, that ought to be fixed. A stronger step would be to modify configure to check the version of the boot JDK and to complain if it doesn't match N-1. Or perhaps even N-1 and update >= 7. What do you think? I was considering filing an RFE. A restriction in configure would probably be more effective at preventing these kinds of errors. s'marks
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 18/06/2013 4:02 PM, Jonathan Gibbons wrote: The only problem with using N is that you don't know whether you have broken building with N-1. Therefore the general recommendation for most people should be to always use N-1. I think Stuart is just searching for ways to make people aware that using N-1 is "the right thing to do". There was certainly an issue here that caused a problem because the code used a JDK8 API that was not available when the source was compiled with JDK7. And sure compiling with 7u boot JDK would have caught that. But we have lots of code that only compiles with JDK8 and that is the way we want it, else JDK8 could not take advantage of any new language features or APIs in JDK8. The real problem here was that the code in question is code that is not built in a way that allows it to use the latest language features or APIs. In which case perhaps the real fix is to use build commands that enforce this restriction ie by using -source 7 -target 7 ? David -- Jon On 06/17/2013 10:04 PM, David Holmes wrote: I thought the only rule was "must be buildable by N-1", not that you must not try to use N! Can the problem preventing a build using JDK8 as the boot JDK not be corrected? I'm assuming it is one of the more unusual parts of the build where we mess with bootclasspath etc? David On 18/06/2013 10:21 AM, Stuart Marks wrote: On 6/17/13 4:02 PM, Kelly O'Hair wrote: Rule #1 Nobody reads the README Rule #2 When things go wrong, blame the README I of course have no objection to the change, however, I'm not convinced it will help much the next time someone runs into this. :^( Hi Kelly! You still read this stuff here? :-) Yeah, I have no illusions that changing the README will prevent many, if any, future occurrences of this problem. However, we had an internal discussion on this incident where the N-1 rule was asserted. There was no dispute about the rule, but I went off to find where it was documented, and found only the fairly weak statement in the README. So, at the very least, that ought to be fixed. A stronger step would be to modify configure to check the version of the boot JDK and to complain if it doesn't match N-1. Or perhaps even N-1 and update >= 7. What do you think? I was considering filing an RFE. A restriction in configure would probably be more effective at preventing these kinds of errors. s'marks
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 6/17/2013 6:22 PM, Jonathan Gibbons wrote: On 06/17/2013 05:21 PM, Stuart Marks wrote: On 6/17/13 4:02 PM, Kelly O'Hair wrote: Rule #1 Nobody reads the README Rule #2 When things go wrong, blame the README I of course have no objection to the change, however, I'm not convinced it will help much the next time someone runs into this. :^( Hi Kelly! You still read this stuff here? :-) Yeah, I have no illusions that changing the README will prevent many, if any, future occurrences of this problem. However, we had an internal discussion on this incident where the N-1 rule was asserted. There was no dispute about the rule, but I went off to find where it was documented, and found only the fairly weak statement in the README. So, at the very least, that ought to be fixed. A stronger step would be to modify configure to check the version of the boot JDK and to complain if it doesn't match N-1. Or perhaps even N-1 and update >= 7. What do you think? I was considering filing an RFE. A restriction in configure would probably be more effective at preventing these kinds of errors. s'marks Stuart, At least make sure you don't break "make bootcycle-images" which deliberately uses the newly built JDK to build itself again. Very important. Eventually N will become the N -1, so passing that boot-cycle test is a major requirement for safely pushing any hotspot update into jdkN -- Alejandro
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
The only problem with using N is that you don't know whether you have broken building with N-1. Therefore the general recommendation for most people should be to always use N-1. I think Stuart is just searching for ways to make people aware that using N-1 is "the right thing to do". -- Jon On 06/17/2013 10:04 PM, David Holmes wrote: I thought the only rule was "must be buildable by N-1", not that you must not try to use N! Can the problem preventing a build using JDK8 as the boot JDK not be corrected? I'm assuming it is one of the more unusual parts of the build where we mess with bootclasspath etc? David On 18/06/2013 10:21 AM, Stuart Marks wrote: On 6/17/13 4:02 PM, Kelly O'Hair wrote: Rule #1 Nobody reads the README Rule #2 When things go wrong, blame the README I of course have no objection to the change, however, I'm not convinced it will help much the next time someone runs into this. :^( Hi Kelly! You still read this stuff here? :-) Yeah, I have no illusions that changing the README will prevent many, if any, future occurrences of this problem. However, we had an internal discussion on this incident where the N-1 rule was asserted. There was no dispute about the rule, but I went off to find where it was documented, and found only the fairly weak statement in the README. So, at the very least, that ought to be fixed. A stronger step would be to modify configure to check the version of the boot JDK and to complain if it doesn't match N-1. Or perhaps even N-1 and update >= 7. What do you think? I was considering filing an RFE. A restriction in configure would probably be more effective at preventing these kinds of errors. s'marks
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
I thought the only rule was "must be buildable by N-1", not that you must not try to use N! Can the problem preventing a build using JDK8 as the boot JDK not be corrected? I'm assuming it is one of the more unusual parts of the build where we mess with bootclasspath etc? David On 18/06/2013 10:21 AM, Stuart Marks wrote: On 6/17/13 4:02 PM, Kelly O'Hair wrote: Rule #1 Nobody reads the README Rule #2 When things go wrong, blame the README I of course have no objection to the change, however, I'm not convinced it will help much the next time someone runs into this. :^( Hi Kelly! You still read this stuff here? :-) Yeah, I have no illusions that changing the README will prevent many, if any, future occurrences of this problem. However, we had an internal discussion on this incident where the N-1 rule was asserted. There was no dispute about the rule, but I went off to find where it was documented, and found only the fairly weak statement in the README. So, at the very least, that ought to be fixed. A stronger step would be to modify configure to check the version of the boot JDK and to complain if it doesn't match N-1. Or perhaps even N-1 and update >= 7. What do you think? I was considering filing an RFE. A restriction in configure would probably be more effective at preventing these kinds of errors. s'marks
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 06/17/2013 05:21 PM, Stuart Marks wrote: On 6/17/13 4:02 PM, Kelly O'Hair wrote: Rule #1 Nobody reads the README Rule #2 When things go wrong, blame the README I of course have no objection to the change, however, I'm not convinced it will help much the next time someone runs into this. :^( Hi Kelly! You still read this stuff here? :-) Yeah, I have no illusions that changing the README will prevent many, if any, future occurrences of this problem. However, we had an internal discussion on this incident where the N-1 rule was asserted. There was no dispute about the rule, but I went off to find where it was documented, and found only the fairly weak statement in the README. So, at the very least, that ought to be fixed. A stronger step would be to modify configure to check the version of the boot JDK and to complain if it doesn't match N-1. Or perhaps even N-1 and update >= 7. What do you think? I was considering filing an RFE. A restriction in configure would probably be more effective at preventing these kinds of errors. s'marks Stuart, At least make sure you don't break "make bootcycle-images" which deliberately uses the newly built JDK to build itself again. -- Jon
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
On 6/17/13 4:02 PM, Kelly O'Hair wrote: Rule #1 Nobody reads the README Rule #2 When things go wrong, blame the README I of course have no objection to the change, however, I'm not convinced it will help much the next time someone runs into this. :^( Hi Kelly! You still read this stuff here? :-) Yeah, I have no illusions that changing the README will prevent many, if any, future occurrences of this problem. However, we had an internal discussion on this incident where the N-1 rule was asserted. There was no dispute about the rule, but I went off to find where it was documented, and found only the fairly weak statement in the README. So, at the very least, that ought to be fixed. A stronger step would be to modify configure to check the version of the boot JDK and to complain if it doesn't match N-1. Or perhaps even N-1 and update >= 7. What do you think? I was considering filing an RFE. A restriction in configure would probably be more effective at preventing these kinds of errors. s'marks
Re: RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
I couldn't find a good way to show me rolling my eyes, but I found this: http://www.tumblr.com/tagged/rolling%20eyes%20gif Rule #1 Nobody reads the README Rule #2 When things go wrong, blame the README I of course have no objection to the change, however, I'm not convinced it will help much the next time someone runs into this. :^( It is important that all developers feel empowered to contribute to this README file. -kto On Jun 17, 2013, at 3:30 PM, Stuart Marks wrote: > Hi all, > > We had a problem in TL the other day [1] [2] that wasn't caught because a > developer was using a JDK 8 build as his boot JDK. Turns out the rule to use > JDK N-1 as the boot JDK for JDK N isn't specified clearly in > README-builds.html. Here's a diff to strengthen the wording in that file. > > Also, is it OK if I push this into TL? > > Thanks, > > s'marks > > [1] http://hg.openjdk.java.net/jdk8/tl/jaxp/rev/659828443145 > > [2] http://hg.openjdk.java.net/jdk8/tl/jaxp/rev/2707f600a096 > > > > > diff -r ea6f3bf82903 README-builds.html > --- a/README-builds.html Tue Jun 04 00:12:51 2013 -0400 > +++ b/README-builds.html Mon Jun 17 15:29:31 2013 -0700 > @@ -355,12 +355,20 @@ > > > Install a > -Bootstrap JDK > - > +Bootstrap JDK. > All OpenJDK builds require access to a previously > released > -JDK, this is often called a bootstrap JDK. > -Currently, for this JDK release we require > -JDK 7 Update 7 or newer. > +JDK called the bootstrap JDK or boot > JDK. > +The general rule is that the bootstrap JDK > +must be an instance of the previous major > +release of the JDK. In addition, there may be > +a requirement to use a release at or beyond a > +particular update level. > + > +Building JDK 8 requires use of a version of > +JDK 7 that is at Update 7 or newer. Do not use > +a build of JDK 8 as the boot JDK for building > +JDK 8. > + > The JDK 7 binaries can be downloaded from Oracle's > href="http://www.oracle.com/technetwork/java/javase/downloads/index.html"; >target="_blank">JDK 7 download site. >
RFR: 8016780: (xs) README-builds.html misses crucial requirement on bootstrap JDK
Hi all, We had a problem in TL the other day [1] [2] that wasn't caught because a developer was using a JDK 8 build as his boot JDK. Turns out the rule to use JDK N-1 as the boot JDK for JDK N isn't specified clearly in README-builds.html. Here's a diff to strengthen the wording in that file. Also, is it OK if I push this into TL? Thanks, s'marks [1] http://hg.openjdk.java.net/jdk8/tl/jaxp/rev/659828443145 [2] http://hg.openjdk.java.net/jdk8/tl/jaxp/rev/2707f600a096 diff -r ea6f3bf82903 README-builds.html --- a/README-builds.htmlTue Jun 04 00:12:51 2013 -0400 +++ b/README-builds.htmlMon Jun 17 15:29:31 2013 -0700 @@ -355,12 +355,20 @@ Install a -Bootstrap JDK - +Bootstrap JDK. All OpenJDK builds require access to a previously released -JDK, this is often called a bootstrap JDK. -Currently, for this JDK release we require -JDK 7 Update 7 or newer. +JDK called the bootstrap JDK or boot JDK. +The general rule is that the bootstrap JDK +must be an instance of the previous major +release of the JDK. In addition, there may be +a requirement to use a release at or beyond a +particular update level. + +Building JDK 8 requires use of a version of +JDK 7 that is at Update 7 or newer. Do not use +a build of JDK 8 as the boot JDK for building +JDK 8. + The JDK 7 binaries can be downloaded from Oracle's href="http://www.oracle.com/technetwork/java/javase/downloads/index.html"; target="_blank">JDK 7 download site.
