Re: RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling
On Thu, Jun 6, 2019 at 10:31 PM Erik Joelsson wrote:
>
> Hello,
>
> On 2019-06-05 20:28, Ao Qi wrote:
> > On Thu, Jun 6, 2019 at 10:49 AM Ao Qi wrote:
> >> On Thu, Jun 6, 2019 at 4:46 AM Erik Joelsson
> >> wrote:
> >>> Here is a new webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.03
> > Another question: should "[bootcycle] " be displayed during make
> > bootcycle-images? If yes, should we remove line 164 "LOG_PREFIX :=" of
> > http://cr.openjdk.java.net/~erikj/8217739/webrev.03/make/common/MakeBase.gmk.html
> > ?
>
> It seems you are right. I assumed that by setting LOG_PREFIX on the make
> command line, it would override. I think it's cleaner to initiate the
> variable to empty so that any accidental setting in the environment
> doesn't interfere. This works for the buildjdk call, which calls
> Main.gmk, but doesn't seem to work for bootcycle, which calls Init.gmk.
>
> For now I will just refrain from initializing it in MakeBase.gmk.
>
> Webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.04
Looks good. Thanks for fixing this!
Cheers,
Ao Qi
>
> >> I (as an Author) am ok with this fix. mips64el zero cross build passed
> >> and javac (to test open a file) worked.
>
> Thanks!
>
> /Erik
>
> >>
> >> UnixConstants.java of build:
> >> .../usr/bin/cc -E -x c -I...
> >>
> >> UnixConstants.java of target:
> >> .../usr/bin/mips64el-linux-gnuabi64-gcc -E -x c
> >> --sysroot=/home/aoqi//chroots/mips64el_stretch/ -I...
> >>
> >> Thanks,
> >> Ao Qi
> >>> The change is in buildjdk-spec.gmk.in where CPP is now also overridden.
> >>>
> >>> /Erik
> >>>
> >>> On 2019-06-05 10:33, Erik Joelsson wrote:
> Never mind, I found the issue. The reason it worked for me was that we
> set SYSROOT_CFLAGS for both the target and the build compiler (from
> using devkits for them). The preprocessor for the target would then
> act correctly given the --sysroot parameter pointing to a sysroot for
> the build system.
>
> I'm working on a patch where PPC gets overridden with the build
> compiler in buildjdk-spec.gmk.
>
> /Erik
>
> On 2019-06-05 08:37, Erik Joelsson wrote:
> > Hello Ao Qi,
> >
> > In my testing, I tried building ppc64le (which I first identified as
> > generating a UnixConstants.java that differed from an x64 build).
> > With this patch, the buildjdk and target version of
> > UnixConstants.java differ the same way. How did you configure your
> > build? I'm suspecting something in the setup of the buildjdk
> > compilers is different from my configuration.
> >
> > /Erik
> >
> > On 2019-06-04 21:48, Ao Qi wrote:
> >> Hi Erik,
> >>
> >> Thank you a lot for taking care of this. I mainly like your approach.
> >> I tested the patch by cross building mips64-zero, by which we found
> >> this problem. However, it seems the problem still exists. The reason I
> >> found is that UnixConstants.java of buildjdk is generated by cross
> >> compiler, so the macro definitions are defined by the target. Here is
> >> a preliminary workaround for your reference only (only worked for
> >> gcc):
> >>
> >> $ hg diff make/gensrc/GensrcMisc.gmk
> >> diff -r f4df9d4201cc make/gensrc/GensrcMisc.gmk
> >> --- a/make/gensrc/GensrcMisc.gmk Tue Jun 04 16:50:25 2019 -0700
> >> +++ b/make/gensrc/GensrcMisc.gmk Wed Jun 05 12:39:43 2019 +0800
> >> @@ -61,6 +61,13 @@
> >> CPP_FLAGS += -nologo
> >>endif
> >>
> >> +ifeq ($(CREATING_BUILDJDK), true)
> >> + # Only worked for gcc
> >> + TEMP_CPP=gcc -E
> >> +else
> >> + TEMP_CPP=$(CPP)
> >> +endif
> >> +
> >># Generate a java source file from a template through the C
> >> preprocessor for the
> >># target system. First extract the copyright notice at the start
> >> of the file.
> >># Run the preprocessor. Filter out the default compiler stderr
> >> output on
> >> @@ -71,7 +78,7 @@
> >>define generate-preproc-src
> >> $(call MakeDir, $(@D))
> >> ( $(NAWK) '/@@END_COPYRIGHT@@/{exit}1' $< && \
> >> - $(CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
> >> + $(TEMP_CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
> >> 2> >($(GREP) -v '^$(&2) \
> >> | $(NAWK) '/@@START_HERE@@/,0' \
> >> | $(SED) -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED
> >> FILE - DO NOT EDIT/' \
> >>
> >>
> >> Cheers,
> >> Ao Qi
> >>
> >> On Wed, Jun 5, 2019 at 12:00 AM Erik Joelsson
> >> wrote:
> >>> Thanks!
> >>>
> >>> During testing, I found an intermittent race issue. The command line
> >>> that runs the interim langtools version of javac still always
> >>> points to
> >>> the buildtools/interim-langtools dir of the normal build. This
> >>> variable
> >>> also needs to be rewritten in buildjdk-spec.gmk.in. New
Re: RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling
Hello,
On 2019-06-05 20:28, Ao Qi wrote:
On Thu, Jun 6, 2019 at 10:49 AM Ao Qi wrote:
On Thu, Jun 6, 2019 at 4:46 AM Erik Joelsson wrote:
Here is a new webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.03
Another question: should "[bootcycle] " be displayed during make
bootcycle-images? If yes, should we remove line 164 "LOG_PREFIX :=" of
http://cr.openjdk.java.net/~erikj/8217739/webrev.03/make/common/MakeBase.gmk.html
?
It seems you are right. I assumed that by setting LOG_PREFIX on the make
command line, it would override. I think it's cleaner to initiate the
variable to empty so that any accidental setting in the environment
doesn't interfere. This works for the buildjdk call, which calls
Main.gmk, but doesn't seem to work for bootcycle, which calls Init.gmk.
For now I will just refrain from initializing it in MakeBase.gmk.
Webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.04
I (as an Author) am ok with this fix. mips64el zero cross build passed
and javac (to test open a file) worked.
Thanks!
/Erik
UnixConstants.java of build:
.../usr/bin/cc -E -x c -I...
UnixConstants.java of target:
.../usr/bin/mips64el-linux-gnuabi64-gcc -E -x c
--sysroot=/home/aoqi//chroots/mips64el_stretch/ -I...
Thanks,
Ao Qi
The change is in buildjdk-spec.gmk.in where CPP is now also overridden.
/Erik
On 2019-06-05 10:33, Erik Joelsson wrote:
Never mind, I found the issue. The reason it worked for me was that we
set SYSROOT_CFLAGS for both the target and the build compiler (from
using devkits for them). The preprocessor for the target would then
act correctly given the --sysroot parameter pointing to a sysroot for
the build system.
I'm working on a patch where PPC gets overridden with the build
compiler in buildjdk-spec.gmk.
/Erik
On 2019-06-05 08:37, Erik Joelsson wrote:
Hello Ao Qi,
In my testing, I tried building ppc64le (which I first identified as
generating a UnixConstants.java that differed from an x64 build).
With this patch, the buildjdk and target version of
UnixConstants.java differ the same way. How did you configure your
build? I'm suspecting something in the setup of the buildjdk
compilers is different from my configuration.
/Erik
On 2019-06-04 21:48, Ao Qi wrote:
Hi Erik,
Thank you a lot for taking care of this. I mainly like your approach.
I tested the patch by cross building mips64-zero, by which we found
this problem. However, it seems the problem still exists. The reason I
found is that UnixConstants.java of buildjdk is generated by cross
compiler, so the macro definitions are defined by the target. Here is
a preliminary workaround for your reference only (only worked for
gcc):
$ hg diff make/gensrc/GensrcMisc.gmk
diff -r f4df9d4201cc make/gensrc/GensrcMisc.gmk
--- a/make/gensrc/GensrcMisc.gmk Tue Jun 04 16:50:25 2019 -0700
+++ b/make/gensrc/GensrcMisc.gmk Wed Jun 05 12:39:43 2019 +0800
@@ -61,6 +61,13 @@
CPP_FLAGS += -nologo
endif
+ifeq ($(CREATING_BUILDJDK), true)
+ # Only worked for gcc
+ TEMP_CPP=gcc -E
+else
+ TEMP_CPP=$(CPP)
+endif
+
# Generate a java source file from a template through the C
preprocessor for the
# target system. First extract the copyright notice at the start
of the file.
# Run the preprocessor. Filter out the default compiler stderr
output on
@@ -71,7 +78,7 @@
define generate-preproc-src
$(call MakeDir, $(@D))
( $(NAWK) '/@@END_COPYRIGHT@@/{exit}1' $< && \
- $(CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
+ $(TEMP_CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
2> >($(GREP) -v '^$(&2) \
| $(NAWK) '/@@START_HERE@@/,0' \
| $(SED) -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED
FILE - DO NOT EDIT/' \
Cheers,
Ao Qi
On Wed, Jun 5, 2019 at 12:00 AM Erik Joelsson
wrote:
Thanks!
During testing, I found an intermittent race issue. The command line
that runs the interim langtools version of javac still always
points to
the buildtools/interim-langtools dir of the normal build. This
variable
also needs to be rewritten in buildjdk-spec.gmk.in. New webrev, only
changed in buildjdk-spec.gmk.in:
http://cr.openjdk.java.net/~erikj/8217739/webrev.02/
/Erik
On 2019-06-04 07:53, Tim Bell wrote:
Erik:
Looks good.
Tim
This is a fix for a problem brought up on this list [1]. The main
issue
is that when cross compiling, and creating a buildjdk during the
build,
the UnixConstants.java cannot necessarily be reused between the
target
JDK and the buildjdk. While investigating this, I came to the
conclusion
that we really should stop trying to reuse anything between the
buildjdk
and target JDK. It's just becoming too complex to maintain (due
to the
intricate interactions with the interim-image,
generate-link-opt-data
and exploded-image-optimize targets, causing circular
dependencies and
endless incremental rebuilds).
So this fix simply removes all the reuse of buildtools, gensrc
and java
compilation when creating a buildjdk. Hopefully this is making
Re: RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling
On Thu, Jun 6, 2019 at 10:49 AM Ao Qi wrote:
>
> On Thu, Jun 6, 2019 at 4:46 AM Erik Joelsson wrote:
> >
> > Here is a new webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.03
Another question: should "[bootcycle] " be displayed during make
bootcycle-images? If yes, should we remove line 164 "LOG_PREFIX :=" of
http://cr.openjdk.java.net/~erikj/8217739/webrev.03/make/common/MakeBase.gmk.html
?
>
> I (as an Author) am ok with this fix. mips64el zero cross build passed
> and javac (to test open a file) worked.
>
> UnixConstants.java of build:
> .../usr/bin/cc -E -x c -I...
>
> UnixConstants.java of target:
> .../usr/bin/mips64el-linux-gnuabi64-gcc -E -x c
> --sysroot=/home/aoqi//chroots/mips64el_stretch/ -I...
>
> Thanks,
> Ao Qi
> >
> > The change is in buildjdk-spec.gmk.in where CPP is now also overridden.
> >
> > /Erik
> >
> > On 2019-06-05 10:33, Erik Joelsson wrote:
> > > Never mind, I found the issue. The reason it worked for me was that we
> > > set SYSROOT_CFLAGS for both the target and the build compiler (from
> > > using devkits for them). The preprocessor for the target would then
> > > act correctly given the --sysroot parameter pointing to a sysroot for
> > > the build system.
> > >
> > > I'm working on a patch where PPC gets overridden with the build
> > > compiler in buildjdk-spec.gmk.
> > >
> > > /Erik
> > >
> > > On 2019-06-05 08:37, Erik Joelsson wrote:
> > >> Hello Ao Qi,
> > >>
> > >> In my testing, I tried building ppc64le (which I first identified as
> > >> generating a UnixConstants.java that differed from an x64 build).
> > >> With this patch, the buildjdk and target version of
> > >> UnixConstants.java differ the same way. How did you configure your
> > >> build? I'm suspecting something in the setup of the buildjdk
> > >> compilers is different from my configuration.
> > >>
> > >> /Erik
> > >>
> > >> On 2019-06-04 21:48, Ao Qi wrote:
> > >>> Hi Erik,
> > >>>
> > >>> Thank you a lot for taking care of this. I mainly like your approach.
> > >>> I tested the patch by cross building mips64-zero, by which we found
> > >>> this problem. However, it seems the problem still exists. The reason I
> > >>> found is that UnixConstants.java of buildjdk is generated by cross
> > >>> compiler, so the macro definitions are defined by the target. Here is
> > >>> a preliminary workaround for your reference only (only worked for
> > >>> gcc):
> > >>>
> > >>> $ hg diff make/gensrc/GensrcMisc.gmk
> > >>> diff -r f4df9d4201cc make/gensrc/GensrcMisc.gmk
> > >>> --- a/make/gensrc/GensrcMisc.gmk Tue Jun 04 16:50:25 2019 -0700
> > >>> +++ b/make/gensrc/GensrcMisc.gmk Wed Jun 05 12:39:43 2019 +0800
> > >>> @@ -61,6 +61,13 @@
> > >>> CPP_FLAGS += -nologo
> > >>> endif
> > >>>
> > >>> +ifeq ($(CREATING_BUILDJDK), true)
> > >>> + # Only worked for gcc
> > >>> + TEMP_CPP=gcc -E
> > >>> +else
> > >>> + TEMP_CPP=$(CPP)
> > >>> +endif
> > >>> +
> > >>> # Generate a java source file from a template through the C
> > >>> preprocessor for the
> > >>> # target system. First extract the copyright notice at the start
> > >>> of the file.
> > >>> # Run the preprocessor. Filter out the default compiler stderr
> > >>> output on
> > >>> @@ -71,7 +78,7 @@
> > >>> define generate-preproc-src
> > >>>$(call MakeDir, $(@D))
> > >>>( $(NAWK) '/@@END_COPYRIGHT@@/{exit}1' $< && \
> > >>> - $(CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
> > >>> + $(TEMP_CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
> > >>> 2> >($(GREP) -v '^$(&2) \
> > >>> | $(NAWK) '/@@START_HERE@@/,0' \
> > >>> | $(SED) -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED
> > >>> FILE - DO NOT EDIT/' \
> > >>>
> > >>>
> > >>> Cheers,
> > >>> Ao Qi
> > >>>
> > >>> On Wed, Jun 5, 2019 at 12:00 AM Erik Joelsson
> > >>> wrote:
> > Thanks!
> >
> > During testing, I found an intermittent race issue. The command line
> > that runs the interim langtools version of javac still always
> > points to
> > the buildtools/interim-langtools dir of the normal build. This
> > variable
> > also needs to be rewritten in buildjdk-spec.gmk.in. New webrev, only
> > changed in buildjdk-spec.gmk.in:
> >
> > http://cr.openjdk.java.net/~erikj/8217739/webrev.02/
> >
> > /Erik
> >
> > On 2019-06-04 07:53, Tim Bell wrote:
> > > Erik:
> > >
> > > Looks good.
> > >
> > > Tim
> > >
> > >> This is a fix for a problem brought up on this list [1]. The main
> > >> issue
> > >> is that when cross compiling, and creating a buildjdk during the
> > >> build,
> > >> the UnixConstants.java cannot necessarily be reused between the
> > >> target
> > >> JDK and the buildjdk. While investigating this, I came to the
> > >> conclusion
> > >> that we really should stop trying to reuse anything between the
> > >> buildjdk
> > >> and target JDK. It's just becoming too complex to maintain
Re: RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling
On Thu, Jun 6, 2019 at 4:46 AM Erik Joelsson wrote:
>
> Here is a new webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.03
I (as an Author) am ok with this fix. mips64el zero cross build passed
and javac (to test open a file) worked.
UnixConstants.java of build:
.../usr/bin/cc -E -x c -I...
UnixConstants.java of target:
.../usr/bin/mips64el-linux-gnuabi64-gcc -E -x c
--sysroot=/home/aoqi//chroots/mips64el_stretch/ -I...
Thanks,
Ao Qi
>
> The change is in buildjdk-spec.gmk.in where CPP is now also overridden.
>
> /Erik
>
> On 2019-06-05 10:33, Erik Joelsson wrote:
> > Never mind, I found the issue. The reason it worked for me was that we
> > set SYSROOT_CFLAGS for both the target and the build compiler (from
> > using devkits for them). The preprocessor for the target would then
> > act correctly given the --sysroot parameter pointing to a sysroot for
> > the build system.
> >
> > I'm working on a patch where PPC gets overridden with the build
> > compiler in buildjdk-spec.gmk.
> >
> > /Erik
> >
> > On 2019-06-05 08:37, Erik Joelsson wrote:
> >> Hello Ao Qi,
> >>
> >> In my testing, I tried building ppc64le (which I first identified as
> >> generating a UnixConstants.java that differed from an x64 build).
> >> With this patch, the buildjdk and target version of
> >> UnixConstants.java differ the same way. How did you configure your
> >> build? I'm suspecting something in the setup of the buildjdk
> >> compilers is different from my configuration.
> >>
> >> /Erik
> >>
> >> On 2019-06-04 21:48, Ao Qi wrote:
> >>> Hi Erik,
> >>>
> >>> Thank you a lot for taking care of this. I mainly like your approach.
> >>> I tested the patch by cross building mips64-zero, by which we found
> >>> this problem. However, it seems the problem still exists. The reason I
> >>> found is that UnixConstants.java of buildjdk is generated by cross
> >>> compiler, so the macro definitions are defined by the target. Here is
> >>> a preliminary workaround for your reference only (only worked for
> >>> gcc):
> >>>
> >>> $ hg diff make/gensrc/GensrcMisc.gmk
> >>> diff -r f4df9d4201cc make/gensrc/GensrcMisc.gmk
> >>> --- a/make/gensrc/GensrcMisc.gmk Tue Jun 04 16:50:25 2019 -0700
> >>> +++ b/make/gensrc/GensrcMisc.gmk Wed Jun 05 12:39:43 2019 +0800
> >>> @@ -61,6 +61,13 @@
> >>> CPP_FLAGS += -nologo
> >>> endif
> >>>
> >>> +ifeq ($(CREATING_BUILDJDK), true)
> >>> + # Only worked for gcc
> >>> + TEMP_CPP=gcc -E
> >>> +else
> >>> + TEMP_CPP=$(CPP)
> >>> +endif
> >>> +
> >>> # Generate a java source file from a template through the C
> >>> preprocessor for the
> >>> # target system. First extract the copyright notice at the start
> >>> of the file.
> >>> # Run the preprocessor. Filter out the default compiler stderr
> >>> output on
> >>> @@ -71,7 +78,7 @@
> >>> define generate-preproc-src
> >>>$(call MakeDir, $(@D))
> >>>( $(NAWK) '/@@END_COPYRIGHT@@/{exit}1' $< && \
> >>> - $(CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
> >>> + $(TEMP_CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
> >>> 2> >($(GREP) -v '^$(&2) \
> >>> | $(NAWK) '/@@START_HERE@@/,0' \
> >>> | $(SED) -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED
> >>> FILE - DO NOT EDIT/' \
> >>>
> >>>
> >>> Cheers,
> >>> Ao Qi
> >>>
> >>> On Wed, Jun 5, 2019 at 12:00 AM Erik Joelsson
> >>> wrote:
> Thanks!
>
> During testing, I found an intermittent race issue. The command line
> that runs the interim langtools version of javac still always
> points to
> the buildtools/interim-langtools dir of the normal build. This
> variable
> also needs to be rewritten in buildjdk-spec.gmk.in. New webrev, only
> changed in buildjdk-spec.gmk.in:
>
> http://cr.openjdk.java.net/~erikj/8217739/webrev.02/
>
> /Erik
>
> On 2019-06-04 07:53, Tim Bell wrote:
> > Erik:
> >
> > Looks good.
> >
> > Tim
> >
> >> This is a fix for a problem brought up on this list [1]. The main
> >> issue
> >> is that when cross compiling, and creating a buildjdk during the
> >> build,
> >> the UnixConstants.java cannot necessarily be reused between the
> >> target
> >> JDK and the buildjdk. While investigating this, I came to the
> >> conclusion
> >> that we really should stop trying to reuse anything between the
> >> buildjdk
> >> and target JDK. It's just becoming too complex to maintain (due
> >> to the
> >> intricate interactions with the interim-image,
> >> generate-link-opt-data
> >> and exploded-image-optimize targets, causing circular
> >> dependencies and
> >> endless incremental rebuilds).
> >>
> >> So this fix simply removes all the reuse of buildtools, gensrc
> >> and java
> >> compilation when creating a buildjdk. Hopefully this is making
> >> Main.gmk
> >> a bit less convoluted.
> >>
> >> While working on this, I came up with the
Re: RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling
Here is a new webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.03
The change is in buildjdk-spec.gmk.in where CPP is now also overridden.
/Erik
On 2019-06-05 10:33, Erik Joelsson wrote:
Never mind, I found the issue. The reason it worked for me was that we
set SYSROOT_CFLAGS for both the target and the build compiler (from
using devkits for them). The preprocessor for the target would then
act correctly given the --sysroot parameter pointing to a sysroot for
the build system.
I'm working on a patch where PPC gets overridden with the build
compiler in buildjdk-spec.gmk.
/Erik
On 2019-06-05 08:37, Erik Joelsson wrote:
Hello Ao Qi,
In my testing, I tried building ppc64le (which I first identified as
generating a UnixConstants.java that differed from an x64 build).
With this patch, the buildjdk and target version of
UnixConstants.java differ the same way. How did you configure your
build? I'm suspecting something in the setup of the buildjdk
compilers is different from my configuration.
/Erik
On 2019-06-04 21:48, Ao Qi wrote:
Hi Erik,
Thank you a lot for taking care of this. I mainly like your approach.
I tested the patch by cross building mips64-zero, by which we found
this problem. However, it seems the problem still exists. The reason I
found is that UnixConstants.java of buildjdk is generated by cross
compiler, so the macro definitions are defined by the target. Here is
a preliminary workaround for your reference only (only worked for
gcc):
$ hg diff make/gensrc/GensrcMisc.gmk
diff -r f4df9d4201cc make/gensrc/GensrcMisc.gmk
--- a/make/gensrc/GensrcMisc.gmk Tue Jun 04 16:50:25 2019 -0700
+++ b/make/gensrc/GensrcMisc.gmk Wed Jun 05 12:39:43 2019 +0800
@@ -61,6 +61,13 @@
CPP_FLAGS += -nologo
endif
+ifeq ($(CREATING_BUILDJDK), true)
+ # Only worked for gcc
+ TEMP_CPP=gcc -E
+else
+ TEMP_CPP=$(CPP)
+endif
+
# Generate a java source file from a template through the C
preprocessor for the
# target system. First extract the copyright notice at the start
of the file.
# Run the preprocessor. Filter out the default compiler stderr
output on
@@ -71,7 +78,7 @@
define generate-preproc-src
$(call MakeDir, $(@D))
( $(NAWK) '/@@END_COPYRIGHT@@/{exit}1' $< && \
- $(CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
+ $(TEMP_CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
2> >($(GREP) -v '^$(&2) \
| $(NAWK) '/@@START_HERE@@/,0' \
| $(SED) -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED
FILE - DO NOT EDIT/' \
Cheers,
Ao Qi
On Wed, Jun 5, 2019 at 12:00 AM Erik Joelsson
wrote:
Thanks!
During testing, I found an intermittent race issue. The command line
that runs the interim langtools version of javac still always
points to
the buildtools/interim-langtools dir of the normal build. This
variable
also needs to be rewritten in buildjdk-spec.gmk.in. New webrev, only
changed in buildjdk-spec.gmk.in:
http://cr.openjdk.java.net/~erikj/8217739/webrev.02/
/Erik
On 2019-06-04 07:53, Tim Bell wrote:
Erik:
Looks good.
Tim
This is a fix for a problem brought up on this list [1]. The main
issue
is that when cross compiling, and creating a buildjdk during the
build,
the UnixConstants.java cannot necessarily be reused between the
target
JDK and the buildjdk. While investigating this, I came to the
conclusion
that we really should stop trying to reuse anything between the
buildjdk
and target JDK. It's just becoming too complex to maintain (due
to the
intricate interactions with the interim-image,
generate-link-opt-data
and exploded-image-optimize targets, causing circular
dependencies and
endless incremental rebuilds).
So this fix simply removes all the reuse of buildtools, gensrc
and java
compilation when creating a buildjdk. Hopefully this is making
Main.gmk
a bit less convoluted.
While working on this, I came up with the idea of adding a log
prefix
functionality, to more easily discern which log lines were
printed by
the buildjdk build and which were part of the normal build. I
liked this
a lot so including that in the same fix, and applied it to the
bootcycle
build as well.
Bug: https://bugs.openjdk.java.net/browse/JDK-8217739
Webrev:
http://cr.openjdk.java.net/~erikj/8217739/webrev.01/index.html
/Erik
[1]
http://mail.openjdk.java.net/pipermail/build-dev/2019-January/024729.html
Re: RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling
Never mind, I found the issue. The reason it worked for me was that we
set SYSROOT_CFLAGS for both the target and the build compiler (from
using devkits for them). The preprocessor for the target would then act
correctly given the --sysroot parameter pointing to a sysroot for the
build system.
I'm working on a patch where PPC gets overridden with the build compiler
in buildjdk-spec.gmk.
/Erik
On 2019-06-05 08:37, Erik Joelsson wrote:
Hello Ao Qi,
In my testing, I tried building ppc64le (which I first identified as
generating a UnixConstants.java that differed from an x64 build). With
this patch, the buildjdk and target version of UnixConstants.java
differ the same way. How did you configure your build? I'm suspecting
something in the setup of the buildjdk compilers is different from my
configuration.
/Erik
On 2019-06-04 21:48, Ao Qi wrote:
Hi Erik,
Thank you a lot for taking care of this. I mainly like your approach.
I tested the patch by cross building mips64-zero, by which we found
this problem. However, it seems the problem still exists. The reason I
found is that UnixConstants.java of buildjdk is generated by cross
compiler, so the macro definitions are defined by the target. Here is
a preliminary workaround for your reference only (only worked for
gcc):
$ hg diff make/gensrc/GensrcMisc.gmk
diff -r f4df9d4201cc make/gensrc/GensrcMisc.gmk
--- a/make/gensrc/GensrcMisc.gmk Tue Jun 04 16:50:25 2019 -0700
+++ b/make/gensrc/GensrcMisc.gmk Wed Jun 05 12:39:43 2019 +0800
@@ -61,6 +61,13 @@
CPP_FLAGS += -nologo
endif
+ifeq ($(CREATING_BUILDJDK), true)
+ # Only worked for gcc
+ TEMP_CPP=gcc -E
+else
+ TEMP_CPP=$(CPP)
+endif
+
# Generate a java source file from a template through the C
preprocessor for the
# target system. First extract the copyright notice at the start of
the file.
# Run the preprocessor. Filter out the default compiler stderr
output on
@@ -71,7 +78,7 @@
define generate-preproc-src
$(call MakeDir, $(@D))
( $(NAWK) '/@@END_COPYRIGHT@@/{exit}1' $< && \
- $(CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
+ $(TEMP_CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
2> >($(GREP) -v '^$(&2) \
| $(NAWK) '/@@START_HERE@@/,0' \
| $(SED) -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED
FILE - DO NOT EDIT/' \
Cheers,
Ao Qi
On Wed, Jun 5, 2019 at 12:00 AM Erik Joelsson
wrote:
Thanks!
During testing, I found an intermittent race issue. The command line
that runs the interim langtools version of javac still always points to
the buildtools/interim-langtools dir of the normal build. This variable
also needs to be rewritten in buildjdk-spec.gmk.in. New webrev, only
changed in buildjdk-spec.gmk.in:
http://cr.openjdk.java.net/~erikj/8217739/webrev.02/
/Erik
On 2019-06-04 07:53, Tim Bell wrote:
Erik:
Looks good.
Tim
This is a fix for a problem brought up on this list [1]. The main
issue
is that when cross compiling, and creating a buildjdk during the
build,
the UnixConstants.java cannot necessarily be reused between the
target
JDK and the buildjdk. While investigating this, I came to the
conclusion
that we really should stop trying to reuse anything between the
buildjdk
and target JDK. It's just becoming too complex to maintain (due to
the
intricate interactions with the interim-image, generate-link-opt-data
and exploded-image-optimize targets, causing circular dependencies
and
endless incremental rebuilds).
So this fix simply removes all the reuse of buildtools, gensrc and
java
compilation when creating a buildjdk. Hopefully this is making
Main.gmk
a bit less convoluted.
While working on this, I came up with the idea of adding a log prefix
functionality, to more easily discern which log lines were printed by
the buildjdk build and which were part of the normal build. I
liked this
a lot so including that in the same fix, and applied it to the
bootcycle
build as well.
Bug: https://bugs.openjdk.java.net/browse/JDK-8217739
Webrev:
http://cr.openjdk.java.net/~erikj/8217739/webrev.01/index.html
/Erik
[1]
http://mail.openjdk.java.net/pipermail/build-dev/2019-January/024729.html
Re: RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling
Hello Ao Qi,
In my testing, I tried building ppc64le (which I first identified as
generating a UnixConstants.java that differed from an x64 build). With
this patch, the buildjdk and target version of UnixConstants.java differ
the same way. How did you configure your build? I'm suspecting something
in the setup of the buildjdk compilers is different from my configuration.
/Erik
On 2019-06-04 21:48, Ao Qi wrote:
Hi Erik,
Thank you a lot for taking care of this. I mainly like your approach.
I tested the patch by cross building mips64-zero, by which we found
this problem. However, it seems the problem still exists. The reason I
found is that UnixConstants.java of buildjdk is generated by cross
compiler, so the macro definitions are defined by the target. Here is
a preliminary workaround for your reference only (only worked for
gcc):
$ hg diff make/gensrc/GensrcMisc.gmk
diff -r f4df9d4201cc make/gensrc/GensrcMisc.gmk
--- a/make/gensrc/GensrcMisc.gmk Tue Jun 04 16:50:25 2019 -0700
+++ b/make/gensrc/GensrcMisc.gmk Wed Jun 05 12:39:43 2019 +0800
@@ -61,6 +61,13 @@
CPP_FLAGS += -nologo
endif
+ifeq ($(CREATING_BUILDJDK), true)
+ # Only worked for gcc
+ TEMP_CPP=gcc -E
+else
+ TEMP_CPP=$(CPP)
+endif
+
# Generate a java source file from a template through the C
preprocessor for the
# target system. First extract the copyright notice at the start of the file.
# Run the preprocessor. Filter out the default compiler stderr output on
@@ -71,7 +78,7 @@
define generate-preproc-src
$(call MakeDir, $(@D))
( $(NAWK) '/@@END_COPYRIGHT@@/{exit}1' $< && \
- $(CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
+ $(TEMP_CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
2> >($(GREP) -v '^$(&2) \
| $(NAWK) '/@@START_HERE@@/,0' \
| $(SED) -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED
FILE - DO NOT EDIT/' \
Cheers,
Ao Qi
On Wed, Jun 5, 2019 at 12:00 AM Erik Joelsson wrote:
Thanks!
During testing, I found an intermittent race issue. The command line
that runs the interim langtools version of javac still always points to
the buildtools/interim-langtools dir of the normal build. This variable
also needs to be rewritten in buildjdk-spec.gmk.in. New webrev, only
changed in buildjdk-spec.gmk.in:
http://cr.openjdk.java.net/~erikj/8217739/webrev.02/
/Erik
On 2019-06-04 07:53, Tim Bell wrote:
Erik:
Looks good.
Tim
This is a fix for a problem brought up on this list [1]. The main issue
is that when cross compiling, and creating a buildjdk during the build,
the UnixConstants.java cannot necessarily be reused between the target
JDK and the buildjdk. While investigating this, I came to the conclusion
that we really should stop trying to reuse anything between the buildjdk
and target JDK. It's just becoming too complex to maintain (due to the
intricate interactions with the interim-image, generate-link-opt-data
and exploded-image-optimize targets, causing circular dependencies and
endless incremental rebuilds).
So this fix simply removes all the reuse of buildtools, gensrc and java
compilation when creating a buildjdk. Hopefully this is making Main.gmk
a bit less convoluted.
While working on this, I came up with the idea of adding a log prefix
functionality, to more easily discern which log lines were printed by
the buildjdk build and which were part of the normal build. I liked this
a lot so including that in the same fix, and applied it to the bootcycle
build as well.
Bug: https://bugs.openjdk.java.net/browse/JDK-8217739
Webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.01/index.html
/Erik
[1]
http://mail.openjdk.java.net/pipermail/build-dev/2019-January/024729.html
Re: RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling
Hi Erik,
Thank you a lot for taking care of this. I mainly like your approach.
I tested the patch by cross building mips64-zero, by which we found
this problem. However, it seems the problem still exists. The reason I
found is that UnixConstants.java of buildjdk is generated by cross
compiler, so the macro definitions are defined by the target. Here is
a preliminary workaround for your reference only (only worked for
gcc):
$ hg diff make/gensrc/GensrcMisc.gmk
diff -r f4df9d4201cc make/gensrc/GensrcMisc.gmk
--- a/make/gensrc/GensrcMisc.gmk Tue Jun 04 16:50:25 2019 -0700
+++ b/make/gensrc/GensrcMisc.gmk Wed Jun 05 12:39:43 2019 +0800
@@ -61,6 +61,13 @@
CPP_FLAGS += -nologo
endif
+ifeq ($(CREATING_BUILDJDK), true)
+ # Only worked for gcc
+ TEMP_CPP=gcc -E
+else
+ TEMP_CPP=$(CPP)
+endif
+
# Generate a java source file from a template through the C
preprocessor for the
# target system. First extract the copyright notice at the start of the file.
# Run the preprocessor. Filter out the default compiler stderr output on
@@ -71,7 +78,7 @@
define generate-preproc-src
$(call MakeDir, $(@D))
( $(NAWK) '/@@END_COPYRIGHT@@/{exit}1' $< && \
- $(CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
+ $(TEMP_CPP) $(CPP_FLAGS) $(SYSROOT_CFLAGS) $(CFLAGS_JDKLIB) $< \
2> >($(GREP) -v '^$(&2) \
| $(NAWK) '/@@START_HERE@@/,0' \
| $(SED) -e 's/@@START_HERE@@/\/\/ AUTOMATICALLY GENERATED
FILE - DO NOT EDIT/' \
Cheers,
Ao Qi
On Wed, Jun 5, 2019 at 12:00 AM Erik Joelsson wrote:
>
> Thanks!
>
> During testing, I found an intermittent race issue. The command line
> that runs the interim langtools version of javac still always points to
> the buildtools/interim-langtools dir of the normal build. This variable
> also needs to be rewritten in buildjdk-spec.gmk.in. New webrev, only
> changed in buildjdk-spec.gmk.in:
>
> http://cr.openjdk.java.net/~erikj/8217739/webrev.02/
>
> /Erik
>
> On 2019-06-04 07:53, Tim Bell wrote:
> > Erik:
> >
> > Looks good.
> >
> > Tim
> >
> >> This is a fix for a problem brought up on this list [1]. The main issue
> >> is that when cross compiling, and creating a buildjdk during the build,
> >> the UnixConstants.java cannot necessarily be reused between the target
> >> JDK and the buildjdk. While investigating this, I came to the conclusion
> >> that we really should stop trying to reuse anything between the buildjdk
> >> and target JDK. It's just becoming too complex to maintain (due to the
> >> intricate interactions with the interim-image, generate-link-opt-data
> >> and exploded-image-optimize targets, causing circular dependencies and
> >> endless incremental rebuilds).
> >>
> >> So this fix simply removes all the reuse of buildtools, gensrc and java
> >> compilation when creating a buildjdk. Hopefully this is making Main.gmk
> >> a bit less convoluted.
> >>
> >> While working on this, I came up with the idea of adding a log prefix
> >> functionality, to more easily discern which log lines were printed by
> >> the buildjdk build and which were part of the normal build. I liked this
> >> a lot so including that in the same fix, and applied it to the bootcycle
> >> build as well.
> >>
> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8217739
> >>
> >> Webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.01/index.html
> >>
> >> /Erik
> >>
> >> [1]
> >> http://mail.openjdk.java.net/pipermail/build-dev/2019-January/024729.html
> >>
> >
> >
> >
Re: RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling
Erik: OK - version 2 looks good. Tim Thanks! During testing, I found an intermittent race issue. The command line that runs the interim langtools version of javac still always points to the buildtools/interim-langtools dir of the normal build. This variable also needs to be rewritten in buildjdk-spec.gmk.in. New webrev, only changed in buildjdk-spec.gmk.in: http://cr.openjdk.java.net/~erikj/8217739/webrev.02/ /Erik On 2019-06-04 07:53, Tim Bell wrote: Erik: Looks good. Tim This is a fix for a problem brought up on this list [1]. The main issue is that when cross compiling, and creating a buildjdk during the build, the UnixConstants.java cannot necessarily be reused between the target JDK and the buildjdk. While investigating this, I came to the conclusion that we really should stop trying to reuse anything between the buildjdk and target JDK. It's just becoming too complex to maintain (due to the intricate interactions with the interim-image, generate-link-opt-data and exploded-image-optimize targets, causing circular dependencies and endless incremental rebuilds). So this fix simply removes all the reuse of buildtools, gensrc and java compilation when creating a buildjdk. Hopefully this is making Main.gmk a bit less convoluted. While working on this, I came up with the idea of adding a log prefix functionality, to more easily discern which log lines were printed by the buildjdk build and which were part of the normal build. I liked this a lot so including that in the same fix, and applied it to the bootcycle build as well. Bug: https://bugs.openjdk.java.net/browse/JDK-8217739 Webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.01/index.html /Erik [1] http://mail.openjdk.java.net/pipermail/build-dev/2019-January/024729.html
Re: RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling
Thanks! During testing, I found an intermittent race issue. The command line that runs the interim langtools version of javac still always points to the buildtools/interim-langtools dir of the normal build. This variable also needs to be rewritten in buildjdk-spec.gmk.in. New webrev, only changed in buildjdk-spec.gmk.in: http://cr.openjdk.java.net/~erikj/8217739/webrev.02/ /Erik On 2019-06-04 07:53, Tim Bell wrote: Erik: Looks good. Tim This is a fix for a problem brought up on this list [1]. The main issue is that when cross compiling, and creating a buildjdk during the build, the UnixConstants.java cannot necessarily be reused between the target JDK and the buildjdk. While investigating this, I came to the conclusion that we really should stop trying to reuse anything between the buildjdk and target JDK. It's just becoming too complex to maintain (due to the intricate interactions with the interim-image, generate-link-opt-data and exploded-image-optimize targets, causing circular dependencies and endless incremental rebuilds). So this fix simply removes all the reuse of buildtools, gensrc and java compilation when creating a buildjdk. Hopefully this is making Main.gmk a bit less convoluted. While working on this, I came up with the idea of adding a log prefix functionality, to more easily discern which log lines were printed by the buildjdk build and which were part of the normal build. I liked this a lot so including that in the same fix, and applied it to the bootcycle build as well. Bug: https://bugs.openjdk.java.net/browse/JDK-8217739 Webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.01/index.html /Erik [1] http://mail.openjdk.java.net/pipermail/build-dev/2019-January/024729.html
Re: RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling
Erik: Looks good. Tim This is a fix for a problem brought up on this list [1]. The main issue is that when cross compiling, and creating a buildjdk during the build, the UnixConstants.java cannot necessarily be reused between the target JDK and the buildjdk. While investigating this, I came to the conclusion that we really should stop trying to reuse anything between the buildjdk and target JDK. It's just becoming too complex to maintain (due to the intricate interactions with the interim-image, generate-link-opt-data and exploded-image-optimize targets, causing circular dependencies and endless incremental rebuilds). So this fix simply removes all the reuse of buildtools, gensrc and java compilation when creating a buildjdk. Hopefully this is making Main.gmk a bit less convoluted. While working on this, I came up with the idea of adding a log prefix functionality, to more easily discern which log lines were printed by the buildjdk build and which were part of the normal build. I liked this a lot so including that in the same fix, and applied it to the bootcycle build as well. Bug: https://bugs.openjdk.java.net/browse/JDK-8217739 Webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.01/index.html /Erik [1] http://mail.openjdk.java.net/pipermail/build-dev/2019-January/024729.html
RFR: JDK-8217739: Cannot reuse java.base UnixConstants.java from target in BuildJDK when cross compiling
This is a fix for a problem brought up on this list [1]. The main issue is that when cross compiling, and creating a buildjdk during the build, the UnixConstants.java cannot necessarily be reused between the target JDK and the buildjdk. While investigating this, I came to the conclusion that we really should stop trying to reuse anything between the buildjdk and target JDK. It's just becoming too complex to maintain (due to the intricate interactions with the interim-image, generate-link-opt-data and exploded-image-optimize targets, causing circular dependencies and endless incremental rebuilds). So this fix simply removes all the reuse of buildtools, gensrc and java compilation when creating a buildjdk. Hopefully this is making Main.gmk a bit less convoluted. While working on this, I came up with the idea of adding a log prefix functionality, to more easily discern which log lines were printed by the buildjdk build and which were part of the normal build. I liked this a lot so including that in the same fix, and applied it to the bootcycle build as well. Bug: https://bugs.openjdk.java.net/browse/JDK-8217739 Webrev: http://cr.openjdk.java.net/~erikj/8217739/webrev.01/index.html /Erik [1] http://mail.openjdk.java.net/pipermail/build-dev/2019-January/024729.html
