On 9/2/12 9:37 PM, David Holmes wrote:
Hi Dan,
On 3/09/2012 12:47 PM, Daniel D. Daugherty wrote:
On 9/2/12 7:26 PM, David Holmes wrote:
The build will create debuginfo/diz files as requested -
that's
fine.
What I'm unclear about is where those files should actually
appear in
our build artifacts, specifically the jdk/jre images that are
created.
Internally when we create binary images RE strips all the
debuginfo/diz files out.
Not quite. The RE makefiles will put debug info files into a
separate debuginfo.zip file. This includes .debuginfo, .diz,
.map and .pdb files. For the regular bundles, the RE makefiles
will not include debug info files. Not quite the same as
stripping them out.
Not trying to split hairs but if the image contains them and
the
RE
bundle of the image does not, then the RE process is stripping
them out.
The above is implying that there is a single RE bundle and that
hasn't
been true for the tar-style universe for quite a long time; the
demo
bundle was added by around JDK8-B20 or so. For the package-style
universe, I don't think that has ever been true.
RE's makefiles take the image generated by the build and package
up the
various files into two or more bundles. I believe each platform
has two
styles of bundles. For Solaris, SVR4 style packages and tar/zip
bundles.
For Windows, an EXE installer and tar/zip bundles. For Linux RPM
packages
and tar/ZIP bundles.
So I'll stick with my assertion that the RE process is placing
debug info
files into a separate bundle and is not stripping them out.
But the build itself seems to treat them in an ad-hoc manner:
- The new build deliberately excludes debuginfo/diz files
associated
with binaries, but will include any related to libraries (via
generic
copying routine). (It's obvious from the comments related to
this that
there is some puzzlement as to this reasoning.)
Sorry I haven't looked at the new build system.
It tries to emulate the old build.
- The old build also tries to exclude the files associated
with
binaries, but only handles .debuginfo not .diz :(
During one round in the FDS project, I included
.debuginfo/.diz
files
right next to some of the binaries. Apparently that's not
allowed
without
explicit permission. The contents of the "bin" directory are
controlled
and there are tests to verify those contents.
I changed the Makefiles for the few binaries that support FDS
to
not
install the debug info files with the binaries, but they are
left in
the normal build artifacts location if someone has the need to
use
them. Of course, I've had queries for debug info files for the
binaries
to be included in the debuginfo.zip bundles. I'm not planning
to
fight
that battle.
Okay that explains the special handling for binaries. So how do
these
then get into the debuginfo.zip? Or don't they?
They don't. They are only available via the build artifacts that
are archived.
Those associated with libraries just seem to get copied if
they
happen
to be there
All of the logic that copies debug info files to the image
should do
so if they happen to be there. Not all component support FDS
now
or
in the future so the logic needs to adapt to what is built.
As I said this all seems very ad-hoc to me. I would expect to
see no
debuginfo/diz files in a created image by default, and have a
separate
target that would produce a tar file of all the debuginfo/diz
files
ready to overlay on an existing image.
Close. The image has to have the debug info or .diz files in
order for
the RE makefiles to generate the debuginfo.zip files. It is
the
RE
makefiles that handle the packaging. Just like the demos are
always
built and it is the RE makefiles that put them in a separate
bundle.
So therein lies the problem. RE lies outside of the OpenJDK
build
system. If the RE Makefiles can copy an image they can copy the
debuginfo files too. I'd much rather see no debuginfo files in
the
built images (unless requested) and a distinct debuginfo
related
target (that RE could use if they choose). That would seem to
be
simpler all round.
My understanding is that "the image" is supposed to be the
complete
image if every bundle is extracted. I believe that is how the
packages
are sanity checked to be complete. RE's Makefiles are meant to
process
"the image" into bundles. They aren't meant to add files of
their
own.
If you are planning to change the meaning of "the image" or if
you
are planning to put files into bundles that are not in "the
image",
then you'll need to coordinate with RE and whoever does the
package
inventory testing.
Also, if you change the way this works, keep in mind that some
of
the
Makefiles generate their objects into a temporary directory and
then
copy things from that temporary directory to the image. Other
Makefiles
generate their objects directly into the image. For the latter,
you'll
have to move the debug info file(s) out of the image into some
other
place where it can be picked up by whatever bundling process you
create
to make the debuginfo.zip file.
Thoughts/comments?
I have to deal with this for the SE Profile work, where we
will
not
want these files present in any of the images.
It should be fine if the Embedded profile does not generate
debug info
files or if the Embedded profile generates debug info, but
does
not
copy the debug info files into the image area. The RE
makefiles
will
handle the presence or absence of debug info files.
These are not embedded profiles, these are just profiles - it
is
all
to be part of SE in Java 8. The build is supposed to create an
image
corresponding to each profile. I can certainly specialise the
handling
of the debuginfo files for the profiles, but it seemed to me
that
the
overall debuginfo-in-image story was rather unclear.
I'm sorry you think that the debuginfo-in-image story is
unclear.
I thought I did a pretty good job integrating FDS into the whole
OpenJDK build process somewhat seamlessly. I guess not.
The key things to remember in all this is that debug info files
(like the demos) have to end up in a separate bundle or bundles.
So if you change the way this is all implemented, you still have
to follow the separate bundle rule.
However, the non-Embedded profile needs to continue to
generate
and
copy debug info files for those components that already
support
FDS.
If you change the non-Embedded logic to not copy the debug
info
files
into the image area, the RE debuginfo.zip bundle will not be
created
and that will break FDS.
RE processes will have to change to handle profiles in any
case.
I
think they'd rather get a debuginfo.zip out of the build than
having
to create it themselves.
Interesting and not something that I have ever heard. I got the
impression that RE needed to have control over what was bundled
and how it was bundled. It's kind of hard to certify/vouch for a
process when you don't do it yourself, but that's not really my
call.
Good luck with the tweaking of the build and packaging
processes.
Just remember: There be dragons here!
Dan
Thanks,
David
Dan
David
