On 9/2/12 9:37 PM, David Holmes wrote:
Hi Dan,
On 3/09/2012 12:47 PM, Daniel D. Daugherty wrote:
On 9/2/12 7:26 PM, David Holmes wrote:
The build will create debuginfo/diz files as requested - that's fine.
What I'm unclear about is where those files should actually appear in
our build artifacts, specifically the jdk/jre images that are
created.
Internally when we create binary images RE strips all the
debuginfo/diz files out.
Not quite. The RE makefiles will put debug info files into a
separate debuginfo.zip file. This includes .debuginfo, .diz,
.map and .pdb files. For the regular bundles, the RE makefiles
will not include debug info files. Not quite the same as
stripping them out.
Not trying to split hairs but if the image contains them and the RE
bundle of the image does not, then the RE process is stripping them
out.
The above is implying that there is a single RE bundle and that hasn't
been true for the tar-style universe for quite a long time; the demo
bundle was added by around JDK8-B20 or so. For the package-style
universe, I don't think that has ever been true.
RE's makefiles take the image generated by the build and package up the
various files into two or more bundles. I believe each platform has two
styles of bundles. For Solaris, SVR4 style packages and tar/zip bundles.
For Windows, an EXE installer and tar/zip bundles. For Linux RPM
packages
and tar/ZIP bundles.
So I'll stick with my assertion that the RE process is placing debug
info
files into a separate bundle and is not stripping them out.
But the build itself seems to treat them in an ad-hoc manner:
- The new build deliberately excludes debuginfo/diz files associated
with binaries, but will include any related to libraries (via generic
copying routine). (It's obvious from the comments related to this
that
there is some puzzlement as to this reasoning.)
Sorry I haven't looked at the new build system.
It tries to emulate the old build.
- The old build also tries to exclude the files associated with
binaries, but only handles .debuginfo not .diz :(
During one round in the FDS project, I included .debuginfo/.diz files
right next to some of the binaries. Apparently that's not allowed
without
explicit permission. The contents of the "bin" directory are
controlled
and there are tests to verify those contents.
I changed the Makefiles for the few binaries that support FDS to not
install the debug info files with the binaries, but they are left in
the normal build artifacts location if someone has the need to use
them. Of course, I've had queries for debug info files for the
binaries
to be included in the debuginfo.zip bundles. I'm not planning to fight
that battle.
Okay that explains the special handling for binaries. So how do these
then get into the debuginfo.zip? Or don't they?
They don't. They are only available via the build artifacts that
are archived.
Those associated with libraries just seem to get copied if they
happen
to be there
All of the logic that copies debug info files to the image should do
so if they happen to be there. Not all component support FDS now or
in the future so the logic needs to adapt to what is built.
As I said this all seems very ad-hoc to me. I would expect to see no
debuginfo/diz files in a created image by default, and have a
separate
target that would produce a tar file of all the debuginfo/diz files
ready to overlay on an existing image.
Close. The image has to have the debug info or .diz files in order for
the RE makefiles to generate the debuginfo.zip files. It is the RE
makefiles that handle the packaging. Just like the demos are always
built and it is the RE makefiles that put them in a separate bundle.
So therein lies the problem. RE lies outside of the OpenJDK build
system. If the RE Makefiles can copy an image they can copy the
debuginfo files too. I'd much rather see no debuginfo files in the
built images (unless requested) and a distinct debuginfo related
target (that RE could use if they choose). That would seem to be
simpler all round.
My understanding is that "the image" is supposed to be the complete
image if every bundle is extracted. I believe that is how the packages
are sanity checked to be complete. RE's Makefiles are meant to process
"the image" into bundles. They aren't meant to add files of their own.
If you are planning to change the meaning of "the image" or if you
are planning to put files into bundles that are not in "the image",
then you'll need to coordinate with RE and whoever does the package
inventory testing.
Also, if you change the way this works, keep in mind that some of the
Makefiles generate their objects into a temporary directory and then
copy things from that temporary directory to the image. Other Makefiles
generate their objects directly into the image. For the latter, you'll
have to move the debug info file(s) out of the image into some other
place where it can be picked up by whatever bundling process you create
to make the debuginfo.zip file.
Thoughts/comments?
I have to deal with this for the SE Profile work, where we will not
want these files present in any of the images.
It should be fine if the Embedded profile does not generate debug info
files or if the Embedded profile generates debug info, but does not
copy the debug info files into the image area. The RE makefiles will
handle the presence or absence of debug info files.
These are not embedded profiles, these are just profiles - it is all
to be part of SE in Java 8. The build is supposed to create an image
corresponding to each profile. I can certainly specialise the handling
of the debuginfo files for the profiles, but it seemed to me that the
overall debuginfo-in-image story was rather unclear.
I'm sorry you think that the debuginfo-in-image story is unclear.
I thought I did a pretty good job integrating FDS into the whole
OpenJDK build process somewhat seamlessly. I guess not.
The key things to remember in all this is that debug info files
(like the demos) have to end up in a separate bundle or bundles.
So if you change the way this is all implemented, you still have
to follow the separate bundle rule.
However, the non-Embedded profile needs to continue to generate and
copy debug info files for those components that already support FDS.
If you change the non-Embedded logic to not copy the debug info files
into the image area, the RE debuginfo.zip bundle will not be created
and that will break FDS.
RE processes will have to change to handle profiles in any case. I
think they'd rather get a debuginfo.zip out of the build than having
to create it themselves.
Interesting and not something that I have ever heard. I got the
impression that RE needed to have control over what was bundled
and how it was bundled. It's kind of hard to certify/vouch for a
process when you don't do it yourself, but that's not really my
call.
Good luck with the tweaking of the build and packaging processes.
Just remember: There be dragons here!
Dan
Thanks,
David
Dan
David
