Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Am Mon, 11 Jan 2010 18:04:21 +0100 schrieb Stephan Bergmann : > Which, at least in its simplest form, would imply that it necessarily > modifies the source tree by adding files to it, something I > personally would prefer moving away from. No. Generated makefiles are _generated_ and thus not source files. They shouldnt be generated to the read-only source tree, but reside in the workdir (those dirs with are located at $MODULE/$PLATFORM in the current build system, but they would not be in the source tree in a new build system). BR, Bjoern -- === Sitz der Gesellschaft: Sun Microsystems GmbH, Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht Muenchen: HRB 161028 Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Wolf Frenkel Vorsitzender des Aufsichtsrates: Martin Haering === - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Am Mon, 11 Jan 2010 17:48:18 +0100 schrieb Thorsten Behrens : > I'd personally use it to dynamically generate the makefiles (i.e. > from configure, or triggered from a smallish global makefile). With GNU make there would not be a need for an "extra" makefile. You can include makefiles that arent generated yet, if there are rules available for their generation: make will then generate the included files and restart itself including the files. From the POV of the user there is no extra step needed to generate the makefiles. BR, Bjoern -- === Sitz der Gesellschaft: Sun Microsystems GmbH, Sonnenallee 1, D-85551 Kirchheim-Heimstetten Amtsgericht Muenchen: HRB 161028 Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Wolf Frenkel Vorsitzender des Aufsichtsrates: Martin Haering === - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Am Mon, 11 Jan 2010 14:07:23 +0100
schrieb Thorsten Behrens :
> Ok, took longer than expected to find enough time for this, but here
> we go:
Hi Thorsten,
great to hear from you about this.
> After my initial performance worries had been sorted out, there are
> basically two points left I'd require from an all-new build system:
> - declarative syntax, as little duplication as possible
> - ability to do cross-platform builds
Valid point, but lets leave cross-platform builds out of the discussion
of a simple DSL for the build tasks as they have little to do with each
other, ok?
> For the former, look at this gnu makefile snippet (taken from
> gnu_make cws):
>
> [...]
>
> $(eval $(call gb_Executable_Executable,rscdep))
>
> $(eval $(call gb_Executable_set_linked_libs,rscdep,\
> $(call gb_OOoLibrary_get_linknames,\
> tl \
> ) \
> ))
>
> $(eval $(call gb_Executable_add_exception_objects,rscdep,\
> tools/bootstrp/appdef \
> tools/bootstrp/command \
>
> [etc.]
>
> Call me purist, but there's a lot of noise in these lines, that
> convey little, if any information. Plus, there's redundancy, in the
> form of name prefixes, gnu make plumbing to call, etc., and also the
> need to exhaustively list all object files (by listing paths names
> of cxx files, stripped from the .cxx extension).
>
> But the actual information contained in the above lines is actually
> this:
>
> rscdep source files: tools/bootstrp/*
> rscdep link libs: tl
Quoting a band from Hamburg: "Jein" (=Yes and No).
Of course, there is a lot of superfluous syntax in the current
description, but you are dropping some information too:
- Where is the information which naming convention the tl-lib is
following?
- What about the files in tools/bootstrp, that are not generating
object files that should be liked into rscdep (sspretty.cxx for
example)?
The first information is hidden in you translation-implementation,
which I think is not better that having it explicit in the makefile.
While "Beautiful is better than ugly.", there is also "Explicit is
better than implicit.".
The cleanest way (which is still dirty) IMHO would be to have the
mapping of libs and naming conventions defined in one global file (in
solenv/inc).
The root problem of course is the use of pretty silly naming conventions
(platform in the filename, wtf?) and the use of different
contradicting naming conventions.
The second information is of course a problem with our source tree, but
unfortunately it seems not at all uncommon to have *.cxx files for
totally different libs in the same dir in the OOo source tree (sw is an
example of a total mess in that regard). If we had "clean
dirs" (only sources for one binary in one dir), it would be rather easy
to implement this in GNU make itself (but it will cost a bit of
performance, as make would need to collect all source files on every
run).
Using a "gnu make generator" for globbing the list of source files
generates a new set of problems when one thinks about how the generator
is run. The generator might actually be run:
a) manually or
b) always (on every make call)
c) only when it needs to be run by checking dependencies
b) likely is a performance hog (needs investigation covering all the
cornercases like idl-compilation etc.).
a) is evil, as it introduces shaky dependencies and little is gained by
it. Lemme explain: We are introducing dynamically collected source
files so that a dev does not need to manually declare the additional
file to the build system. With b) however, we traded "manually
declaring the source file" against "manually needing to regenerate the
makefiles", which I feel to be not much gain as both will be forgotten
by the careless dev.
c) would be nice. Unfortunately, it is not cheap to find out if a
makefile needs to be regenerated as it depends on the dirstate of the
source tree (or to be more specific: the parts of the source tree that
are referenced in it). Finding out _if_ a makefile needs to be
regenerated is likely just as or more expensive that just generating
it, thowing us back to solution b) with its drawbacks.
> So I hacked up some python to parse a sort of declarative input
> file, and generate gnu makefiles with Björn's eval framework out of
> that. Corresponding to the above lines, this would be the input
> snippet:
>
> targets: [
> {
> executable: {
> name: 'rscdep'
> arch: host
> }
> sources: {
> glob: '*.cxx'
> paths: ['tools/bootstrp']
> }
> linking: ['tl']
> } ]
>
> I think it's cleaner, and there's definitely not much (should I say
> any?) redundancy left. Additionally, one can enhance the script to
> generate makefiles for pretty much every make tool of this world,
> including eclipse/netbeans/visual studio project files.
> ...
>
> What do you think?
Looking a lot better that the GNU make input files, that are currently
used in the GNU make prototype for sure. However, I wonder how the
deeply nested list
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
On Mon, Jan 11, 2010 at 7:57 PM, Thorsten Behrens wrote: >> functionality? Even if CMake eventually turns out to be too slow, >> would it not make more sense to write your own custom CMake back >> end rather than the configuration/generation front end? >> > I guess it's now my turn to ask for sample code here. ;) For a backend? No, sorry. I have never looked into that. But the issue raised earlier was that because CMake's Makefiles are recursive (or something) they are too slow, probably because automake does it this way and is slow. I personally do not think this will be an issue. When running on Windows, the time taken by makefiles when changing directories is insignificant compared to the time taken by the compiler. But I have only tried it under Virtualbox and not at all thoroughly. If, however, both the Makefile and MSBuild generators turn out to be too slow, the CMake developers are very responsive and would most likely work with you to improve the performance. Only after this fails would you need to look into writing your own backend. > Mostly marketing. I'm not too interested who else is using it, > whether it has reached a tipping point etc. etc., but rather how an > actual cmake solution to the requirements mentioned before may look > like (scaffolding, drafting, etc. totally acceptable - just need the > gist of it). :) Well, my CMake build env experiment was posted here again just now. Is that sufficient for evaluation or do you need something more? - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
On Mon, Jan 11, 2010 at 8:11 PM, Martin Hollmichel wrote: > I started some time ago a cmake prototype for OOo in my spare time > (http://hg.services.openoffice.org/hg/cws/mh6bc/) for the latest status > please see the latest ReadMe.txt in the Source root for the most recent > status). You're invited to join this prototype, but be warned: it does not > work and kills your cat when try to run that :-). You might want to look look at my attempt: http://lists.freedesktop.org/archives/ooo-build/2009-August/000181.html It goes quite a lot further and solves some of the issues listed in your readme (detecting STLPort and Boost, etc). There's also the dmake -> cmake converter script so you don't have to keep writing the files by hand. - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Hi, I think it's cleaner, and there's definitely not much (should I say any?) redundancy left. Additionally, one can enhance the script to generate makefiles for pretty much every make tool of this world, including eclipse/netbeans/visual studio project files. I would like to point out that what you are doing is generating your own language and a build tool/generator based on that. There's nothing wrong with it as such, but this is reinventing the wheel again (just like Google's GYP). Instead of custom dmake/build.pl you would have custom gnumakegen/gnumake_or_something. What is the benefit you get from this instead of using something like CMake that already has a mature implementation of this functionality? Even if CMake eventually turns out to be too slow, would it not make more sense to write your own custom CMake back end rather than the configuration/generation front end? I second the demand for simple, readable description files. ath the first glance, this seem very easy with cmake, it was fun prototyping this on Linux for the OOo tool-chain up to the idlc. regmerge level. A lot of writing could be saved in comparisons to Ooo current build system. during the prototyping I also was surprise about all the stuff we're doing in Ooo's makefile's (some superfluous, some really needed) and I was surprised that much of the stuff already is covered by cmake (just proved by reading the documentation, not all tested in reality). Things are getting a bit more complicated, if you're looking on some old grown specialties of OOo, e.g the generations of import libraries on Win32. At some stage it is useful, if not even required to have an cmake expert with the OOo project who can determine with some experience how and where to implement such one-off's of OOo. so we still have three valid alternatives: * renew and modernize our old dmake environment * re-write the build environment with a more modern tool and use some more modern patterns * reuse and enhance already abstract buildenv like cmake. I'm not yet sure what will be the best way to go. For further information here is a Google Tech Talk about CMake and all related things (testing, code coverity, packaging, etc, etc) by one of the creators. If the build tool decision is not yet final, it is worth watching. The CPack and CTest enhancements of cmake are indeed quite impressive, a transition from scp to CPack would be very interesting (and kicking build.pl / deliver.pl and solver at the same time). I started some time ago a cmake prototype for OOo in my spare time (http://hg.services.openoffice.org/hg/cws/mh6bc/) for the latest status please see the latest ReadMe.txt in the Source root for the most recent status). You're invited to join this prototype, but be warned: it does not work and kills your cat when try to run that :-). Martin - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Jussi Pakkanen wrote: > > I think it's cleaner, and there's definitely not much (should I say > > any?) redundancy left. Additionally, one can enhance the script to > > generate makefiles for pretty much every make tool of this world, > > including eclipse/netbeans/visual studio project files. > > I would like to point out that what you are doing is generating your > own language and a build tool/generator based on that. > Yes. I'm piggy-backing on Björn's own rewrite, adding the (IMHO) crucial features that a new build system *should* have, once we're going for this endeavour. > There's nothing wrong with it as such, but this is reinventing the > wheel again (just like Google's GYP). Instead of custom dmake/ > build.pl you would have custom gnumakegen/gnumake_or_something. > What is the benefit you get from this instead of using something > like CMake that already has a mature implementation of this > functionality? Even if CMake eventually turns out to be too slow, > would it not make more sense to write your own custom CMake back > end rather than the configuration/generation front end? > I guess it's now my turn to ask for sample code here. ;) I've no strong opinion on cmake, except for the fact that it sucks at cross-building; the input syntax is ~ok, though not really good on enforcing structure; no idea how much effort there is writing a custom backend vs. having a dedicated approach in the first place (i.e. how much code could we share, e.g. from the eclipse/visual studio output targets in cmake, vs. having it all custom anyways?) > For further information here is a Google Tech Talk about CMake and all > related things (testing, code coverity, packaging, etc, etc) by one of > the creators. If the build tool decision is not yet final, it is worth > watching. > > http://www.youtube.com/watch?v=8Ut9o4OdSC0 > Mostly marketing. I'm not too interested who else is using it, whether it has reached a tipping point etc. etc., but rather how an actual cmake solution to the requirements mentioned before may look like (scaffolding, drafting, etc. totally acceptable - just need the gist of it). :) Cheers, -- Thorsten pgpBJfu60iKAW.pgp Description: PGP signature
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
On 01/11/10 17:48, Thorsten Behrens wrote: Stephan Bergmann wrote: So I hacked up some python to parse a sort of declarative input file, and generate gnu makefiles with Björn's eval framework out of that. At what stage of the development/build process would that script be called? I'd personally use it to dynamically generate the makefiles (i.e. from configure, or triggered from a smallish global makefile). Which, at least in its simplest form, would imply that it necessarily modifies the source tree by adding files to it, something I personally would prefer moving away from. -Stephan - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Stephan Bergmann wrote: > >So I hacked up some python to parse a sort of declarative input > >file, and generate gnu makefiles with Björn's eval framework out of > >that. > > At what stage of the development/build process would that script be called? > I'd personally use it to dynamically generate the makefiles (i.e. from configure, or triggered from a smallish global makefile). -- Thorsten pgprzdC1BzTxq.pgp Description: PGP signature
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
On 01/11/10 14:07, Thorsten Behrens wrote: So I hacked up some python to parse a sort of declarative input file, and generate gnu makefiles with Björn's eval framework out of that. At what stage of the development/build process would that script be called? -Stephan - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
On Mon, Jan 11, 2010 at 3:07 PM, Thorsten Behrens wrote: > I think it's cleaner, and there's definitely not much (should I say > any?) redundancy left. Additionally, one can enhance the script to > generate makefiles for pretty much every make tool of this world, > including eclipse/netbeans/visual studio project files. I would like to point out that what you are doing is generating your own language and a build tool/generator based on that. There's nothing wrong with it as such, but this is reinventing the wheel again (just like Google's GYP). Instead of custom dmake/build.pl you would have custom gnumakegen/gnumake_or_something. What is the benefit you get from this instead of using something like CMake that already has a mature implementation of this functionality? Even if CMake eventually turns out to be too slow, would it not make more sense to write your own custom CMake back end rather than the configuration/generation front end? For further information here is a Google Tech Talk about CMake and all related things (testing, code coverity, packaging, etc, etc) by one of the creators. If the build tool decision is not yet final, it is worth watching. http://www.youtube.com/watch?v=8Ut9o4OdSC0 - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
Re: [tools-dev] Re: Building OpenOffice.org with GNU make
Mathias Bauer wrote:
> > Understood. I pretty much agree with the goals, assume you did your
> > due diligence on verifying that gnu make does not trip over on the
> > full input set (that was my point of cautioning you, with the bjam
> > story), and am now trying to explore ideas on how to make the actual
> > makefiles appealing - the current state is not convincing, just
> > plainly because they're not substantially different from the ideal
> > dmake makefile in the existing system - and with their redundancies,
> > will suffer the same bitrot as the old system.
> Sounds good. :-)
> Any improvement would be welcome.
>
Ok, took longer than expected to find enough time for this, but here
we go:
After my initial performance worries had been sorted out, there are
basically two points left I'd require from an all-new build system:
- declarative syntax, as little duplication as possible
- ability to do cross-platform builds
For the former, look at this gnu makefile snippet (taken from
gnu_make cws):
[...]
$(eval $(call gb_Executable_Executable,rscdep))
$(eval $(call gb_Executable_set_linked_libs,rscdep,\
$(call gb_OOoLibrary_get_linknames,\
tl \
) \
))
$(eval $(call gb_Executable_add_exception_objects,rscdep,\
tools/bootstrp/appdef \
tools/bootstrp/command \
[etc.]
Call me purist, but there's a lot of noise in these lines, that
convey little, if any information. Plus, there's redundancy, in the
form of name prefixes, gnu make plumbing to call, etc., and also the
need to exhaustively list all object files (by listing paths names
of cxx files, stripped from the .cxx extension).
But the actual information contained in the above lines is actually
this:
rscdep source files: tools/bootstrp/*
rscdep link libs: tl
So I hacked up some python to parse a sort of declarative input
file, and generate gnu makefiles with Björn's eval framework out of
that. Corresponding to the above lines, this would be the input
snippet:
targets: [
{
executable: {
name: 'rscdep'
arch: host
}
sources: {
glob: '*.cxx'
paths: ['tools/bootstrp']
}
linking: ['tl']
} ]
I think it's cleaner, and there's definitely not much (should I say
any?) redundancy left. Additionally, one can enhance the script to
generate makefiles for pretty much every make tool of this world,
including eclipse/netbeans/visual studio project files.
The "arch: host" line is not yet fleshed out in the script, nor in
the gnu_make cws, but is indented for doing cross-builds: just
annotate every target with one out of "arch: [host:target:both]" to
later advise the build system which output architecture to compile
for.
Prototypical script, input files & patch against CWS gnu_make
attached (to handle input file globs). For testing, call it like
this: ./gmakegen.py tools_rscdep.in
What do you think?
-- Thorsten
gnumakegen.tgz
Description: application/compressed-tar
pgpJpcsblWlGK.pgp
Description: PGP signature
Re: [tools-dev] makefile 'export' supported by dmake?
Hi Christopher, > Is the makefile 'export' directive supported by dmake? If so, what would be > the proper syntax? http://hg.services.openoffice.org/DEV300/file/d90568f4aa0c/dmake/man/dmake.nc Ciao Frank -- - Frank Schönheit, Software Engineer frank.schoenh...@sun.com - - Sun Microsystems http://www.sun.com/staroffice - - OpenOffice.org Basehttp://dba.openoffice.org - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Sitz der Gesellschaft: - - Sun Microsystems GmbH, Sonnenallee 1, D-85551 Kirchheim-Heimstetten - - Amtsgericht München: HRB 161028 - - Geschäftsführer: Thomas Schröder, Wolfgang Engels, Wolf Frenkel - - Vorsitzender des Aufsichtsrates: Martin Häring - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - To unsubscribe, e-mail: dev-unsubscr...@tools.openoffice.org For additional commands, e-mail: dev-h...@tools.openoffice.org
