On Tue, Jun 05, 2007 at 09:52:49PM +1000, John Pye wrote: > Sorry for a bit of a delay on this one. I thought I would follow up with > some comments though. > > Firstly, > > Jean-Marc Lasgouttes wrote: > > John> The current system seems (from my rather superficial inspection) > > John> to revolve around lots of little script files written in Python; > > > > It produces these scripts actually. The logic is in C++ and needs some > > information from the data structures that may not be available outside. > > > > Perhaps a better approach would be for LyX to export an API (for example > via SWIG) that allowed *static* scripts to access the information they > need at run time. Added advantage, other than all the text file > generation, is that this would allow the scripts to be byte-compiled.
This would mean we need to keep some kind of API stable over time in order not to break external scripts. This is close to impossible as it basically forbids core changbes without introduction of compatibility layers. This might work for things that have been designed as libraries and long life cycles, but it is not feasible for a project in constant flux as we have with LyX. > Making the scripts static (rather than generated from inside C++) would > make solving problems such as my DIA one more straightforward and obvious. This is true, however, the price is far too high. > The concept I had here was that one might be writing some software with > a tarball that contains both C code and supporting LyX documentation. > The documentation ultimately gets embedded into the application file in > the form of let's say a windows .CHM help file. Imagine that a developer > want to make some changes to a small part of the program and perhaps > some small change in the documentation. If the LyX build process were > somehow integrated with scons, then developer could just call 'scons' > and rebuild the necessary bits of the program code plus the missing bits > of the documentation. The dependency checker would have everything it > needed without launching the full-blown LyX application to build the > documentation. In this case, LyX would just do the job of perhaps > transforming .lyx to .tex; scons would do the job of compiling the .tex > file together with .png images together into a set of .html files and > then convert the .html files to .chm files... People have successfully used plain Makefile for this purpose. > At present, because lyx uses its own build system, the efficiencies of > the above approach are not possible. ? Which 'own' build system? > I think that perhaps it does need a build system. My thesis takes a good > five minutes to build, and there is absolutely no indication of how far > through it is, or what it's doing at any given point in time. You know that it takes five minutes, and if it takes an average of two minutes to fill your coffee mug you'll know when 40% are done... > Building of images is not necessarily a simple process. For example, > what happens if I have an Asymptote-to-EPS filter and I am trying to > build a PDF file? And what if I have an SVG-to-PNG file and I am > producing a DVI? A proper build system would allow the missing 'tools' > to be clearly identified. Put it in a Makefile. > When I change things in 'sub-files' in my thesis (ie chapters) LyX > doesn't always correctly detect that something's changed, and it ends up > producing the same PDF file as before I made my changes. It seems that > there could be a missing link in the dependency tracking. Makefile. > In response to the above, I usually re-start lyx when I want to > re-create the PDF. But this seems to trigger a complete rebuild of > everything, even the bits such as image conversions (SVG-to-EPS etc). > Surely there should be caching of these things, and it should be robust > enough that LyX doesn't have to start over again on each instance. There > should be some kind of database of file hashes to check that source > files have not been modified -- and this sort of thing is provided for > example by SCons. You can warm up any cache you wantm e.g, from within a Makefile. > Processors are tending to go parallel these days, so there is going to > be a greater need for parallel builds. That is something that SCons > would allow, and this would make a big difference particularly for > preparing images for embedding in large documents. make -j 8 > Another problem is that the current system copies 'source' files into a > temporary directory before running the script. This causes files with > relative file references within them to break, as for example with DIA > files as I noticed recently. SCons fixes this problem because, at your > preference, it can either build files in-place, or can generate a > parallel 'build directory' into which all necessary dependency files are > copied before they are needed. You can provide 'copiers' that do the Right Thing. > Finally you mentioned that the scripts are generated from C++ code. > Writing out script files from C++ is usually not the ideal architecture > in my experience. Surely not. But the conclusion that therefore an all-script solution is preferable looks strange as well. > So you mentioned that the build process is trivial but I think that the > above examples show that it's perhaps not so trivial, and perhaps if a > full-scale build system were used it might allow the LyX development > effort to reduce the amount of 'support code' that needs to be > maintained, so that you can focus on some of the more core LyX > functionality? The point is that LyX is not a build system, cannot be, and does not want to be. If you need one (and you have shown good reasons for that) just use one. If you prefer Scons over make, so be it, but in both cases this is a solution _wrapping_ LyX, not a solution integrated in LyX. Andre'
