Re: [PATCH] serious doubts about waf
On Tue, Nov 24, 2009 at 11:02:37PM +0100, John Mandereau wrote: > Le mardi 24 novembre 2009 à 21:20 +, Graham Percival a écrit : > > On the 16th, I gave up and fixed this one myself. However, I > > discovered that I couldn't copy files from srcdir into the blddir > > without changing their name, which kind-of borks > > Documentation/pictures/. > > I'm quite certain we can avoid copying pictures from outdir to blddir; > however, if more generally we can't transform a file in srcdir to a file > in blddir that has the same name, This is a current limitation of waf. The author stated that fixing it was on his TODO list, but that was over a year ago. We can either do weird stuff to avoid this limitation, ask him to fix this bug, or try to fix it ourselves. I personally favor the latter two options; I'm sick of doing wierd stuff just to fit within the limitations of a build system. > It's not for making the pictures available to Texinfo > formatters, as a simple -I srcdir (which we already have in current > build system IIRC) would do the job. For some pictures, we resize them, > but then for clarity we could give a different name to the source. I suppose so. > For other pictures, IIRC we just symlink them so that they're > caught by mirrortree (a module used by www_post.py that links > files into *-root). I don't want to have mirrortree in the new build system if we can avoid it. > Is it worth going on investigating this? I think so, so I'll > have a look. Waf will not be used for 2.14. If there are truly no translation issues that need addressing, then go ahead, but it'll be a few weeks or months before we'll try any switching. Oh, for that matter, we should probably re-sync dev/waf with master (to get the recent manual renamings), copy the wscript* files from dev/gperciva, then delete dev/gperciva. I'm not in any kind of rush to touch any of this until after 2.14, though. Cheers, - Graham ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: [PATCH] serious doubts about waf
Le mardi 24 novembre 2009 à 21:20 +, Graham Percival a écrit : > On the 16th, I gave up and fixed this one myself. However, I > discovered that I couldn't copy files from srcdir into the blddir > without changing their name, which kind-of borks > Documentation/pictures/. I'm quite certain we can avoid copying pictures from outdir to blddir; however, if more generally we can't transform a file in srcdir to a file in blddir that has the same name, let's sit down two minutes to see why we do this. It's not for making the pictures available to Texinfo formatters, as a simple -I srcdir (which we already have in current build system IIRC) would do the job. For some pictures, we resize them, but then for clarity we could give a different name to the source. For other pictures, IIRC we just symlink them so that they're caught by mirrortree (a module used by www_post.py that links files into *-root). Is it worth going on investigating this? I think so, so I'll have a look. Best, John signature.asc Description: Ceci est une partie de message numériquement signée ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: [PATCH] serious doubts about waf
On Tue, Nov 24, 2009 at 09:42:05PM +0100, John Mandereau wrote: > Le jeudi 12 novembre 2009 à 17:54 +, Graham Percival a écrit : > > 1) if you can't duplicate the problem after checking out > > dev/gperciva, then I'll check a third time to see if there's any > > old version of waf floating around on this machine. > > No, I couldn't duplicate. But, it might be necessary to clear some > cache using distclean, then reconfigure, so you can have a command like I clear the cache with rm -rf waf-out On the 16th, I gave up and fixed this one myself. However, I discovered that I couldn't copy files from srcdir into the blddir without changing their name, which kind-of borks Documentation/pictures/. That message is on -devel somewhere. Waf is dead until at least after 2.14. Cheers, - Graham ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: [PATCH] serious doubts about waf
Sorry for the delay, it often takes me a while to catch all traffic on Lily lists when I'm back from a trip, even a few-days one. Le jeudi 12 novembre 2009 à 17:54 +, Graham Percival a écrit : > 1) if you can't duplicate the problem after checking out > dev/gperciva, then I'll check a third time to see if there's any > old version of waf floating around on this machine. No, I couldn't duplicate. But, it might be necessary to clear some cache using distclean, then reconfigure, so you can have a command like waf-light distclean configure build Best, John signature.asc Description: Ceci est une partie de message numériquement signée ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: [PATCH] serious doubts about waf
Ok, it seems the the key was to call bld.rescan(node) after adding it. I'm off to a short conference tomorrow. I expected to have internet access, but when I mentioned it to one of the professors today, he just laughed at me. :(so if I'm not around for a few days, that's why. I'm going to work on 1) the waf build 2) merging the two init-pl files. in the dev/gperciva branch. I don't know how much time I'll have, but it's traditional for me to do build system stuff at conferences. I have no plans on touching anything else, so if you're waiting for me to do anything, either do it yourself or keep on waiting. Cheers, - Graham On Thu, Nov 12, 2009 at 5:54 PM, Graham Percival wrote: > On Thu, Nov 12, 2009 at 10:21:16AM +0100, John Mandereau wrote: >> Le jeudi 12 novembre 2009 à 00:21 +0100, John Mandereau a écrit : >> > Does the attached patch convince you to go on on Waf? >> >> Actually, please ignore my patch and checkout a SVN copy of Waf instead >> >> svn checkout http://waf.googlecode.com/svn/trunk/ && mv trunk waf >> >> then invoke waf-light from this checkout. > > I'm already working from waf. I did an svn update. It still > fails the first time I run it, while working perfectly the second > time. > > In case there's any misunderstanding, I'm talking about the > dev/gperciva branch. The minimal example that Thomas Nagy sent > works just fine. > > > 1) if you can't duplicate the problem after checking out > dev/gperciva, then I'll check a third time to see if there's any > old version of waf floating around on this machine. > > 2) if you can duplicate it in dev/gperciva, then please either > fix the problem in my wscripts, or follow up with the waf people > (if you can extract another small example). > > Cheers, > - Graham > ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: [PATCH] serious doubts about waf
On Thu, Nov 12, 2009 at 10:21:16AM +0100, John Mandereau wrote: > Le jeudi 12 novembre 2009 à 00:21 +0100, John Mandereau a écrit : > > Does the attached patch convince you to go on on Waf? > > Actually, please ignore my patch and checkout a SVN copy of Waf instead > > svn checkout http://waf.googlecode.com/svn/trunk/ && mv trunk waf > > then invoke waf-light from this checkout. I'm already working from waf. I did an svn update. It still fails the first time I run it, while working perfectly the second time. In case there's any misunderstanding, I'm talking about the dev/gperciva branch. The minimal example that Thomas Nagy sent works just fine. 1) if you can't duplicate the problem after checking out dev/gperciva, then I'll check a third time to see if there's any old version of waf floating around on this machine. 2) if you can duplicate it in dev/gperciva, then please either fix the problem in my wscripts, or follow up with the waf people (if you can extract another small example). Cheers, - Graham ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: [PATCH] serious doubts about waf
Le jeudi 12 novembre 2009 à 00:21 +0100, John Mandereau a écrit : > Does the attached patch convince you to go on on Waf? Actually, please ignore my patch and checkout a SVN copy of Waf instead svn checkout http://waf.googlecode.com/svn/trunk/ && mv trunk waf then invoke waf-light from this checkout. Best, John signature.asc Description: Ceci est une partie de message numériquement signée ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: serious doubts about waf
On Wed, 11 Nov 2009, Graham Percival wrote: > What's the advantage of automake over the current system? (I'm not > even certain what the current system is called!) If anyone goes into this, could they PLEASE, PLEASE, document it slightly better? [I'm almost certainly I'm doing insane things when I'm building the packages in Debian.] Don Armstrong -- Some pirates achieved immortality by great deeds of cruelty or derring-do. Some achieved immortality by amassing great wealth. But the captain had long ago decided that he would, on the whole, prefer to achieve immortality by not dying. -- Terry Pratchet _The Color of Magic_ http://www.donarmstrong.com http://rzlab.ucr.edu ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: [PATCH] Re: serious doubts about waf
Le mercredi 11 novembre 2009 à 23:36 +, Graham Percival a écrit : > Hmm. I don't completely follow, but does this explain why waf > always succeeds on the second attempt? If not, it might be > something weirder... but then again, if you're following it up on > the waf-user list, then it's no longer my concern. I can't answer your question, because I haven't worked out the order Waf executes code, especially what code is called when reading the wscripts and what code is called when actually building. > I'll check it out tomorrow. If it works, I'll spend another few > hours on waf -- particularly, trying out things that are more > likely to fail. If it wasn't likely to be true, I'd suspect some sarcasm here :-) > If you're willing to follow up on any more cases like this, then I > think I'll be willing to keep on waffing. This means being almost completely idle on the academic side (because learning Italian and new math tools already takes much time), but as my PhD starts in January I feel like I can afford going for it; it will make me more aware of Waf code than be productive on wscripts for LilyPond, but this seems necessary to make those maoing wscripts actually work. Best, John signature.asc Description: Ceci est une partie de message numériquement signée ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: serious doubts about waf
On Wed, Nov 11, 2009 at 02:38:12PM -0800, Patrick McCarty wrote: > I don't have much experience *using* automake, but from what I've > read: > > - A Makefile.am is easier to maintain than an equivalent handcoded > makefile. > - The generated makefiles will be very portable (not reliant on GNU > make). > - Automake integrates well with Autoconf, probably more so than the > current stepmake system. One problem with automake (and autoconf) is that it's incredibly difficult to debug if something goes wrong. Ciao, Kili ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: [PATCH] Re: serious doubts about waf
On Thu, Nov 12, 2009 at 12:21:47AM +0100, John Mandereau wrote: > After hours of debugging and code inspection, I think you did nothing > weird, it's a bug in Waf code: Node.py:Node.find_dir(), which looks up > directories, call bld.rescan() which among others removes nodes of > directories that haven't been yet created, like our call to > bld.path.ensure_dir_node_from_path. Hmm. I don't completely follow, but does this explain why waf always succeeds on the second attempt? If not, it might be something weirder... but then again, if you're following it up on the waf-user list, then it's no longer my concern. > Does the attached patch convince you to go on on Waf? I'm not going to > do more Waf advocacy that I already did, because you may rightly accuse > me of fluffing around :-) I'll check it out tomorrow. If it works, I'll spend another few hours on waf -- particularly, trying out things that are more likely to fail. If you're willing to follow up on any more cases like this, then I think I'll be willing to keep on waffing. Cheers, - Graham ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: serious doubts about waf
Le mercredi 11 novembre 2009 à 23:07 +, Graham Percival a écrit : > Given that most of our users are on windows, I defined > "portability" as "it will run on operating system X with Y extra > software installed", where X should be large and Y should be > small. Then SCons and Waf are reasonable choices in this respect. > In all > seriousness, I think that make (possibly including automake) is > the best build system. All the "next generation" build systems > seem to shoot their feet off in various ways... cmake has the > strictly-defined "open source" (i.e. "closed documentation") as > well as its own invented scripting language... Build systems tend to be complicated that specialized scripting languages are no longer suitable, and I have neither the skills nor the motivation to extend GNU Make or consorts (cmake) by hacking its source code. > waf has the > oddities details here... Waf is not mature, it's kind of experimental, but come on, we already almost managed to live with another piece of software that has a moving target in place of a stable API, and you even convinced us to choose it to output the website in HTML :-) > apparently scons is slow, although I > personally don't care about the speed issue. I don't care either at the moment, but if we went for Scons we would probably all sing the same rant like a choir, like "Cygwin is slow!" a few years ago :-P (sorry for the wrong analogy between a build system and end-user software, but I really couldn't help) Best, John signature.asc Description: Ceci est une partie de message numériquement signée ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
[PATCH] Re: serious doubts about waf
Le mardi 10 novembre 2009 à 23:34 +, Graham Percival a écrit : > The basic problem is that waf doesn't like to have directories in the > build tree that aren't present in the source tree. After asking for > help, the main author pointed out a not-really-documented function > that allowed me to create an extra directory: > > http://groups.google.com/group/waf-users/browse_thread/thread/1e3774fdf387e083 > > However, this special function doesn't seem to work instantaneously... > or else I'm doing something really weird in > Documentation/wscript_build After hours of debugging and code inspection, I think you did nothing weird, it's a bug in Waf code: Node.py:Node.find_dir(), which looks up directories, call bld.rescan() which among others removes nodes of directories that haven't been yet created, like our call to bld.path.ensure_dir_node_from_path. Does the attached patch convince you to go on on Waf? I'm not going to do more Waf advocacy that I already did, because you may rightly accuse me of fluffing around :-) I'm following this up on waf-users too... > In the discussion (shown in the above link), it turns out that the waf > API is going to change in version 1.6. This new version is only in > the planning stages; no code exists for it... but I'm really starting > to question if we really want to use waf. I have to admit that I was > really shaken by the directory thing. I assumed that since it was > python, I could do anything that python could do -- so making > directories, moves files from directory to directory, etc. would be > easy. Apparently not. It's not easy because custom directories and files management has to be done in a suitable place in a build system, and as you just discovered Waf has much less support for this that we'd naturally expect for building docs the way we want. Best, John Index: wafadmin/Node.py === --- wafadmin/Node.py (révision 6794) +++ wafadmin/Node.py (copie de travail) @@ -276,6 +276,9 @@ current = prev.childs.get(name, None) if current is None: current = self.__class__(name, prev, DIR) +new_path = os.path.join (prev.name, name) +if not os.path.isdir (new_path): + os.mkdir (new_path) return current # FIXME: remove in waf 1.6 signature.asc Description: Ceci est une partie de message numériquement signée ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: serious doubts about waf
On Wed, Nov 11, 2009 at 02:38:12PM -0800, Patrick McCarty wrote: > On 2009-11-11, Graham Percival wrote: > > What's the advantage of automake over the current system? (I'm > > not even certain what the current system is called!) > > I don't have much experience *using* automake, but from what I've > read: > > - A Makefile.am is easier to maintain than an equivalent handcoded > makefile. If we were doing this from scratch, I'd definitely push for makefile.am rather than the stepmake system. However, we already have an almost-working stepmake system; at the moment I doubt that the cost of switching to makefile.am is worth the benefits. > - The generated makefiles will be very portable (not reliant on GNU > make). Given that most of our users are on windows, I defined "portability" as "it will run on operating system X with Y extra software installed", where X should be large and Y should be small. Although I'd like to, I can't claim that cygwin (or manually installing the GNU tools on windows) is a "small Y". I'm quite annoyed at the build system situation. In all seriousness, I think that make (possibly including automake) is the best build system. All the "next generation" build systems seem to shoot their feet off in various ways... cmake has the strictly-defined "open source" (i.e. "closed documentation") as well as its own invented scripting language... waf has the oddities details here... apparently scons is slow, although I personally don't care about the speed issue. Cheers, - Graham ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: serious doubts about waf
On 2009-11-11, Graham Percival wrote: > On Tue, Nov 10, 2009 at 06:36:37PM -0800, Patrick McCarty wrote: > > On Tue, Nov 10, 2009 at 3:58 PM, Graham Percival > > wrote: > > > On Tue, Nov 10, 2009 at 11:34:35PM +, Graham Percival wrote: > > >> I'm just about ready to give up on waf. > > > > > > As Jan put it, "a new build system should be a breath of fresh > > > air". I'm not feeling fresh. > > > > Though I haven't been following these discussions very closely, I have > > looked into automake a little bit. It sounds like we could make it > > work. At least, I think it would simplify makefile maintenance once > > we have initially set everything up. > > What's the advantage of automake over the current system? (I'm > not even certain what the current system is called!) I don't have much experience *using* automake, but from what I've read: - A Makefile.am is easier to maintain than an equivalent handcoded makefile. - The generated makefiles will be very portable (not reliant on GNU make). - Automake integrates well with Autoconf, probably more so than the current stepmake system. > I used to maintain it (and the duplicate qmake build system) for > marsyas before I got so annoyed at having two build systems and > switched to cmake. Given that windows users couldn't use > automake, I'm not certain it's worth the effort of switching to > it. Well, there's always the option of keeping the generated Makefile.in files alongside the Makefile.am automake files in the source tree. Then Windows users should be able to build stuff if they have some incarnation of "make" of their system, since Automake-generated makefiles do not depend on GNU Make. -Patrick ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: serious doubts about waf
On Tue, Nov 10, 2009 at 06:36:37PM -0800, Patrick McCarty wrote: > On Tue, Nov 10, 2009 at 3:58 PM, Graham Percival > wrote: > > On Tue, Nov 10, 2009 at 11:34:35PM +, Graham Percival wrote: > >> I'm just about ready to give up on waf. > > > > As Jan put it, "a new build system should be a breath of fresh > > air". I'm not feeling fresh. > > Though I haven't been following these discussions very closely, I have > looked into automake a little bit. It sounds like we could make it > work. At least, I think it would simplify makefile maintenance once > we have initially set everything up. What's the advantage of automake over the current system? (I'm not even certain what the current system is called!) I used to maintain it (and the duplicate qmake build system) for marsyas before I got so annoyed at having two build systems and switched to cmake. Given that windows users couldn't use automake, I'm not certain it's worth the effort of switching to it. Cheers, - Graham ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: serious doubts about waf
On Tue, Nov 10, 2009 at 3:58 PM, Graham Percival wrote: > On Tue, Nov 10, 2009 at 11:34:35PM +, Graham Percival wrote: >> I'm just about ready to give up on waf. > > In case anybody thinks this was rather fast, I spent 3-4 hours on > Saturday doing waf, and 7-9 hours today. I'm not overly eager to > throw away approximately 10 hours of work, but if it's not going > to pan out, then better to stop it now rather than later. > > As Jan put it, "a new build system should be a breath of fresh > air". I'm not feeling fresh. Though I haven't been following these discussions very closely, I have looked into automake a little bit. It sounds like we could make it work. At least, I think it would simplify makefile maintenance once we have initially set everything up. -Patrick ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: serious doubts about waf
On Tue, Nov 10, 2009 at 11:34:35PM +, Graham Percival wrote: > I'm just about ready to give up on waf. In case anybody thinks this was rather fast, I spent 3-4 hours on Saturday doing waf, and 7-9 hours today. I'm not overly eager to throw away approximately 10 hours of work, but if it's not going to pan out, then better to stop it now rather than later. As Jan put it, "a new build system should be a breath of fresh air". I'm not feeling fresh. Cheers, - Graham ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
serious doubts about waf
I'm just about ready to give up on waf. (in particular, I'm not doing anything else unless somebody convinces me otherwise) The dev/gperciva branch can build various manuals in HTML and pdf, including the texi2html init files and css stuff. However, you need to run it twice -- the first time you run waf, it will die. The second time it's just fine. The basic problem is that waf doesn't like to have directories in the build tree that aren't present in the source tree. After asking for help, the main author pointed out a not-really-documented function that allowed me to create an extra directory: http://groups.google.com/group/waf-users/browse_thread/thread/1e3774fdf387e083 However, this special function doesn't seem to work instantaneously... or else I'm doing something really weird in Documentation/wscript_build In the discussion (shown in the above link), it turns out that the waf API is going to change in version 1.6. This new version is only in the planning stages; no code exists for it... but I'm really starting to question if we really want to use waf. I have to admit that I was really shaken by the directory thing. I assumed that since it was python, I could do anything that python could do -- so making directories, moves files from directory to directory, etc. would be easy. Apparently not. I'm therefore considering wholeheartedly embracing automake. Yes, it's a bit clunky, but it's already a completely-working system (other than 2 or 3 things in the Issue tracker). I could replace bits and pieces of it to make it easier to understand + maintain. Cheers, - Graham ___ lilypond-devel mailing list lilypond-devel@gnu.org http://lists.gnu.org/mailman/listinfo/lilypond-devel
